Blogging Workflow

In this post, I want to talk about some of the technical details behind this site and the workflow that I have for writing.

This site is built with Jekyll and hosted on a private Github repository, with two branches. The master branch is where the published site lives (basically, the contents of the _site folder), while the source branch is where the working files are kept, under version control.

“Doesn’t Github Pages build your site for you automatically?”

Yes, it’s supposed to, but not if you use custom plugins that are not supported by Github Pages. I have a custom Ruby plugin that generates category pages, so there’s that.

I use fish, so I wrote a function sy that acts on the surface like a command line binary.

For the purposes of this post, I’ve renamed the directories. ~/website is the directory where my website source files live, and ~/website.publish is the directory where the published files live. There is also a ~/website.publish.backup folder which contains a backup of the previous version of the published files. (Yes, everything is managed under VC, but still.)

This is how sy looks like, in its entirety:

function sy

    switch (echo $argv[1])
        case "--serve"
            subl ~/website
            cd ~/website
            rvm use 2.1.1
            jekyll serve -w
            cd
        case "--publish"
            rm -rf ~/website.publish.backup
            rvm use 2.1.1
            cd ~/website | jekyll build
            cp -fR ~/website.publish ~/website.publish.backup
            cd ~/website.publish/
            rm -r *
            cp -fR ~/website/_site/  ~/website.publish/
            cd
        case "--new"
            set newdate (date +%Y-%m-%d)
            set categories "[$argv[2]"
            for x in $argv[3..-2]
                set categories $categories","$x
            end
            set categories $categories"]"
            echo -e "---\ntitle:\nlayout: post\ncategories: $categories\ndate: $newdate\n---" > ~/website/_posts/$argv[2]/$newdate-$argv[-1].markdown
            subl ~/website/_posts/$argv[2]/$newdate-$argv[-1].markdown
        case "--gallery"
            set python_arguments ""
            for x in $argv[3..-2]
                set python_arguments $python_arguments $x
            end
            set path ""
            for x in $argv[3..-2]
                set path $path"/"$x
            end
            python ~/website/_plugins/gallery.py $argv[2] $python_arguments $argv[-1] >> ~/website/_posts/$argv[3]/$argv[5].markdown
            mv $argv[2] ~/website$path
        case ""
            echo -e "\nUsage:\n"
            echo -e "--serve\nOpens the website folder in Sublime Text, and opens a local server at localhost:4000\n"
            echo -e "--publish\nBuilds the Jekyll files from source and copies it to the publish folder. The existing publish folder is backed up in the same directory.\n"
            echo -e "--new [main-category] [nested-categories] [post-name]\nCreates a new post in the [main-category] folder titled [post-name]. The date defaults to today's date, and the categories tag is populated with the [main-category] followed by [nested-categories] if any.\n"
            echo -e "--gallery [folder] [main-category] [nested-categories] [post-name]\nRuns gallery.py on [folder] to generate HTML markup and append to [post-name] (which must exist). Specified folder is then moved to appropriate location.\n"
        case "*"
            echo "Invalid command."
    end
end

Let’s go through each option one by one.

case "--serve"
    subl ~/website
    cd ~/website
    rvm use 2.1.1
    jekyll serve -w
    cd

sy, when used with the --serve flag, opens the website directory in Sublime Text and runs jekyll serve, which opens a local copy for development preview. Straightforward enough.

case "--publish"
    rm -rf ~/website.publish.backup
    rvm use 2.1.1
    cd ~/website | jekyll build
    cp -fR ~/website.publish ~/website.publish.backup
    cd ~/website.publish/
    rm -r *
    cp -fR ~/website/_site/  ~/website.publish/
    cd

The --publish option builds the source files in ~/website and copies it to ~/website.publish, but not before it makes a copy of original published files in ~/website.publish/backup. There’s a lot of potentially dangerous rm -rf-ing here, and I’ll take another crack at refactoring it when I have the time.

case "--new"
    set newdate (date +%Y-%m-%d)
    set categories "[$argv[2]"
    for x in $argv[3..-2]
        set categories $categories","$x
    end
    set categories $categories"]"
    echo -e "---\ntitle:\nlayout: post\ncategories: $categories\ndate: $newdate\n---" > ~/website/_posts/$argv[2]/$newdate-$argv[-1].markdown
    subl ~/website/_posts/$argv[2]/$newdate-$argv[-1].markdown

The --new option is where it starts to get interesting. In the description, it says:

The date defaults to today's date, and the categories tag is populated with the [main-category] followed by [nested-categories] if any.

--new takes three (or more) options: main-category, nested-categories, and post-name. It takes these options and creates a new Markdown file in the appropriate location in the _post directory, with the necessary YAML front matter appended. For example, if the following command is run (today):

sy --new coding ruby i-made-a-new-plugin

A new post will be created in ~/website/_posts/coding/2014-12-02-i-made-a-new-plugin.markdown with the following YAML front matter:

---
title:
layout: post
categories: [coding,ruby]
date: 2014-12-02
---

Finally, this file is opened in Sublime Text for immediate editing.

case "--gallery"
    set python_arguments ""
    for x in $argv[3..-2]
        set python_arguments $python_arguments $x
    end
    set path ""
    for x in $argv[3..-2]
        set path $path"/"$x
    end
    python ~/website/_plugins/gallery.py $argv[2] $python_arguments $argv[-1] >> ~/website/_posts/$argv[3]/$argv[5].markdown
    mv $argv[2] ~/website$path

--gallery is by far the most complex bit of sy. Its arguments are similar to --new’s, except it takes in an additional [folder] argument. Its main purpose is to generate HTML markup for the photo galleries on the site, by acting as a wrapper for a Python script that I wrote, gallery.py.

Its first argument is the full location of a folder, located anywhere, containing the images to be used in the post. The images in the folder must be arranged in a specific manner:

Inline images live in the root of the folder.
Gallery images should be placed in a subfolder called gallery.
Thumbnails for the gallery above should be placed in a subfolder called thumbs. gallery.py will check and match all the images in gallery with those in thumbs and shout if there’s a mismatch.
The cover image should be placed in a subfolder called cover.

The subsequent arguments are the main and nested categories, as well as the name of the post (which must already exist). gallery.py then generates the HTML markup for all of the different types of images, which sy appends to the post according to the arguments supplied. Lastly, it moves the folder to the correct place in the source directory.

I’ll spare you the details of gallery.py, but the last thing I’ll like to mention in the section is that it leverages on the pyexiv2 library to extract the title metadata from the images if it can find any.

Why do I do this?

Lightroom 5 is my photo management tool of choice, and it allows me to enter the title and caption of each image. When exported, this information continues to live in the metadata of the image. When gallery.py is generating the markup, it looks to see if each image’s title field is populated, and if so, enters it as the title attribute in the img tag. This title attribute is recognized by Magnific Popup and will then be displayed beneath the lightbox.

So yeah, that’s the gist of my blogging workflow. There’re a couple more details I didn’t go into (such as my Lightroom workflow, and how it segues into the Jekyll workflow), and I’ll append to this post if I feel like documenting the rest of it.