Writing a Novel Using Markdown: Part Two

In my previous post, I spoke a little about my Markdown workflow. I ended it prematurely because I didn’t have a prepared version of my book, Red Star Falling, that I could share with you.

Now I do.

You can download Multimarkdown / Markdown1 source of the complete book here. This ZIP archive contains the text of the book (cheekily, I’ve redacted most of the story), the Writer as a Stranger Cascading Style Sheet, the cover image, and a little Möbius vector graphic I’m using as a section break.

As I mentioned in my previous post, once you have your book in Markdown (i.e. a plain text file), you can use whatever tools you wish to produce your ebook–in whatever format(s) that might be.

I use a Mac OS X feature called a service. This does nothing more than run a bash script, which is a series of commands telling the computer what to do with the Markdown files. You can download the service here. It works on OS X Mountain Lion, but it will probably work on earlier versions of the Mac. The raw source for the bash script (i.e. just the series of commands) is available here, but I’ve also appended it to this post. All computers, including those running Windows, can execute bash scripts–but they might need some tweaking.

Here are the files that make up the text of the book (i.e. those in the ZIP):

Screen Shot 2013 06 23 at 20 33 23

Note that the files are numbered to preserve their order. Each Markdown file ends with a few blank lines to pad them out when they’re combined into a single file 2.

All the important information about the book–who the author is, the title, revision number, where the cover file is located–all of that is declared in the file ‘1_meta_data.mmd’. This means that, if you wish to use this method to produce your own book, the script will work as long as you’ve updated the metadata in this file.

What extra stuff you need

It would be nice if OS X came with ebook-publication tools built-in. Alas, it doesn’t. My script uses two powerful programs: ebook-convert and multimarkdown. You need to install these locally first. Note that my script tells the computer exactly where the multimarkdown command lives (it’s in /usr/local/bin/multimarkdown). I don’t know why I had to give this path to make the script work, but I did.

Using the OS X Service

This is really straightforward. On your Mac, put the service into Home > Library > Services. If your Library folder is hidden, you’ll need to Option-click the ‘Go’ menu in Finder and select it. Logout, login, and then you should see service (called • [Toy] Create MOBI-EPUB-PDF from multiple MMD, using Metadata copy) in the contextual menu when any Finder items are highlighted.

Download my example archive, highlight all the Markdown files, then right-click and select the service. It will then create .mobi, .epub and .pdf versions of Red Star Falling.

These files will be placed in the sub-folder called Publication_ready. A log file in that folder, called Log.txt, will be updated with each new conversion; look there if you want to find out why things have gone wrong. Intermediate .html files will be deleted.

Because I publish (for now) only to the Kindle, I’ve made sure that the .mobi file works perfectly on the Kindle for long-form prose3. The .epub is serviceable but not pretty. The .pdf likewise. One advantage of scripting the process is that you can tweak the settings for .epub output until you’re happy and then apply those to all your books using the same script.

Cool things you can now do

  • Because this is a script, you can tweak it to:

    • Pass the .epub file to a validation tool. I’d recommend epubcheck. Once this is in the script, it’ll happen every time.

    • Push the finalised files to a central folder where you keep all your ‘publication ready’ books.

  • You can use a single style sheet for all your books. The format–.epub, .mobi, or whatever–should not change the look and feel of your final books. Once set, you don’t need to worry about this again.

  • Creating a new book is as simple as duplicating the directory and changing the metadata. The main text of your story is simply a Markdown file: human-readable, portable, and small.

  • Use aliases. In this book, I have a section–i.e. a Markdown file–that tells the reader about my other books. I don’t want to replicate this for every new book, and I want any future books to compile with an update list of my works. So I use an alias to point to a central Markdown file that contains this information. When I update that file, any book will compile with an updated list of my works.

  • Set a watch on the folder. Perhaps trigger the script to run (suitably modified) when the main story file is updated. That way, all the publication-ready files will self-update.

What are the implications of this workflow for the writing process?

None. The act of writing itself should be completely independent of all the complexities that happen to the Markdown file later, when it’s processed. It’s more convenient in several ways. You’ll not have to worry about smart quotes, or font sizes, or anything like that. All you have to concentrate on is writing the story in this format:

## Red Star Falling

In the moment before Saskia Brandt awoke, she had a vision of red chrysanthemums falling. The flowers looked unreal. Their stems were too straight and their falls too slow. Their *Gestalt* was artful sadness.

Then the sky beyond them wintered and the dream faded.

For that, I’d recommend any number of cool programs, including ByWord and iAWriter.

Coda: The script itself

In case you’re here just for this script and want to have a look at it, here it is.

## MULTIMARKDOWN FILES > MOBI, PDF AND EPUB

## IH, 23 JUNE 2013

## This script takes a group of files in Multimarkdown,
## combines them into a single file, and then produces
## MOBI, EPUB and PDF. Also runs an epub check program.

## Expects metadata in a file called 1_meta_data.md

## Expects to have a directory called Publication_Ready.
## Will pipe command output to Publication_Ready/Log.txt

## Unusual commands used (i.e. you'll need to install these):
## ebook-convert [<http://manual.calibre-ebook.com/cli/ebook-convert.html>]
## multimarkdown [<http://fletcherpenney.net/multimarkdown/use/>]

###########################################

# Find out the directory that this script is working within
DIR=`dirname "$1"`

# Go there
cd "$DIR"

# Extract meta data from the file called '1_meta_data.mmd'
title=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Title"`
revision=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Revision"`
short_title=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="ShortTitle"`
author=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Author"`
series=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Series"`
series_index=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="SeriesIndex"`
tags=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Tags"`
cover=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Cover"`

# LOG Date
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo `date` $revision >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt

# Concatenate all the files passed to this script
cat "$@" > $short_title.mmd 

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo ******* Compiling HTML from Multimarkdown... >> Publication_Ready/Log.txt

# Run 'multimarkdown' to produce HTML, smart mode, appending result to log
/usr/local/bin/multimarkdown $short_title.mmd --output=$short_title.html --smart >> Publication_Ready/Log.txt

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo ******* Compiling MOBI… >> Publication_Ready/Log.txt

# Make MOBI
ebook-convert $short_title.html Publication_Ready/$short_title[$revision].mobi --authors="$author" --series="$series" --series-index=$series_index --title="$title" --tags="$tags" --output-profile=kindle >> Publication_Ready/Log.txt

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo ******* Compiling PDF… >> Publication_Ready/Log.txt

# Make PDF
ebook-convert $short_title.html Publication_Ready/$short_title[$revision].pdf --authors="$author" --series="$series" --series-index=$series_index --title="$title" --tags="$tags" >> Publication_Ready/Log.txt

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo ******* Compiling EPUB… >> Publication_Ready/Log.txt

# Make EPUB
ebook-convert $short_title.html Publication_Ready/$short_title[$revision].epub --remove-first-image --authors="$author" --series="$series" --series-index=$series_index --title="$title" --tags="$tags" --output-profile=ipad >> Publication_Ready/Log.txt 

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo Cleaning up temporary files... >> Publication_Ready/Log.txt

# Clean up the temporary HTML and mmd files
rm $short_title.html $short_title.mmd >> Publication_Ready/Log.txt

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo Job done bish bash `date` >> Publication_Ready/Log.txt


  1. Feel free to use this CSS with your own books. Think of it as the Writer as a Stranger ‘house style’. It will do things like indent all paragraphs unless they follow a chapter heading or section break and deal with blockquotes appropriately. I’ve iterated it a few times over the past few weeks to get it looking how I like it. ↩

  2. I’m aware that I can do this programmatically. (And, interestingly, the Multimarkdown command line tool has an option to concatenate multiple input files; however, it didn’t work for me and I couldn’t figure out why. Hence the old-school solution.) ↩

  3. You should always check your .mobi file using the Kindle Previewer application. ↩

Published by

Ian Hocking

Writer and psychologist.

19 thoughts on “Writing a Novel Using Markdown: Part Two”

  1. Wonderful articles – nice to know I’m not the only crazy out here who has been seduced by Markdown. Your workflow seems to add the much-needed flexibility that basic Gruber Markdown does lack – I’ll shamelessly steal some of your ideas ;o)

    My own workflow is an attempt at the other extreme – Gruber Markdown only, although I’m then getting what flexibility I can from Calibre conversion (Calibre will take raw Markdown as input).

    Briefly, I do this :

    – Write in pure Gruber Markdown
    – Convert w/ Calibre to ePub
    — have Calibre remove blank lines between paragraphs and add indents
    — specify page break for all h6’s under *Structure Detection*
    — add one line of “CSS” under “Look & Feel”
    (h1,h2,h3,h4,h5,h6{text-align:center;})
    – Use ePub as basis for converting to any further formats needed – Mobi, PDF, you name it

    The resulting ePub will go fault-free through http://validator.idpf.org/ *unless* one has a number first in the text following any of the Markdown # to ###### tags (causes an “id” in the ePub with number as first character in name, and that’s a no-no).

    The ePub can be uploaded as-is to Amazon KDP, and the only errors I’ve ever gotten are typos ;o)

    As always with Markdown, you can add any and all html you like (I do use the occasional ), but you can get a quite readable layout without using any.

    Title page and start of first chapter of your example could be done as shown below. You might get a smile out of taking it through Gruber’s online Dingus converter – just remember that Calibre will take care of centering and indenting.

    Have fun – and thanks again for the article and the encouragement to keep using Markdown ;o)

    *************************** start of example *****************************

    ### IAN HOCKING

    # RED STAR FALLING

    ##### (c) Ian Hocking 2013

    ######

    ## One

    **I**n the moment before Saskia Brandt awoke, she had a vis­ion of red chrys­an­them­ums fall­ing. The flowers looked unreal. Their stems were too straight and their falls too slow. Their *Gestalt* was art­ful sadness.

    Then the sky beyond them wintered and the dream faded.

    ####

    **S**he awoke to freezing darkness. Her throat was dry. She turned and coughed.

    *************************** end of example *****************************

  2. Ahem … your comment system does some conversion of its own, it seems. Anyway, in the above example, pls replace — with triple hyphens (- – -) and add an extra blank line after the “(c) Ian Hocking” line. Sorry about the fuss. Just shows no formatting is foolproof ;o)

  3. Great stuff, Thomas! Your method looks interesting. I’ve actually made my own workflow a little difficult since I wrote this article, so it might well be time for me to update it… As you imply, the more minimal, the better…

  4. Hi ?an,

    Thanks for cool infos on your website. I am CPA and I’d like to write some ebooks for businessman. I decided to use markdown but ? dont know so much thing.

    Could you please offer me some easy editor software for markdown ? And where can ? buy markdown book template ?

    Thanks in advance

    Yavuz Gurkan

  5. Hi, Ian.

    Thanks for the very detailed look at how you went about using markdown. It got me writing my own version, using multimarkdown, node.js, and grunt. Basic principle is the same as yours but not as advanced. I’m thinking of extending the idea by incorporating some of the ideas in Scrivener into a programmer’s text editor (In my case, Sublime Text) to give some of the goodies that make managing files and folders a bit easier.

    https://github.com/stevecooperorg/Subliviner

    Still very early days, but just wanted to say thanks for sharing.

    Steve

  6. Hi Yavuz

    I’m afraid I don’t know of too many places for templates. The templates I offer her are the only ones I know about. It’s possible that LaTex might be a better fit for you. Meantime, in terms of software, I tend to use Textmate 2, Writer Pro, and Byword.

    With best wishes

    Ian

  7. Hi Steve

    Great work! Looks interesting. I’ll have a play with it at the weekend. I’m particularly pleased to see it on GitHub, which is the right place for this sort of thing (and beyond my expertise right now).

    Cheers

    Ian

  8. Cheers, Ian. At the mo, as I say, it’s really early days. I’ve not got round to the publishing part particularly — it just concatenates the markdown into a single file and produces the HTML version — but it’s a start.

    I’m still thinking about restructuring manuscripts somewhat, still. Maybe something like this;

    manuscripts/
    Draft 0.Book name/
    A.Front matter/
    B.Chapters/
    01.Chapter Title/
    scene 1.mdd
    scene 2.mdd
    02.Chapter title
    C.End Matter/

    Do you ever deal with multiple drafts? Unplaced fragments, plot summaries, story bible? And how do you do backups or source control? Interested to find out what you’ve learned while actually *finishing* books…

    Steve

  9. Yes, it looks like the difference might be that I don’t break my chapter files into scenes, but I can see that might be a good idea.

    As for previous versions, I give all my drafts (which are of the entire book, incidentally) a random version number, and when I make a big change the draft, I move the entire file to archive folder and then create an entirely new draft file with an incremented version number.

    Would prefer to do this with some kind of subversion/version control software…

    Cheers

    Ian

  10. Thanks for the thoughts, Ian. I’ve just moved all my notes into one location and I don’t quite know how to order it all yet — I’ll evolve something soon, I think.

    Source control is *theoretically* straightforward — just a matter of storing all your .mmd files — but moving from ‘possible’ to ‘easy and painless’ is a big old gap.

  11. Thanks for this! I’ve written a number of ebooks and in my experiments with Markdown I’ve been able to get book files accepted without an issue in both Amazon and Smashwords but I couldn’t get the indents right. Your CSS file works wonderfully for this. Thank you!

    I generally write my drafts in iaWriter, but I am experimenting with Markdown Pro. I generally code my ToC manually using HTML. Your CSS file and tips are incredibly helpful. Thank you again!

  12. Great, Annie! Glad it helped. I’ve been using a few apps recently for writing (though most of what I’ve been doing is editing, so I’ve used Scrivener). Otherwise, I’ve been using WriterPro.

  13. Thks much for posting all this – it convinced us to do all our blogs / newsletters / books in markdown.

    Markdown text files make it simple to store our publishing in our database as json (we use Postgresql).

    Since you like markdown – you might try jekyll – a simple blog site engine based on markdown. Takes about 30 secs to install (no kidding!).

    Git hub book has a free markdown editor, which plugs into their version control system.

    For those writers not on mac – all your scripts can all be run on linux as well – it’s really simple to dual boot a Windows machine with Ubuntu Linux.

    Since pandoc is real simple to install on our linux systems we will be trying that out but noted your prev comments on pandoc

  14. I have a question … how do you handle character dialouge in Markdown? I ask, because a carrige-return followed by an indent (I use three spaces) for a line of dialouge is not rendered like that when converting it — the indent is ignored, and the dialouge is appended to the previous line.

    Thanks.

  15. Thanks for your question, Brett. The three-character space is probably being interpreted as ‘format as-is’. For true Markdown, each new paragraph needs to be on a separate line. The quotes you use will not be smart quotes initially; your Markdown rendering program will need to take care of that.

Leave a Reply

Your email address will not be published. Required fields are marked *