Category Archives: prouctivity

Writing a Novel Using Markdown: Part Two

In my pre­vi­ous post, I spoke a little about my Markdown work­flow. I ended it pre­ma­turely because I didn’t have a pre­pared ver­sion of my book, Red Star Falling, that I could share with you.

Now I do.

You can down­load Multimarkdown / Markdown1 source of the com­plete book here. This ZIP archive con­tains the text of the book (cheekily, I’ve redac­ted most of the story), the Writer as a Stranger Cascading Style Sheet, the cover image, and a little Möbius vec­tor graphic I’m using as a sec­tion break.

As I men­tioned in my pre­vi­ous post, once you have your book in Markdown (i.e. a plain text file), you can use whatever tools you wish to pro­duce your ebook–in whatever format(s) that might be.

I use a Mac OS X fea­ture called a ser­vice. This does noth­ing more than run a bash script, which is a series of com­mands telling the com­puter what to do with the Markdown files. You can down­load the ser­vice here. It works on OS X Mountain Lion, but it will prob­ably work on earlier ver­sions of the Mac. The raw source for the bash script (i.e. just the series of com­mands) is avail­able here, but I’ve also appen­ded it to this post. All com­puters, includ­ing those run­ning Windows, can execute bash scripts–but they might need some tweaking.

Here are the files that make up the text of the book (i.e. those in the ZIP):

Screen Shot 2013 06 23 at 20 33 23

Note that the files are numbered to pre­serve their order. Each Markdown file ends with a few blank lines to pad them out when they’re com­bined into a single file 2.

All the import­ant inform­a­tion about the book–who the author is, the title, revi­sion num­ber, where the cover file is located–all of that is declared in the file ‘1_meta_data.mmd’. This means that, if you wish to use this method to pro­duce your own book, the script will work as long as you’ve updated the metadata in this file.

What extra stuff you need

It would be nice if OS X came with ebook-publication tools built-in. Alas, it doesn’t. My script uses two power­ful pro­grams: ebook-convert and mul­ti­mark­down. You need to install these loc­ally first. Note that my script tells the com­puter exactly where the mul­ti­mark­down com­mand lives (it’s in /usr/local/bin/multimarkdown). I don’t know why I had to give this path to make the script work, but I did.

Using the OS X Service

This is really straight­for­ward. On your Mac, put the ser­vice into Home > Library > Services. If your Library folder is hid­den, you’ll need to Option-click the ‘Go’ menu in Finder and select it. Logout, login, and then you should see ser­vice (called • [Toy] Create MOBI-EPUB-PDF from mul­tiple MMD, using Metadata copy) in the con­tex­tual menu when any Finder items are highlighted.

Download my example archive, high­light all the Markdown files, then right-click and select the ser­vice. It will then cre­ate .mobi, .epub and .pdf ver­sions of Red Star Falling.

These files will be placed in the sub-folder called Publication_ready. A log file in that folder, called Log.txt, will be updated with each new con­ver­sion; look there if you want to find out why things have gone wrong. Intermediate .html files will be deleted.

Because I pub­lish (for now) only to the Kindle, I’ve made sure that the .mobi file works per­fectly on the Kindle for long-form prose3. The .epub is ser­vice­able but not pretty. The .pdf like­wise. One advant­age of script­ing the pro­cess is that you can tweak the set­tings for .epub out­put until you’re happy and then apply those to all your books using the same script.

Cool things you can now do

  • Because this is a script, you can tweak it to:

    • Pass the .epub file to a val­id­a­tion tool. I’d recom­mend epubcheck. Once this is in the script, it’ll hap­pen every time.

    • Push the final­ised files to a cent­ral folder where you keep all your ‘pub­lic­a­tion ready’ books.

  • You can use a single style sheet for all your books. The format–.epub, .mobi, or whatever–should not change the look and feel of your final books. Once set, you don’t need to worry about this again.

  • Creating a new book is as simple as duplic­at­ing the dir­ect­ory and chan­ging the metadata. The main text of your story is simply a Markdown file: human-readable, port­able, and small.

  • Use ali­ases. In this book, I have a section–i.e. a Markdown file–that tells the reader about my other books. I don’t want to rep­lic­ate this for every new book, and I want any future books to com­pile with an update list of my works. So I use an alias to point to a cent­ral Markdown file that con­tains this inform­a­tion. When I update that file, any book will com­pile with an updated list of my works.

  • Set a watch on the folder. Perhaps trig­ger the script to run (suit­ably mod­i­fied) when the main story file is updated. That way, all the publication-ready files will self-update.

What are the implic­a­tions of this work­flow for the writ­ing process?

None. The act of writ­ing itself should be com­pletely inde­pend­ent of all the com­plex­it­ies that hap­pen to the Markdown file later, when it’s pro­cessed. It’s more con­veni­ent in sev­eral ways. You’ll not have to worry about smart quotes, or font sizes, or any­thing like that. All you have to con­cen­trate on is writ­ing the story in this format:

## Red Star Falling

In the moment before Saskia Brandt awoke, she had a vision of red chrysanthemums falling. The flowers looked unreal. Their stems were too straight and their falls too slow. Their *Gestalt* was artful sadness.

Then the sky beyond them wintered and the dream faded.

For that, I’d recom­mend any num­ber of cool pro­grams, includ­ing ByWord and iAWriter.

Coda: The script itself

In case you’re here just for this script and want to have a look at it, here it is.

## MULTIMARKDOWN FILES > MOBI, PDF AND EPUB

## IH, 23 JUNE 2013

## This script takes a group of files in Multimarkdown,
## combines them into a single file, and then produces
## MOBI, EPUB and PDF. Also runs an epub check program.

## Expects metadata in a file called 1_meta_data.md

## Expects to have a directory called Publication_Ready.
## Will pipe command output to Publication_Ready/Log.txt

## Unusual commands used (i.e. you'll need to install these):
## ebook-convert [<http://manual.calibre-ebook.com/cli/ebook-convert.html>]
## multimarkdown [<http://fletcherpenney.net/multimarkdown/use/>]

###########################################

# Find out the directory that this script is working within
DIR=`dirname "$1"`

# Go there
cd "$DIR"

# Extract meta data from the file called '1_meta_data.mmd'
title=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Title"`
revision=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Revision"`
short_title=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="ShortTitle"`
author=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Author"`
series=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Series"`
series_index=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="SeriesIndex"`
tags=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Tags"`
cover=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Cover"`

# LOG Date
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo `date` $revision >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt

# Concatenate all the files passed to this script
cat "$@" > $short_title.mmd 

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo ******* Compiling HTML from Multimarkdown... >> Publication_Ready/Log.txt

# Run 'multimarkdown' to produce HTML, smart mode, appending result to log
/usr/local/bin/multimarkdown $short_title.mmd --output=$short_title.html --smart >> Publication_Ready/Log.txt

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo ******* Compiling MOBI… >> Publication_Ready/Log.txt

# Make MOBI
ebook-convert $short_title.html Publication_Ready/$short_title[$revision].mobi --authors="$author" --series="$series" --series-index=$series_index --title="$title" --tags="$tags" --output-profile=kindle >> Publication_Ready/Log.txt

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo ******* Compiling PDF… >> Publication_Ready/Log.txt

# Make PDF
ebook-convert $short_title.html Publication_Ready/$short_title[$revision].pdf --authors="$author" --series="$series" --series-index=$series_index --title="$title" --tags="$tags" >> Publication_Ready/Log.txt

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo ******* Compiling EPUB… >> Publication_Ready/Log.txt

# Make EPUB
ebook-convert $short_title.html Publication_Ready/$short_title[$revision].epub --remove-first-image --authors="$author" --series="$series" --series-index=$series_index --title="$title" --tags="$tags" --output-profile=ipad >> Publication_Ready/Log.txt 

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo Cleaning up temporary files... >> Publication_Ready/Log.txt

# Clean up the temporary HTML and mmd files
rm $short_title.html $short_title.mmd >> Publication_Ready/Log.txt

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo Job done bish bash `date` >> Publication_Ready/Log.txt


  1. Feel free to use this CSS with your own books. Think of it as the Writer as a Stranger ‘house style’. It will do things like indent all para­graphs unless they fol­low a chapter head­ing or sec­tion break and deal with block­quotes appro­pri­ately. I’ve iter­ated it a few times over the past few weeks to get it look­ing how I like it. ↩

  2. I’m aware that I can do this pro­gram­mat­ic­ally. (And, inter­est­ingly, the Multimarkdown com­mand line tool has an option to con­cat­en­ate mul­tiple input files; how­ever, it didn’t work for me and I couldn’t fig­ure out why. Hence the old-school solu­tion.) ↩

  3. You should always check your .mobi file using the Kindle Previewer applic­a­tion. ↩

Writing a Novel Using Markdown: Part One

This story begins with a per­son like me. When I pub­lish ebooks, I want them to turn out exactly as I intend. That means tak­ing the text of the story, sand­wich­ing it with front mat­ter (like copy­right) and more text at the end (like acknowledgments).

Until recently, I wrote my books using the word pro­cessor Apple Pages. I would export this to a Microsoft Word doc­u­ment for my edit­ors and proofread­ers. Then, when the book was ready to be con­ver­ted into an ebook, I would do a lot of the work by hand.

And therein lies the prob­lem. There’s a lot of manual effort involved when you take a Word doc­u­ment and pro­duce some­thing that Amazon or iBooks will accept as an ebook.

In this post I’m going to talk about using plain text files to con­sist­ently and pain­lessly pro­duce ebooks. I won’t assume much knowledge.

Styling Text

There should be a sep­ar­a­tion between style and con­tent. Style is font size, para­graph spa­cing, and all that jazz. The con­tent is the text itself. Word pro­cessors like Pages or Microsoft Word com­bine style and con­tent into a single file. These files are com­plex, con­tain ambi­gu­ities, and are dif­fi­cult to read without a work­ing copy of the pro­gram that pro­duced them.

When you sep­ar­ate style and con­tent, many advant­ages res­ult. The con­tent file doesn’t need to be in a pro­pri­et­ary format. It can be plain text. The con­tent file will be eas­ily read by a com­puter, which means the com­puter can auto­mate cer­tain things for you: cre­ate an ebook, a nice-looking PDF, a PowerPoint present­a­tion, a web page…all from the same con­tent file.

But how do you make the con­tent file?

Use Markdown

Markdown was cre­ated by John Gruber as a way of writ­ing for the web without includ­ing the dis­trac­tion of mark-up tags (more below). It has been exten­ded by sub­sequent programmers.

The best way to describe Markdown is to show you an example. Here is the open­ing of my novella, Red Star Falling:

## Red Star Falling

In the moment before Saskia Brandt awoke, she had a vis­ion of red chrys­an­them­ums fall­ing. The flowers looked unreal. Their stems were too straight and their falls too slow. Their *Gestalt* was art­ful sadness.

What you’re look­ing at above is a plain text snip­pet. The # char­ac­ter means that the text Red Star Falling should be treated as a head­ing (just like a head­ing in Word). Two ## char­ac­ters means that this is a head­ing level two. Immediately, you can see that this is a fool­proof way of telling a com­puter that the text is head­ing level two; at the same time, it’s easy for a human to read.

After

## Red Star Falling

there is a blank line. This tells the com­puter to treat the next text it encoun­ters as the begin­ning of a new para­graph. Notice that, in this new para­graph, I’ve used a

*

char­ac­ter to enclose the German word Gestalt. This tells the com­puter to emphas­ise the word. In this case, it makes it italic.

OK. So I’ve writ­ten the first para­graph of my glor­i­ous story. What do I do with it?

Well, the Amazon Kindle format is based on HTML, which is the simple tag­ging format that most webpages use. So I want to pro­duce HTML from the Markdown. There are many tools that will help you do this. I’ll talk about some of them later. When I con­vert the above text to HTML I get the following:

<h2>Red Star Falling</h2>

<p>In the moment before Saskia Brandt awoke, she had a vision of red chrysanthemums falling. The flowers looked unreal. Their stems were too straight and their falls too slow. Their <em>Gestalt</em> was artful sadness.</p>

If you’re not famil­iar with HTML, this will look a bit weird. The import­ant thing to note is that the ori­ginal Markdown format text will reli­ably pro­duce the HTML.

Why this is exciting

How much do I love Markdown? Let me count the ways.

  1. The thing that I, as a writer, look at, is a plain text file writ­ten in Markdown. It doesn’t get much sim­pler. I don’t need to worry about smart quotes and dash sizes–all this is done when cre­at­ing the HTML.

  2. Because it’s a plain text file, I can work on it from a com­puter of vir­tu­ally any type (desktop, laptop, mobile), any age (even ancient PCs can read text files) and any plat­form (plain text files are very portable).

  3. Helper pro­grams can take this Markdown put it into any form I like: PDF, .mobi, epub, and so on. It can take some time to set up these helper pro­grams, but you only have to do it once.

  4. In ten years’ time, the file will be super easy to open. It’s plain text, not a pro­pri­et­ary word-processing format.

My Workflow

Most of my writ­ing takes place on two devices: A MacBook Air and an iPad. My work­flow is there­fore Mac-centric. However, the approach should work on any system.

Here is the point where I real­ise that my own work­flow files are hope­lessly hard-coded to my own sys­tem, and provid­ing you with a work­ing example of Red Star Falling that you can down­load and play with is impossible without edit­ing my files. So, instead of giv­ing you these files, I’ll explain the pro­cess in plain English.

  • My book is a series of plain text files, all of which are in Markdown format. Each file rep­res­ents a sec­tion of the book that I’ve isol­ated for con­veni­ence. (Note that file num­ber six is actu­ally an alias to a file held in a more cent­ral cent­ral loc­a­tion, which is the same across all my books.)

image

  • I then pass the files to an Automator Service. The Automator script does the following:

    • Adds them together to make a single, large Markdown file.
    • Uses the Multimarkdown command-line tool by Fletcher Penney to cre­ate an HTML ver­sion of the book. A sep­ar­ate style sheet (see Useful resources) is used to cre­ate the final ebook style that I like. This tool also smartens quotes, cre­ates proper dash char­ac­ters, etc.
    • Passes the HTML to the ebook-converter command-line tool by the indefatig­able Kovid Goyal, which cre­ates an Amazon-compatible .Mobi, iBooks-friendly .epub, and a PDF for good measure.

I’ve now writ­ten a follow-up to this post that includes a detailed example.

Useful resources

A Markdown cheat­sheet.

My Writer as a Stranger CSS (Cascading Style Sheet). Use this in com­bin­a­tion with a Markdown doc­u­ment to pro­duce a book with the same final format­ting as Red Star Falling.

Markdown tools from Brett Terpstra.

My Desktop

[See the lar­ger pic­ture]

I’m fas­cin­ated by the series My Desktop, which is a Guardian column by my friend and writer-abite-tine Ben Johncock. The series fea­tures a snap­shot of a writer’s com­puter screen, which allows the word-botherer in ques­tion to riff on their work­flow and habit; some­thing all writers itch to know about, I would suggest.

Here’s mine. You’ll notice that I use a Mac. This is OS X Lion run­ning on my dad’s old Mac Mini. My writ­ing is done within a sep­ar­ate user account. (I have an exact duplic­ate of this account on my MacBook Air.) This means that (i) I can set it up in a way par­tic­u­lar to my wordly habits (it does not, for instance, have a con­figured email cli­ent) and (ii) it provides an uncluttered envir­on­ment that I can rely upon, no mat­ter how untidy my main user account gets. The back­ground was a mono­chrome grey until last year, when I changed it to a more friendly, phys­ical grey. The text ‘Be ori­ginal’ is a reminder that I would like to write books that stand apart from oth­ers. The image itself comes from InterfaceLift.

At the bot­tom of the screen, in the OS X dock, there are no ‘pinned’ applic­a­tions. Minimally does it. The blue com­pass is the Mac browser, Safari. I keep this run­ning for quick checks of Google maps, Wikipedia, and vari­ous pages rel­ev­ant to the cur­rent work. I try to observe Stephen King’s excel­lent rule that you write with the door closed and edit with the door open, so Safari gets an out­ing only when I’m edit­ing, as I am now. Otherwise, I enter [’TC’] in the text if I feel that I need to insert some­thing that I can’t at that moment. ‘TC’ means ‘to come’. For instance, I can never remem­ber the full name of one of my char­ac­ters, so I often write ‘[TC Pasha’s name’], which I’ll replace with Pavel Eduardovitch Nakhimov later.

The pen-and-ink icon is Pages, where I keep the text of the work. I’ve tried writ­ing in applic­a­tions like Byword (where I’m writ­ing this post) but I prefer the feel of Pages. To the right of this is a strange, double-paned icon: OmniFocus.

OmniFocus is a task-manager applic­a­tion that I find use­ful for main­tain­ing lists of edits. Although I haven’t opened it for a few weeks, it was great for work­ing through the edits sug­ges­ted by my agent. I cat­egor­ised them as major or minor and kept a record of what I’d done.

On the left of the desktop, in the win­dow called ‘Amber Rooms 101′, you’ll see three files. One of these is my cur­rent draft. Another is the pre­vi­ous draft, which I’m ran­sack­ing. The third is a text file con­tain­ing notes about what I want to do with the cur­rent draft. This folder is actu­ally a search. It’s look­ing for ‘AR101’ in the title of the file. When I move onto another draft, I’ll cre­ate new files with ‘AR102’.

In the Finder bar at the top, you’ll see a small blue box icon. This is DropBox. I use it to syn­chron­ise files between my machines. I have a Writing folder access­ible to this user account, but my other DropBox files are hidden.

Music? Never listen to it dur­ing writ­ing or edit­ing. The only other things on my phys­ical desk, at which I stand rather than sit, are late-nineteenth-century maps of St Petersburg.