Branching Out

In April of 2011, I published my only novel, Déjà Vu to the Kindle store. I was at the end of my tether with the intransigent (for me) traditional publishing industry. I had a story I believed in and nowhere to place it. Finally, I chose the Kindle store, which was the only game in town. Over the next two years, I made enough money to edit, proof and decorate my four remaining books (Flasback, The Amber Rooms, and Proper Job. So my professional writing has sustained itself financially.

One of the great aspects of the Kindle store is the Kindle Select Programme. This allows authors to make books free for a period (five days in every ninety) in return for exclusivity. Even though the percentage of people downloading and, crucially, reading free books is not high, the deal worked out well for me. Off the back of these promotions, my books tended to ride high in the paid chart afterward, and many of the best reviews of Déjà Vu begin with “I downloaded this for free and wasn’t expecting much, but…”

But. The Kindle Select programme isn’t what it used to be. Probably as a result of Amazon moving its focus away from Select (and because the algorithm might suppress older books), the programme is one of diminishing returns. Eventually, the exclusivity isn’t worth it.

At the same time, I’ve been moving towards a more automated way of producing books, as you can see in these posts. I’ve now reached the point where a single text file (the novel, styled using Markdown) can be passed to a program designed to generate validated ebooks (complete with table of contents, about the author, and so on) for Kindle, PDF, and iBooks.

About iBooks. I did use the Smashwords meat-grinder, once upon a time, to publish Déjà Vu in many ebook stores, including the iBookstore. There was one advantage to this and several disadvantages. The advantage: It was free, and that’s great. The disadvantages:

  • The produced versions looked like crap (inconsistent placement of images; variable line height within body text; even–saints preserve us–failure to indent at the start of chapters).

  • At the time, the Amazon Kindle store was huge, and even good sales for these others amounted to pennies.

The first disadvantage was the killer for me, though the second might have iced my cake. I delisted the book from Smashwords. This meant that Déjà Vu appeared briefly on iBooks. Enough to pick up a couple of reviews, then disappear.

Déjà Vu is now back on the iBookstore. It looks good. I designed it (and all my other books) from the ground up using Markdown, CSS and Apple’s proofing tools. My precious indenting is correct and the overall impression is much closer to a professional one. I’ve even added my own fleurons.

We’ll see what happens with regards to sales. My suspicion is that they remain quiet because Apple’s sales are driven almost entirely by brand name and exposure on the front page of the site. Amazon, by contrast, is much more aggressive about pushing unknown authors on the basis of matchmaking to the reader’s book history. M’acquaintance Scott Pack, relates his experience of publishing Confessions of a GP on iBooks, which benefitted extremely from front-of-store promotion by Apple. Advertising is the Achilles’ Heel of the independent author; we just don’t have the clout. Everything has to be word of mouth, and the iBookstore isn’t good at amplifying that.

Are you asking yourself: “But, Ian, how can I help you out?” If so, you are very kind. The main difficulty for me is the lack of reviews. Each book starts afresh. (I’m not sure what the iBooks policy is on including Amazon customer reviews in their product descriptions; I might check that out.) But if you’ve read any of my books and feel disposed towards rating them once again on iBooks, I will feel briefly warm and fuzzy.

You can see all my iBookstore books here.

Writing a Novel Using Markdown: Part Two

In my previous post, I spoke a little about my Markdown workflow. I ended it prematurely because I didn’t have a prepared version of my book, Red Star Falling, that I could share with you.

Now I do.

You can download Multimarkdown / Markdown1 source of the complete book here. This ZIP archive contains the text of the book (cheekily, I’ve redacted most of the story), the Writer as a Stranger Cascading Style Sheet, the cover image, and a little Möbius vector graphic I’m using as a section break.

As I mentioned in my previous post, once you have your book in Markdown (i.e. a plain text file), you can use whatever tools you wish to produce your ebook–in whatever format(s) that might be.

I use a Mac OS X feature called a service. This does nothing more than run a bash script, which is a series of commands telling the computer what to do with the Markdown files. You can download the service here. It works on OS X Mountain Lion, but it will probably work on earlier versions of the Mac. The raw source for the bash script (i.e. just the series of commands) is available here, but I’ve also appended it to this post. All computers, including those running Windows, can execute bash scripts–but they might need some tweaking.

Here are the files that make up the text of the book (i.e. those in the ZIP):

Screen Shot 2013 06 23 at 20 33 23

Note that the files are numbered to preserve their order. Each Markdown file ends with a few blank lines to pad them out when they’re combined into a single file 2.

All the important information about the book–who the author is, the title, revision number, where the cover file is located–all of that is declared in the file ‘1_meta_data.mmd’. This means that, if you wish to use this method to produce your own book, the script will work as long as you’ve updated the metadata in this file.

What extra stuff you need

It would be nice if OS X came with ebook-publication tools built-in. Alas, it doesn’t. My script uses two powerful programs: ebook-convert and multimarkdown. You need to install these locally first. Note that my script tells the computer exactly where the multimarkdown command lives (it’s in /usr/local/bin/multimarkdown). I don’t know why I had to give this path to make the script work, but I did.

Using the OS X Service

This is really straightforward. On your Mac, put the service into Home > Library > Services. If your Library folder is hidden, you’ll need to Option-click the ‘Go’ menu in Finder and select it. Logout, login, and then you should see service (called • [Toy] Create MOBI-EPUB-PDF from multiple MMD, using Metadata copy) in the contextual menu when any Finder items are highlighted.

Download my example archive, highlight all the Markdown files, then right-click and select the service. It will then create .mobi, .epub and .pdf versions of Red Star Falling.

These files will be placed in the sub-folder called Publication_ready. A log file in that folder, called Log.txt, will be updated with each new conversion; look there if you want to find out why things have gone wrong. Intermediate .html files will be deleted.

Because I publish (for now) only to the Kindle, I’ve made sure that the .mobi file works perfectly on the Kindle for long-form prose3. The .epub is serviceable but not pretty. The .pdf likewise. One advantage of scripting the process is that you can tweak the settings for .epub output until you’re happy and then apply those to all your books using the same script.

Cool things you can now do

  • Because this is a script, you can tweak it to:

    • Pass the .epub file to a validation tool. I’d recommend epubcheck. Once this is in the script, it’ll happen every time.

    • Push the finalised files to a central folder where you keep all your ‘publication ready’ books.

  • You can use a single style sheet for all your books. The format–.epub, .mobi, or whatever–should not change the look and feel of your final books. Once set, you don’t need to worry about this again.

  • Creating a new book is as simple as duplicating the directory and changing the metadata. The main text of your story is simply a Markdown file: human-readable, portable, and small.

  • Use aliases. In this book, I have a section–i.e. a Markdown file–that tells the reader about my other books. I don’t want to replicate this for every new book, and I want any future books to compile with an update list of my works. So I use an alias to point to a central Markdown file that contains this information. When I update that file, any book will compile with an updated list of my works.

  • Set a watch on the folder. Perhaps trigger the script to run (suitably modified) when the main story file is updated. That way, all the publication-ready files will self-update.

What are the implications of this workflow for the writing process?

None. The act of writing itself should be completely independent of all the complexities that happen to the Markdown file later, when it’s processed. It’s more convenient in several ways. You’ll not have to worry about smart quotes, or font sizes, or anything like that. All you have to concentrate on is writing the story in this format:

## Red Star Falling

In the moment before Saskia Brandt awoke, she had a vision of red chrysanthemums falling. The flowers looked unreal. Their stems were too straight and their falls too slow. Their *Gestalt* was artful sadness.

Then the sky beyond them wintered and the dream faded.

For that, I’d recommend any number of cool programs, including ByWord and iAWriter.

Coda: The script itself

In case you’re here just for this script and want to have a look at it, here it is.

## MULTIMARKDOWN FILES > MOBI, PDF AND EPUB

## IH, 23 JUNE 2013

## This script takes a group of files in Multimarkdown,
## combines them into a single file, and then produces
## MOBI, EPUB and PDF. Also runs an epub check program.

## Expects metadata in a file called 1_meta_data.md

## Expects to have a directory called Publication_Ready.
## Will pipe command output to Publication_Ready/Log.txt

## Unusual commands used (i.e. you'll need to install these):
## ebook-convert [<http://manual.calibre-ebook.com/cli/ebook-convert.html>]
## multimarkdown [<http://fletcherpenney.net/multimarkdown/use/>]

###########################################

# Find out the directory that this script is working within
DIR=`dirname "$1"`

# Go there
cd "$DIR"

# Extract meta data from the file called '1_meta_data.mmd'
title=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Title"`
revision=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Revision"`
short_title=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="ShortTitle"`
author=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Author"`
series=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Series"`
series_index=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="SeriesIndex"`
tags=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Tags"`
cover=`/usr/local/bin/multimarkdown 1_meta_data.mmd --extract="Cover"`

# LOG Date
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo `date` $revision >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt

# Concatenate all the files passed to this script
cat "$@" > $short_title.mmd 

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo ******* Compiling HTML from Multimarkdown... >> Publication_Ready/Log.txt

# Run 'multimarkdown' to produce HTML, smart mode, appending result to log
/usr/local/bin/multimarkdown $short_title.mmd --output=$short_title.html --smart >> Publication_Ready/Log.txt

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo ******* Compiling MOBI… >> Publication_Ready/Log.txt

# Make MOBI
ebook-convert $short_title.html Publication_Ready/$short_title[$revision].mobi --authors="$author" --series="$series" --series-index=$series_index --title="$title" --tags="$tags" --output-profile=kindle >> Publication_Ready/Log.txt

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo ******* Compiling PDF… >> Publication_Ready/Log.txt

# Make PDF
ebook-convert $short_title.html Publication_Ready/$short_title[$revision].pdf --authors="$author" --series="$series" --series-index=$series_index --title="$title" --tags="$tags" >> Publication_Ready/Log.txt

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo ******* Compiling EPUB… >> Publication_Ready/Log.txt

# Make EPUB
ebook-convert $short_title.html Publication_Ready/$short_title[$revision].epub --remove-first-image --authors="$author" --series="$series" --series-index=$series_index --title="$title" --tags="$tags" --output-profile=ipad >> Publication_Ready/Log.txt 

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo Cleaning up temporary files... >> Publication_Ready/Log.txt

# Clean up the temporary HTML and mmd files
rm $short_title.html $short_title.mmd >> Publication_Ready/Log.txt

# LOG
echo "" >> Publication_Ready/Log.txt
echo "" >> Publication_Ready/Log.txt
echo Job done bish bash `date` >> Publication_Ready/Log.txt


  1. Feel free to use this CSS with your own books. Think of it as the Writer as a Stranger ‘house style’. It will do things like indent all paragraphs unless they follow a chapter heading or section break and deal with blockquotes appropriately. I’ve iterated it a few times over the past few weeks to get it looking how I like it. ↩

  2. I’m aware that I can do this programmatically. (And, interestingly, the Multimarkdown command line tool has an option to concatenate multiple input files; however, it didn’t work for me and I couldn’t figure out why. Hence the old-school solution.) ↩

  3. You should always check your .mobi file using the Kindle Previewer application. ↩

Writing a Novel Using Markdown: Part One

This story begins with a person like me. When I publish ebooks, I want them to turn out exactly as I intend. That means taking the text of the story, sandwiching it with front matter (like copyright) and more text at the end (like acknowledgments).

Until recently, I wrote my books using the word processor Apple Pages. I would export this to a Microsoft Word document for my editors and proofreaders. Then, when the book was ready to be converted into an ebook, I would do a lot of the work by hand.

And therein lies the problem. There’s a lot of manual effort involved when you take a Word document and produce something that Amazon or iBooks will accept as an ebook.

In this post I’m going to talk about using plain text files to consistently and painlessly produce ebooks. I won’t assume much knowledge.

Styling Text

There should be a separation between style and content. Style is font size, paragraph spacing, and all that jazz. The content is the text itself. Word processors like Pages or Microsoft Word combine style and content into a single file. These files are complex, contain ambiguities, and are difficult to read without a working copy of the program that produced them.

When you separate style and content, many advantages result. The content file doesn’t need to be in a proprietary format. It can be plain text. The content file will be easily read by a computer, which means the computer can automate certain things for you: create an ebook, a nice-looking PDF, a PowerPoint presentation, a web page…all from the same content file.

But how do you make the content file?

Use Markdown

Markdown was created by John Gruber as a way of writing for the web without including the distraction of mark-up tags (more below). It has been extended by subsequent programmers.

The best way to describe Markdown is to show you an example. Here is the opening of my novella, Red Star Falling:

## Red Star Falling

In the moment before Saskia Brandt awoke, she had a vision of red chrysanthemums falling. The flowers looked unreal. Their stems were too straight and their falls too slow. Their *Gestalt* was artful sadness.

What you’re looking at above is a plain text snippet. The # character means that the text Red Star Falling should be treated as a heading (just like a heading in Word). Two ## characters means that this is a heading level two. Immediately, you can see that this is a foolproof way of telling a computer that the text is heading level two; at the same time, it’s easy for a human to read.

After

## Red Star Falling

there is a blank line. This tells the computer to treat the next text it encounters as the beginning of a new paragraph. Notice that, in this new paragraph, I’ve used a

*

character to enclose the German word Gestalt. This tells the computer to emphasise the word. In this case, it makes it italic.

OK. So I’ve written the first paragraph of my glorious story. What do I do with it?

Well, the Amazon Kindle format is based on HTML, which is the simple tagging format that most webpages use. So I want to produce HTML from the Markdown. There are many tools that will help you do this. I’ll talk about some of them later. When I convert the above text to HTML I get the following:

<h2>Red Star Falling</h2>

<p>In the moment before Saskia Brandt awoke, she had a vision of red chrysanthemums falling. The flowers looked unreal. Their stems were too straight and their falls too slow. Their <em>Gestalt</em> was artful sadness.</p>

If you’re not familiar with HTML, this will look a bit weird. The important thing to note is that the original Markdown format text will reliably produce the HTML.

Why this is exciting

How much do I love Markdown? Let me count the ways.

  1. The thing that I, as a writer, look at, is a plain text file written in Markdown. It doesn’t get much simpler. I don’t need to worry about smart quotes and dash sizes–all this is done when creating the HTML.

  2. Because it’s a plain text file, I can work on it from a computer of virtually any type (desktop, laptop, mobile), any age (even ancient PCs can read text files) and any platform (plain text files are very portable).

  3. Helper programs can take this Markdown put it into any form I like: PDF, .mobi, epub, and so on. It can take some time to set up these helper programs, but you only have to do it once.

  4. In ten years’ time, the file will be super easy to open. It’s plain text, not a proprietary word-processing format.

My Workflow

Most of my writing takes place on two devices: A MacBook Air and an iPad. My workflow is therefore Mac-centric. However, the approach should work on any system.

Here is the point where I realise that my own workflow files are hopelessly hard-coded to my own system, and providing you with a working example of Red Star Falling that you can download and play with is impossible without editing my files. So, instead of giving you these files, I’ll explain the process in plain English.

  • My book is a series of plain text files, all of which are in Markdown format. Each file represents a section of the book that I’ve isolated for convenience. (Note that file number six is actually an alias to a file held in a more central central location, which is the same across all my books.)

image

I’ve now written a follow-up to this post that includes a detailed example.

Useful resources

A Markdown cheatsheet.

My Writer as a Stranger CSS (Cascading Style Sheet). Use this in combination with a Markdown document to produce a book with the same final formatting as Red Star Falling.

Markdown tools from Brett Terpstra.