Reverse engineering, trial by combat, and listening
So, beware: navel-gazing alert. What follows is a post about the process of writing, and might only be of interest to other writers.
Now then. Let's say you're a writer. Or, at least, you like to write. You can string words together. You are sensitive to noun-verb agreement and you write active sentences. You know that adverbs are not your friends. One day, you think, you want to be as good as Hemmingway, or Faulkner, or Wodehouse or Adams. (Let's leave aside the fact that you are unlikely to achieve these artistic heights. That kind of pessimism is too pessimistic.) How do you get from where you are to where they are? Or, if you prefer, how do you improve your product, and keep improving it, until you have entered the league of Big Players?
The process of writing has one curious aspect. It looks easy. Why? Because your instrument is your mouth or your fingers, and words are your notes. Everybody is an expert with this instrument. It's not like playing a piano, where the physical interface between human and music will take years to learn competently. Writing, then, looks easy, because verbal communication is a task that you are genetically programmed to do, and have spent almost all of your life learning to master. I'd argue that this makes it difficult to learn to write well - in relation to those far-away targets, Hemmingway, Wodehouse et al. - because it is not immediately obvious what the routes of development are. A pianist can emulate increasingly complex music, can learn scales, music theory, and get online feedback from a hovering music teacher. A writer...but here the analogy breaks down. A better analogy for a writer is the musical composer. The composer might have the principles of opening bars, middle eights, verses and choruses. What are the principles of composition for a story? It is the difference in application of these compositional principles that characterizes the gap between the apprentice writer, such as myself, and the masters. I think that point is uncontroversial. I'd like to talk about how the writer goes about acquiring these principles.
Improvement by formal instruction
When I started out writing fiction serious-like, I did what came naturally as a conscientious student. I looked it up in a book. Several books, in fact. But at the back of my mind, I had already dismissed the intellectual route as a way of picking up these principles. When I read McKee's Story, for example, I reacted strongly against the precise rhythms and almost formulaic descriptions of drama. For every example he came up with, I could produce a counter example. Surely, I thought, I'm an expert in story already. I've seen countless films, read countless books, and even sat through plays. These principles are for people without the touch. Me, I've got a finely-tuned storyteller brain already.
Improving by doing
I wrote a bloody awful novel. I keep it my filing cabinet to remind me how bad it is. It suffers from a failure to apply basic principles of composition. Scenes should be points of conflict that move the story forward. These scenes should build towards a larger conflict, a turning point or revelation, and this marks the end of an act. Not all stories do this, of course. Compositional principles aren't something you can plug in. Some stories react against these principles - but, generally speaking, these aren't the kind of stories I'm interested in writing.
Anyway, in the course of writing my first novel, I learned that my immersion in Western culture was not enough to make me a writer. I would not burst into literature precocially like Mailor or Ellis. My storytelling 'expertise' was only sufficient for me to recognise the problems with my book. The answers - practical craft-based methods of solving these problems - could only be found in the principles of composition. Once you use them, they click into place. A competent writer can produce a novel. A better writer can recognise what's wrong with it. A good writer knows how to fix it. A great writer is capable of fixing it. 'All first drafts are shit' -- Hemmingway.
Improving by reverse engineering other stories
The other method of improving my craft is to doggedly analyze every piece of fiction I'm exposed to (the corrollary is that I doggedly catalogue every piece of nonfiction too, including my life experience, so that whereas another person might say, "Ooh, that's interesting/sad", I think "Right, I'll nick that forthwith"). Why, for example, did I think that 'Serenity' was such a great film? Good characters, believable motivations, high jeopardy, satisfying relationships, and an undercurrent of humanity. How did the writer/director Joss Whedon manage that? How did the 10% of the movie iceberg suggest the 90% of a whole world beneath, of values, and how did it manipulate my emotions? There are always concrete answers to these questions. It might be possible to trace a character's motivation to a single, startling exchange of dialogue, or an object he picks up and muses over; a tiny manipulation leads to a great effect. I think a writer can pull lessons of craft from both good and bad films. But to do so, it is immeasurably helpful to have a shorthand way of characterizing the story: conflicts, inciting incidents, acts one, two, three, four, whatever. This draws upon formal instruction in composition and personal successes and failures an applying the principles.
Getting your shit together
Somewhat predictably, my conclusion is that there needs to be an interaction between reverse engineering classic stories, reading about formal composition and learning by doing. You already knew that, didn't you? But there is a chance you are stuck in one of those three right now in your writing career. You might be leaning heavily on McKee or Vogler or Campbell, and formulizing your story. You might be making mistakes by crashing on, undisciplined, into a novel. You might be agonizing over the clear brilliance of Hemmingway or Tolstoy or Basho against the dire crapness of your own novel or book. I've recently overturned my disdain for the structured approached to composition (or, at least, approached these principles with an open mind) and I think that my current novel has benefitted hugely. If you're writer too, you might consider that, to become a master of fictional composition, you should become a jack of all the routes to it.