I just finished Richard Bookstaber’s A Demon of Our Own Design: Markets, Hedge Funds, and the Perils of Financial Innovation and my head is spinning from the packed pages with dire implications for our financial system. Bookstaber’s decades of running risk-management for Morgan Stanley, Salomon Brothers and assorted hedge funds make this 2007 book seem almost prescient in light of today’s subprime-induced meltdown.

Bookstaber’s central argument is that financial innovation – Wall Street geniuses creating new products – increases the complexity of the market to an extent that it is impossible to understand and manage, leading to the dramatic events of the past twenty years. These are dramas Bookstaber saw first-hand, and by his own admission, helped engineer as an MIT PhD working on the cutting edge of finance.

The book is both history and theory, though being far from a finance expert (I often relied on my Dad’s explanations over the past two days), the theory was far more interesting and will be the topic of this post.

The history, though, is also compelling. It chronicles the numerous catastrophes of the past 25 years including 1987′s crash, the Internet bubble and Long Term Capital Management’s implosion. Bookstaber explains how these disasters were not due to outside forces (say, an oil shock), but due to the financial system itself. As investment strategies became increasingly sophisticated, aided by top mathematical models and cutting-edge technology, failure increased. The opposite should have been true: Nobel Prize winning academics were devising financial products and investment strategies to more fully quantify the market. However, fundamental, endogenous differences prove to be insurmountable barriers to financial perfection. The reasons are two-fold: the normality of accidents and the limits of human knowledge.

Complexity, Tight Coupling and Normal Accidents

I first encountered Charles Perrow’s book, Normal Accidents, while reading about another interest of mine: climbing. In his impressive book, Deep Survival, Laurence Gonzalez draws heavily upon Perrow’s insights to explain why accidents in outdoor sports are not only likely, they are standard. The idea showed up again when studying nuclear reactors in my Science, Technology and the Global Arena course last semester. A theory that applies equally well to climbing Mt. Hood, Three Mile Island and the fall of Long Term Capital Management should clearly be required reading, but many remain unfamiliar with Perrow’s brainchild, now decades old.

A normal accident is one which unavoidable due to the structure of the system. Systems which lead to normal accidents are complex and tightly coupled. Complex systems are nonlinear structures where actions in one area cause events among the system. No individual completely understands a complex system, whether it be a space shuttle or modern market, and no amount of testing will ever uncover every possible outcome. By itself, a complex system isn’t prone to catastrophic accidents; the problem comes when the system is also tightly-coupled, meaning processes are time-dependent. If a book isn’t reshelved at a library immediately, no serious problem emerges because a library isn’t tighly-coupled. If, however, a nuclear reactor safeguard doesn’t kick in immediately, a cascade of errors will quickly lead to disaster. There is little slack in a tightly-coupled system, so exactness is important.

Today’s financial markets are both complex and tightly-coupled. They are complex, in large part, because banks hold leverage around the world – meaning that problems in Japan can quickly reach Brazil, even if no direct economic connection exists. “The tight coupling in financial markets comes from the nonstop information flow and unquenchable demand for instant liqudity.” The result is the presence of normal accidents of historic proportions. The history of financial markets also shows this: the famous Dutch tulip mania only reached manic proportions when someone had the bright idea of creating forward contracts, so traders could “buy” tulips that didn’t even exist and proceed to trade those pieces of paper on the expectation of a flower crop.

Financial markets, complex and tightly-coupled, are bound to have failures like the one crippling the economy today. The easy answer, resonating especially hard today, is to regulate the industry. Yet, the most common forms of regulation only add to complexity, and thus accidents. Bookstaber argues that regulation should seek to reduce complexity in the first place, rather than try to control it after the fact.

The Limits of Knowledge

The second theoretical discussion, and where I think Bookstaber is at his strongest, is in his discussion of the limits of human knowledge. In it, he channels some of Nassim Nicholas Taleb‘s “black swan” proposition, but curiously fails to mention his fellow quant-turned-critic. Bookstaber takes three decently well-known concepts and molds what I found to be a spectacular chapter about epistomology (can epistomology be spectacular?).

The first of those is Kurt Godel’s proof that nothing can be proven invariably. The primary example are the self-referential statements known as the Liar Paradox. “This statement is a lie” cannot be true because doing so would contradict it. Godel’s point that not everything followed a logical path was magnified by Wener Heisenberg’s famous Uncertainty Principle which showed that by simply observing a particle, it was changed. The result was a recognition that it was impossible to objectively know something precisely, and the harder we tried, the more variance resulted. The ability to know the future disappeared because we could not accurately know the past or present. Bookstaber writes,

“This metaphor extens neatly into the world of financial markets. In the purely mechanistic universe of classical physics, we could apply Newtonian laws to project the future course of nature, if only we knew the location and velocity of every particle. In the world of finance, the elementary particples are the financial assets. In a purely mechanistic financial world, if we knew the position each investor has in each asset and the ability and willingness of liquidity providers to take on those assets in the event of a forced liquidation, we would be able to understand the market’s vulnerability… Practically, it wouldn’t work. Just as the atomic world turned out to be more complex than Laplece [Heisenberg's predecessor] conceived, the financial world may be similarly comlex and not reducible to a simple causality.”

The reasons are manifold, but rest primarily on the fact that traders do not exist in a vacuum and do not hold perfect information. Transparency increases result in liquidity decreases – as we seek to observe (by increasing transparency), we change the market (by altering the supply of liquidity).

The final piece of Bookstaber’s argument for accepting human ignorance is the work of Edward Lorenz, commonly known as the “butterfly effect.” So-called because a hypothetical flap of a butterfly’s wings may be magnified over weeks to effect the weather across the globe, it comes out of the recognition that minuscule perturbations can have astronomic repercussions. Indeed, because because Heisenberg showed that we cannot measure without some error, “for many dynamic systems our forecast errors will grow to the point that even an approximation will be out of our hands.” As Kevin Kelly pointed out, increased understanding only leads to increased awareness of ignorance.

Coarse Behavior

Then what about the billions of dollars run through advanced algorithmic trading schemes? How should an investment firm structure to avoid normal accidents? Bookstaber is weakest when it comes to recommendations (possibly because he is not a policy wonk, possibly because he runs a hedge fund of his own), but he does include some interesting discussion of the value of coars behavior.

Looking to biology, Bookstaber sees the cockroach as the ideal risk-manager; after all, it has survived tectonic changes in environment and continues to outwit human predators. The reason is because its defense mechanism consists of the rather unsophisticated rule: if a puff of air is detected by its fiber nervous system, it scurries. The puffs may or may not signal an approaching predator, but the cockroach runs anyways. As Bookstaber writes, “This risk-management structure is extremely coarse; it ignores a wide set of information about the environment – visual and olfactory cues, for example – that one would think an optimal risk management system would take into account.” That is, the cockroach filters out all but the essential, if at times inaccurate, indicator of doom. As Clay Shirky said at this year’s Web 2.0 Expo, it’s not information overload, it’s filter failure.

Compare this to animals like the oddly named furu, a finely-tuned product of Darwinian selection – specialized in almost every way for its environment. The furu, though, suffered near extinction when an alien species was introduced to its lake. The parallels are clever: highly specialized financial models and strategies cannot last in a complex, tightly-coupled market where change comes fast and furious. Although fine-tuning may yield short-term payoff, the inevitable result (2-3 years in Bookstaber’s observation) is extinction.

Instead, he advocates an investment strategy configured for the unknown. In addition, the strategist seeking to avoid the fate of the book cover’s Icarus should simplify organizational complexity and introduce slack when possible by decoupling processes. The take-aways are many, but as endogenous pressures continue to wreack havoc on the market, Bookstaber provides a powerful argument against increased innovation which only introduces more normal accidents.

If you liked The Black Swan or Fooled by Randomness, or want to better understand today’s economy, I cannot recommend this book enough.