Zipf's law

Greater knowledge of a danger permits greater safety. For centuries, shipbuilders have put care into the design of their hulls and sails. They know that, in most cases, the sea is moderate. But they also know that typhoons arise and hurricanes happen. They design not just for the 95 percent of sailing days when the weather is clement, but also for the other 5 percent, when storms blow and their skill is tested.

 I am, of course, a true believer in the power of probability. I have seen it and applied it in economics, physics, information theory, metallurgy, meteorology, neurology, anatomy, taxonomy, and many other seemingly improbable fields. As a graduate student at the University of Paris more than fifty years ago, I wrote my doctoral thesis on an ignored byway of applied probability: the power law that rules the mathematical frequency with which individual words occur in common language. With such a background I would hardly be one to refute the usefulness of probability theory in yet another field, finance. In financial markets, God can appear, anyway, to play with dice. What I know is that the ruler of chance can create what I call several distinct “states” or types of chance. And what I contest is the way today’s financial theorists, in their classrooms and their writings, calculate the odds. It may seem to some an academic quibble—but as will be seen, it can be the difference between winning and losing a fortune.

It was a review of a book by an academic “character,” George Kingsley Zipf. Zipf, independently wealthy, was a university lecturer at Harvard in a self-invented field he called statistical human ecology. His book, Human Behavior and the Principle of Least Effort, saw power laws as an omnipresent pattern in the social sciences. Such power laws are common in physics, and are a form of what I now call fractal scaling. Seismologists have a mathematical formula that shows the number of earthquakes varying by a power law with their intensity, on the famous Richter scale. Put another way: Small quakes are common while big ones are rare, with a precise formula relating intensity to frequency. But at that time only a few examples were known—to very few persons. Zipf, an encyclopedist obsessed by an idée fixe, claimed that power laws do not occur only in physical sciences but are the rule in all manner of human behavior, organization, and anatomy—even in the size of sexual organs.

Most fortunately, the book review my uncle gave me limited itself to one unusually elaborate example: word frequencies. In text or speech, some words such as “the” or “this” occur often; others, “milreis” or “momus,” appear rarely if ever (for the curious: the first is an old Portuguese coin, the second a synonym for critic).