Gauss vs. Pareto

Tuesday, June 5, 2007

Read this brilliant article on John Hagel’s blog.  Excerpts –

We’re shifting from a Gaussian world to a Paretian world, with profound implications for business.  Johann Gauss was a famous mathematician in the 18th century and Vilfredo Pareto was a great economist who lived across the cusp of the 19th and 20th centuries.  So, what possible relevance do these dead white men have for business today?  Gauss contributed the Gaussian distribution, also known as the normal distribution, as a way to characterize the probability of events – most of us know it as the familiar bell curve with a significant hump in the middle and two relatively modest tails on either side of the hump.  Pareto, on the other hand, inspired the Pareto, or power law, probability distribution.  Chris Anderson’s The Long Tail offers a great contemporary example of the Pareto probability distribution – a few extreme events or “blockbusters” on the left hand side of the curve and a very long tail of much less popular events on the right hand side of the curve.  The Pareto distribution has also been popularized as the “80/20” rule.  These are two very different ways of viewing the world, with some events following a Gaussian distribution (classic example: the heights of individual human beings) and other events following a Pareto distribution (classic examples: frequency of word use, size of human settlements, distribution of Internet traffic and intensity of earthquakes).

PowerLaws TN
(click thumbnail for larger image)

Bill McKelvey (a professor at UCLA’s business school) in a recent journal article written with Pierpaolo Andriani, highlights a crucial distinction between the Gaussian and Paretian worlds: Gaussian and Paretian distributions differ radically.  The main feature of the Gaussian distribution . . . can be entirely characterized by its mean and variance . . . A Paretian distribution does not show a well-behaved mean or variance.  A power law, therefore, has no average that can be assumed to represent the typical features of the distribution and no finite standard deviations upon which to base confidence intervals…..Andriani and McKelvey focus on the desperate efforts of social “scientists” to fit social phenomena into Gaussian distributions…..But it is not just social scientists who fall prey to this temptation to adopt a Gaussian view of the world.  Business executives also are drawn to a Gaussian world.  At one level it is much simpler – there is a meaningful “average consumer” that can be used to scale products and operations around – and it is a much more predictable world.  In many respects, the history of Western business in the twentieth century represents an effort to build scalable operations through standardization designed to serve “average consumers”.  As McKelvey observes in another paper (“Extreme Events, Power Laws, and Adaptation”) co-authored with Max Boisot:  Organizations can be shaped or forced into a Gaussian form.  The large hierarchies that managers work in, and the procedures that they impose on their organizational members – the division of labor, single-point accountability, cost accounting, etc. – aim to achieve control by isolating and objectifying.  These managers inherit from the industrial economy a belief that, even where the world is not yet Gaussian, it can be made so through design…..Here’s the problem (or opportunity).  Gaussian distributions tend to prevail when events are completely independent of each other.  As soon as you introduce the assumption of interdependence across events, Paretian distributions tend to surface because positive feedback loops tend to amplify small initial events.  For example, the fact that a website has a lot of links increases the likelihood that others will also link to this website.  McKelvey and Andriani suggest that Gaussian distributions can morph into Paretian distributions under two conditions – when tension increases and when the cost of connections decreases.  In our globalizing economy, tension rises as competitive intensity increases and as business landscapes evolve faster than the capacity of most organizations to adapt.  At the same time, costs of connections are rapidly decreasing as public policy shifts towards freer movement of goods, money and ideas and rapid improvements in the price-performance of IT infrastructures dramatically reduce the cost of information transmission.  Bottom line: Paretian distributions become even more prevalent…..

As McKelvey and Andriani point out, companies like Google and Microsoft have achieved enormous concentration of economic value creation that defies the averages of the Gaussian world.  These extreme events have an interesting property – they emerge first in the “fat tail”, on the edge of conventional business activity, driven by a different view of business opportunity, and then gather momentum until they eventually break into the head of the distribution and change the game for everyone else.  The challenge for business managers is to sort out the signal from the noise in the fat tail and spot early on the emergent extreme events that could reshape the business landscape.  The Gaussian focus on averages obscures these events, treating them as meaningless “outliers” until it is too late.  There’s another form of extreme event that also becomes more prominent in a Paretian world – this is the tendency for extreme forms of clustering in social networks, whether it takes the form of clustering in mega-cities in physical space or clustering of links and traffic on web sites in virtual space.  Economic value inexorably follows these social clusters.  This also has powerful implications for business, ranging from where to locate operations in physical space to how to redesign institutional architectures to accommodate thousands of business partners.  There’s also a public policy implication – in many domains we are likely to see degrees of concentration and consolidation of economic power that is unprecedented (although Pareto just over 100 years ago observed that 20% of the population in Italy owned 80% of the land).  Now, of course, all three of these extreme events are related – the clustering events generate and amplify the positive feedback loops that lead to both sudden and severe negative events as well as the more gradual, but no less significant, positive events…..

Later in the paper, McKelvey and Boisot offer a suggestion about different strategies for achieving understanding between the Gaussian and Paretian worlds: Processing dots is appropriate to what we label the routinizing strategy.  Processing patterns, on the other hand, better serves what we call the Pareto-adaptive strategy.  Processing dots means processing data, a low-level cognitive activity.  By contrast, processing patterns – pattern recognition – is a high-level cognitive activity, one that involves selecting relevant patterns from among myriad possibilities…..In a Paretian world, surface events can become a distraction, diverting attention from the deep structures molding these surface events.  Surfaces are extraordinarily complex and rapidly evolving while the deep structures display more simplicity and stability.  These deep structures are profoundly historical in nature – they evolve through positive feedback loops and path dependence.  Snapshots become misleading and understanding requires a dynamic view of the landscape…..But, as with most things in business (and in life), mindsets become a key stumbling block.  McKelvey and Boisot describe the “Gaussian perspective of the world” as one built on atomism, privileging “stability over instability, structure over process, objects over fields, and being over becoming.”  Not a bad summary of the way most Western executives view the business landscape.  There is a natural and very human tendency to seek out the typical or the average and to search for more predictability.  By implication, a Paretian world requires a much more dynamic view of the world, one that looks for patterns in evolving relationships, rooted deeply in context, and that understands how these changing patterns reshape who we are as well as our opportunities for growth.  McKelvey’s provocative work will help to challenge and shift our mindsets.

2 Responses to “Gauss vs. Pareto”

  1. Austin Bob Says:

    Great post. I think I just found another “daily read” (as well as some more references for my own blog if/when I find time to maintain it).

  2. MMM Says:

    Hey thanks man. Will check in on Hither and Yon, especially for your reflections on PM (I’m a recent PMP). Keep posting…

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: