Essay | 12.09.99
Why Everything Is All That
Steven Johnson asks why the web isn't growing smarter with old age.
SO YOU'RE ON a book tour, promoting a little pamphlet you've written on interface design and contemporary culture, and you're subsisting on the standard book tour diet of radio call-in shows and alternative weekly interviews -- not to mention the occasional two-minute paroxysm on "Good Morning, Portland!" Only your publisher also happens to specialize in a line of contemporary spiritual titles, and so the in-house publicist has sent galleys of your digital manifesto to every New Age radio station, print zine, and ashram in the country. What's more, some of them have actually taken the bait. And so before you know it, you find yourself -- hard-bitten secularist that you are -- sandwiched between the Harmonic Convergence Hour and Millennium Watch on the drivetime segment, fielding questions from the listeners of Radio Deepak.
The thing is, the questions turn out to be consistently as smart and forward-thinking and technologically adept as any you've encountered on the tour. They're sensitive to the nuances of your argument, and refreshingly indifferent to the latest IPO pricing. You start kicking yourself for embarking on the interview with such prejudice -- until your interlocutors roll out their Final Question, and you find yourself stammering into the microphone, looking for exit signs. "You've written a great deal about the web and its influence on modern society. Do you think, in the long term, that the rise of the web is leading towards a single, global, holistic consciousness that will unite us all in Godhead?"
It's a question with only one responsible answer: "I'm not qualified to answer that." And I confess that's the response I gave the first five times it was posed to me, though each time I thought to myself that there was something fundamentally flawed about the concept, something close to a category mistake. For there to be a single, global consciousness, the web itself would have to be getting smarter, and the web wasn't any single unified thing -- it was just a vast sum of interlinked data. You could debate whether the web was making us smarter, but whether the web itself was slouching towards consciousness seemed ludicrous.
Two years later the question is still bouncing around in my head, and I have to admit I'm warming up to it, in a roundabout way. Is the web getting smarter? I still think the answer is no. The only difference now is that I think it's worth asking why not.
"SMARTER," OF COURSE, is a contentious word, and probably not the right one to use in this context. The web certainly isn't getting smarter in the Terminator sense of a network becoming self-aware. (Though if the web does ever achieve consciousness, here's hoping the first thing it does is destroy Ron Harris.) A better -- and less Extropian -- way of posing the question might be: is the web becoming more organized as it grows? Is it becoming more structured with use, or less structured? Plenty of decentralized systems in the real world spontaneously generate structure as they increase in size: Cities organize into neighborhoods or satellites; the neural connections of our brains develop extraordinarily specialized regions without any master planner drawing up the blueprints. Has the web followed a comparable path of development over the past few years?
You need only take a quick look at the NASDAQ most active list to see that the answer is an unequivocal no. The portals and the search engines exist in the first place because the web is a tremendously disorganized space, a system where the disorder grows right alongside the overall volume. Yahoo and AltaVista function, in a way, as man-made antidotes to the web's natural chaos -- an engineered attempt to restore structure to a system that is incapable of generating structure on its own. This is the oft-noted paradox of the web: The more information that flows into its reservoirs, the harder it becomes to find any single piece of information in that soup.
One way to wrestle with this concept is to think about the web as an old-fashioned Gibsonian virtual metropolis. Imagine the universe of HTML documents as a kind of city spread out across a vast landscape, with each document representing a building in that space. Only this particular city is more anarchic than any real-world city on the planet. There are no patches of related shops and businesses -- no meat-packing or theater districts; no bohemian communities or upscale brownstones; not even the much-lamented edge city clusters of Los Angeles or Tyson's Corner. The web's city is simply an undifferentiated mass of data that grows more confusing with each new "building" that's erected -- so confusing, in fact, that the mapmakers (the Yahoos and Googles of the world) generate almost as much interest as the city itself.
IF THE WEB turns out to be so inept at organizing itself, then how do cities manage to pull it off? Granted, actual cities are heavily shaped by top-down forces, like zoning laws and planning commissions, but for the sake of the analogy, let's restrict ourselves to the bottom-up forces that create neighborhoods and other unplanned population clusters. For some time now complexity theorists have speculated about a mystical process of "order emerging out of chaos" in research papers and books that were as much descriptions of this behavior as they were explanations. But in recent years, some of those theorists -- not to mention a handful of mainstream economists -- have developed more precise models that recreate the neighborhood-formation process with startling precision.
Paul Krugman's wonderful 1995 lectures, "The Self-Organizing Economy" -- published as a book the following year -- include a remarkably simple mathematical model that can account for the "polycentric, plum-pudding pattern of the modern metropolis." Building on the segregation models of celebrated game theorist Thomas Schelling, Krugman's system assumes a simplified city made up only of businesses, each of which makes a decision about where to locate itself based on the location of other businesses. There are centripetal forces drawing businesses closer to one another (because firms may want to share a customer base or other local services) and there are centrifugal forces driving businesses further apart (because firms compete for labor, land, and in some cases customers.) Within that environment, Krugman's model relies on two primary axioms:
1. There must be a tension between centripetal and centrifugal forces, with neither too strong.
2. The range of the centripetal forces must be shorter than that of the centrifugal forces: business must like to have other businesses nearby, but dislike having them a little way away. (A specialty store likes it when other stores move into its shopping mall, because they pull in more potential customers; it does not like it when stores move into a rival mall 10 miles away.)
And that's all that we need. In any model meeting these criteria, any initial distribution of business across the landscape, no matter how even (or random), will spontaneously organize itself into a pattern with multiple, clearly separated business centers.
Krugman even provides a chart demonstrating the city's self-organization in time -- an image that captures the elegance of the model. Scatter a thousand businesses across this landscape at random, then turn on the clock and watch them shuffle around the space. Eventually, no matter what the initial configuration, the firms will gather into a series of distinct clusters evenly spaced from each other. There's no rule for clustering that the businesses are directly obeying: their own motives are strictly local. But those micromotives nevertheless combine to form "macrobehavior" -- to use Schelling's language -- a high-level order that exists on the level of the city itself. Of course, if the centripetal forces are too strong, the landscape quickly contracts into a single, dense "central place" -- as cities have done for millenia. Looking at the world through the lens of Krugman's model, you can see the historical rise of the edge city as a secondary-effect of centrifugal forces growing stronger, due to technological and infrastructural changes in post-war urbanism.
Can the web get smarter? Share your thoughts in the Loop.