Blockbusters and the long tail (part 2)

In their 1995 book The Winner-Take-All Society, Robert Frank and Phillip Cook argue that many markets, including those for entertainment, have feedback loops that cause popular products to become more and more popular. Frank and Cook see three main factors driving this process: (1) People are naturally drawn to greater talent. (2) People like to consume the same content as their friends and peers. (3) Products with high fixed costs and low marginal costs are more profitable when sold in large quantities.

William McPhee’s 1963 book Formal Theories of Mass Behavior makes a similar point about the natural advantages of popular products, arguing that obscure products face a “double jeopardy” in markets and are thus likely to remain obscure: Most consumers aren’t aware of their existence, and those consumers who are aware of them tend disproportionately to be experts who will also be aware of superior options.

On the other hand, even if a small number of blockbusters have dominated the entertainment markets in the past, that doesn’t mean they will continue to do so. What seemed in the past to be natural market concentrations may actually have had more to do with the limitations of physical channels than with limitations in consumers’ preferences. After all, you can’t buy what you can’t find. When consumers are offered a greater breadth of content, as they are on the internet, what if it turns out that they have interests and appetites that are much more diverse than was previously assumed? From this perspective, the theories of McPhee and Frank and Cook both have significant limitations when applied broadly to entertainment markets. Consider the case of product differentiation.

Economists recognize two main types of product differentiation: vertical and horizontal. In vertically differentiated markets, products exhibit a commonly agreed ordering of value. (Think BMW versus Chevy, or Hilton versus Holiday Inn, or hardcover versus paperback.) For entertainment goods, one might argue that vertical differentiation exists in the context of James Joyce versus E.L. James, the Grateful Dead versus the Dead Milkmen, or Tom Hanks versus just about everyone else. But even here there is room for debate.

And that’s the point. Many (perhaps most) entertainment goods don’t have a commonly agreed upon ordering. This puts them in the category of horizontally differentiated products. Thus, while Frank and Cook’s theory relies on consumers being drawn to “greater talent,” and McPhee’s relies on experts’ awareness of “superior options,” who can say whether your favorite book, movie, or song is “greater” or “lesser” than mine?

Frank and Cook’s second and third points are equally problematic when it comes to how technology might change the consumption of entertainment. Although people like to consume what their friends consume, online social networks allow us to receive recommendations from a wider circle of friends, potentially broadening our perspectives and allowing us to discover previously undiscovered niches. And although high fixed costs and low marginal costs naturally favor blockbuster products, digital technologies can lower the fixed costs of production for many types of entertainment, reducing the scale necessary to profitably create some types of content.

So what should you do when the underlying theory is inconclusive? You should study the data. That’s exactly what we did in 2000, with Erik Brynjolfsson and Yu Jeffrey Hu. And the answer that emerged was clear. Online access to niche products creates an enormous amount of value for consumers.

Our study began with an approach suggested by Madeline Schnapp, then the director of market research at O’Reilly Books. Previously, Schnapp had collected weekly sales data from Amazon for a set of O’Reilly’s titles and matched these data to the sales ranks reported on Amazon’s product page. With these data, she developed a model that, given knowledge of the sales rank of any of Amazon’s titles, could predict the weekly sales of that title fairly accurately. Using a similar empirical approach and a dataset provided by an anonymous publisher, we replicated Schnapp’s results and found strong evidence that online consumers had a great appetite for obscure titles. In our estimates, between a third and a half of Amazon’s sales during that period came from titles that wouldn’t be available in even the largest brick-and-mortar bookstores.

To calculate how much economic value was generated when consumers were given access to obscure titles, we turned to an approach for measuring the value of “new goods” developed by Jerry Hausman and Gregory Leonard. The main advantage of this approach is that it doesn’t rely on theoretical views of consumer behavior or on judgments about the relative worth of mainstream and obscure titles. Instead, it focuses on the economic reality of what consumers purchase, and on their revealed willingness to pay for those purchases.