Can Computers Pick the Next Big Thing?

They can, to a point—but human interaction can't be reduced to algorithms

By Barrett Sheridan

In the early 2000s, a handful of entrepreneurs became convinced that machines could mimic human taste and effectively predict popularity. This was a revolutionary notion, suggesting that the talents of legendary tastemakers—the Harvey Weinsteins and Clive Davises—could be replicated by silicon and algorithms. In melodramatic terms, the idea represented an escalation in the war between humans and machines, furthering the debate over what skills and faculties, if any, are unique to homo sapiens.

With each passing year, the humanists appear to lose ground. In 1997 an IBM (IBM) supercomputer named Deep Blue beat the world's best chess player, Garry Kasparov, in a six-game match. Kasparov later wrote off Deep Blue and its relatives as "brute-force programs" that played chess with no creativity, no concern for "hundreds of years of established theory."

Kasparov would have had little patience for the would-be hit predictors, who, for the last decade or so, have tried to do for art and culture what Deep Blue did for chess. Generally, they distilled a piece of content to its numerical essence. Songs were easiest, because their underlying structure is mostly math. Companies and research centers such as The Echo Nest and the International Society for Music Information Retrieval built up databases and correlated variables like pitch, tempo, and melody. By correlating them with historical information on how the song fared in the market, the hit predictors could make an educated guess about whether a brand-new song stood a chance of topping the charts.

One company trying to do this was called Hit Song Science, founded in Barcelona in 2001. Hit Song Science had some early success. In 2002, as the team was fine-tuning its algorithms, HSS determined that 8 of the 14 tracks on an album by a then-obscure singer had genuine hit material. That album, Come Away with Me by Norah Jones, subsequently sold more than 10 million copies.

The same year, an executive at BMG who was promoting a new band, Maroon 5, got in touch with Mike McCready, one of HSS's co-founders. The band's single, Harder to Breathe, was going nowhere, and the BMG executive needed help. Running the album through his software, McCready determined that another track, This Love, had much greater hit potential. The executive sent the new single to radio stations, and Maroon 5's album, Songs About Jane, went triple platinum.

As a business, HSS was not quite as successful. Many predictions turned out to be duds. The algorithms rated Michael Jackson's Billie Jean a flop and a six-minute instrumental a surefire hit. "We discovered we couldn't make the bold kind of claims we were hoping we could make with this technology," says McCready, who left the company in 2006. He now runs a website called Music Xray that helps match music executives and musicians. "The technology might get there someday, but it's not there now."

The number of different possible chess games is 10 to the 120th power—a staggering number, and a surmountable one, if you have the right "brute-force programs." Popularity, by contrast, is a social phenomenon. Making predictions without accounting for human interaction and influence is like programming a computer to play chess and ignore the queen.

Duncan Watts knows this better than anyone. Now Yahoo!'s (YHOO) chief research scientist, Watts was a professor at Columbia University in 2006 when he and two graduate students performed a study that confirmed what marketers have long known: Humans are deeply susceptible to persuasion, and there's no way to predict their tastes unless social factors are considered. In the study, the researchers asked 14,000 people to rank songs by bands they'd never heard of. Some of the participants had no information to go on other than their own taste; others were grouped into pools and shown what the rest liked.

Business Exchange

Track and share business topics across the Web.