The Policy Analysis Market (and FutureMAP) Archive

by Robin Hanson

This website contains information on the Policy Analysis Market, a DARPA-funded research project, within the FutureMAP program, which was suddenly cancelled amid a media storm on July 29, 2003.

Information on the original Policy Analysis Market (PAM)

PAM (and FutureMAP) before the media storm.

In 2000 Michael Foster, who ran (and still runs) the National Science Foundation quantum computing research program, convinced DARPA (Defense Advanced Research Projects Agency, the blue-sky research arm of the U.S. Defense Department) to fund research on prediction markets starting in 2001. Prediction markets are speculative markets created for the purpose of aggregating information on topics of interest. Previous field studies had found that such markets out-predict co-existing institutions regarding the weather, printer sales, movie sales, elections, and much more.

This research program was eventually named "FutureMAP", but the first DARPA call for proposals went out under the name "Electronic Market-Based Decision Support." This call basically said "We've heard this works elsewhere; show us it works for problems we care about." The call went out in May 2001, for proposals due in August, and by December two firms had won SBIR (Small business independent research) grants. The winners were Neoteric Technologies, subcontracting to Martek and professors at the University of Iowa, and Net Exchange, founded by a Caltech professor (John Ledyard) and subcontracting to professors at George Mason University (myself and David Porter), and later to the Economist Intelligence Unit. The Net Exchange project came to be called the "Policy Analysis Market" (PAM).

The plan was for two firms to get $100K for a six month Phase I, and after which one of them would be awarded $750,000 to continue Phase II over two more years. There was also the possibility of applying to get $100,000 of funding for the six months between these phases. More money became available than initially planned, so in fall 2002 both firms were funded to continue to Phase II, and Net Exchange applied for and won interim funding. Also during 2002, the infamous John Poindexter (who I have never met) became a DARPA executive, and Foster's FutureMAP program was placed within Poindexter's organization, the Information Awareness Office (IAO). In December 2002, DARPA called for proposals for related research, at this point using the name FutureMAP. In summer 2003 a half dozen teams, at Penn State, Metron, ICT, GMU (including me), Sparta, and BBN, were awarded $100,000 each.

Neotek sponsored an end of phase I conference in June 2002, and showed a few demonstration markets, using their pre-existing software, on SARS and the color security threat level. When FutureMAP was cancelled, Neotek had still not identified their market topics, and had surely spent less than half of their Phase II funding. Net Exchange spent about two thirds of their Phase II funding, and the new small projects had spent little of their funding. Michael Foster had asked for, but not received, $8,000,000 more in FutureMAP funding over the next few years.

From the very start, the Net Exchange team began laboratory experiments to study the issue of price manipulatoin, as this was a widely expressed concern. Also from the start, we planned to focus on forecasting military and political instability around the world, how US policies would effect such instability, and how such instability would impact US and global aggregates of interest. The Net Exchange president, Charles Polk, named this the Policy Analysis Market (PAM). We later had to narrow our focus to a smaller region, the Mideast, because the Economist Intelligence Unit charged a high price to judge after the fact what instability had actually occurred in each nation.

We planned to cover eight nations. For each nation in each quarter of a year, we planned to have traders predict its military activity, political instability, economic growth, US military activity, and US financial involvement. In addition traders would predict US GDP, world trade, US military casualties, and western terrorist casualties, and a few to-be-determined miscellaneous items. This would require a hundred or so base markets. Most important, we wanted to let our traders predict combinations of these, such has how moving US troops out of Saudi Arabia would effect political stability there, how that would effect stability in neighboring nations, and how all that might change oil prices.

For many years before PAM, Net Exchange had specialized in combinatorial markets, where buyers and sellers can exchange complex packages of items. So from the start of PAM, we planned to see how far we could go in developing combinatorial prediction markets. In Phase I Net Exchange put together a combinatorial market similar to their previous markets, and at the end of Phase I we ran a complex simulation where a dozen students traded over a few days for real money. Unfortunately, only about a dozen trades occurred, a serious failure.

In the interim phase, the Net Exchange team prepared for and ran lab experiments comparing two new combinatorial trading mechanisms with traditional mechanism. These experiments, where six traders set 255 independent prices in five minutes, found that a combinatorial market maker was the most accurate. Phase II was mostly being spent implementing a scaleable production version of this market maker. It requires a net subsidy to traders, and so because we had budgeted $50,000 for this subsidy, individual bets were limited to a few tens of dollars.

We were concerned that we might not attract enough traders to achieve a meaningful test. While we had considered running markets within government agencies, we choose public markets due to legal problems with conditional transfers of money between agencies and the absence of a single agency strongly interested in collaborating. On May 20, 200, DARPA reported to congress on the IAO, and described FutureMAP in terms of predicting a bioweapons attack against Israel. In June 2003 we began to tell people about our webpage, and to give talks drum up interest. Charles Polk created the PAM website, wherein in the faint background sample screen, he included as colorful examples of miscellaneous items the assassination of Arafat, and a missile attack from North Korea.

In the summer of 2003, the Senate but not the House had cancelled IAO funding, which included all FutureMAP funding, because of privacy concerns with another IAO project, "Total Information Awareness." Due to this funding uncertainty, when the media storm hit our plans were to start on September 1 with one hundred testers to which we had each given $100. Registration to be one of those testers was to open August 1, with public trading to being January 1, 2004.

The media storm hit on July 28, 2003, when two senators complained that we were planning to let people bet on terrorist attacks. The next morning the secretary of defense announced that FutureMAP was cancelled. In the intervening day, no one from Congress asked us if the accusations were correct, or if the more offending aspects could be cut from the project. DARPA said nothing. The next day, John Poindexter resigned, and two months later all IAO research was cancelled.

Other PAM pages