How internal prediction markets can help media outlets

Future News 67

The American public were quickly and unceremoniously introduced to the world of prediction markets in 2003. Word had reached two senators of the blandly branded Policy Analysis Market (PAM), part of the Future Markets Applied to Prediction (FutureMAP) project from another tedious acronym, the Defense Advanced Research Projects Agency (DARPA). 

The initiative was launched in May 2001, just months before al Qaeda’s September 11th attacks on the US, and was designed to help improve decision-making by forecasting military and political (in)stability around the world. 

Created between George Mason University’s Dr Robin Hanson and private outfit NET Exchange, PAM traders would be able to place stakes on a set of different geopolitical scenarios. The idea was that price movements could help inform the American military, intelligence and diplomatic machine.

In July 2003, around a month after the PAM project went public with its prototype, it was quickly shuttered as Ron Wyden and Byron Dorgan, two outraged democrat senators, memorably re-named the project as a ‘terror market’. The ambitious initiative was then as good as dead, with then Deputy Defense Secretary Paul Wolfowitz pulling the plug right at the end of that month. 

But PAM’s demise only proved to be a setback for prediction research within the US’ defense and intelligence communities. Intelligence Advanced Research Projects Activity (IARPA), for instance, held forecasting tournaments up to November 2019. The agency also has “the authority to run tournaments in the future”, a source close to the organization told FN. And, understandably, it is very much interested in predictive analytics around Covid-19. 

The corporate world has also had its own flirtation with internal prediction markets. Researchers Bo Cowgill and Eric Zitzewitz looked at the impact of similar projects within Google, Ford and an unnamed basic materials conglomerate (Firm X), concluding: 

Despite large differences in market design, operation, participation, and incentives, we find that prediction market prices at our three companies are well calibrated to probabilities and improve upon alternative forecasting methods. Ford employs experts to forecast weekly vehicle sales, and we show that contemporaneous prediction market forecasts outperform the expert forecast, achieving a 25% lower mean-squared error (p = 0.104).

For all of that, though, prediction markets haven’t caught on. Hanson, who is also a research associate at Oxford University’s Future of Humanity Institute, told FN that this is because some businesses don’t have the appetites for such data-driven initiatives and “don’t want to be at the will of some randomly fluctuating forecast”.

“The main attraction has been showing that you are on the cutting edge by doing a cool, sexy thing,” he added. “It’s mainly big companies who want to do that. They are not very interested in actually using it.” The problem, as Hanson sees it, is that the internal prediction market could become too “politically disruptive” for decision-makers and that is why they have been “killed” in the past.

Setting one up, though, can be fairly straightforward and can be simply kicked-off by sending out an all-team email with a login alongside an initial stake in the market, which, for companies, would usually have scenarios based on the outcomes of different projects. It is also important to provide anonymity, while the organization will benefit from being able to see prices. “People closest to a project will know if it’s not going to make a deadline,” Hanson explained.

As for using real or fake money, the academic suggested there was some evidence that the former increases participation in internal prediction markets, but, unlike opinion polling, it doesn’t have to be representative. 

For Hanson there are “huge gains” to be made. But should companies and governments adopt another system, the Good Judgement Project’s superforecasting approach, as popularized by Pennsylvania University’s Professor Philip Tetlock? “Going from the status quo to anything I or Tetlock would make would be a huge improvement,” he diplomatically said. 

💼 Jobs and business

🎧 Podcasts

🤖 Technology 

📧 Contact

For high-praise, tips or gripes, please contact the editor at iansilvera@gmail.com or via @ianjsilvera. Follow on LinkedIn here. 

Illustration: Freepik