How internal prediction markets can help media outlets
Future News 67
The American public were quickly and unceremoniously introduced to the world of prediction markets in 2003. Word had reached two senators of the blandly branded Policy Analysis Market (PAM), part of the Future Markets Applied to Prediction (FutureMAP) project from another tedious acronym, the Defense Advanced Research Projects Agency (DARPA).
The initiative was launched in May 2001, just months before al Qaeda’s September 11th attacks on the US, and was designed to help improve decision-making by forecasting military and political (in)stability around the world.
Created between George Mason University’s Dr Robin Hanson and private outfit NET Exchange, PAM traders would be able to place stakes on a set of different geopolitical scenarios. The idea was that price movements could help inform the American military, intelligence and diplomatic machine.
In July 2003, around a month after the PAM project went public with its prototype, it was quickly shuttered as Ron Wyden and Byron Dorgan, two outraged democrat senators, memorably re-named the project as a ‘terror market’. The ambitious initiative was then as good as dead, with then Deputy Defense Secretary Paul Wolfowitz pulling the plug right at the end of that month.
But PAM’s demise only proved to be a setback for prediction research within the US’ defense and intelligence communities. Intelligence Advanced Research Projects Activity (IARPA), for instance, held forecasting tournaments up to November 2019. The agency also has “the authority to run tournaments in the future”, a source close to the organization told FN. And, understandably, it is very much interested in predictive analytics around Covid-19.
The corporate world has also had its own flirtation with internal prediction markets. Researchers Bo Cowgill and Eric Zitzewitz looked at the impact of similar projects within Google, Ford and an unnamed basic materials conglomerate (Firm X), concluding:
Despite large differences in market design, operation, participation, and incentives, we find that prediction market prices at our three companies are well calibrated to probabilities and improve upon alternative forecasting methods. Ford employs experts to forecast weekly vehicle sales, and we show that contemporaneous prediction market forecasts outperform the expert forecast, achieving a 25% lower mean-squared error (p = 0.104).
For all of that, though, prediction markets haven’t caught on. Hanson, who is also a research associate at Oxford University’s Future of Humanity Institute, told FN that this is because some businesses don’t have the appetites for such data-driven initiatives and “don’t want to be at the will of some randomly fluctuating forecast”.
“The main attraction has been showing that you are on the cutting edge by doing a cool, sexy thing,” he added. “It’s mainly big companies who want to do that. They are not very interested in actually using it.” The problem, as Hanson sees it, is that the internal prediction market could become too “politically disruptive” for decision-makers and that is why they have been “killed” in the past.
Setting one up, though, can be fairly straightforward and can be simply kicked-off by sending out an all-team email with a login alongside an initial stake in the market, which, for companies, would usually have scenarios based on the outcomes of different projects. It is also important to provide anonymity, while the organization will benefit from being able to see prices. “People closest to a project will know if it’s not going to make a deadline,” Hanson explained.
As for using real or fake money, the academic suggested there was some evidence that the former increases participation in internal prediction markets, but, unlike opinion polling, it doesn’t have to be representative.
For Hanson there are “huge gains” to be made. But should companies and governments adopt another system, the Good Judgement Project’s superforecasting approach, as popularized by Pennsylvania University’s Professor Philip Tetlock? “Going from the status quo to anything I or Tetlock would make would be a huge improvement,” he diplomatically said.
💼 Jobs and business
Bloomberg has launched Quicktake, a streaming news channel.
NYT’s flagship Dealbook event is going virtual (FN 66 covered the issue here).
Shortlist for the 2020 British Journalism Awards has been announced.
Connected TVs are the new target for digital publishers.
Understanding how the 2020 Election polls performed.
🎧 Podcasts
BBC Panorama has teamed up with PBS’ Frontline to produce an investigative podcast, I’m Not A Monster.
🤖 Technology
Facebook and Twitter bosses face Judiciary Committee grilling on Tuesday.
Parler makes a pitch for conservatives.
India moves to regulate digital media.
📧 Contact
For high-praise, tips or gripes, please contact the editor at iansilvera@gmail.com or via @ianjsilvera. Follow on LinkedIn here.
FN 66 can be found here
FN 65 can be found here
FN 64 can be found here
FN 63 can be found here
FN 62 can be found here
FN 61 can be found here
FN 60 can be found here
FN 59 can be found here
FN 58 can be found here
FN 57 can be found here
FN 56 can be found here
FN 55 can be found here
FN 54 can be found here
FN 53 can be found here
FN 52 can be found here
FN 51 can be found here
FN 50 can be found here
Illustration: Freepik