Swarm AI Produces 57% Return on Investment in Season-Long NBA Study
Swarm AI technology has enabled groups of fans and enthusiasts to produce some fairly remarkable results, and the weekly NBA Swarm is no exception. Working together as an AI system that combines their wisdom, experience, and intuition, the NBA Swarm went 9-1 last week, and has produced multiple perfect forecasts, including a ten-team parlay that scored a 2,700% Return on Investment (ROI). Of course, it’s one thing to catch lightning in a bottle on a given night, but quite another to achieve consistently strong results week in and week out. Forecasting consistency is one of the hallmarks of good forecasting, and another is the self-awareness to know when to make a prediction ( or a wager ) and when not to.
With that in mind, Unanimous AI has recently completed a season-long study of the NBA Swarm’s ability to forecast basketball matchups. These results examine the AI’s performance over a 25-week sample of games and test a machine learning model (derived from years of Unanimous AI’s sports forecasting swarms) to determine the relationship between the Swarm’s confidence and its ability to produce a successful Return on Investment. This study – Dense Neural Network used to Amplify the Forecasting Accuracy of real-time Human Swarms – will be presented at the Computational Intelligence and Communication Networks (CICN) Conference in January, 2019.
Swarm AI technology is modeled after the way natural groups form intelligent systems in order to amplify their intelligence and solve complex problems. Fish schooling, bees swarming, and birds flocking are all examples of these real-time systems, and Unanimous AI brings this power to human groups via its Swarm AI platform. When a group of NBA fans and enthusiasts comes together to make a forecast, they form a dynamic, real-time system, moderated by AI algorithms, that enables them to converge on an optimal answer. Every magnet that you see in the replay below represents an individual fan. Together they negotiate and navigate the puck to the answer that best represents the collective will and insight of the group.
For this study, Unanimous AI formed a series of 25 weekly swarms over the course of the NBA season. Those swarms ranged in size from 28 to 43 participants, and in total they forecast 238 NBA games in 25 weeks. The statistical benefits of this type of long-term study should be quite clear, as the ability to reproduce a result is central to the idea of demonstrably accurate and reliable forecasting. But, unlike a previous long-term study of the NHL Swarm, which showed that a group of hockey fans were able to produce a 170% ROI over 20 weeks, this study introduces a Conviction Index to test multiple wagering strategies made possible by the NBA Swarm.
Forecasting the Swarm AI’s Forecast
The Conviction Index is a measure of the Swarm AI’s confidence in a given prediction, and the goal of the study was to see if that confidence could be positively correlated to predictive success. In other words, does the AI know when it has enough information and insight to make a good prediction? And, if we know that, can we take the betting odds into account to estimate the Swarm AI’s expected ROI for a given strategy?
To find out, researchers at Unanimous AI trained a dense neural network using aggregate behaviors displayed by 424 NFL and NHL swarms. This network, known as the Swarm Conviction Estimator was used to evaluate every prediction the NBA Swarm made throughout the study. At the same time, an ROI Estimator was developed using odds from Sportsbook.ag, a well-known bookkeeper. In this way, researchers were able to determine tiers of confidence for every forecast NBA Swarm and the risk/ reward profile of wagering on those games. From that information, the results of those 25 swarms could be analyzed to determine which strategies would produce the best return on investment.
If consistency is the first pillar of good forecasting, and discretion is the second, then self-awareness about when a forecast should – and should not – be trusted is paramount. In the context of this study, that means that the Conviction Index + the ROI Estimator can be used to forecast the results of AI’s own forecast into an Expected ROI value.
AI Optimization More Than Doubles Forecast Success
The most basic strategy, of course, would be to simply wager on every game in the forecast, disregarding the Conviction Index entirely. Naturally, this strategy would violate the second pillar of good forecasting, so we would expect that – if the Conviction Index is doing its job, it would also be the least accurate and successful method. In contrast, strategies that pare down the 238 available betting opportunities into buckets of increasingly selective Expected ROI, and compare those results. The chart below details the increasingly profitable results of wagering only on games where the Expected ROI is above certain thresholds.
As you can see, the Swarm AI’s probability of producing positive ROI was just under 90%, no matter which selection method is being used. And yet, as the threshold for placing a bet increases from “bet on every game” to “bet on games with an Expected ROI above 20%” the calculated Return on Investment at the end of the NBA season more than doubles.
In other words, the most naive betting strategy possible from the NBA Swarm’s forecast nevertheless produced a 24.7% ROI, and was 88.8% likely to produce positive ROI. Those kinds of number reflect a spectacular ability to cap downside risk alongside the sort of return that would make a hedge-fund manager drool.
But, if you were to wager only on the games where a positive ROI could be expected, the likelihood of victory remains constant, and the profit improve to nearly 30%. And so on, until, if one were to wager only one NBA games where the Swarm AI’s expected ROI was above 20%, the results are incredible – 89% chance of positive ROI, and an actual 56.6% Return on Investment.
Finally, it’s useful to compare the NBA Swarm’s ability to predict winners against the frequency with which Vegas favorites emerge victorious. Using the data accumulated over the course of this study and simulated 10,000 times, Unanimous found that Vegas favorites could be expected to win 66.5% of the time, while the NBA Swarm would be accurate 72% of the time. That 5% increase in accuracy is significant, but also underscores the importance of knowing which of the NBA Swarm’s forecasts to put your hard-earned money behind.
In this way, the NBA Swarm combined with the Conviction Index algorithm was able to deliver on both pillars of good forecasting. First, the AI proved to be reliable over a long-term study. That the NBA Swarm was able to deliver a perfect forecast on one night is undoubtedly impressive, but it’s much more important that the system reliably produce amplified intelligence. To that end, this season-long study demonstrates that, over the course of an entire season, the NBA Swarm was able to outperform Vegas reliably, and forecast winners with 72% accuracy. Then, when that full set of predictions is pared down using the Conviction Index, the NBA Swarm’s ROI more than doubles, suggesting that the AI-optimized self-awareness had a major impact on the expected return.
About Unanimous AI
Unanimous AI pioneers technologies that amplify the intelligence of business teams, combining the power of AI with the depth of human insight. Its Swarm AI technology is currently being used by Fortune 500 companies for a wide range of applications, from forecasting sales to optimizing marketing campaigns. In 2018, Swarm AI was recognized as SXSW’s “Artificial Intelligence Innovation of the Year.” For more about Unanimous AI, see the TED talk, “Hope for Humans in an AI World” or visit http://unanimous.ai
Would you like to participate in future predictions, experiments and studies? Unanimous AI holds several weekly Swarm Sessions across a variety of topics, from the NFL and English Premier League, to elections, the Oscars, and more.
Want to learn more about our Swarm AI technology? Check out our TED talk below…
