A legal scholar says he and colleagues have developed an algorithm that can predict, with 70 percent accuracy, whether the US Supreme Court will uphold or reverse the lower-court decision before it.
"Using only data available prior to the date of decision, our model correctly identifies 69.7 percent of the Court’s overall affirm and reverse decisions and correctly forecasts 70.9% of the votes of individual justices across 7,700 cases and more than 68,000 justice votes," Josh Blackman, a South Texas College of Law scholar, wrote on his blog Tuesday.
While other models have achieved comparable accuracy rates, they were only designed to work at a single point in time with a single set of nine justices. Our model has proven consistently accurate at predicting six decades of behavior of thirty Justices appointed by thirteen Presidents. It works for the Roberts Court as well as it does for the Rehnquist, Burger, and Warren Courts. It works for Scalia, Thomas, and Alito as well as it does for Douglas, Brennan, and Marshall. Plus, we can predict Harlan, Powell, O’Connor, and Kennedy.
Given that there isn't much wagering action out there for Supreme Court decisions, Blackman says there's other real-world applications, like helping high court litigators develop strategies to overcome the model.
“If you have intelligence that’s reliable about how the court will decide the case, you can make a more informed litigation decision,” Blackman told the American Bar Association.
But are humans more accurate than an algorithm? Blackman, who created FantasySCOTUS—a Supreme Court fantasy league of sorts—said "power predictors" in the league had hit a 75 percent accuracy level.
But all of that is history. He plans a competition between the algorithm and the players in his Supreme Court fantasy league. "So what’s next? I’m sure you’re wondering how our model will do with future cases decided in the upcoming term. That’s the plan," he said. "This year we will be hosting a tournament where the players of FantasySCOTUS will compete against our algorithm. What IBM’s Watson did on Jeopardy, our model aims to do for the Supreme Court."
The algorithm, he said, uses the "extremely randomized trees" method, with more than 90 variables [PDF]. The source code is available here.
"Our model generates many randomized decision trees that try to predict the outcome of the cases, with different variables receiving different weights. This is known as the “extremely randomized trees” method," he said. "Then, the model compares the predictions of the trees to what actually happened, and learns what works, and what doesn’t. This process is repeated... many, many times, to calculate the weights that should be afforded to different variables. In the end, the model creates a general model to predict all cases across all courts."
reader comments
83