How Predictive is Oral Argument Questioning?

By David Yin

The mainstream media and legal blogosphere have covered the oral arguments in the Patient Protection and Affordable Care Act litigation (Dept. of Health and Human Services v. Floridaad nauseam. The NYTimes certified, ”[t]here is no tea-leaf reading like Supreme Court tea-leaf reading” and excerpted commentary from notable sources:

Lyle Denniston from SCOTUSBlog determined that the argument ended “with Kennedy, after first displaying a very deep skepticism, leaving the impression that he might yet be the mandate’s savior.” Other observers, listening to the same argument, were not so optimistic about the law’s chances. Andrew Cohen of The Atlantic reported “[t]he initial wisdom from the commentariat coming out of Tuesday’s argument, after all the barking by court conservatives, is that the insurance mandate is in trouble.” Jeffrey Toobin of the New Yorker declared, in a much-criticized rant, “This is a train wreck for the Obama administration… This law looks like it’s going to be struck down. I’m telling you all of the predictions, including mine, that the justices would not have a problem with this law were wrong.” Dahlia Lithwick of Slate penned a piece titled, “A Moment of Silence for Obamacare.”

All this analysis elides the antecedent question, which is how much stock should we be putting into these pointed questions?

Rex E. Lee, a former Solicitor General, was often asked just how much an oral argument affected a case. His response was always “a confident ‘I don’t know.’” While Lee’s point was not intended to imply that oral argument is not useful, one might deduce that even one of the greatest oral advocates could not reliably predict whether his side was helped, or hindered, by the Court’s questioning. Lee went on to note, “It should not be inferred, however, that comments in oral argument always reflect the justices’ views. In the case of some members of the present Court, what you see and hear at oral argument is what you get at the conference vote. In other cases, it is not.” In other words, justices often play devil’s advocate, and trying to divine their position is just that–a game of astrologers and soothsayers, correct on some days, and wrong on others. And there are many reasons why a justice might want to play devil’s advocate. For this politically-charged case in particular, it may be important for optical reasons for conservative justices to be seen as aware of the government’s shortfalls before begrudgingly upholding the law on moderate grounds. In general, justices may want to test one side’s argument as far as it will go to develop an opinion that best outlines the limits of their rule.

A number of political scientists have attempted to statistically prove the contrary. A paper from Epstein,  Landes, and Posner; for example, concludes that justices tend to ask the most questions (and most wordy questions) of the side they tend to vote against. I haven’t done the math for the PPACA oral arguments, but given Breyer’s absurd 472-word question to Clement (transcript pages 61-63), I’d agree his skepticism is pretty much set in stone. Yet as much as general trends might be extracted from years of oral arguments, Lee’s point stands: for any one case, the confidence one has in predicting the outcome based solely on the questions should be low.

This term, the Supreme Court decided an important political question doctrine case, Zivotofsky v. Clinton (also known as MBZ v. Clinton).  The legal blogs turned out in full force, with most of the forecasts based on the oral argument going in favor of [Sec. State] Clinton/the Executive. SCOTUSBlog’s Lyle Denniston wrote, “the Court appeared to be leaning toward giving the White House a more modest victory over Congress.” Slate’s Dahlia Lithwick concluded, “The court appears poised to decide the substantive question in this case and to support the president’s expansive view of executive authority.” A few months later, much to the surprise of apparently everyone but Zivotofsky’s attorney, the Court came down with an 8-1 decision against the Executive.

Another example is the case of Helling v. McKinney, a 1992 case about whether housing a nonsmoking inmate with a smoking inmate constituted cruel and unusual punishment under the 8th Amendment. Although the lawyer for Nevada’s prisons was admittedly atrocious, amicus curiae was the U.S. Government, represented by then-Deputy SG John G. Roberts. Roberts delivered a killer analogy to bolster Nevada’s case: “If a parent smokes and exposes children to secondary tobacco smoke, we don’t brand that as child abuse. It’s difficult to imagine how what we allow parents to do to children in the home is somehow cruel and unusual when done to a convicted felon in prison.” Roberts’ analogy clearly piqued the Court, which posed it to said felon’s lawyer, and received an unconvincing response. Yet despite apparently “winning” oral argument as much as Clement “won“ on Tuesday, Roberts’ side went down 7-2.

The problem with divination from oral argument is that it makes too much of too little. As any lawyer or law student knows, the briefs have much greater weight for the bench than oral argument. Paul Clement explained, “I’m a big believer that oral argument makes a difference, but I’m also a big believer that comparably the briefs make even more of a difference.” Of course, a justice’s understanding of the law trumps both. I notice that despite his usual recalcitrance, we are all very confident in Justice Thomas’ vote in the healthcare case. And even a terrible oral argument from one’s opponent may not spare the other side from the cold hard merits (e.g. the respondent’s argument in U.S. v. Alvarez was cringe-worthy, but I’d bet Alvarez wins nonetheless).

Ultimately, the commentary on the healthcare oral arguments is interesting and informative to read, but should be taken with a heavy brick of salt. I suspect if any journalists or law professors could actually predict outcomes of cases with great accuracy and confidence, they would have long ago retired their pens to make fortunes in financial markets.

Old Paper by