What Predictive Coding Court Rulings Can Teach Us

Given the efficiencies and benefits of predictive analytics – it’s no surprise that it has been getting a lot of press over the past few years. Yet, there are relatively few cases where the case law explicitly references predictive analytics. Judge Andrew Peck has been an important influence in advocating for the use of predictive analytics within his written opinions. However, his recent ruling in Hyles vs. City of New York again highlighted how the intersection between this technology and the legal realm is still evolving.Predictive coding offers the promise of significant cost savings where expenditures are at their highest. Download these case studies that show the strategic value of getting to the data faster.

Whatever you call it, the basic concept of predictive analytics is the same — decisions that are made on a statistically significant smaller sample can then be used to forecast like decisions on the rest of the document universe.  Yes, there are differences in the math and underlying technology for the various methods (TAR, predictive coding, CAR, etc.) but they all provide a prioritized approach which is much better than traditional linear review.  Predictive analytics is here to stay, so how does one best position predictive analytics with the opposing party or more importantly, the bench?  This article addresses that issue.

Properly positioning the use of predictive analytics starts with considering what has been opined in previous cases and current predictive analytics industry trends.

Court rulings reinforce the importance of cooperation between parties

There is a trend towards judicial awareness of predictive analytics and support of parties’ agreements to use advanced technologies. Additionally, a few seminal decisions provide guidance on the defensible use of predictive analytics and the importance of agreed-upon protocols and disclosure – both of the processes utilized and the results achieved.

Global Aerospace, Inc. v. Landow Aviation

A court acceded to the defendant’s request the use of predictive analytics over the plaintiff ’s objection in Global Aerospace, Inc. v. Landow Aviation. The plaintiff argued that the court’s approval of predictive analytics constituted its agreement that Defendants need only produce a certain percentage of responsive documents. The court allowed the use of predictive analytics, but also preserved the plaintiff’s ability to challenge the completeness and sufficiency of the production.

Kleen Products LLC v. Packaging Corp. of America

In Kleen Products LLC v. Packaging Corp. of America, Plaintiffs sought to compel the defendants to redo their previous productions using predictive coding technology. Defendants countered that their use of keyword searching and advanced analytics (such as email threading) was precise and thorough. After a protracted struggle involving two separate hearings and multiple written submissions, Plaintiffs withdrew their demand as to the existing document request. For future production requests, the parties agreed to meet and confer regarding production methods, raising the possibility that Plaintiffs might renew their demands about predictive analytics.

EORHB, Inc. v. HOA Holdings (Hooters), LLC

Hooters is known for its wings, beer, and well, you know. The U.S. restaurant chain was involved in a lawsuit stemming from the sale of the company. What’s unique about the case, however, is that Vice Chancellor J. Travis Laster, the Delaware judge presiding, recommended the use of predictive analytics to resolve the legal dispute. By requesting that the plaintiff and defendant “show cause why this is not a case where predictive coding is the way to go,” the court order marks the first time a judge has demanded serious consideration be given to predictive coding.

The court also issued an order directing both parties to demonstrate why they should not use a single predictive coding provider in EORHB, Inc. v. HOA Holdings, LLC. The court’s decision in EORHB was especially bold at the time, because it essentially prescribed the use of predictive analytics software (and one provider thereof) unless the parties could show cause not to do so. However, after the show cause hearing, the court lifted the order. This allowed Defendants to proceed with the vendor of their choice and Plaintiffs to proceed without using predictive coding, given the low volume of documents they were expected to produce.

In re Biomet

In re Biomet was a products liability case in which the defendant began review by using search terms to cull nearly 20 million documents over the objection of Plaintiffs’ Steering Committee. The Committee urged the defendant to await a ruling from the Judicial Panel on Multi-district Litigation before proceeding with document productions. 

When Defendant thereafter used predictive coding, Plaintiffs argued that the initial use of search terms had tainted the document pool and asked the court to require the defendant to start over, using only predictive coding. The court refused to honor Plaintiffs’ request on the grounds that Defendant had fulfilled discovery obligations using reasonable methods.

Pyrrho Invs. Ltd. v. MWB Prop. Ltd

The United States is not the only country where the judiciary is embracing predictive analytics.  In February 2016, the England and Wales High Court, Chancery Division approved the use of predictive coding to satisfy discovery requests in Pyrrho Invs. Ltd. v. MWB Prop. Ltd.Although the court acknowledged that the use of predictive coding may not be a good fit for every case, it decided that—where it is possible and appropriate—this method can and should be used alongside manual review to fulfill discovery requests for electronically stored information.  We expect to see more decisions like this from other countries in the near future.

Tips for establishing predictive coding “rules of engagement”

Although courts have not mandated or prohibited the use of predictive analytics outright, they have specified acceptable rules of engagement for the use of predictive analytics. Perhaps the most well-known case is Da Silva Moore v. Publicis Groupe. 

Da Silva Moore is touted as the first published judicial opinion that implicitly approved the use of predictive analytics for reviewing documents. Both parties assented to the use of predictive analytics, but argued over the manner in which it would be employed; per the parties’ wishes, the court permitted predictive analytics usage and provided guidance to establish rules of engagement. 

U.S. Magistrate Judge Andrew Peck emphasized that parties intending to use predictive analytics must choose a reliable eDiscovery vendor and program, and must design an “appropriate process” that includes “appropriate quality control testing.” The opinion specified a number of steps that parties should consider, including the following suggestions:

1. Bring both vendor experts to a court hearing to respond to the magistrate judge’s questions.
2. Allow the requesting party to view the documents that were used to train the predictive analytics system, both those that were marked responsive and those that were marked not responsive.
3. Allow the requesting party to view the additional documents that were used to stabilize the predictive coding system, whether or not they were marked responsive.
4. Do not adhere to an arbitrary number of documents that will be produced without reference to the statistical results.
5. Do not limit the number of iterative reviews used to “train” the system up front, but rather assess whether the system has stabilized before stopping the iterative reviews.

U.S. District Judge Andrew Carter later approved Judge Peck’s protocol on the basis that it was reliable, standards-based, and allowed participation by Plaintiffs, who were given the ability to object once production was made if they believed it to be insufficient.

The Da Silva Moore principles of cooperation, transparency and attention to a strong workflow had a concrete and defined role in the case of In re Actos. In Actos, the parties entered into a detailed protocol, wherein three experts from each side met, collaboratively reviewed the training set of Defendants’ documents, and agreed upon relevance determinations for all non-privileged documents in the set. Robust protections to guard documents subject to privilege and confidentiality were included. This approach may not be right for every case in which predictive analytics is used, but it does provide an example of a transparent process.

Although there are a few definitive decisions regarding predictive analytics, there is certainly room for optimism that these applications will succeed in gaining widespread judicial support. As lawyers and judges being to trust the reliability of predictive analytics they will fully realize and understand when how to best deploy advanced technologies in different scenarios.  

Learn from those that have gone before and adopt the guidelines Judge Peck outlines in Da Silva Moore to ensure the use of Predictive Analytics provides a solid foundation for your next case.