Predictive Coding vs. Search Terms: Who Determines the Method of Review?

This blog was originally published on The Daily Record.

Two recent cases in two different courts dealt with the issue of technology assisted review (“TAR”) and its use in discovery. At issue in each case was the plaintiffs desire for defendants to use TAR instead of relying on search terms to identify relevant documents. Both courts denied the requests.

Technology Assisted Review in eDiscovery

Before we go further I will provide a brief refresher on TAR and its place in the electronic discovery process.  TAR, or predictive coding as it’s sometimes called, is a technology and a process that involves a subject matter expert (a.k.a. human) coding documents as responsive or non-responsive. Those decisions are fed into predictive coding software and after a number of iterations the software is able to determine the responsiveness level of the remaining documents.

Yes, this is more of the “machines are replacing humans” you have been reading about, which has been happening since the Luddite riots. TAR has been around for a number of years and has been widely used by parties on all types of matters. Its application has been proven in many studies and in practice to be a more efficient and effective manner of identifying and producing relevant documents compared to reliance on key terms. Many courts have advocated and celebrated the use of predictive coding. One of those courts is the same who denied the plaintiffs request to force defendants to use it.

Hyles vs. New York City

In Hyles vs. New York City (S.D.N.Y 08/01/2016), Plaintiff requested the Court to force the City to utilize TAR. Plaintiff proposed that the City should use TAR as a “more cost-effective and efficient method of obtaining ESI from Defendants.” The City claimed that using TAR would not be cost-effective and due to previous disagreements on discovery issues, felt that coming up with an agreeable set of seed documents would be difficult. They staunchly opposed the use of TAR and wanted to utilize keywords and conduct a traditional document review for responsiveness and privilege.

Judge Peck, Predictive Coding, and the Obligation to Determine Document Review Workflow

The judge in the matter, Andrew Peck, is a strong proponent of TAR and agreed with the plaintiff that it is far superior to key terms. Judge Peck stated: “Hyles absolutely is correct that in general, TAR is cheaper, more efficient and superior to keyword searching.”

However, Judge Peck also strongly believes in the Sedona Conference Principles and referenced the sixth principle that states: “[r]esponding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.” He went on to state that “it is not up to the Court, or the requesting party (Hyles), to force the City as the responding party to use TAR when it prefers to use keyword searching. While Hyles may well be correct that production using keywords may not be as complete as it would be if TAR were used, the standard is not perfection, or using the ’best’ tool, but whether the search results are reasonable and proportional.”

If it’s a fact that TAR is a more efficient and cost-effective process for discovery, then why would any party refuse or be reluctant to use it? Maybe the value of the case is too small to warrant the use of TAR. While TAR is more cost-effective, those savings typically occur during the review phase. The processing and application of predictive coding technology is often pricier than indexing a document set in preparation for keyword searching.

If you have a small dataset or access to inexpensive review resources, then TAR may not be the path for your matter. It also may be that TAR is seen as black-box technology or an esoteric process that attorneys don’t trust or understand. Maybe they want eyes on every documents and using keywords is like a warm blanket and hot cocoa on a cold winter’s night. Or perhaps a party already has what it believes to be a sound process for identifying and reviewing documents… A process similar to Pfizer’s.

A Responding Party Can Choose its Method of Document Selection

Pfizer’s Viagra products liability litigation, an order was issued by the Court (N.D. California) in October of this year regarding the use of TAR. Once again the plaintiffs requested that the Court force “Pfizer to use technology assisted review (“TAR”) and/or predictive coding with Plaintiffs’ input to identify the locations of relevant information and the responsive ESI from those locations. Plaintiffs argue that TAR and/or predictive coding is a more sophisticated tool than the traditional search term or search query approach, and that using that suggested approach would save time and money for both sides.”

This sounds like a lot like the Hyles matters and it warms my heart that the plaintiffs are concerned with Pfizer saving money. However, there is a cost saving potential for parties on the receiving end as well. Pfizer’s use of TAR may result in a smaller number of documents produced to the plaintiffs, which lessens their cost of review. But maybe Pfizer has found a better way or one that it feels works better than TAR. Their process is outlined in the order:

“Pfizer describes its preferred methodology an iterative process—albeit not the same iterative process as TAR and/or predictive coding—in which Pfizer tests search terms and validates them using rigorous sampling of potentially responsive documents. Initially, Pfizer tests and validates a number of search terms by applying those terms to a sample set of documents, reviewing the documents for responsiveness, and verifying that the search terms yield high rates of response.”

Effective perhaps, old-school perhaps, but a process nonetheless. And one that seems to work for Pfizer.

In the end the Court denied the Plaintiff’s request and cited the Hyles matter.

“As the court reasoned in Hyles, the responding party is the one best situated to decide how to search for and produce ESI responsive to discovery requests. The responding party “can use the search method of its choice. If [the propounding party] later demonstrates deficiencies in the … production, the [responding party] may have to re-do its search. But that is not a basis for Court intervention at this stage of the case.”

Making TAR the Standard

Whatever the reason(s) the defendants in these two matters had for not using TAR, both courts decided that a party can’t be forced to use one method over another. I think these decisions are sound ones. Absent proof that there has been a deficiency in discovery, a party should be able to choose its method of document selection, review and production.

However, I do feel that more parties should consider TAR and other technologies to assist with document review. The future is now and it may not be long before TAR is the norm. Judge Peck is certainly someone who would like that day to come sooner rather than later.

“The Court would have liked the City to use TAR in this case. But the Court cannot, and will not, force the City to do so. There may come a time when TAR is so widely used that it might be unreasonable for a party to decline to use TAR. We are not there yet.” 

What do you think? Should courts be able to force a party to use TAR or any other method it deems superior?

Attract & Retain Top Talent

With a rapidly changing industry, it's vital to offer the right compensation and set the right expectation. With our Salary Guide, get detailed job descriptions, industry insights and local salary data to equip your managers with hiring confidence and expertise.

Get your copy »

Get email updates about more content like this.


| Next articles in The Column blog |

Get the foundation you need to hire the best legal talent.

Request your copy of our 2021 Salary Guide »
March 01, 2018

Looking for the Summer Internship of a Lifetime?

If a summer internship that invites you to meet, travel and learn from Fortune Global 500 company executives sounds like your idea of a valuable experience, our CEO for One Month program is the one you've been searching for. This program invites top interns to compete for a chance to go straight to the top of our Global Fortune 500 parent company, the Adecco Group, as the CEO for One Month. And we should mention - the top contender takes home a $10,000 paycheck. Here's how it works.
Read Post »