Data Proliferation, Process Controls and the Role of Project Managers in Document Review

The proliferation of eData gives any document review the potential to become enormous. From eDiscovery pundits to “woe is me” defense attorneys, we have all heard the familiar, “there is just too much data” refrain. But what does that actually mean?

The amount of data created in just two days in 2011 equals roughly all of the data created from recorded history to 2003. The amount of data created in 2010 is the equivalent of 700-plus trillion copies of the recently enacted Healthcare Reform Act—a 2000-plus page document. If one were to stack each of those copies one on top of another, the documents would stretch from Earth to Pluto and back 16 times.

Thankfully, the days of banker boxes filling up walls of war rooms are long gone, although the statistics above could cause one to yearn for the days of finite and knowable information stores. Now, it seems, information is everywhere. From e-mail to LinkedIn, from Facebook to Snapfish, information storage and sharing is ubiquitous. The “cloud” is all around us.

What are the implications for traditional document review? Fortunately, as man’s ability to easily create and store data has expanded greatly, so has our ability to create the technology required to cull this information. Whether through the use of early case assessment or advanced searching techniques, the resulting reviewable data is typically only 5-10% of the original collection. Although seemingly small in comparison, this set of data is still so large that it can mean a great deal more money spent in the review process. Or not.

When looking at a review as a process, i.e., as a systematic series of actions directed to some end, we can perceive it as the simple act of taking something through an established and routine set of procedures to convert it from one form to another. In this case, the new form is usable, reviewable data that we hope is relevant to our case.

Viewing the review as a process, we can assuredly add efficiencies along the way—one of the goals of a managed review. Enter the project management professional. With a project manager on the case, you gain the expertise of a process engineer, who will work with you in a way very different than legal professionals trained solely in the practice of law. True project managers are trained in the process of projects.

Roughly 100 years ago, Henry Ford visited a Chicago slaughterhouse and recognized the efficiency of each person performing the same task over and over again. The inspiration for the modern assembly line was born.

In an assembly line, parts are added to a product in a sequential manner using optimally planned logistics to create a finished product much faster than with traditional “one-at-a-time” methods. Project-managed document review is akin to an assembly line, and efficiencies can be added if we pay close attention to the process. The first goal is to define any problems. The tenured project manager will work with you to define project goals and create the case overview manual. Once the workflow and communication processes are established and the project begins, the project manager will immediately begin to measure key aspects of the process and collect relevant data. Then, the project manager will analyze the data to look for cause-and-effect relationships. For example, if Y reviewer reviews all of John Smith’s documents, then Y increases output by X%. Like the worker on the assembly line, the repetition of similar tasks breeds a greater familiarity and ease. With this heightened familiarity of one custodian or topic, the reviewer is able to go through the same mental motions over and over, making quicker and more accurate decisions.

If there are any outliers in the review process, the project manager will seek out the root cause of the defect, ensuring that all factors have been considered. The manager will verify that document sets are substantially similar among all reviewers when determining the individuals’ speed and accuracy. This leads to changes or improvements designed to optimize the current process, based on the data collected, using techniques such as design experiments.

One such design experiment that led to significant changes in Special Counsel’s management of the batching process during a review was a matter of human nature. Twelve contract attorneys, all within a 1% deviation from each other in review speed and within .5% of one another in accuracy on previous review matters, were split into groups of four. Each group was given the exact same data set. However, Group A was given documents in sets of 1,000; Group B received sets of 500; and Group C was given documents in sets of 200. Speed and accuracy were measured over the course of two days and a clear correlation was found between the size of the document set initially distributed and the speed of the review.

On average, the reviewers in group C reviewed 72 documents an hour while those in group A reviewed only 68 documents per hour. Thus, by simply changing the traditional batching practice, a cost savings of 6% could be realized. Armed with this information, the project manager would make that batching practice standard on all future review processes.

A primary goal of traditional and legal project management is to ensure that any deviations from target are corrected before they result in defects, such as an error in judgment that causes a document to be coded inaccurately. Rather than waiting until batches are finished, Special Counsel recommends immediate quality checks of judgment calls made by a reviewer in order to provide real-time feedback to the team on any deviations. In many reviews the quality check is based on random sampling that occurs well after the review has started. This prevents the implementation of process improve-ments resulting from reviewer feedback until much later in the cycle, thereby increasing miscoding of documents.

Using a remote supervisory tool such as Special Counsel’s SightManager®, a project manager can begin the QC process immediately and in real time, thereby minimizing mistakes and allowing for immediate remediation. We also recommend a mix of both random and targeted quality checks. In a non-randomized quality assurance check, reviewers who fall outside of the average review velocity and/or accuracy rates (ahead of as well as behind the bell curve), would find their documents reviewed for accuracy at an increased rate. Additionally, those reviewing documents of high profile custodians or those likely to have “hot docs” in their batches would see an increased percentage of their work checked to ensure consistency with stated protocols.

Through the implementation of control systems such as statistical process control, production boards and visual workplaces, as well as continuous monitoring of the reviewers’ work pace and flow, project managers are able to modernize the document review space just as Henry Ford revolutionized process engineering in the early 20th century with his assembly line.

If you would like more information about project manager certification or the managed review process, please contact Special Counsel.

Christopher Gallagher, Esq. is a Senior Vice President of Special Counsel.