Machine Learning For Document Review: The Numbers Don’t Lie

http://ow.ly/9Ehbj

An article by James Shook, Esq. posted on the Kazeon.com website.

This article discusses predictive coding technology, and looks at the limits for the use of this technology, and looks at two important recent cases.

The article states, "Perhaps more important, according to recent studies the predictive coding process is also more effective than human or keyword review. Unfortunately, it is difficult to determine the true accuracy of human review because opinions, even among experts, can vary on whether a document is relevant to a case. (Maura Grossman & Gordon Cormack, Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, XVII Rich. J.L. & Tech. 11 92011 at 9). But the bulk of available information implies that machine coding is better. In fact, some studies put a human reviewer’s recall– the percentage of relevant documents actually located – at less than 50%. The use of basic keywords is even worse, dropping recall to about 25%. (Grossman/Cormack at 18-19). Some predictive coding studies indicate that the process is far more accurate, in the range of 70% recall (Grossman/Cormack at 36-37). Given the lower cost and speed, recall that’s even close to the human level should be acceptable. (Note that other measures, such as precision and F1 (the harmonic mean of recall and precision) – are also important in this process. For more information see Grossman/Cormack at 9)."  A link to the referenced article is provided by the author.

In addition, the article looks at the recent opinions in the cases of Magistrate Judge Andrew Peck’s recent decision in Da Silva Moore v. Publicis, (where the parties were in agreement regarding the use of predictive coding, and the court sought a protocol from the parties), and the Kleen Products LLC v. Packaging Corp. of America, in which Magistrate Judge Nan Nolan is addressing an issue where one party has objected to the use of such technology, and the court is being asked to compel the use of predictive coding over the objection.