Search, Forward Will manual document review and keyword searches be replaced by computer- assisted coding?

http://ow.ly/6Ju9h

An article by Hon. Andrew Peck published by law.com on LTN webpage (article dated October 1, 2011...I guess we are getting an advanced peek).

The article discusses traditional linear review, and then comments as to whether other means of technology enhanced reviews can provide similar results.

As the article states, "Despite its flaws, many senior lawyers (and some clients) still consider manual review to be the "gold standard" against which other review techniques are compared. While the volume of electronically stored information (and concomitant expense) has largely eliminated manual review as the sole method of document review, manual review remains used along with, for example, keyword screening. Let us consider whether manual review as the gold standard is myth or reality."

The article goes on to state, "Two recent research studies clearly demonstrate that computerized searches are at least as accurate, if not more so, than manual review. Herb Roitblatt, Ann Kershaw, and Patrick Oot, of the Electronic Discovery Institute, concluded that "[o]n every measure, the performance of the two computer systems was at least as accurate (measured against the original review) as that of human re-review." ("Document Categorization in Legal Electronic Discovery: Computer Classification vs. Manual Review," Journal of Am. Society for Information Science & Technology, 61(1):70-80 (2010).)

Likewise, Wachtell, Lipton, Rosen & Katz litigation counsel Maura Grossman and University of Waterloo professor Gordon Cormack, using data from the Text Retrieval Conference Legal Track, concluded that "[T]he idea that exhaustive manual review is the most effective — and therefore the most defensible — approach to document review is strongly refuted. Technology-assisted review can (and does) yield more accurate results than exhaustive manual review, with much lower effort. ("Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review," Richmond J. of Law & Tech., Vol. XVII, Issue 3, 1-48 (2011))."

Links to the case studies cited above are provided in the article.

Judge Peck points out issues associated with keyword searching, and the various limitations associated with that technique, "The problems with keyword search are well known. Lawyers are used to doing keyword searches in "clean" databases, such as Westlaw and Lexis, which use full sentences, full words (not abbreviations), and largely the same words to describe the same concept. E-mail collections are not clean databases. People use different words to describe the same concept; even business e-mails are informal, rampant with misspellings, abbreviations, and acronyms."

The article cites case law that has criticized the results of keyword searching, and also provides the following example: "How effective is keyword searching? In 1985, scholars David Blair and M.E. Maron collected 40,000 documents from a Bay Area Rapid Transit accident, and instructed experienced attorney and paralegal searchers to use keywords and other review techniques to retrieve at least 75% of the documents relevant to 51 document requests. Searchers believed they met the goals, but their average recall was just 20%. This result has been replicated in the TREC Legal Track studies over the past few years."

The article reiterates remarks Judge Peck has made in the past regarding the use of "predictive coding" techniques, and the fact that there is no case he is aware of approving such use.  However, he states that counsel need not wait for such judicial approval in a specific opinion, rather he suggests the following: "
Of course, the best approach to the use of computer-assisted coding is to follow the Sedona Cooperation Proclamation model. Advise opposing counsel that you plan to use computer-assisted coding and seek agreement; if you cannot, consider whether to abandon predictive coding for that case or go to the court for advance approval.

Until there is a judicial opinion approving (or even critiquing) the use of predictive coding, counsel will just have to rely on this article as a sign of judicial approval. In my opinion, computer-assisted coding should be used in those cases where it will help "secure the just, speedy, and inexpensive" (Fed. R. Civ. P. 1) determination of cases in our e-discovery world."

P.S.  Have questions or comments about this topic, the writers of this blog would love to hear them. This is an area of great interest to SRM Legal, and we would be happy to get your insight to share with others.