Friday, December 30, 2011

Secrets of Search – Part III



http://ow.ly/8eoB1

An article by Ralph Losey on his blog e-Discovery Team.

This article provides additional information regarding the use of keyword searching, and follows two earlier articles on this same topic.

The article begins recapping the earlier secrets that were discussed in the prior series, including many inadequacies of keyword search methodologies.  The previously revealed secrets included the following:  keyword searching is ineffective; gold standards used to measure the success of attorney reviews were flawed; and humans are very poor at performing effective review for relevancy and privilege.

The article goes on to further state, "Requesters who demand production with only machine review, and any responders foolish enough to comply, have not understood the third secret. It is way too risky to turn it allover to the machines. They are not that good! The reports of their excellence have been grossly over-stated. Humans, there is need for you yet."

The article goes on to provide a fourth secret....relevant is irrelevant. "...in big data collections, I could care less about merely relevant documents. Their only purpose is to lead me to highly relevant documents. Moreover, as we will see in the fifth and final secret, I only care about a handful of those."

The article goes on to provide analysis of the Federal Rules of Civil Procedure
and Federal Rules of Evidence, and how they impact the requirements for disclosure of evidence.  The author makes a case why "all relevant" evidence is not what the rules will always impose as the standard.

The article then turns of the fifth secret...the rule of 7...Seven Plus or Minus Two.  As the author states, "...this is the first time I have written at length on the magic power of seven, plus or minus two. I hesitate to go to this deep place of information transmission and cognitive limitations, but, in order to keep the search for truth and justice on track, we really have no choice. We must, like the Pythagoreans of old, consider the significance of the number seven and its impact on our work, especially on our conceptions of proportionality.

The fifth secret of search is based on the legal art of persuasion and the limitations of information transmission. The truth is, no jury can possibly hold more than five to nine documents in their head at a time.

It is a waste of time to build a jury case around more documents than that. Judges who are trained in the law, and are quite comfortable with documents, can do a little better, but not that much. In a bench trial you might be able to use eight to twelve documents to persuade the skilled judge. But even then, you may be pushing your luck."

The article goes on to discuss the fact that computers can add to the document count, however this is not necessarily the goal that a proper document review should be seeking to accomplish.

The article goes on to point out, "There is a key lesson for e-discovery in the trial lawyer wisdom of seven. To be useful discovery must drastically cull down from the millions of ESI files that may be relevant, to the few hundred that are useful, and the five or nine really needed for persuasion. Culling down from millions to only tens of thousands is not serving the needs of the law. It is a pointless waste of resources, a waste of client money. A production of tens of thousands of documents, not to mention hundreds of thousands, is unjust, slow and inefficient."

In addition, further wise advice is provided, "Even if well-intentioned, many vendors (and lawyers) don’t understand that the law requires only reasonable efforts, and proportional efforts, not perfect or exhaustive efforts. They don’t understand the basic limitations of a trial or cumulative evidence."  As the author states, the review needs to be proportionate to the specific circumstances of the case, and often this thought is lost during the effort to produce relevant information to the opposing side.

No comments:

Post a Comment