Conrad Jacoby's E-Discovery Update: Attorneys, Experts, and E-Discovery Competence

By Conrad J. Jacoby, Published on May 7, 2008

Attorneys and judges alike are increasingly sophisticated about the discovery of electronically stored information ("ESI"). Unfortunately, while one consequence of this development should be increased competence in the legal team preserving, collecting, and producing these materials, another consequence is an increase in the number of complaints issued against producing parties for problems in their management of ESI materials. Two recent cases emphasize the credibility problems that counsel can face in the context of e-discovery - and suggest that outside assistance may be the only way for some counsel to demonstrate that these materials are being managed in a competent and trustworthy way.

On February 18, 2008, Magistrate Judge John Facciola issued an opinion in the ongoing criminal matter, United States v. O'Keefe, No. 06-249, 2008 WL 449729 (D.D.C. Feb. 18, 2008). Defendants, the requesting party for purposes of this opinion, expressed dissatisfaction with several aspects of the government's production of hardcopy and electronic documents. The government, in turn, produced a fairly detailed affidavit from the employee who managed the search for responsive documents. The affidavit described which hardcopy and electronic archives were searched, and, with respect to ESI repositories, the specific search terms that were used to identify potentially relevant documents. However, the requesting party renewed some of its objections on grounds that the affidavit did not disclose the software tool/technology that had been used to conduct the searches, nor did the affidavit describe the analysis and decision-making process that was used to develop the relatively limited search terms the government used to identify potentially relevant documents.

The Court's opinion did not separately discuss the requesting party's demand that the government describe with specificity the search technology it had used, but the opinion did directly address issues regarding the adequacy of the government's search strategy. Recognizing that an armchair quarterback can always identify additional actions that might have yielded a different result, the Court found that such opinions, without special knowledge or experience that gives them particular weight, are insufficient grounds for relief:

Whether search terms or "keywords" will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics…This topic is clearly beyond the ken of a layman and requires that any such conclusion be based on evidence that, for example, meets the criteria of Rule 702 of the Federal Rules of Evidence. Accordingly, if defendants are going to contend that the search terms used by the government were insufficient, they will have to specifically so contend in a motion to compel and their contention must be based on evidence that meets the requirements of Rule 702 of the Federal Rules of Evidence.

Slip Op. at 16-17, 2008 WL 449729 at *__ (internal citations omitted).

Software programs and search engines use a variety of algorithms to execute search queries, and results will vary depending on the software that is used or on how the search system is configured. Equally significant, electronic search technology constantly balances speed against accuracy. Most people believe that search engines index all text in the documents to which they are applied. Many times, however, this is not the case. In the interest of speed, many search and indexing engines analyze only a fixed amount of data per document. For short documents, this may include the full text. For documents longer than the target length, however, significant portions of body text may never be indexed. Depending on how they are configured, some search engines also don't ordinarily index all document or e-mail message-associated metadata. Thus, the exact mix of what electronic information is indexed - and therefore searched for potential relevance - varies depending on technology and the documents to which it is applied. And, the significance of search queries as applied to a specific document collection is not usually readily apparent.

However, if it is extended beyond its case-specific facts, the O'Keefe opinion breaks significant new ground. Implicit in its reasoning is a belief that though lawyers have been making arguments for many years about the adequacy of a litigant's efforts to comply with its discovery obligations, requesting counsel no longer possesses sufficient expertise to successfully establish the factual foundation by which the adequacy of a producing party's efforts regarding electronic documents can be measured. Indeed, for some lawyers, this analysis may be quite accurate -relatively few lawyers are adequately equipped to discuss the intricacies of complex computer search technology based exclusively on their own research. Instead, it does seem more appropriate to rely on the expert testimony of someone with professional expertise in these areas. O'Keefe goes one step further, though, by suggesting that e-discovery motion practice now requires such supporting testimony as a matter of course, regardless of the technological sophistication of counsel.

A second recent opinion, In re World Trade Center Disaster Site Litigation, 2008 WL 793578 (March 24, 2008 S.D.N.Y.), also speaks to the need for specialized expertise to competently work with discovery materials. Judge Alvin Hellerstein, already working with a Special Master to streamline discovery, ordered a third party vendor not affiliated with the parties to prepare and maintain a centralized database of discovery materials (e.g., relevant discovery documents and e-mail messages) for use by all litigants. Defendants consented to the creation of an outsourced database, but plaintiffs challenged the order on grounds that it unnecessarily duplicated document databases that they had already created, and that the cost of these outsourced operations was an unreasonable burden on plaintiffs. The Court rejected these arguments:

There is no substitute for an independently created and managed database. Without a common core of reliable information, the approximately 10,000 cases that are before me will become increasingly difficult to manage. Disputes will proliferate, and progress will be slowed. Depositions and other discovery will become necessary, as each side tries to prove facts that their separate databases treat in different ways...The importance of these cases is clear, to the litigants and to society. The ability of the court and counsel to cause them to progress towards resolution with due speed, economy and fairness depend very much on the ways that complex information can be organized and reported, reliably, responsively and consistently.

2008 WL 793578 at *2.

Serial litigation involving ten thousand related actions will involve substantially similar, if not identical, fact evidence to decide common issues. In re World Trade Center Disaster suggests that a lawyer's freedom to develop the case according to the best interests of the client can be counterbalanced by the court's need to efficiently manage its case load, even at the expense of individuality in discovery requests.

Creating a master document repository to apply equally within a cluster of related litigation matters makes a great deal of sense. However, the Court's opinion is somewhat noteworthy in that this master discovery database will be maintained under the direction of the Court, not by one or more of the producing parties. The significance is subtle, but important. Often, a Court will order a primary producing party to suggest (and fund) a system for making materials available on a consistent basis to multiple requesting parties. Here, however, litigants were not involved in the system design process. Instead, the Court's Special Masters, aided by an independent legal technology consultant, analyzed the litigation to conclude that a centralized discovery database was and chose the vendor to provide these services. Litigants were permitted to voice their opinions only after the decision-making process had been completed by information management specialists.


As electronically stored information continues to increase in complexity and volume, it is increasingly logical that attorneys may no longer be the best managers of discovery information. Specialists with greater understanding of the search and retrieval technology may well do a better and more cost-effective job of managing objective information. However, such increased reliance upon technical experts should not be considered a limitation on a lawyer's freedom to develop a case. Applying law to the facts - and identifying the most probative facts in the first place - remain key attorney tasks. In addition, it is also difficult to believe that the O'Keefe opinion stands for the proposition that expert opinion must be used to support all allegations of incomplete or inadequate discovery. Certain situations, such as refusing to produce any relevant materials, should remain plainly obvious, even without supporting expert testimony. However, by the same token, questions regarding the adequacy of affirmative e-discovery actions taken by a producing part are highly nuanced, and the O'Keefe opinion may be only the first of many opinions speaking to the burden of proof required to successfully win or defend such arguments.