Features - "Show Me the Money" - Measuring the Return on Knowledge Management

By Kingsley Martin, Published on October 15, 2002

Kinglsey Martin has 25 years experience in field of legal practice, including 15 years dedicated to law office technology. Martin started his professional career as a lawyer, having graduated with law degrees from Oxford University and Harvard Law School. He practiced law in the United Kingdom and the United States for five years, specializing in tax law.  Martin has worked with a number of law firms, including Dorsey & Whitney, Jones Day and Kirkland & Ellis, serving in the roles of practice support, strategic planning and chief information office. He is currently working as a consultant focusing on knowledge management.

Valuing technology investments has confounded many organizations. It is especially complex for law firms given prevailing compensation systems. Lawyers and administrators are rightly questioning rising technology budgets but often lack the tools to effectively evaluate software decisions. And, in the absence of proof, many firms have held back investment in knowledge management (KM) technology but, as a result, have not taken advantage of new technologies designed to improve efficiency.

Fueling concern is the fact that it has been well documented that few knowledge management initiatives have been successful, unless firms are willing to invest substantial resources to manually collect and organize research material. But most firms in the United States are unwilling to invest non-billable time in this effort, especially when the benefits of such investment are uncertain. In order to help law firms evaluate their KM strategies, this article proposes a framework to measuring the return on investment (ROI), the cost of information (COI), and proposes tools to evaluate alternative knowledge-sharing strategies.

The first point that should be made is that advanced knowledge management tools must be justified on a different basis compared to technology used to support the firm’s operational needs, such as word processing, communications and even document management. Basic operational software is increasingly required in order to conduct business, and can, therefore, be justified as a cost of doing business. Knowledge management technologies, and other advanced systems, are justified if they reduce expense, improve productivity or enhance value.

What methods are available to law firms to measure, and more important, recover the cost of their investment? How can the firm, and its clients, feel comfortable that the costs of the technology are justified and offer value for money? The answer to these questions can be found by considering two different charge-back mechanisms available to law firms – cost recovery through disbursement charges and cost justification through overhead expense.

1. Cost Recovery

The first method of cost recovery allocates the cost of the software to the firm’s clients based on a prorated or fixed-usage charge. This method is, for example, used by many firms to charge clients for the costs of online research. In the case of online research, these firms presumably believe it is more equitable to charge the specific costs of research incurred in connection with a representation rather than treat the cost as an overhead expense effectively incorporating the cost into the billable-hour charge and, therefore, charging to all clients whether or not online research was used in a particular matter.

The factors that a firm may wish to consider in determining whether to charge an expense are (a) whether the charge can be shown to increase the value of representation (from the client’s viewpoint), (b) whether the cost can be accurately calculated and (c) whether the cost can be allocated to the client on an equitable basis. Many other factors should also be considered: Are clients willing to pay for research costs? Will disbursement charges restrict usage? Can the firm adjust the billable rate to capture the costs of the technology?

All technology costs are ultimately charged to the firm’s clients, whether specifically disbursed or captured as part of the firm’s overhead recovered through the billable-hour charge. For example, in a similar circumstance, most firms have determined that it is more equitable to charge the exact number of photocopies made by the firm in connection with a particular client representation, rather than apportion a pro rata charge based on the entire costs of operating the firm’s photocopy system. Accordingly, while law firms have not generally disbursed charges for use of technology systems, this circumstance is primary attributable to the fact that most technologies lack the ability to fairly calculate usage charges.

2. Cost Justification

Some law firms may alternatively decide to treat the cost of KM technology as an internal expense. Irrespective of the decision regarding charge-back, all firms need tools to effectively evaluate their technology investments.

a) Internal Expense

The benefits of KM can be directed at reducing internal costs and/or enhancing profit. Looking first at internal expense, it is well understood that maintaining document collections is a significant expense for organizations. In fact, many recent studies have shown that the introduction of technology has yet to reduce the expense of filing systems. Forrester Research, for example, estimates that more than 80 percent of a corporation’s business-critical information is locked in unstructured formats. A study commissioned by Lucent Technologies found that the volume of unstructured information in large organizations doubles every two months. More important, other studies have illustrated the inefficiencies of existing electronic document storage and retrieval systems.

  • Records Management Quarterly reported that 80 percent of the electronic data on a typical network has not been retrieved or used in 30 days; more than 50 percent has not been accessed in several months; and only 20 percent of network storage is active data, while the remaining 80 percent is inactive1.
  • Tony McKinley, in an article titled "Managing All Information Assets,2" claims that 85 percent of documents filed are never retrieved, while 50 percent or more are duplicates.
  • A study by the multinational firm BAE Systems discovered that 80 percent of employees waste an average of 30 minutes per day retrieving information, while 60 percent are spending an hour or more duplicating the work of others.

Effective knowledge management tools can help firms reduce internal costs of maintaining electronic filing systems and reduce the administrative expense of locating documents. By itself, however, internal efficiencies may not justify the cost of knowledge management because it is difficult to effectively translate efficiencies into reduced labor costs3. But, when combined with improvements to the firm’s bottom line, such internal efficiencies offer additional incentive to better manage electronic files.

b) Increasing Profit

KM can also be targeted to improve law firm profit margins. First, effective knowledge sharing has been shown to substantially increase productivity. Virtually all documents drafted in law firms are based on precedent contained in explicit documentation or tacit knowledge. Unfortunately, in many cases, attorneys cannot quickly locate useful material and must accordingly “reinvent the wheel,” work from inadequate precedent or occupy the non-billable time of others through broadcast e-mail messages requesting sample forms.

Second, just as efficiency can improve profitability, leverage can have the same effect where knowledge transfer enhances the quality of work performance, and therefore, its value. In this context, leverage is the ability to delegate work to the most cost-effective resource. The transfer of knowledge is, in fact, the essence of knowledge management. While many commentators have touted the productivity goals of KM, the true value of knowledge sharing will likely be found in its ability to leverage knowledge and enhance the value of professional services. Using knowledge management technologies to capture prior work product, exemplars and other materials developed by senior and skilled practitioners, law firm associates, legal assistants and non-attorneys can perform valuable client work and charge a rate commensurate with their enhanced performance. Accordingly, the application of leverage is simply recognition of the fact that billing rates increase with experience, and such experience can be gained through hands-on practice, training and the use of technology. As a result, computer systems are being enlisted as part of professional development to supplement mentoring programs and provide training opportunities for attorneys. In addition to such broad-based skill development, KM initiatives can focus knowledge transfer on specific practice areas, developing the skills of associates, legal assistants and non-lawyers to perform higher-level work through the development of forms, practice guides and document assembly.

c) Measuring “Hard” Returns

The primary tool used by businesses to measure the value of technology acquisition is the return on investment, or ROI. In financial circles, ROI is a measure of a company’s performance. It is the company’s total income (before interest, taxes or dividends) divided by its total capital. However, most business managers think of ROI as the return (or incremental gain) from a project minus its cost. Additional precision can be achieved by discounting the return to its present value, reflecting the fact that future income is less valuable than current income, because money can be invested and generate interest. The formula for ROI can be expressed as follows:

ROI = PV (Incremental Gain) – Total Cost of Project

  • PV is present value.

  • An alternative measure of technology investment is known as the “payback period,” defined as the period required to recover the cost of an investment. In practice, this measurement device is an example of a rule of thumb stating that value is maximized and risk is mitigated if the payback period is reduced. Frequently, this tool is used to compare alternative investments. Applying this standard, it can be generalized that projects based on manual coding will likely have a longer payback period than initiatives that rely more heavily on technology, because large numbers of documents are not required to be annotated before the system attains critical mass.

    i) Projecting Costs

    The costs of a KM initiative can be calculated, although in some cases certain expenses must be estimated. Where the project is based on in-house development of KM systems, combined with manual document collection and profiling, expense calculations must total hardware acquisition, software development and the ongoing cost of lawyers or content managers to organize research archives. Alternatively, law firms can build their KM initiative on a turnkey program, such as West km™, developed by West, that provides the software and maintenance for a fixed per-user charge.

    ii) Projecting Revenue

    The gains or benefits of KM comprise “hard” or financial returns and “soft” or intangible benefits. Depending on the firm’s specific knowledge management goals, it may target and measure expense reduction, productivity gains or leverage improvement, or focus on the “soft benefits” of knowledge management.

    The application of ROI to law firms, where compensation is largely based on the billable hour, requires some modification to the basic profit formula. Moreover, by understanding the formula, law firm administrators will see that only a small level of improvement in efficiency or enhanced value of services is needed to offset the cost of a knowledge management system. As shown by David Maister, author of The Trusted Advisor and other books on law firm management, profitability is calculated by the following formula:
    Net Profit Per Partner = (Leverage +1) (Billable Rate) (Utilization) (Realization) (Margin)
  • Leverage is the ratio of non-partner professionals (associates, paralegals and of counsel who have billable hours) to partners.

  • Billable Rate is the average billable rate of all fee earners.

  • Utilization is the average number of billable hours for fee earners in a year.

  • Realization is the percentage of work done that is billed and collected.

  • Margin is the ratio of gross income to net income, calculated by subtracting expense for each partner from gross revenue received for each partner, and expressing the net income as a percentage of the gross income.

  • This formula demonstrates that law firms can achieve positive financial results from any one or more of the following: (a) increasing utilization (shorthand for working longer hours); (b) increasing billing rates; (c) increasing leverage either by adding associates or by delegating work to least-cost personnel; (d) improving realization; or (e) decreasing expense. For example, consider a firm with a leverage ratio of 2, an aggregate billable rate of $250, working on average 2,000 hours per year, having a realization rate of 93 percent and an expense margin of 43 percent. In these circumstances, net profit per partner (NPP) will be $599,8504. If the firm were to focus on realization rates and, through improvements in productivity and knowledge sharing, realize a small increase in the realization in the amount of two percent (from 93 percent to 95 percent), then NPP will increase by $12,900.

    In many cases, however, it is difficult to measure ROI derived from technology projects. A number of studies have shown there is no direct correlation between profitability and technology investment5. The relationship is complex and indirect. First, analysis must account for causation, because in most situations, technology serves a role within a larger context of numerous other processes, and its cause-effect relationship cannot be considered in isolation. Second, any measurement tool must handle the issue of equivalency, because many of the benefits – the so-called “soft benefits” – cannot be easily converted into financial value6. For this reason, a comparative measurement tool, called the cost of information, is proposed and described on the next page7.

    d) Soft Benefits and Comparative Evaluation

    In addition to applying ROI as a measure of technology value, a firm can also focus directly on the effectiveness of the investment in generating intellectual capital by measuring the rate at which technology expense is converted into valuable information assets. This computation, called the cost of information, calculates the expense of knowledge sharing by comparing the per-document cost of the system to the average rate of document reuse. The formula can be expressed as follows:
    Cost of Information = Document Preparation Cost / Rate of Reuse
  • Document Preparation Cost is either the unit cost of profiling each document or the total cost of the system divided by the number of documents organized by such system.

  • Rate of Reuse is the average number of documents in the system divided by the total number of times a document is reused in a designated period, such as a year.
  • The formula allows law firms to determine how much it costs to prepare a document for reuse. For example, if a firm spends on average $30 to prepare a document for inclusion in the firm’s work product retrieval system, and documents in the system are used on average three times in one year, the cost of information is $10, or, in other words, the law firm must spend $10 for each occasion the document is reused.

    As experienced strategists understand, a successful knowledge-sharing program requires an understanding of three main components: the technology platform, the information structure and the work flow process. The last element, sometimes referred to as the “cultural” aspects of KM, has proved to be most challenging. In fact, it has prompted many observers to note that “KM is not about technology.” In fact, all three elements are critical to sustaining a successful strategy. Moreover, by automating the work flow process, law firms can likely reduce the impact of such “cultural” barriers and become less dependent on manual labor. The effect can be seen by comparing various strategies.

  • Bibliographic Coding. The cost of efforts to code documents with bibliographic data can be estimated based on the experience of litigation support systems. A number of companies offer services to code or profile documents produced in the discovery process of litigation. The coding of bibliographic information may record the author, date, title, recipients and other objective data about the document. Depending on the number of items of data collected and the average length of the documents, the cost may range from $2 to $10 per document. While litigation support systems serve vital roles in the process of document production control, thereby justifying the cost, it can be fairly observed that few attorneys use the software. We may therefore conclude that although the costs of bibliographic coding are low, the value in the opinion of attorneys may not justify the expense. For example, if coding costs were $5 per document, but attorneys use only one in five documents, the cost of information would be $25. This analysis partially explains why a document management system containing bibliographic information alone has not become a viable platform for work product retrieval.

  • Subject Matter and Context Coding. Law firms that have been successful in developing work product retrieval systems have generally relied on trained lawyers with no or low billable-hour requirements to collect and maintain precedent collections. Documents in these systems are profiled with subject matter and context information. Estimates of the cost of document preparation range from $25 to $35 per document8. We can therefore assume that firms continuing their investment in such work product systems consider the value to be returned. In fact, despite the higher document preparation cost, the firm’s COI may be lower than bibliographic coding systems. Based on the costs stated above, and assuming documents are used on average five times per year, the firm’s COI would range from $5 to $7.

  • Advanced Search Tools. Finally, COI analysis can also be applied to advanced search technologies that may be used to provide access to very large document collections. Calculating the cost of document preparation can be computed by dividing the number of documents maintained on the system by the total cost of the search technology. For example, if the search system costs $300,000 to install and provides access to 1 million documents, the unit cost per document will be 30¢. But given the breadth of the collection, it may be assumed that only a portion of the documents will be retrieved and used. If, in our example, 100,000 documents are retrieved and used, the COI will be $3.

  • Although the analysis uses hypothetical values, the comparison shows first that cost is relative when compared to usage. Second, a firm can maximize its return on information by combining breadth of information sources with high usage.

    3. Conclusion

    By examining law firm profit models, the costs of KM systems, and document reuse statistics, law firms can better understand the value of knowledge-sharing systems and develop sustainable strategies. While the analysis cannot yield definitive answers, it can more clearly illustrate the benefits of KM, allowing firms to focus their KM strategies. Working through the analysis, many firms will discover that the value of knowledge sharing is its ability to leverage the skills and capabilities of law firm personnel and enhance the value of professional services.


    [1]  Records Management Quarterly became The Information Management Journal as of the January 1999 issue and it available at www.arma.org/publications/journal/journal_about.cfm. <back to text>

    [2]  http://www.infologics.com/media2.htm <back to text>

    [3]  It can be argued that law firms have been able to reduce secretary-to-lawyer ratios using technology. However, in most cases, reduced secretary ratios is more attributable to lawyers undertaking work previously performed by secretarial staff, rather than through improved efficiency. <back to text>

    [4]  For firms that are less highly leveraged, for example, having a leverage ratio of 1, NPP will be $399,900 and a 1 percent increase in realization will yield additional profit in the amount of $8,600. However, a firm with a lower leverage ratio can spread the cost of the KM system over a smaller number of users in order to calculate the per-user expense. <back to text>

    [5]  American Lawyer Technology, March 2000, pp. 23-27. <back to text>

    [6]   Some studies have suggested that soft benefits should be discounted to better quantify the returns. However, any such discount factor is necessarily arbitrary. <back to text>

    [7]  Because of the difficulties of evaluating causation and equivalency, a number of ROI studies performed for law firms leave the reader unconvinced. <back to text>

    [8]  Based on the assumption that a content manager is valued at the rate of a junior associate and can process three to five documents per hour.  <back to text>