One of the questions firms and legal departments will need to grapple with over the coming three years is whether to renew their licenses with existing providers or swap out their technology stack to make room for solutions powered by advanced AI. In the next few articles, I will be exploring this question in light of current developments.
———————–
It’s time to level-set about advanced AI: it can’t do everything. Or perhaps more practically, a large language model can’t replace all of the other technology you already have.
One of the main reasons for this is the importance of an interface and a built-out user experience (UX) that offers a journey through the system that is aligned with the way users actually work. There are other reasons a large language model (LLM) won’t replace all of your technology (one of which being advanced AI is simply unnecessary to do all things), but this article will focus on UX.
The goal of UX designers is to understand their users, how they work, and what their needs are, and to then map a logical and functional user journey that results in an interface that meets those needs. Some of the core technology that has been used in law firms for years is successful because it aligns with the way that lawyers work (making it easier to adopt). Most of the new advanced AI solutions that have popped up in the past months are too new to have really worked out the user journey in a way that aligns the technology with user needs.
In order to dig deeper, let’s take a look at some examples of use cases where LLMs could excel but will nonetheless lose out to incumbents for the time being.
Contract Review
Advanced LLMs that have been reinforced with legal content are exceptionally good at recognizing legal concepts in a way that allows them to identify clauses across contracts. This new technology is much better at consistently identifying clauses that are highly variable in language than previous machine learning technology, because it “understands” language in a different, more human way. Previous applications built with older machine learning technology required that the algorithm be manually trained to recognize a new clause, often by showing the algorithm hundreds of examples of that clause in order to reach a level of precision and recall high enough to satisfy human reviewers. Advanced LLMs do not require this training.
You would think, therefore, that it’s a no-brainer to swap out your existing contract review solution for an LLM that has been built out to allow for contract review. CoCounsel, for example, includes contract review as one of its “skills” and numerous CLM solutions have added LLM technology to support ad hoc contract review.
But would these solutions be a good replacement, for example, for Litera’s Kira, or eBrevia, or Luminance, solutions that are used not just for ad hoc contract review but for due diligence?
My answer: not yet.
Taking Kira as an example, since it is the leading incumbent in many large law firms, the reason it’s hard to displace is because it was built with a deep understanding of how lawyers work. Kira’s interface follows the user journey in a way that none of the new LLM solutions yet do. Within Kira, users are able to not just identify clauses from within a set of contracts, but also flag them according to theme or issue and add notes alongside the clause. Users can work collaboratively within the system, assigning review tasks to one another and tracking progress. A due diligence report can be developed in sync with review, and downloaded from the system afterwards, eliminating the need for a separate workflow involving the drafting of a report as one works through contract review.
The time-saving from this type of contract review tool, in other words, comes not just from the technology itself but also from the way the user journey has been catered for within the system. Kira also comes with a pre-trained selection of thousands of clauses, which means the issue around training the algorithm on new clauses is moot if you’re using it for standard M&A due diligence.
Over time, the market will be owned either by the incumbent who first successfully builds new LLM architecture into its back end, combining it with existing technology so that it underpins the interface and makes it more powerful, or by the new entrant who does enough UX design work to build a slick interface that allows users to automate their full due diligence workflow in the system.
For now, none of the LLM solutions I’ve seen is built out enough to displace incumbents for the purpose of due diligence review. If what you’re looking for, however, is a solution that allows you to quickly identify and extract, say, termination clauses or force majeure clauses from a set of contracts, simply so you can compare them or see what’s in them, it’s worth looking at the newer solutions.
Drafting
It’s clear that one of the main use cases for generative AI is its ability to generate text, in other words to draft. Does that mean the new solutions will or should replace incumbent document automation or smart drafting products?
Again, my answer is: Not yet.
Even with LLM products that have been reinforced with a strong set of legal data, automatically generating a share purchase agreement bespoke to a particular deal is a big ask. Drafting letters or NDAs is perhaps another matter, but as our LTH Expert for Document Automation Catherine Bamford recently posted on LinkedIn: “Why would you want your lawyers to start with a different template each time? A better question is how can GenAI speed up the process of Document Automation?”
Document automation is powerful not just because it improves the speed of drafting but also because it improves quality of output. Rather than having to start from scratch each time a lawyer drafts, an automation tool allows them to start from a firm-approved template or their own preferred template for an agreement and automate the parts of it relevant to a specific matter. By working with existing precedents, users can be sure that the document they are generating is of the quality required for their work and their firm – and that it’s compliant with any firm formatting guidelines or style guide.
As Catherine says, it’s not that useful to generate an entirely new agreement every time you draft. Generative AI works by generating new text every time it is prompted to do so, which means the output will be different each time even if you use the same prompt. Using such a tool for drafting means you will not get any consistency across your agreements, and will need to read, review, and edit extensively every time you generate a draft. The time required to do that extensive reviewing and rewriting is likely to be longer than the time it would have taken you to reach a good draft by using an existing document automation solution coded for your own templates.
Looking at newer recent solutions such as smart drafting tools, which operate by drawing upon existing databases of content and suggesting clauses for inclusion in a contract as you draft, the reason these have been successful is the way that they fit within user workflows. Often developed as a plug-in to Word, the best of these tools have evolved to include a knowledge base or clause library that users can build as they work, tagging clauses so they can easily find their preferred language at a later date and pull it into the contract they’re drafting.
A simple language generation tool simply can’t compare to that kind of complex functionality. Smart drafting solutions are already building in generative AI to add features to their existing products, and similar to contract review, the document automation solution that is able to combine advanced generative AI technology with standard documents or clause libraries, adding the right workflows and interface for drafting to enable consistency and quality of output, will be the one to watch for.
Search
Large language models are great at finding the proverbial “needle in a haystack”, and retrieval of information from a database is one of the superpowers of this type of technology. You would think, therefore, that it’s a no-brainer to swap out your existing search technology for a LLM solution. Is it, though?
The answer: It depends.
The primary questions to ask in considering whether this makes sense for your firm or organization is what you need to search for, and what existing search technology you are currently using.
Search is an area where the simplicity of a good experience belies the complexity of the underlying architecture. Users commonly approach search with different intents (for example, to find a specific document, to find information for research purposes, or to locate expertise), and depending on that intent, a different type of search response or search architecture may be relevant. Search engines like Google and now Bing combine different kinds of search architecture in combination to accommodate multiple search intents, and the best search products for legal do the same. It’s therefore critical to understand your users’ needs for search before you make a buy decision.
If your users frequently need to perform ad hoc searches across single large databases of content in order to find a specific answer, then an LLM-based targeted search tool might suit you just fine.
If, however, your users have more complex search needs, especially if you have a firm-wide enterprise search solution deployed at your organization and it is well adopted, an LLM alone cannot replace it.
Enterprise search and even federated search systems are highly nuanced, providing users not just with simple search but with contextualized knowledge (there’s a reason these are commonly regarded as knowledge management solutions). The power of an enterprise search system is its ability to pull together structure and unstructured data from multiple systems at a firm, combining it in a way that provides the user with a journey that allows them to find not just exactly what they’re looking for, but also additional results and associated information that improves their quality of work.
An example of this is a user who is looking for a sample agreement from a matter similar to the one they’re currently working on. They’re not just looking for a share purchase agreement, they’re looking for one that was drafted in the context of a cross-border deal with Canada, where they represented the buyer and the vendor was a pharmaceutical company. Performing a search in the enterprise search system allows them to look for matters and filter specifically by all kinds of details about the matter – the relevant party, the industry, the currency and amount of the deal. Once they find a similar matter, the search engine allows them to drill down into the documents to see the contracts and open them immediately in Word. It will also show them who worked on the matter (pulling data from timekeeping and financial systems), what office that person is located in and how to contact them so the user has someone they can turn to who has the exact experience they need to draw upon in order to ensure they produce the best possible work. In the same search, the user might also have come across several other matters or documents that are relevant to what they’re doing but might not have come up in a targeted search for a particular document (the element of serendipity that can sometimes lead to unexpected yet excellent results).
In a targeted search, even if it’s underpinned by advanced AI, neither the context nor the nuance of the above scenario is possible. Or in other words, a targeted search can only accommodate one type of search intent, usually in one database, whereas the incumbent solutions have been built to accommodate numerous search intents across multiple datasets that are regularly indexed.
There are other details that differentiate more sophisticated search solutions from simple LLM-based targeted search too. Things like the fact that search terms are highlighted in results, or the ability to preview documents without clicking into them seem like small details but take on significance in search workflows.
The winners here, therefore, will be companies who have already developed nuanced UX around search, combining different types of search – knowledge graphs, semantic search, targeted search, and more – as well as workflow features to satisfy different search intents, and who now add LLM technology to the back-end tech stack (whilst retaining and building on their existing architecture).
The recent announcements around iManage Insight+ indicates that the team at iManage might be a nose ahead here. Insight+ builds upon existing search functionality and comes with built-in knowledge curation. The Insight product has included AI technology since its inception but has been rebuilt for the cloud, and forms just one part of a multi-phase roll-out for iManageAI, which will ultimately include generative AI in certain workflows according to Alex Smith, iManage Product Director for Search, Knowledge, and AI.
Conclusion
An LLM provider who tells you they can solve complex problems rapidly by building on advanced AI is not likely to be factoring in the UX that will allow you drive proper adoption of the solution across the firm. While generative AI is groundbreaking and LLMs can do many things well, the decision to replace existing systems with something new simply because the underlying technology is more advanced is not as simple as it sounds. Keeping user needs, legal workflows and genuine use cases at the heart of such decision-making is critical to ensure you’re making sound decisions.
Editor’s Note: This article is republished with permission of the author with first publication on Legaltech Hub.