Legal AI, token charging and bread machines!
Legal AI heralds a seismic change in the delivery of legal services. But now the bedding in period is over the results are mixed and implementation has been patchy. Some law firms are rolling out extensive training programmes and sophisticated integration schemes, whilst others relying on an out of the box product, found it wanting.
There is an art to achieving all the benefits legal AI promises, and one which requires a lot of human input and preparation. When AI licences were charged on a per seat or subscription basis, that human input could be done through experimentation and trial and error. Yes, that experimentation was inefficient, the cost was measured in wasted lawyer time, expensive but something easy to measure, manage and control. This is changing fast. Recent news from The Lawyer tells us Legora is ditching the subscription model and moving to consumption-based pricing. This follows general AI providers, such as Open AI and Anthropic, who are bringing pricing into line with their cost base by charging for tokens or usage rather than subscription.
It is likely other legal AI products will follow suit, making the cost of experimentation untenable. Suddenly if a solicitor spends hours iteratively working with an AI to fine-tune a desired output the work could quickly become unprofitable, it might have been cheaper for the lawyer to work just using their brain and a traditional precedent bank.
The key distinction is this, under a subscription model you can get an (often) unlimited use of the product. Under a usage or token billing model you get charged on how much you use the AI. Neither model charges based on how good the output is, only how you get to the output. Moving from the first system to the second not only will increase costs hugely but it makes it harder to predict the costs.
The solution is tacit knowledge… it is also the problem.
The dirty secret is that whilst AI is impressive, getting the best out of it means training it to your specific needs. This is where the problem starts. Tacit knowledge is the know-how which is so internalised through experience and practise it is hard to recognise, explain or teach. It is the secret sauce needed to make something work properly. The most famous example of tacit knowledge is the Panasonic Breadmaker story. The makers of the bread machine could not get it to create a good loaf until one of the team volunteered as an apprentice to a master baker and realised that the bakers were not only stretching the dough during kneading but twisting it. The action of twisting the stretch was so internalised no-one realised it was happening.
How much twisting is done by lawyers in their legal work without them realising? In my experience a great deal. You can train AI on all the documents in the world, but it is the information no-one bothered to write down that is the crucial data. That is where Legal AI implementation comes unstuck.
If tacit knowledge is the solution, how do we get to it?
Maybe you want the AI to review Non-Disclosure Agreements; a simple, common agreement type, relatively low risk, fixed format commodity work. But could you write down every single step and thought process you have in doing such a review, and could you do it in the specific, narrow ‘criteria’ language demanded by certain AI platforms - some of which require specialist training, and the granting of relevant permissions, from the vendors themselves before modification of the NDA review parameters are permitted?
I spent the last 20 years working with lawyers to dig that information out of their brains, codify it and systematise it. Originally, I did it to train paralegals, now I do it to train AI. It is the same process for both. I have interviewed a great many lawyers about how they do what they do, step by step. Creating a set of processes and playbooks that get the work done. All of them hold some secret, automatic knowledge in their mind which is important to the task at hand. I look at a piece of legal work and break it down like this:
What is the commercial context/goal of the task?
What are all the tasks and parts of tasks needed to complete the work (including the ‘obvious’ ones and the ones you do without thinking)?
In what order do those tasks happen?
Which tasks are contingent on others being completed first?
Who currently does those tasks?
What is the strategic plan of the organisation and what levels of delegation best support that plan (i.e. a senior solicitor currently does the ‘know your client check’ but it is more cost efficient for a paralegal because this practice area has more price sensitive clients)?
Who could do those tasks, what training and other support would they need to do them?
Where in that process should we have escalation processes and what do they look like?
Where in the process are the quality checks and what are they?
What are common errors and pitfalls in the work process and how to avoid them?
How do we judge if the work has been a success? What metrics do we use?
In what format should the final work be delivered?
This involves a deep investigation of the legal work and processes; understanding how and when tasks can be broken down into smaller tasks and writing playbooks codifying that process. The playbook is essential and needs particular features for success; especially if the task is to be completed by a paralegal or AI.
The beauty of this work and the processes and playbook created is that it is always valuable regardless of what comes next.
It can be used to:
train an AI,
train a paralegal,
make the work of senior lawyers more efficient,
standardise transactions thereby reducing risk,
onboard new staff, and
ensure company knowledge is maintain and passed on.
Without it the AI will always be working on half the data it needs.
I strongly recommend documenting the knowledge base like this first. In a world where Legal AI is charging on usage not subscription, this is how you keep control of the costs. This method means there is no wasted time, the AI tasks are clearly defined, the training material is created by humans first according to your legal strategy instead of by expensive AI experimentation and when you want to change your AI provider, you own everything you need to train its replacement. This process might seem granular but it is the fastest and most reliable way to get full value out of your legal AI.