The generative game

ChatGPT and other generative AI systems are the hot new thing. But after one lawyer fell foul of the court by relying on it, Neil Rose looks at what it means for the profession – and asks ChatGPT itself how Chartered Legal Executives can use it

In February 2022, Roberto Mata issued proceedings in New York alleging that he was injured when a metal serving cart struck his left knee during an Avianca flight from El Salvador to John F. Kennedy Airport.

Some 11 months later, lawyers for the Colombian airline filed a motion to dismiss on the grounds that the claim was time-barred under the Montreal Convention. They were then confused by the reply from Mr Mata’s lawyers, New York firm Levidow Levidow & Oberman, as they were unable to locate most of the case law mentioned, and the few they could find did not back up the propositions for which they were cited.

The court conducted its own search but was unable to find many of them either.

The reason, it turned out, was that Steven A Schwartz had consulted ChatGPT – because the legal research tool the firm subscribed to did not cover the relevant law – and the much-hyped artificial intelligence (AI) system had simply made the cases up. It then insisted they were real when Mr Schwartz double-checked.

Judge Kevin Castell was not impressed. Fining Mr Schwartz, a colleague and their firm $5,000 each, he said: “In researching and drafting court submissions, good lawyers appropriately obtain assistance from junior lawyers, law students, contract lawyers, legal encyclopedias and databases such as Westlaw and LexisNexis.

“Technological advances are commonplace and there is nothing inherently improper about using a reliable artificial intelligence tool for assistance. But existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings.”

Here, he said, the lawyers “abandoned their responsibilities when they submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT, then continued to stand by the fake opinions after judicial orders called their existence into question”.

He also ordered the lawyers to inform their client and the judges whose names were wrongfully invoked by ChatGPT of the sanctions imposed on them.

Mr Schwartz told the judge that he was “operating under the false perception” that ChatGPT “could not possibly be fabricating cases on its own”.

He said: “My reaction was, ChatGPT is finding that case somewhere. Maybe it’s unpublished. Maybe it was appealed. Maybe access is difficult to get. I just never thought it could be made up.”

Early adoption

The story has made headlines around the world, as have many others about ChatGPT and similar new generative AI systems, like Bard (GPT stands for generative pretrained transformer).

So, what is generative AI? The World Economic Forum explains: “Generative AI refers to a category of AI algorithms that generate new outputs based on the data they have been trained on. Unlike traditional AI systems that are designed to recognise patterns and make predictions, generative AI creates new content in the form of images, text, audio, and more.”

“In early 2023, global giant Allen & Overy became the first law firm to adopt an AI assistant similar to ChatGPT” 

Business consultancy Accenture says: “The technology underpinning the powerful new chatbot is one of the biggest step changes in the history of AI… Across business, science and society itself, it will enable groundbreaking human creativity and productivity.”

It has caught the attention of lawyers just as much as anyone else and we’ve already seen some adopt it. In early 2023, global giant Allen & Overy (A&O) became the first law firm to adopt an AI assistant similar to ChatGPT to help speed up the work of more than 3,500 lawyers.

The tool, Harvey, uses natural language processing, machine learning and data analytics to automate and enhance different aspects of legal work. It can operate in multiple languages. A&O said it could help generate insights, recommendations and predictions based on large volumes of data, “enabling lawyers to deliver faster, smarter and more cost-effective solutions to their clients”.

An A&O spokesman stressed that the software did not aim to replace lawyers and that Harvey’s results needed to be carefully assessed for accuracy. It was an initial research and drafting tool which, when used properly, could save lawyers a few hours a week each.

But the spokesman pointed out that, in such a large firm, small savings of time amounted to a significant cost saving overall. As time went on, Harvey would speed up, making the financial benefit greater.

This is not just a big City firm phenomenon. Southport-based Fletchers, the biggest specialist personal injury firm in the country, recently announced that, having successfully built and used two versions of an AI tool to help lawyers make initial case decisions, work was beginning on a third.

Dan Taylor, head of integration at Fletchers, said SIDSS (Structured Information Decision Support Systems) had saved “tens of thousands of hours” in staff time previously spent deciding whether cases should continue to be investigated.

He said the first iteration of SIDSS was based on a “random forest” of decision trees, while the second introduced machine learning and neural networks. Work on the third version would involve discovering “how we can harness the power of ChatGPT-type models and apply them to our data”.

SIDSS compares initial client details of medical negligence claims with the tens of thousands of previous cases in the law firm’s database to deliver a “confidence score”, expressed as a percentage, as to whether the firm should continue to investigate the case.

Every case, however large, is put through SIDSS, as long as clients give their consent. Rejected cases are sent to an “overturn team” to be checked to see whether they should still be investigated. Mr Taylor said that the overturn teams agreed with SIDSS in an “overwhelming number of cases”.

Lawyers working on the case can also decide whether they want to ignore the confidence score. “This gives clients confidence that decisions are not being made just by AI.”

Mr Taylor added: “Early decisions are really important to people with medical negligence claims. ChatGPT-type models are an enormous opportunity to improve things even further.”

A recent survey by Thomson Reuters found that 54% of in-house lawyers believe ChatGPT and other generative AI should be used for legal work, although only 11% of them were currently using or planning to use ChatGPT, compared to just 5% of law firms.

At the same time, in-house lawyers were concerned about the potential risks, particularly relating to accuracy, privacy, confidentiality and security.

Cautionary tale

Before getting carried away, let’s not forget the cautionary tale of Steven Schwartz. Other lawyers have tested ChatGPT and found it wanting. For example, Daniel Chard, a partner and head of property at Bird & Co in the East Midlands, asked it a range of questions relating to first-time buyers.

He found that the chatbot slipped up on stamp duty and first-time buyer ISAs, though he was “impressed with elements of its capabilities” in answering popular queries.

Mr Chard said it was “clear to see how ChatGPT could be disruptive for many sectors, including the legal sector” but it could not replace the advice of a legal expert, especially in complex situations such as buying a house. “This comes down to the nuances of the law, which changes frequently, as well as the human element of interactions with lawyers, which simply cannot be replaced by a chatbot.”

Mr Chard said the most “striking issue” with ChatGPT “appearing to give” legal advice was that it was “not able to tailor any advice given to the person asking”, which “could easily lead first-time buyers down a path” where they wasted time and money.

Meanwhile, a test by Shane McCann, managing director of SG Murphy Solicitors in Belfast, found that ChatGPT struggled to cope with a dozen legal scenarios, all based on the kind of questions faced by high street lawyers.

Some lawyers have tested ChatGPT and found it wanting

He said that, as well as producing one factually incorrect answer (about prescriptive easements in the UK), three-quarters of the chatbot’s responses were below the standard of a “real lawyer”, but it was better at drafting simple legal agreements. Like Mr Chard, he said the lack of advice tailored to the client was another challenge.

The Master of the Rolls, Sir Geoffrey Vos, referred to Mr Schwartz’s case in a recent speech. “The risks of litigants in person using ChatGPT to create plausible submissions must be even more palpable,” he observed. “And indeed such an event was reported as having happened in Manchester only a few days ago.”

He said the case showed lawyers could not use generative AI to cut corners. “I suspect that non-specialised AI tools will not help professional lawyers as much as they may think, though I have no doubt that specialised legal AIs will be a different story.”

He said Spellbook was already claiming to have adapted “GPT-4 to review and suggest language for your contracts and legal documents”.

In litigation at least, Sir Geoffrey went on, “the limiting factor may be the court or tribunal adjudicating on the dispute”.

He said: “One can envisage a rule or a professional code of conduct regulating whether and in what circumstances and for what purposes lawyers can use large language models to assist in their preparation of court documents, and be properly held responsible for their use in such circumstances.

“Those will be things that the existing rules committees, regulators, and the new Online Procedure Rules Committee… will need to be considering as a matter of urgency.

The MR added that the way ChatGPT answered Mr Schwartz’s questions to confirm the cases it cited indicated two issues: that ChatGPT and AIs more generally “need to be programmed to understand the full import of a human question”, and humans using them “need to be savvier in checking their facts”.

This meant both asking more precise questions and programmers explaining to “the AIs they are creating what humans mean when they ask something as open textured as ‘is this a real case’”.

He said: “This requires careful and detailed programming beyond what might be required in other fields of activity… If GPT-4 (and its subsequent iterations) is going to realise its full potential for lawyers… it is going to have to be trained to understand the principles upon which lawyers, courts and judges operate.

“As Mr Schwartz found to his cost, the present version of ChatGPT does not have a sufficiently reliable moral compass. In the meantime, court rules may have to fill the gap.”

But Sir Geoffrey recognised that generative AI may well have a significant role to play. “We may even hopefully turn it to the advantage of access to justice and effective and economical legal advice and dispute resolution.”

Job hunter

The worry for lawyers is whether AI will take their jobs. At the moment, the answer appears to be a resounding no – although it is always worth remembering that generative AI is still in its relative infancy and will only get better from here.

As Sir Geoffrey Vos noted in another recent speech, that while the previous iteration of ChatGPT scored in the bottom 10% when it took the US Bar exam, the most recent version was in the top 10%.

“GPT-4 and other advanced machine learning is likely to transform the work that lawyers need to do and possibly even, in the slightly longer term, the business of judging,” he said.

Solicitor Simon Gibson, chief executive of the Spirant Group – which provides a range of services around personal injury claims – harks back to Mr Schwartz in New York and points out: “ChatGPT was trained on a diverse range of internet text. However, it does not have access to an up-to-date, exhaustive, or accurate legal database. It is not designed to be relied upon for legal advice, let alone case citations.

“The worry for lawyers is whether AI will take their jobs. At the moment, the answer appears to be a resounding no” 

“This incident has underscored the significance of understanding the functionality, reliability and limitations of AI before integrating it into professional practices.”

The law, he says, is “intricate, complex and forever evolving”, hence the reliance on the human ability to interpret, strategise and present it accurately.

“AI, as it stands today, lacks the capacity to comprehend context fully, interpret nuances, or exercise judgement. It cannot replace years of legal expertise, understanding and the ability to engage in sophisticated legal thinking.”

The legal sector needs to spend time learning about AI and its benefits, and not rush to purchase or implement it prematurely. “AI is not a panacea for all legal challenges; instead, it should be seen as an evolving tool designed to augment, not replace, human capabilities. At least in legal work, it is likely that the AI will always need to be underpinned by a knowledgeable, human touch.”

A similar message comes from Chris Tart-Roberts, chief knowledge and innovation officer at City law firm Macfarlanes, which recently briefed journalists on its experience of piloting generative AI.

“The reality with this technology is that it is a long way from being perfect. Anything you get out of it requires an awful lot of checking,” he said. The firm’s assessment was that AI was “probably 50-60% accurate”.

“The second thing that becomes very clear when you start using this technology in anger is that it’s not sentient. It’s a really obvious point but clearly it does not have human experience, it cannot demonstrate empathy, it can’t quantify risk, it can’t understand commercial circumstances or the bargaining power of the sort of clients that we act for.

“When you see people using the technology in the context of their day-to-day work, you realise what a big part those things play in what lawyers actually do.”

The third finding was that, at the moment, the technology “is only useful in relation to certain, very specific things”.

Macfarlanes has found it “very good at summarising things”, such as documents, cases or pieces of legislation, “and it’s pretty good at doing some first cuts of emails and notes”.

It could “do a decent job at research” but not legal research. Mr Tart-Roberts explained how AI really struggled with technical legal analysis, well below the accuracy average. For example, in analysing a court ruling, it could not differentiate between the decisions cited in a judgment, such as one at first instance and one on appeal.

The technology has “a huge amount of potential – but as a support tool. We really need to be clear about that”, he stressed.

He observed that lawyers already have access to a lot of support technologies, such as know-how and precedents, and “those things have not replaced our lawyers, they’ve augmented them”. Generative AI “will be no different from that”, he forecast.

But Mr Tart-Roberts stressed that generative AI is “here to stay”. He said: “When you get your hands on it, it is really, really impressive. The genie is out of the bottle. We will see, over probably the next three years, that this technology will become ubiquitous across the profession. We are already seeing the start of that.”

He pointed out that how it was already being implemented by existing technology suppliers to the profession, like LexisNexis, Practical Law and Microsoft.

“We will also see some firms, like us, that will invest in developing technologies themselves and looking at how they can use these open-source models within their firm on their own datasets to do very specific things that the organisation needs.

“We need to start planning for those things now, exploring those use cases, understanding what the technology is good at and not good at, establishing our checks and balances, and establishing good governance.”

There also needed to be a discussion about wider issues, such as regulation, data and AI’s environmental and societal impacts, while legal regulators too would need to provide “more clarity and guidance”, particularly around client protection.

The second certainty was that the profession needed to widen its skillset. “We are going to need more legal technologists, more AI specialists, and more data and computer science specialists.”

Third, he said: “Whilst I do not think that AI is going to replace lawyers any time soon, I do absolutely think that lawyers who use AI in the future will replace lawyers who don’t use AI.”

We asked ChatGPT: How can Chartered Legal Executives use generative AI?

It replied:

“Chartered Legal Executives can leverage generative AI in various ways to enhance their legal practice and improve efficiency. Here are a few examples:

- Document Generation: Generative AI can be used to automate the creation of legal documents, such as contracts, agreements, and legal letters. By training AI models on a large dataset of existing documents, Chartered Legal Executives can use AI-powered tools to generate draft documents quickly, saving time and reducing manual labour.

- Legal Research: AI-powered tools can assist Chartered Legal Executives in conducting legal research. These tools can analyse vast amounts of legal texts, case law, and statutes to identify relevant precedents and legal arguments. By using natural language processing and machine learning algorithms, these tools can help legal executives find relevant information more efficiently, saving valuable research time.

- Due Diligence and Contract Review: Generative AI can aid in due diligence processes and contract review. By using machine learning algorithms, AI tools can analyse large volumes of contracts and legal documents to identify potential risks, anomalies, or missing clauses. This can help Chartered Legal Executives streamline their review process, identify critical issues, and ensure compliance.

- Legal Chatbots: Chatbots powered by generative AI can be used to provide basic legal information and guidance to clients. These chatbots can be programmed with legal knowledge and can interact with clients to answer common legal questions, provide initial advice, or direct clients to appropriate resources. This can free up the time of Chartered Legal Executives, allowing them to focus on more complex legal matters.

- Case Prediction and Strategy: AI models trained on historical legal cases can assist Chartered Legal Executives in predicting case outcomes and developing legal strategies. By analysing patterns and similarities between current and past cases, AI can provide insights on the likelihood of success, potential arguments, and recommended approaches. This can aid in decision-making and help Chartered Legal Executives provide more informed advice to their clients.

It's important to note that while generative AI can be a valuable tool, it should not replace the expertise and professional judgment of Chartered Legal Executives. It is meant to enhance their work, improve efficiency, and provide valuable insights. Legal professionals should carefully review and validate the outputs generated by AI tools and ensure compliance with legal and ethical standards.”