It started in May 2023 when a news story broke that a lawyer had used ChatGPT to write a legal brief for a court case. While this may not look like an issue in itself, the problem arose when the contents of the brief were examined by the opposing side.

A brief summary of the facts

The matter pertains to the case Roberto Mata v Avianca Inc, which involves an Avianca flight (Colombian airline) from San Salvador, El Salvador, to New York, United States, on 27 August 2019. The claimant – Mr Roberto Mata – claimed that during the flight he was hit on the knee by a serving cart and this caused serious injury. Thus, Mr Mata wanted to bring a claim for damages under the Convention for the Unification of Certain Rules for International Carriage By Air (Montreal Convention of 1999). The Convention is an international treaty that governs liability and compensation in cases of air travel-related accidents caused by airlines. It establishes the liability of air carriers for death, injury, or delay of passengers, as well as loss, damage or delay of baggage and cargo during international flights.

Avianca’s legal counsel asked the judge to toss out the case as the two-year statute of limitations period had passed pursuant to Article 35 of the Montreal Convention. However, Mr Mata’s lawyers objected and submitted a brief that cited several court decisions to counter this argument. The counter claim was that New York, where the case was being heard, has a three-year statute of limitation period and that the bankruptcy of Avianca had paused the period. While such arguments in civil proceedings are neither new nor controversial, the fact that the cited court decisions were nowhere to be found in the legal archives raised some eyebrows. Why was this? It was because the lawyers relied on ChatGPT and it made the cases up. It was an instance of ‘AI hallucination’.

Human-machine interaction

Legal professionals are increasingly utilising AI to streamline tasks that are typically labour-intensive. For example, it can be used during the discovery process in litigation cases; for predictive legal analytics, such as assessing the likely chances of success for a claim; legal research to help find laws, cases and articles; and legal language processing to ensure that the verbiage is correct. These tools can save time, reduce costs, and enhance consistency which brings benefits to lawyers, clients, and courts. While there are certainly benefits and opportunities for the legal profession in making use of AI, this does not come without risks and challenges, as exemplified by this recent case.

The lawyers in question stated that they had used ChatGPT ‘to supplement’ their work and even asked ChatGPT to verify whether the cases were real. As ChatGPT confirmed the authenticity of the court cases, the lawyers did not check further. When the lawyers representing Avianca checked the brief, they were unable to find the cases and, consequently, requested Mr Mata’s lawyers to provide copies. This was, of course, impossible, and the lawyers were called to court to explain.

In the end, an affidavit was signed admitting the use of AI, claiming there was no intention to deceive the court or act in bad faith. It further stated that this was the first time the lawyer responsible for drafting the brief had used AI for legal research and that he had believed it to be a reliable source for legal work. This proved incorrect and could have been avoided if the lawyers had checked the cases, which would have been a straightforward activity as the names and full citations were provided by ChatGPT.

The outcome was that the lawyers involved were each ordered to pay a $5,000 fine with the court stating that there is ‘nothing inherently improper about using a reliable artificial intelligence tool for assistance’. Thus, the court acknowledged the relevance of AI tools. However, it concluded that the case at hand was unacceptable.

A silver lining is that it was discovered early on in the proceedings, so the fabricated cases did not impact the final outcome. Further, the incident has become a public affair so, while this may have been embarrassing for the parties involved, it serves a lesson to all.

Knock-on effects of the case

One knock-on effect is that a Federal judge in the Northern District of Texas issued a standing order stating that persons appearing before the court must attest that ‘no portion of any filing will be drafted by generative artificial intelligence’ or highlight AI-generated text so it can be checked for accuracy. The judge wrote that while these ‘platforms are incredibly powerful and have many uses in the law’, briefings are not one of them as the platforms are ‘prone to hallucinations and bias’. Thus, it is not banned, but its use must be flagged and checks must be performed so that ‘bogus judicial decisions, with bogus quotes and bogus internal citations’ do not make it into the courtroom.

While AI can supplement legal work, the human lawyer remains essential as AI tools struggle to interpret complex legal nuances, provide strategic counsel, and uphold ethical standards, as well as provide the human touch in often difficult circumstances. There must be human-machine interaction.

AI will not replace you, but a competent person using AI will.

Photo by Franz Harvin Aceituna on Unsplash.