Skip to content
By lawyers, for lawyers · Since 1993
Clarence
Resources / Read the Fine Print on AIBy Leonie Cranney, Head of Marketing at Clarence · Part 1 of 5 · Checked 15 October 2026

Large language models: what they are, who owns them, and what happens to what you type

Every AI tool on your desk runs on the same kind of engine. Start with what it is, who makes it and where your words go.

If you've asked ChatGPT a legal question and got back a confident answer with a citation, you've watched a large language model (LLM) at work. You may also have watched it invent the case.

What is an LLM?

A program that predicts words. An LLM is trained on an enormous amount of text and learns the patterns of language well enough to predict what comes next. Anthropic's own documentation describes training as teaching a model "to predict the next word, given the previous context". When you ask a question, the model writes its answer a fragment at a time (these fragments are called tokens), each one chosen because it is likely to follow the last. ChatGPT, Claude, Gemini and Copilot are products built around one.

There's no case database behind the answer. When a general chatbot gives you a citation, it has produced something shaped like a citation because citations in its training data look that way. Sometimes the case exists. Sometimes it's a plausible fake with a real-sounding party name and a sensible year. The industry calls this hallucination. It's a soft word for a built-in behaviour: the model is built to produce an answer whether or not it has one.

Australian courts have seen plenty of it. Damien Charlotin, a researcher who tracks court and tribunal decisions dealing with AI-generated errors, had logged more than 2,100 decisions worldwide as at 2 October 2026, 112 of them Australian. In Dayal [2024] FedCFamC2F 1166, a solicitor handed up a list of authorities that didn't exist; the Victorian Legal Services Board later restricted his practising certificate and put him under supervision for two years. In August 2025, senior counsel in a Victorian Supreme Court murder trial apologised for submissions containing fabricated quotes and non-existent citations. Justice Elliott said it was "not acceptable for artificial intelligence to be used unless the product of that use is independently and thoroughly verified."

Who owns them?

Four companies make the models most lawyers will meet. OpenAI makes the GPT models behind ChatGPT, and Microsoft owns about 27% of it. Anthropic makes Claude, with Amazon and Google as its major investors. Google DeepMind makes Gemini. Microsoft's Copilot runs on OpenAI's models, and for business customers in Australia Anthropic's Claude is now switched on inside it by default.

The legal research tools borrow them too. Thomson Reuters' next-generation CoCounsel runs on Claude, and LexisNexis is adding Claude to Lexis+. The name on the product tells you who you've contracted with. The model underneath may belong to someone else.

Exactly what happens to your data

It depends on the plan you're on. Three separate questions sit under that: whether your text is used to train the model, how long it's kept, and where it's stored. Typing into a chat doesn't train the model on the spot. Training happens later, from stored conversations, if the setting allows it.

Personal plans train by default. On the free and individually paid plans for ChatGPT, Claude and Copilot, your conversations can be used to train the models unless you switch that off. Gemini on a personal Google account keeps chats for 18 months, and a sample is read by human reviewers. Google's own privacy page asks you not to enter confidential information you wouldn't want a reviewer to see.

Business plans don't. ChatGPT Business and Enterprise, Claude Team and Enterprise, Gemini in Google Workspace and Copilot on a work account all say your content isn't used for training by default. Retention is a separate setting and varies by plan, so check both. Rating an answer with a thumbs up or down can be the exception to the training rule.

Storage is mostly overseas. None of the four keeps personal-plan data in Australia. Some business plans store content at rest here (OpenAI's Enterprise plan and Copilot for business do), and the processing still happens offshore. Anthropic's own platform stores data in the US, and Workspace data regions are the US or the EU.

Parts 2 to 5 go through each tool's plans and settings and quote the terms, so you can see where client work can safely go.

The other kinds of AI you'll hear about

Reasoning models are LLMs that work through a problem in steps before answering. OpenAI describes them as using "internal reasoning tokens" to break down the prompt and consider multiple approaches. They're better at multi-step analysis and slower to respond. They still make things up.

Small, on-device models run on a phone or laptop. Apple's runs at around 3 billion parameters, a fraction of the size of the cloud models. They're less capable, but your data doesn't have to leave the device.

Tools that search before they answer (the industry calls this retrieval-augmented generation, or RAG) are how most legal research tools work. The tool first searches a defined set of documents (a case law database, or your own matter files), then hands what it finds to the LLM to write the answer. A 2024 Stanford study found general chatbots hallucinated between 69% and 88% of the time on specific legal queries. Legal research tools built on RAG did much better, but still hallucinated between 17% and 33% of the time. Those tests used 2023 and 2024 models, and both numbers have likely moved since.

Agents are systems built around an LLM, with tools and permission to take steps on their own: search, open files, draft, fill in forms. Anthropic describes them as systems where LLMs "dynamically direct their own processes and tool usage". They save more time and need closer supervision.

What the courts now expect

Several Australian courts have now set rules. NSW Supreme Court Practice Note SC Gen 23 (in force since 3 February 2025) says generative AI must not be used to generate the content of affidavits or witness statements, and citation checks can't be done by an AI tool alone. The Federal Court's GPN-AI (16 April 2026) requires that legal authorities cited "exist and support the proposition stated". The Supreme Court of Victoria issued SC GEN 25 in May 2026, and Queensland's Supreme Court Practice Direction 5 of 2025 treats your name on a submission as confirmation that you've checked every reference.

Before you type

Use it for language, verify anything that's a fact. LLMs are very good at drafting, summarising, restructuring and explaining. The checks that matter are short.

  • Check which account you're signed into. Personal and business plans have different rules.
  • Switch training off on any personal plan, and leave the thumbs alone.
  • Know how long your chats are kept, where, and whether a reviewer at the provider can read them.
  • Treat every authority, date, figure and quote as a lead to check against a primary source, the same way you'd treat a first-year's research memo.

Next in the series: ChatGPT, who owns it and what happens to what you type.

Read the Fine Print on AI

Five parts, one a week.

Part 2 looks at ChatGPT: who owns it and exactly what happens to what you type. Then Claude, Gemini and Microsoft Copilot, one each Tuesday.

See the series →

Sources

Every fact above was checked against the source listed here on 15 October 2026. Data settings, prices and model names change often, so check the current page before you rely on one.

  • Anthropic, Glossary (training, tokens, RAG): https://platform.claude.com/docs/en/about-claude/glossary
  • Anthropic, Building effective agents, 19 Dec 2024: https://www.anthropic.com/engineering/building-effective-agents
  • OpenAI, Reasoning models guide: https://developers.openai.com/api/docs/guides/reasoning
  • Apple Machine Learning Research, Apple foundation models 2025 update, 9 Jun 2025: https://machinelearning.apple.com/research/apple-foundation-models-2025-updates
  • Stanford HAI, Hallucinating law, 11 Jan 2024: https://hai.stanford.edu/news/hallucinating-law-legal-mistakes-large-language-models-are-pervasive
  • Magesh et al, Hallucination-free? Assessing the reliability of leading AI legal research tools, Stanford RegLab, 2024 (J Empirical Legal Studies 2025): https://reglab.stanford.edu/publications/hallucination-free-assessing-the-reliability-of-leading-ai-legal-research-tools/
  • Damien Charlotin, AI Hallucination Cases database, read 2 and 4 Oct 2026: https://www.damiencharlotin.com/hallucinations/
  • Victorian Legal Services Board + Commissioner, Statement on the Dayal matter, 2 Sep 2025: https://www.lsbc.vic.gov.au/news-updates/news/statement-mr-dayal-matter
  • ABC News, Victorian lawyer apologises after AI-generated submissions, 15 Aug 2025: https://www.abc.net.au/news/2025-08-15/victoria-lawyer-apologises-after-ai-generated-submissions/105661208
  • OpenAI, Our structure: https://openai.com/our-structure/
  • Axios, OpenAI completes recapitalization, 28 Oct 2025: https://www.axios.com/2025/10/28/openai-completes-recapitalization
  • Fortune, Amazon and Google's billions in Anthropic, 4 Jun 2026: https://fortune.com/2026/06/04/amazon-google-billions-anthropic-ipo/
  • Microsoft Learn, AI providers operating as Microsoft subprocessors (read 5 Oct 2026): https://learn.microsoft.com/en-us/microsoft-365/copilot/connect-to-ai-subprocessor
  • Thomson Reuters, press release, 12 May 2026: https://www.thomsonreuters.com/en/press-releases/2026/may/thomson-reuters-and-anthropic-expand-partnership-to-connect-claude-with-cocounsel-legal
  • LexisNexis, press release, 13 May 2026: https://www.lexisnexis.com/community/pressroom/b/news/posts/lexisnexis-expands-lexis-with-protege-by-integrating-anthropics-claude-legal-plugin-suite
  • OpenAI Help Centre, How your data is used to improve model performance (read 4 Oct 2026): https://help.openai.com/en/articles/5722486
  • OpenAI Help Centre, Data residency and inference residency (read 4 Oct 2026): https://help.openai.com/en/articles/9903489
  • Anthropic Privacy Centre, Is my data used for model training (read 4 Oct 2026): https://privacy.claude.com/en/articles/7996868-is-my-data-used-for-model-training
  • Claude Platform docs, Data residency: https://platform.claude.com/docs/en/manage-claude/data-residency
  • Google, Gemini Apps Privacy Hub, updated 24 Sep 2026 (read 4 Oct 2026): https://support.google.com/gemini/answer/13594961
  • Google Workspace, Generative AI in Google Workspace Privacy Hub, updated 14 Aug 2026: https://knowledge.workspace.google.com/admin/gemini/generative-ai-in-google-workspace-privacy-hub
  • Google Workspace Admin Help, Data regions (updated 1 Oct 2026): https://knowledge.workspace.google.com/admin/compliance/choose-a-geographic-location-for-your-data
  • Microsoft Support, Privacy FAQ for Microsoft Copilot (read 5 Oct 2026): https://support.microsoft.com/en-us/microsoft-copilot/privacy-faq-for-microsoft-copilot
  • Microsoft Learn, Enterprise data protection in Microsoft Copilot, updated 18 Aug 2026: https://learn.microsoft.com/en-us/microsoft-365/copilot/enterprise-data-protection
  • Microsoft Learn, Data residency for Copilot, updated 3 Sep 2026: https://learn.microsoft.com/en-us/microsoft-365/enterprise/m365-dr-service-copilot-offerings
  • Supreme Court of NSW, Practice Note SC Gen 23: https://supremecourt.nsw.gov.au (Practice Notes, SC Gen 23)
  • Federal Court of Australia, GPN-AI, 16 Apr 2026: https://www.fedcourt.gov.au/law-and-practice/practice-documents/practice-notes/gpn-ai
  • Supreme Court of Victoria, SC GEN 25, 14 May 2026: https://www.supremecourt.vic.gov.au/areas/legal-resources/practice-notes/sc-gen-25-the-use-of-artificial-intelligence-by-court-users
  • Queensland Courts, Supreme Court Practice Direction 5 of 2025, 24 Sep 2025: https://www.courts.qld.gov.au

General information, not legal advice. Clarence Workspaces is a workspace provider, not a law firm.