The study claims that although GPT-4 is more reliable than GPT-3.5, both models can generate fabricated bibliographic citations and make errors in genuine references. This is because 55% of GPT-3.5’s citations and 18% of GPT-4’s citations were fabricated, while 43% and 24% of their respective genuine citations contained substantial errors.[1]
[1] Fabrication and errors in the bibliographic citations generated by ChatGPT - Scientific ReportsAcademic Paper
Nature · 6 Oct 2026
Passage from the sourceAbstract Although chatbots such as ChatGPT can facilitate cost-effective text generation and editing, factually incorrect responses (hallucinations) limit their utility. This study evaluates one particular type of hallucination: fabricated bibliographic citations that do not represent actual scholarly works. We used ChatGPT-3.5 and ChatGPT-4 to produce short literature reviews on 42 multidisciplinary topics, compiling data on the 636 bibliographic citations (references) found in the 84 papers. We then searched multiple databases and websites to determine the prevalence of fabricated citations, to identify errors in the citations to non-fabricated papers, and to evaluate adherence to APA citation format. Within this set of documents, 55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated. Likewise, 43% of the real (non-fabricated) GPT-3.5 citations but just 24% of the real GPT-4 citations include substantive citation errors. Although GPT-4 is a major improvement over GPT-3.5, problems remain.
2023 models and selected prompts; not a current universal hallucination rate.
Tests on verifiable US legal questions found frequent incorrect and invented legal claims. Shows citation-like factual failures in a high-stakes domain.[2]
[2] Large Legal Fictions: Profiling Legal Hallucinations in Large Language ModelsAcademic Paper
arXiv.org · 6 Oct 2026
Legal hallucinations include more than fabricated citations; task mix affects rates.
The study claims that AI-driven legal research tools still produce significant hallucinations despite using technologies such as retrieval-augmented generation (RAG). This is because Lexis+ AI, Westlaw AI-Assisted Research, and Ask Practical Law AI were found to hallucinate between 17% and 33% of the time, showing that their claims of being “hallucination-free” are overstated.[3]
[3] Hallucination-Free? Assessing the Reliability of Leading AI Legal Research ToolsAcademic Paper
Magesh et al. / Journal of Empirical Legal Studies · 6 Oct 2026
Passage from the sourceABSTRACT Legal practice has witnessed a sharp rise in products incorporating artificial intelligence (AI). Such tools are designed to assist with a wide range of core legal tasks, from search and summarization of caselaw to document drafting. However, the large language models used in these tools are prone to “hallucinate,” or make up false information, making their use risky in high-stakes domains. Recently, certain legal research providers have touted methods such as retrieval-augmented generation (RAG) as “eliminating” or “avoid[ing]” hallucinations, or guaranteeing “hallucination-free” legal citations. Because of the closed nature of these systems, systematically assessing these claims is challenging. In this article, we design and report on the first preregistered empirical evaluation of AI-driven legal research tools. We demonstrate that the providers' claims are overstated. While hallucinations are reduced relative to general-purpose chatbots (GPT-4), we find that the AI research tools made by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time. We also document substantial differences between systems in responsiveness and accuracy. Our article makes four key contributions. It is the first to assess and report the performance of RAG-based proprietary legal AI tools. Second, it introduces a comprehensive, preregistered dataset for identifying and understanding vulnerabilities in these systems. Third, it proposes a clear typology for differentiating between hallucinations and accurate legal responses. Last, it provides evidence to inform the responsibilities of legal professionals in supervising and verifying AI outputs, which remains a central open question for the responsible integration of AI into law.
Product versions and tested questions are specific; retrieval can still reduce errors.