Claude is described as “the trusted assistant for you to do your best work.” Other generative AI tools use similar language that humanizes systems and emphasize the trustworthiness of tools. The marketing of trust is creating issues with users who trust the conversational outputs of these systems over the experience and knowledge of human librarians. Recently, the International Committee of the Red Cross (ICRC) published a notice concerning AI-generated archival references. The notice stated, in part, “We are aware that some AI chatbots (such as ChatGPT, Gemini, Copilot, Bard and others) may generate incorrect or fabricated archival references. These systems do not conduct research, verify sources, or cross-check information.” Notably, the notice states “If a reference cannot be found, this does not mean that the ICRC is withholding information;” which seems to suggest that some ICRC Library patrons might trust AI chatbots more than human librarians. A Scientific America article discussing the ICRC’s notice, highlights the difficulties AI models create for researchers and librarians. False sources create more work for librarians who spend more time trying to find non-existent sources. Sarah Falls, of the Library of Virginia, states “it is much harder to prove that a unique record doesn’t exist.” Librarians, already spending more time on trying to find non-existent sources, must also contend with a patron possibly trusting the AI model’s answer over the human librarian.

In my last post I said law librarians must have a seat at the table in important AI discussions. The ICRC’s notice and other recent developments highlight just how important librarians are. A Futurism article discussing the ICRC notice and Scientific American article is titled “Librarians Dumbfounded as People Keep Asking for Materials That Don’t Exist.” I’m glad the author is covering this issue, but I would argue that librarians are not dumbfounded. Librarians are on the frontlines of knowledge creation and information access. We saw many of these issues coming. Unlike AI-generated references, librarians know how to “conduct research, verify sources, [and] cross-check information.” We also know how to help our patrons do these things, but only if our patrons trust us more than they trust technology.

Trust in technology over trust in humans is not a new issue,[1] but it has been super charged by generative AI.[2] Prior to the generative AI boom, search technologies and technology enhanced information garnered misplaced trust through aesthetically pleasing interfaces; proxies (like Google featured snippets) that act as surrogates for deep reading and analysis; and the general anthropomorphizing of technology. Even before generative AI chatbots, search systems, like Google, were built to mimic simple two-actor communicative queries. Users input search terms or queries into a search box and receive an output in the form of a ranked list of results. Early empirical studies of user search behavior showed that most users didn’t look beyond the first pages of Google results and were more likely to trust the higher-ranked results even when those results weren’t relevant to their research. Misplaced trust in Google search and other search technologies was priming us for misplaced trust in generative AI. We’re more prone to trust ChatGPT and other chatbots because they’re designed to mimic more complex two-person communicative queries. Rather than getting a list of results, we receive a detailed response, often in fully formed sentences with seemingly correct citations, and can ask follow-up questions. We are lulled into feeling like we’re interacting in a duologue with a “trusted assistant” or a “thinking partner.”

Cat Moon recently wrote on The AI of Law blog about ways we can “stop acting like spectators in our own industry” and start seeing the “AI revolution” as “a tool implementation challenge that we have the agency to solve.” One way is to stop anthropomorphizing AI. Instead of treating AI as a colleague or “thinking partner” we should treat it as a utility. AI is a “what” not a “who.” This shift in perspective is essential to ensuring users approach AI with a librarian’s mindset. Users can’t trust “what” (AI) is giving them a response, but they can trust “who” (librarians) is helping them verify the information.

Hana Lee Goldin recently published “How to Spot AI Hallucinations Like a Reference Librarian” on Card Catalog – republished here on LLRX. I won’t summarize it here because you should read the whole post for yourself, but I was especially interested in her analysis of why AI citations are so convincing. Goldin writes, “The tell isn’t that fake citations look wrong. It’s that they look too right. Too convenient. Too perfectly aligned with whatever point the AI is making. Real academic citations are messy.” Law librarians know about the messiness of citations. We can help walk our patrons through the three-layer verification method Goldin describes.

• Layer One: check if the source actually exists
• Layer Two: if the source exists, does it actually state what the AI claims it states
• Layer Three: if the source exists and it states what the AI claims, is the source contextually correct. For example, is one divergent theory being presented as consensus.

We can also help patrons understand that AI’s confidence doesn’t equal competence. As law librarians we are naturally skeptical of certainty. The law teaches us to question everything, and library school teaches us how to verify everything. We, not generative AI, are the trusted human assistants that need to help our patrons effectively use technology tools.

[1] For example, see Niel Kerssens, When Search Engines Stopped Being Human: Menu Interfaces and the Rise of the Ideological Nature of Algorithmic Search, 1 Internet Histories 219 (2017); Matthew Reidsma, Masked by Trust: Bias in Library Discovery (2019); Devesh Narayanan & David De Cremer, “Google Told Me So!” On the Bent Testimony of Search Engine Algorithms, 35 Phil & Techn. 1 (2022); Bing Pan, Helene Hembrooke, Thorsten Joachims, Lori Lorigo, Geri Gay & Laura Granka, In Google We Trust: Users’ Decisions on Rank, Position, and Relevance, 12 J. Computer-Mediated Communication 801 (2007); Siva Vaidhyanathan, The Googlization of Everything (and Why We Should Worry) (2011); Jennifer Chapman, Teaching Critical Use of Legal Research Technology, 28 J. Legal Writing Institute 123 (2024).

[2] Some studies are starting to come out that examine trust in generative AI models, like ChatGPT. See, e.g., Joy Buchanan & William Hickman, Do People Trust Humans More than ChatGPT?, 112 J. Behavioral & Experimental Econ. 102239 (2024); Mengmeng Zhang, Jian-Hong Ye & Xiantong Yang, Watch Out For Errors! Factors Related to ChatGPT Skepticism: A Cognitive Perspective, 33 Interactive Learning Environments 3985 (2025); Zhongwang Lyu, Junshen Lin & Wei Cheng, Public Trust in ChatGPT: A BERT-Based Framework for Analyzing Social Media Discourse, Int’l J. Human-Computer Interaction (2025), https://doi.org/10.1080/10447318.2025.2545454.