Arvind Narayanan and Sayash Kapoor’s new book AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference is a timely wake-up call amidst today’s AI hype. Narayanan and Kapoor are on a mission to help readers separate genuine AI advances from “snake oil” – the bogus or overhyped claims that too often swirl around artificial intelligence.

Narayanan is a professor of computer science at Princeton University and director of its Center for Information Technology Policy. Kapoor is a Princeton computer science doctoral candidate. TIME magazine included both authors in their list of the 100 most important people in the AI space.

In an era when AI is alternately hailed as a solution to all problems or vilified as an existential threat, AI Snake Oil strikes a balanced tone. The authors don’t oppose AI across the board – in fact, they celebrate certain real advances – but they are deeply skeptical of inflated promises. For lawyers and regulators grappling with AI, Narayanan and Kapoor’s analysis provides a much-needed reality check on both the technology’s potential and its pitfalls.

Myths, Hype, and the AI Marketing Machine

AI Snake Oil calls out the major “hype superspreaders” fueling today’s AI bubble:

• Big Tech Companies: Eager to attract investment, tech companies frequently overstate AI’s capabilities. From software firms touting the latest “gen AI” tool as a revolution, to cloud providers bragging about infinite AI compute power, corporate marketing sets unrealistic expectations.
• Researchers and Benchmark Gaming: The academic AI community is not blameless. Pressures to publish and get media attention tempt researchers to overstate findings or game benchmarks. For example, dozens of papers claimed to predict court decisions with high accuracy, but many exploited “data leakage”– e.g., using words from a judge’s opinion that only appear after the decision is known. Once that leak was patched, the predictive power vanished. Always dig into how an AI was evaluated – was it a controlled, rigorous test, or are we looking at inflated numbers?
• Journalists and Media: Sensational headlines often amplify AI myths. Reporters sometimes uncritically reprint company press releases or anthropomorphize AI for clicks. One New York Times column went viral by describing a chatbot that “wanted to be alive.” Narayanan and Kapoor argue that such stories sow public confusion about sentient algorithms that don’t exist. They also criticize “access journalism,” in which tech reporters soften coverage to stay in companies’ good graces. The result: every incremental lab result is hailed as world-changing, while lurid tales of AI misbehavior (often misunderstood) go viral.
• Public Figures and Pundits: Celebrities, CEOs, and even policymakers sometimes spread misleading narratives. Flashy keynotes proclaim that AI will “transform everything,” while doomsayers warn of an imminent robot apocalypse. Grandiose rhetoric usually serves the speaker’s agenda – attracting funding, shaping legislation, or simply grabbing attention.

AI Is Not One Big Thing: Predictive vs. Generative

A core theme of AI Snake Oil is that “AI” is an umbrella term covering very different technologies – and conflating them causes endless confusion. The authors liken it to having a single word, “vehicle,” for bikes, cars, and rockets. Debates turn absurd.

For clarity, Narayanan and Kapoor break modern AI into two types:

1. Predictive AI – algorithms that analyze past data to forecast future outcomes (fraud detection, insurance pricing, risk scores).
2. Generative AI – models that create what appears to be new content (ChatGPT, DALL-E).

Each type has different capabilities and limitations, but the public and even policymakers often don’t distinguish between them. Wild breakthroughs in one domain (say, generative AI’s leap in fluency) get incorrectly applied in our minds to others (“If ChatGPT is so smart, surely an AI judge is around the corner!”). This conflation fuels confusion and lets hucksters thrive.

By defining these categories, AI Snake Oil aims to demystify the buzzword. It debunks the myth that a single monolithic “intelligence” is improving inexorably. Predictive algorithms use old statistical methods, while recent AI excitement is due to generative models with large neural networks. And some “AI” isn’t really AI at all, in the transformative sense – an automated content moderation filter or a glorified Excel model is a far cry from sci-fi depictions. Whenever someone claims an AI system will solve X problem, the authors encourage us to ask what kind of AI, exactly?

Predictive AI in the Dock: Crime, Courts, and Other Case Studies

One of the book’s strongest messages is that predictive AI often simply does not work as advertised. Here are just a few of the sobering examples:

• Criminal Justice Risk Tools: Many jurisdictions use AI risk scores to decide bail or sentences. Studies show that these tools embed racial bias but are barely better than random guesses. Judges may trust a printout that is statistically only a hair better than flipping a coin.
• Predictive Policing & Fraud Screening: In the Netherlands, an algorithm flagged immigrant women for welfare fraud at vastly disproportionate rates. In U.S. cities, predictive-policing software sends officers to minority neighborhoods, perpetuating feedback loops.
• Healthcare Triage Algorithms: Medicare insurers have used AI to predict discharge dates. One 85-year-old still in pain lost coverage after the model insisted she’d be ready on Day 17.
• The “Suckers List”: Allstate’s 2013 algorithm for Maryland auto rates pinpointed drivers 62+ as price-inelastic and proposed steep hikes – a form of automated age discrimination. Regulators blocked it locally, yet the tactic surfaced elsewhere.
• Hiring & HR Screening: Vendors claim they can read personality from a 30-second video, but independent audits find performance little better than chance – and sometimes correlated with lighting or background noise rather than human talent.

Narayanan and Kapoor argue these failures aren’t patchable bugs but structural limits of trying to predict complex human behavior from historical data. People game metrics, data are incomplete, and “objective” automation can erase compassionate discretion.

For all these reasons, the authors recommend a heavy dose of evidence-based skepticism: don’t trust a predictive AI tool’s claims “unless they are accompanied by strong evidence.” In practice, that means demanding rigorous validation studies and audits before such systems are used to decide anything important – a lesson lawyers and policymakers can certainly appreciate.

AI Cannot Single-Handedly Fix Social Media

Broken organizations often reach for AI as a quick fix. In 2018 Mark Zuckerberg told Congress that Facebook’s content-moderation woes would be solved by AI. Six years later, moderation remains messy because the core questions are human and political, not purely technical.

Co-author Sayash Kapoor’s stint as a Facebook researcher gives Chapter 6 (“Why Can’t AI Fix Social Media?”) unusual insider depth.

Hyping the Risk of Out-of-Control AI

Many purveyors of AI snake oil delight in forecasting that Skynet is right around the corner. They suggest that Generative AI is so close to AGI (artificial general intelligence, meaning AI apps that can perform most or all tasks as effectively as any human being). They claim that we should expect a Terminator-style revolt any day now. This is the flip side of the AI-Is-Our-Savior pitch.

Narayanan and Kapoor have a more realistic view:

We’re not saying that AGI will never be built, or that there is nothing to worry about if it is built. But we think AGI is a long-term prospect, and that society already has the tools to address its risks calmly. We shouldn’t let the bugbear of existential risk distract us from the more immediate harms of AI snake oil.

AGI is definitely a possibility we should take seriously. The questions we should ask are when we might reach it, what it will look like, and what we can do to steer it in a more beneficial direction.

I agree with the authors that we can get giant benefits from AI safely if we build in the right safeguards. My only concern is whether our polarized politics will allow us to implement the safeguards the authors recommend?

The Bottom Line

From predicting case outcomes to drafting legal documents, AI promises abound. But as Narayanan and Kapoor compellingly argue, separating AI fact from fiction is now a critical skill for professionals.

AI Snake Oil has a few warts. The authors will never be candidates for the Nobel Prize for Literature. It is repetitive and seems too negative at times.

Despite its stylistic shortcomings, AI Snake Oil is a crucial guide for navigating the complex and often misleading landscape of artificial intelligence. It is essential reading for lawyers and policymakers struggling to make sense of how to deal with AI. It belongs on the shortlist with Ethan Mollick’s Co-Intelligence: Living and Working with AI and Richard Susskind’s How to Think About AI: A Guide for the Perplexed.

=======

Arvind Narayanan and Sayash Kapor, AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference (Princeton University Press, Princeton, and Oxford, September 2024). Available from Princeton University Press, Barnes and Noble, Amazon, Google Play (Ebook) and Audible (audio recording).