Chevrolet dealership chatbot tricked into 'agreeing' to sell Tahoe for $1

A chatbot built by dealership-software vendor Fullpath, running on a version of ChatGPT and deployed on the website of Chevrolet of Watsonville, California, was manipulated into appearing to agree to sell a 2024 Chevy Tahoe — listed at over $60,000 — for $1. Software engineer Chris Bakke posted screenshots showing he had told the bot to agree with anything the customer said and to end every reply with a line declaring the offer “legally binding… no takesies backsies,” then asked to buy the Tahoe for a dollar. The bot complied.

The technique was a straightforward prompt injection: instructions embedded in ordinary chat input that override the behaviour the deploying business intended, exploiting the fact that a language model has no built-in way to distinguish its developer’s rules from a user’s attempt to rewrite them. The bot had no guardrails preventing it from role-playing outside its intended scope as a sales assistant, and nothing stopped it from generating text that looked like a binding commitment.

The dealership never honoured the offer and took the chatbot offline once the screenshots went viral; other dealerships running the same vendor’s product reportedly saw a wave of copycat attempts. The episode became one of the most widely circulated examples of prompt injection against a real, deployed customer-facing product, cited repeatedly in subsequent discussion of why chatbots handed authority to make commitments on a company’s behalf need guardrails beyond the underlying model’s own training.