It’s just you, ChatGPT, and your human reviewer

TL;DR: It turns out that hundreds of human contractors could be reading your ChatGPT conversations in order to help improve the AI. Many users likely don’t know about this step—or how it works. It shows how much consumers are in the dark when it comes to AI data retention and utilization.

What happened: A stranger could be making upwards of $50 an hour to analyze your ChatGPT logs with a fine-tooth comb, per a new 404 Media investigation. The contractors, who work for Mercor—an AI data firm that suffered a major data breach this past spring—rate AI responses based on how well the chatbot fulfilled the user’s prompt (being sycophantic and using too many emoji is bad, for example).

The goal of these chat reviews is to make OpenAI’s models better, but there’s little public documentation on how the process works. Per OpenAI’s help center, humans (usually) only access convos if the user has the “Improve the model for everyone” setting toggled on—it’s on by default for all individual ChatGPT plans. One of its help pages briefly mentions human review, but the company didn’t respond to 404’s question asking whether it has explicitly told users about such routine inspection.

How the review works: If you allow ChatGPT to train on your convos, potentially any ordinary exchange could get a human once-over. All chats are first run through OpenAI’s Privacy Filter model, which attempts to remove personal info—but the AI lab notes that this model “can make mistakes.” Reviewers might also see additional info about a user, including what they’ve previously tried to use ChatGPT for and, in some cases, even where they live.

The data retention fine print: Your ChatGPT convos don’t necessarily disappear just because you delete them, either. According to OpenAI’s privacy policy, that data could remain in its systems if the “Improve model” setting is on and the data’s “already been de-identified and disassociated from your account.”

The haziness around how long AI labs keep your data—and who is looking at it—isn’t just a concern for consumers. Nvidia and Palantir, among others, are now reportedly considering scaling back their use of OpenAI and Anthropic models unless these labs can guarantee their intellectual property won’t be retained, per The Information.

Bottom line: It’s not just OpenAI that asks humans to review and rate user chats—Anthropic told 404 that it has a similar system. So before you unload every little detail about your finances and personal life to an AI chatbot, consider what the human on the other end will think. —WK