Humans Read Some AI Chats. So, Is ChatGPT Private?

What happened

What is Project Lily?

On September 14, 404 Media reported that OpenAI had hired hundreds of contractors to read real user prompts, sometimes whole conversations, and rate and critique ChatGPT’s replies. The effort is codenamed Project Lily. Contractors do not see usernames, and OpenAI says it tries to strip personal information first. The company acknowledged that sensitive details can still get through.

International Business Times (September 16, 2026), summarizing the report, adds detail. Reviewers rate several versions of a reply on a scale of 1 to 7, and some excerpts arrive with a summary of the user’s saved memories, which can include personal context and sometimes a location. OpenAI says a privacy filter removes names, addresses, emails and phone numbers, but it can miss less common identifying details. One goal of the work was making ChatGPT less sycophantic and less full of AI-speak.

The story is still moving. A proposed class action filed September 16 in a U.S. district court in San Francisco claims users were not told, according to Open Class Actions. Those claims are unproven, and the first case management conference is set for December 18. On October 16, WBUR’s Endless Thread devoted a full episode to it, noting that many people assume their chats are private.

What’s actually true

So, is ChatGPT private?

Private from the public, yes. Private from the company, not entirely. Here is what OpenAI’s own help page on model improvement says today:

• On individual plans (Free, Plus and Pro), OpenAI may use your content to train its models unless you turn off Improve the model for everyone in Settings, Data controls.
• Opting out covers your new conversations. Per the reporting, earlier chats can still be reviewed.
• Tapping thumbs up or thumbs down may let OpenAI use that entire conversation, even if you opted out.
• Temporary Chat is not used to improve models, does not show in your history and does not create memories.
• Business, Enterprise and Edu accounts are not used for training by default.

This is not only a ChatGPT question. Google’s Gemini Apps Privacy Hub (updated October 13, 2026) says trained reviewers read a subset of conversations, keeps reviewed ones for up to three years even if you delete your activity, and advises you not to enter anything confidential you would not want a reviewer to see. Anthropic told 404 Media it also uses human review, for Claude users who allow it in their settings. A Stanford HAI study (October 2025) of six major AI developers found all of them train on chats by default.

Both sides

Should humans review AI chats at all?

The case for human review. Machines cannot grade themselves well. People are how AI companies catch answers that are wrong, unsafe or simply flattering. Some of Project Lily’s goals, like cutting sycophancy, are things users asked for. OpenAI and Google both say identifying details are filtered or disconnected from accounts before reviewers see them, and every major app offers a way to opt out.

The case against how it is done. Privacy researchers argue the defaults point the wrong way. The Stanford HAI team recommends that training on chats require clear opt-in consent, plus filtering personal details by default. Critics note that filters miss things, opt-outs are not retroactive, and many people never open their settings. Some users in the reviewed prompts had even asked ChatGPT to keep things confidential, International Business Times reported.

Both sides agree on one thing: you deserve to know. That is the line Pixel Pinkscape holds too. AI is a tool you direct, and you get to decide what it sees.

Your takeaway

How to make ChatGPT more private: a 5-minute check

1. Turn off training. ChatGPT: Settings, Data controls, “Improve the model for everyone.” Gemini: turn off Keep Activity. Claude: check the model-improvement setting under Privacy.
2. Use Temporary Chat for anything sensitive. A health question, a money worry, a personal message.
3. Skip the thumbs. On ChatGPT, feedback can share the whole conversation.
4. Review your memories. Delete saved memories you would not want summarized for anyone.
5. Swap details for placeholders. Write [CLIENT] instead of a name and [AMOUNT] instead of a number. The answer is just as useful.

I want your help without sharing private details.
I will use placeholders like [CLIENT], [CITY], [AMOUNT]
and [DATE] instead of real information.

My situation: [describe it with placeholders]
What I need: [a reply, a plan, a summary]

1. Help me with the task using only the placeholders.
2. If you need more context, ask me a general question
   instead of asking for a name, number or address.
3. At the end, list the spots where I should fill in
   real details myself before I send or use this.

If you run a business, the stakes are higher: client notes, invoices and staff details often end up in chat windows. A short written AI policy, plus a team plan with training turned off, solves most of it. Our AI training for teams walks your people through exactly these settings and habits, and our guide on whether AI is safe covers the other everyday checks.

Free resources

Free resources for keeping your AI chats private

• OpenAI Help: How your data is used — the official, free explanation of the training setting, Temporary Chat and feedback.
• Gemini Apps Privacy Hub — Google’s free guide to human review, Keep Activity and how long chats are kept.
• Stanford HAI: Be careful what you tell your AI chatbot — a free, plain-language summary of how six AI companies handle your chats.
• The Pixel Pinkscape Dreamers community — a free Facebook group where you can ask how others set up their AI tools.

FAQ

Frequently asked questions