Lesson: Why your worry is rational: three things to actually worry about

Star

Aisha teaches eighth-grade English at a public school in Portland. Last Tuesday her principal sent an email: starting next week, every teacher will use a new AI tool to draft progress reports. Aisha is not against the tool. She wants to know what happens to a student’s name, a student’s grade, a parent’s phone number when she types them into a box on a screen she did not choose.

The worry has a shape. It is not “AI is scary.” It is what happens to the things I type when I type them into something I do not control? That is a specific question. It has specific answers. By the end of this lesson, you will be able to name three of those answers in plain language and separate them from the headline-noise version of “AI privacy.”

Aisha is one shape of the wider Sarah pattern in this track: a non-technical person being handed an AI tool whose data-handling she did not choose. The rest of the lesson speaks to that pattern. Aisha returns as the recurring concrete case; “you” is whoever is reading.

Three things, not infinity

When the word “privacy” enters the news, the coverage usually arrives as a cloud. Surveillance, breaches, training data, deepfakes, identity theft, what your kids’ apps know, what your car’s app knows, what your insurance company is doing with the car’s app. By the time the cloud disperses, the reader is left with a feeling, not a list. Feelings do not lead anywhere. Lists do.

There are not infinity privacy concerns. For most of what Sarah does with AI tools, there are three. Naming them is the work of this lesson. Acting on them is the work of the rest of the track.

The three:

1. Surveillance. Every interaction with the tool is observed. What you type passes through real systems on the way to the model and back.
2. Storage and leak risk. What the tool keeps about you can be exposed later, by accident or by attack.
3. Vendor lock-in. The rules you signed up under can change after you have already trusted the tool with your information, and getting out is harder than getting in.

These are not new. They predate AI. What AI changes is the intimacy of what now sits inside each of them. Aisha is being asked to put student names and parent phone numbers into a system that will see all three of these forces apply.

Worry 1: Surveillance

When Aisha types a student’s name into the new tool, the text does not stay on her laptop. It travels across the network to the AI company’s systems, where a model processes it and a reply comes back. At several points along that path, the message is handled by software written by people whose job is to keep that software running. Some of them can read it if they have a reason and the access. Some logging systems record pieces of it automatically. The path’s specific shape (which networks, which handoffs, which data centers) differs from vendor to vendor; Phase 2 of this track will walk a real example end-to-end. The point of this lesson is the upstream one: the moment Aisha presses Enter, what she typed is no longer only on her own machine.

This is not a conspiracy theory. It is how the system works. The Electronic Frontier Foundation, a long-running digital rights group, makes the point clearly: tens of millions of people use chatbots to brainstorm, test ideas, and explore questions they might never post publicly or even admit to another person, and these conversations reveal people’s most sensitive information. EFF’s worked examples sit at the heavier edge of the spectrum: someone asking sensitive questions about their health, their safety, or how to escape a difficult personal situation. Aisha’s case is on the everyday edge of the same spectrum: a student’s name plus a grade plus a comment about that student’s behavior is, in aggregate, a small intimate fact about a child. The pattern is the same; the gravity is calibrated to the person. A single chat thread can expose the kind of intimate detail once locked away in a handwritten diary, whether the diary entry is heavy or routine.

Mozilla’s privacy research team frames this as the constant surveillance worry: many features of modern AI tools require constant listening and constant video processing. Wearable AI is the extreme version (a person in a coffee shop, glasses on, quietly asking the model who that person across the room is). Chatbots are the everyday version. Each interaction is observed.

What Sarah can do. Surveillance is the easiest of the three to act on, because it is the most upstream. The shorter answer: if it is sensitive, do not paste it into a tool whose data-handling you have not checked. The slightly longer answer: prefer tools that let you opt out of training, prefer tools with shorter retention windows, and treat the chatbot like a postcard rather than a sealed envelope. Postcards are fine for “remind me what year the Magna Carta was signed.” They are wrong for “here is a parent’s full name and phone number.” Phase 2 of this track will teach the data-flow trace in detail; for now, the rule is enough.

Worry 2: Storage and leak risk

The second worry is what happens after the tool has the data. Surveillance is about what gets seen on the way. Storage is about what gets kept.

Mozilla’s privacy research is direct about this: AI chatbot requests usually need to be processed over the internet on a cloud server and may be stored by the company. Beyond what the company plans to do with that data, nothing on the internet can be 100% secure or private. Your conversations with an AI chatbot are valuable to hackers, which increases the risk of that data being hacked in transit or from where it is stored. The likelihood depends on how strong the company’s security measures are.

There is also a softer version of “leak” that is not a hack at all. Mozilla notes that exchanges with an AI chatbot may also be reviewed by humans. The example: if a conversation is flagged for potentially violating the tool’s policies, or when training is mandatory, a real person at the company can read it. So the content you provide to a chatbot could be seen by a person, either by design (because it is subject to human review) or by accident (because the system was attacked or the data was exposed).

For Aisha, the consequence is concrete: a student’s report card comment is now, possibly, in a system that has been read by a reviewer she will never meet, and that could be exposed by a breach she will never see coming. The likelihood of any individual record being exposed in any given month is small. The likelihood that some records in a large company’s database are exposed eventually is not.

What Sarah can do. Treat storage-and-leak as a “what if this leaked tomorrow” test. For each thing you are about to paste, ask: if this conversation appeared in a news article next year, with my name attached, what would the consequence be? If the answer is “nothing serious,” paste away. If the answer is “I would lose my job” or “my client would have a real complaint” or “a child’s parent would be angry,” do not paste it. The test is not paranoid. It is the same test a careful person already applies to email. The tool is a new surface; the test is old.

Two related moves help. First, learn where the tool’s settings let you shorten retention or turn off training (Mozilla maintains an ongoing series on these settings for the most popular tools). Second, prefer tools whose architecture limits what they can store in the first place; lessons 5.1 and 5.2 of this track teach how to recognize those tools.

Worry 3: Vendor lock-in

The third worry is the slowest of the three to bite, and the one Sarah is least likely to think about on her own. The rules under which she trusted the tool can change after she has trusted it.

Mozilla’s researchers put it this way about AI vendors broadly: by the time the team analyzes one product, there is a new update that sends them back to the drawing board. The tools themselves are moving targets. The legal documents that govern them are moving targets too. The team notes one major chatbot provider has 18 privacy documentation links covering privacy policies, usage policies, terms of use, model cards, system cards. “I’m more confused than when I started,” one of the researchers writes. If a privacy researcher with a working knowledge of the field is confused, the question of whether Aisha can confidently track changes across 18 documents over a school year answers itself.

There are three pieces of vendor lock-in to be aware of, in plain language. First, the tool can change its policy. A vendor’s training-data default can shift from “off” to “on,” and the opt-out can become a setting buried several menus deep. Second, the data you have already given the tool does not unsend. Some of it can be deleted on request; some of it cannot, because it has already been used to train a model that has already learned from it in a way that is not easily undone. Third, the longer you use a tool, the more your routines depend on it, which makes it harder to leave when the policy you don’t like is the one you have now.

This is not a conspiracy either. It is the general pattern of any service whose pricing depends on dependency; AI tools are no exception. The point is not to refuse all such services. The point is to know which lever the vendor is holding, so the choice to use the tool is informed rather than absent.

What Sarah can do. Vendor lock-in is the worry Sarah cannot solve in the moment of using the tool. She solves it earlier and later. Earlier: prefer tools whose privacy posture is architectural (the system cannot retain certain data because of how it is built) over tools whose posture is promise-based (the company says it will not retain the data). Promises are easier to change than architectures. Later: every few months, check whether the tool’s policy has changed in a way that affects you, and be willing to switch when the answer is yes. The five-question rubric in Phase 4 is the tool that makes this check fast.

Common pitfalls

Three mistakes a first-time privacy-aware reader is likely to make in the next week:

Treating “privacy mode” as a privacy guarantee. Many tools now have a “temporary chat” or “incognito” mode. Read what each one actually promises. Mozilla’s research finds that these modes typically protect against some specific things (such as the conversation being used for future training, or being saved into your visible history) while still storing the conversation for some period of time for safety or abuse review. The specific protections are real; they are narrower than the word “incognito” suggests. Phase 4 of this track teaches the rubric for reading these claims quickly.

Confusing “they cannot identify me” with “they cannot reach me.” Many privacy choices are about reducing identifying information. They are not the same as not being reachable. A tool can know a great deal about how you use it without knowing your legal name. A breach of that data can still affect you. The two threats are managed differently, and lessons in Phase 3 and Phase 4 will keep them separated.

Stopping at the first privacy setting you find. The first toggle is rarely the whole picture. Mozilla’s research enumerates several layers of settings that interact: account-level settings, app-level settings, browser-level settings, and tool-specific feature toggles (like training opt-out and conversation memory features). Phase 4 will teach a five-question rubric that lets you map them in under ten minutes per vendor, but the habit is the point: never assume the visible setting is the only setting.

What’s next

This lesson named three worries. The next lesson, 1.2, will turn the three into the seed of a personal threat model. The exercise is one paragraph: what am I protecting, and from whom? You write the paragraph in your own situation, in your own words. You will revisit and revise it in lesson 6.6 of this track, after every Phase has added a piece of the model. The first paragraph is the seed; the final plan is the harvest.

Aisha now has the same three nameable concerns about her new school AI tool that you have about whatever tool you are about to use. She is no longer staring at an undirected cloud. She is looking at three specific questions she can ask, and three specific protections she can choose. That is the difference between worry and action.

The cloud disperses when you name what is inside it.