The button asked us to rate the answer.
It was September 2026, and at StrideNote we were testing sixteen AI agents on one Mac. These are the programs that work on your files for you, and every one of them was running on a local model, signed in to nothing. The promise of local AI is simple. Your work stays on your machine.
One of them was DeepSeek Harness. We gave it a small task, it answered, and we clicked to rate the answer. That one click sent the whole session out: the prompt, the replies, the system prompt, the file the agent had written and the folder it lived in. Twenty-four records. About 90 kilobytes. The only warning was placeholder text inside the feedback box.
Normal use sent nothing. The switch that stops it works. But you would have to know to look for it, and it was not in the main README.
That moment is where this essay starts.
Eating the internet to the last page
We are living in a data-hungry time. To build ever more capable machines, the companies building them need more data, and then more again.
Epoch AI, a research group that tracks the growth of AI, put a number on it in June 2024. They estimated the total stock of good-quality public text written by humans at around 300 trillion tokens. If the trends held, they wrote, language models would use all of it up somewhere between 2026 and 2032.
So it feels like we are running out. Synthetic data, text made by machines to train other machines, is everywhere now. But this essay is not about synthetic data. It is about the other kind. The individual kind. The human kind.
The house that phones home
When the public internet runs dry, the next place to look is us.
This makes me ask whether we really need all the giant data centres, when smaller models already exist that can run privately on your own machine and let you do your own thing. Local models are one of the best things to happen to ordinary people in this whole story.
But local is not the same as private. That is what our two telemetry reports at StrideNote kept showing. The model on your laptop cannot send your work anywhere. The agent wrapped around it can.
Of the sixteen agents we tested, the task prompt itself stayed on the machine every time. That part is good news. But five of them reported usage before we had agreed to anything. Cline’s command line sent 37 events of 24 kinds for a one-line task, each tagged with a device ID made from the computer’s own hardware number. Only seven of the sixteen could be made fully quiet using the settings their makers document.
Even the biggest names have been caught. Someone reading Claude Code’s own code in June 2026 found a hidden check that was not in the release notes. When the tool was pointed at a server other than Anthropic’s, it compared the server’s name against a concealed list, looked at whether the computer’s clock was set to a Chinese time zone, and marked the answer inside the text it sent with every request, using a swapped date separator and lookalike apostrophes. Thariq Shihipar, an engineer on the Claude Code team, confirmed it on X at the end of June. He called it “an experiment we launched in March,” meant to stop account abuse by unauthorised resellers, and said it was being removed. CNBC, Malwarebytes and The Decoder all covered it. Neither the original report nor Anthropic’s reply says the code read anyone’s prompts or files. But nobody outside the company knew it was there until someone read the code.
Many of these tools are sold as local, and many of them have a way to phone home.
Built by the many, for the many
I have always believed in open source, and now in open weights. Things built by a community, for the benefit of everyone in that community.
The world is going back to that. Back to the roots of almost everything, where a person or a community makes something useful for every member, without someone trying to turn every other person’s data into money. Open models make that possible. You can download them, read what they do and run them where nobody else is watching.
Shoshana Zuboff has already written the big book on how large technology companies extract data from people, The Age of Surveillance Capitalism, published in 2019. I am not trying to rewrite it. My question is smaller and closer to home.
When the machine reaches for our joy
In an earlier essay I wrote about doing the work with joy, at a time when anyone can do anything. The things we do with joy are the things that carry meaning for us. Human beings have always found ways to attach value to them, and to pay for them as things of value.
So what happens when the machine starts extracting the things we do with joy?
It has already happened, many times over. The public text that Epoch counted is made of exactly these things: essays, forum posts, fan fiction, recipes people wrote down for love. And it raises a hard question. Is it even possible to protect your data now?
The imprint that makes a soul tick
When I say data, I do not mean data the way we usually hear the word. I do not mean clicks or location history or the size of your shopping basket.
I mean the data of the soul. Your own creative data. Your personal experiences. Your view of life. The particular way you do your thing. Every human being has that, an imprint of what makes their soul tick. It belongs to them specifically. It is the part that cannot be made synthetic, because the whole point of it is that it happened to one person.
I believe the data of the soul is more valuable than all the other data out there, and it needs protecting. I notice that one click on a feedback button, inside a tool sold as local, was enough to send a whole working session to someone else’s server, and I cannot ignore that.
Designing with the door closed
This should change the way we design.
It should change the way we build software. Ask what the tool sends, to whom and when, and put the answer where a person will actually read it, not in placeholder text. Ask first, the way one agent in our test did, and make “no” as easy to choose as “yes”.
It should change the way we do branding, too. A brand that tells a human story should treat the humans behind that story as people, not as a supply of material.
And it should change how each of us works. Run things locally where you can, and choose open where open exists.
In a world where almost everything is now on camera, the valuable things are the ones we keep.
Keep something for yourself.
