“I only use AI to polish my care notes.”
It’s a phrase I hear more and more. On the surface it sounds harmless — a care worker copying an entry into ChatGPT to tidy up the spelling and grammar before it goes into the system. No harm done. Right?
Not quite.
What’s actually in a care note?
Care notes document what happened during a visit, shift, or interaction. They’re written to be anonymous — no full names, just “the service user” or initials. But they contain details: behaviours, moods, medication responses, personal circumstances, family dynamics. Over time, a pattern builds.
When a worker pastes note after note into the same AI conversation — or even across separate sessions with the same account — that AI may be building a surprisingly detailed picture of the individual being cared for. Conversation memory, inference from context, and the accumulation of detail mean that what started as tidying the grammar becomes something much closer to processing personal health data.
Where does that data go?
Most consumer AI tools store conversation data on servers outside the United Kingdom. Under UK GDPR, transferring personal data to a country without an adequate data protection framework is a breach. It doesn’t matter that the notes looked anonymous when they were pasted in.
There’s a second issue: many providers use user inputs to train future versions of their models. That means details about your service users — however indirectly captured — may end up embedded in a commercial AI system. The right to erasure under GDPR becomes almost impossible to enforce once data is woven into a model’s weights.
The risk is growing, not shrinking
As AI models become more capable, their ability to infer identity from partial information increases. What is genuinely unidentifiable today may not be tomorrow. Employers cannot assume the risk stays static.
Governance is the answer — not a ban
The solution isn’t to prohibit AI use. It’s to govern it properly, using what I call the triangle of governance — three things that only work when all three corners are in place:
- Policy — A clear, written AI policy specifying which tools are approved for use with service user data, and which are not. “Don’t use ChatGPT for care notes” needs to be explicit, not implied.
- Training — Staff need to understand why the policy exists, not just that it does. A worker who understands the GDPR risk is far less likely to cut corners than one who has simply been told no.
- Oversight — Spot checks, audit trails, and a culture where workers feel comfortable raising concerns. Policy without oversight is just a document. Training without oversight drifts.
If you’re responsible for care delivery, AI governance isn’t optional any more — it’s part of your duty of care.