Last week, I wanted to buy some un-FDA-approved Ozempic online, because who doesn’t. The site was called Mochi. I paid $39 to become a member (turns out they don’t have a member’s lounge). Then I was told I needed to meet with a “Doctor of Medicine”.
To book this appointment — which I’d already paid for — I had to check a box that read: “By confirming, you are consenting to our notetaker AI transcribing this meeting.”
Wait, what? Isn’t this — purely technically speaking — a medical appointment?
But the only (completely-inconvenience-free) way to get the Ozempic was the appointment. And the only way to get the appointment was by checking the box. And whatever, I thought, the whole thing is a farce anyway -- I know it; the Doctor knows it; even the AI notetaker is probably sick of the roleplay at this point.
As I checked the box, I made a mental micro-plan: Say the things to get the script. Don’t say too many other things. Don’t want to accidentally mention the time I accidentally overdosed on un-FDA-approved Mounjaro and had to drink a protein shake on the floor of a Whole Foods car park to make it home. Best to strip all humanity from the interaction and become an optimally legible Patient of Medicine, just in case.
My dad was a GP in England (the equivalent of a primary care physician, for Americans). Patients loved him. And I loved them. He used to take me, as a jammy-fingered five-year-old, to visit them on his rounds — I held his stethoscope against the wheezing chests of old men; I got more involved in injections and blood draws than was probably safe or appropriate; I froze grapes in stolen liquid nitrogen, for educational reasons. My dad schlepped to visit patients for unnecessary examinations and real cups of tea. He prescribed sanitary pads to women who couldn’t afford them.
I think about this when I think about my own doctors. Dr Evans1, my university doctor — a kind yet formidable man who prescribed me anti-anxiety medications mid finals-meltdown, instructing me, questionably, to: “not read anything about the side effects because you’ll only convince yourself you have them” (later, as he was about to retire, Ebola broke out, and he went to treat patients on the frontline).
Dr Bennet, my GP in Oxford for almost a decade, sat me down at my last appointment and told me to stop trying to solve my severely disabled sister’s medical needs single-handedly. I had gone in with a folder full of medical records, requests for a thousand referrals, and anxiety to the point of tremors. I left with no medication and a profound sense of relief.
This kind of kindness works. Across medicine and rehabilitation, warmer, more collaborative doctor-patient relationships are associated with better adherence, better patient experience, and, in many cases, better health outcomes (obviously). There are hundreds of studies to this effect, but you don’t need to read them. In the same way you don’t need me to tell you that the love of your friends makes you happier and healthier. Relationships heal. So what happens when an AI notetaker enters the room?
Well, no biggie. Just imagine a neatly dressed man waiting in the corner of every medical appointment you’ll ever have again. He is quiet, polite, genuinely invisible.
Most of the complaints about him so far are that he’s bad at writing things down.
And, to be fair, he’s been bad at writing things down. He’s done things like record the wrong side for breast cancer; confuse “have epilepsy” with “does not have epilepsy”; write “patient consented”, when no such thing happened (x 100,000, see Saucedo v. Sharp Healthcare); and invent entire neurological exams from beginning to end.
But — as is the answer to most common gripes about AI — give it six months. Or give it a year. These are real harms, and I think they’ll mostly be fixed.
Other harms I’m less sure about. A friend of mine is a doctor at a regional hospital, where AI note-takers are about to save him hours a day on documentation. I told him about the AI transcription on my fake appointment. He told me about a patient of his with three months to live.
The patient needed an organ transplant that his hospital couldn’t perform. My friend took him aside. “Look,” he said. “The wait from here is going to be longer than you have. I’m not telling you to do this — but if I were you, next time this happens, I’d go to this specific hospital and tell them these exact things.”
The patient did. The patient lived.
“If we’d had AI in the room,” my friend said, “there’s no way I could have said that.”
I worry the man in the room will degrade the kind of discretion that kept my friend’s patient alive. The kind of human, unguarded, technically indefensible action that allows institutions to serve humanity, and not the other way around.
To this day, I remember a teacher conspiring with me to keep my little sister in an English-speaking school, even when we couldn’t pay the bills and “the board” was against it. I remember a train conductor letting me on without a ticket when I was panicked and late and somehow in charge of a gaggle of German children. I remember off-the-record reassurance from my surgeon that the procedure was “essentially risk free”.
The man might get involved here, and he will not be limited to your doctor’s office. Or your workplace. Or your school. Or your bank, or your train station, or your spouses’ AI therapist, or the man across the pool wearing those weird-looking glasses.
In a couple of years, I think asking where the man is not might be a better question.
One response is: good. Discretion is just a polite word for unaccountable power. If a doctor can go off-script to save a life, they can also go off-script to harm one. Record everything, and you get fairness.
But that’s not what this kind of legibility produces. As my aunt who lived in Soviet-era Bulgaria told me: authoritarianism isn’t just jackboots and rallies. It’s someone else in the room, always, everywhere. It’s the script you need to mentally switch on to account for them — how would this look on a transcript? It’s the distance that running script creates between you, your experience, and other people. It’s the inability to bend the rules in service of humanity. As Hannah Arendt wrote, this kind of domination “destroys private life as well”, and loneliness becomes unbearable when the self loses the “trusted and trustworthy company of its equals.”
The relationships that sustain resistance and vitality — the ones built on unguarded honesty, on being fully known, on the rare and hallowed ground of relational safety — are harder to form in a world where someone else is so often in the room; where so much of relational life is captured by a third party, whose interests are not entirely your own. By the time a despot tries to concentrate power with transformative AI — by the time someone tries (searches Project 2025) “loyalty purges” and “unitary executive control” — I hope our muscle for aliveness and rebellion has not been been atrophying for too long.
There is a version of all of this that’s fine, obviously. Hopefully the man is taught discretion. Hopefully we build rooms he can’t enter. Hopefully the organ transplant story becomes a sad artefact of a more tragic time, and we’re all better off for it.
But I notice that when I checked that box — when I consented to the man in the corner of my fake medical appointment — I didn’t think about any of this. I just checked it with a flicker of annoyance. The way you check a cookie banner. The way you agree to Apple’s terms of service. The way you stop switching off a camera.
I got the Ozempic, by the way. I feel sick.
Doctors’ names changed to protect anonymity
I read this after the “Got Wrecked” post. I was expecting less. This feels readable, purposeful, and satisfying as-is.
SUCH A GOOD POST! Thanks for discussing important questions publicly.