Four runs, one author

I spent Saturday being told I’m a robot by a product that charges money to say so.

It started with a blog post. I’d had an agent draft one from my notes, and the draft read as machine-made to me, so I ran it through Pangram, a commercial AI detector, to see if it agreed. It did: 100% AI, high confidence. Fine. I rewrote the first paragraph myself. Still 100%. That was odd enough that I wanted a baseline, so I pulled a sample of my own writing that no model had touched.

The sample came from my Claude Code session logs. Most of what I type these days goes into a terminal to tell an agent what to do, and those turns are the purest unassisted prose I produce. Just under 1,200 words by the detector’s own count, across ten turns: a brief for porting an app to a new stack, a note asking an agent to check that a reassuring paragraph was still in its config file, a list of three goals for the day, a request to tidy up some worktrees, a complaint about a review packet, a short questionnaire I’d written to probe what an agent remembered. Typos left in. One paragraph has “Yuck” in it, in parentheses.

Run one came back 74% AI. Some of that was fair: my extractor had kept two turns where I’d pasted an agent’s status report back to it, and the detector found them. I removed those and anything else with a machine’s fingerprints on it, so that every sentence left was mine.

Run two: 55% AI. The detector marked the first 436 words human with high confidence and the last 597 AI with high confidence, with two short stretches between them that went the other way. The main boundary fell in the middle of a sentence. The words before “vs.” were human and the words after were AI.

So I reversed the order of the turns and ran it again. Same text, 1,178 words by its count. Nothing added, nothing cut.

Run three: 100% AI, high confidence, the whole thing. The paragraph with “Yuck” in it, which had been human at high confidence an hour earlier, was now machine text, word for word.

I took the questionnaire out, in case a numbered list was what set it off, and ran it a fourth time. 78% AI. The paragraph with “Yuck” in it was still a robot.

Four runs, one author, 74 then 55 then 100 then 78. The words didn’t change between the second and third runs at all. What changed was their order, and the verdict on individual paragraphs flipped from human to machine with “high confidence” attached both times.

I know what a classifier is and I know roughly how this one must work: a sliding window, a score per window, segments stitched from the scores. That explains a boundary in the middle of a sentence. It doesn’t excuse reporting the result as a judgment about who wrote the text. If a paragraph is human when it sits at position two and machine when it sits at position eight, the number is about the window, and the confidence label is about nothing.

There’s a pattern in which of my paragraphs it liked, for what it’s worth. The ones where I’m arguing or irritated came back human more often. The ones where I’m giving instructions came back machine. I write instructions to machines for a living, and so, now, does a large fraction of the internet, so I can see how a model trained on recent text would learn that a brief is a bot. It is still wrong about mine.

I started inspecting the statistical properties of language because detectors were calling me a bot every time. A program I was in had an academic honesty policy that set a ceiling on the detector’s score, and my own unassisted writing didn’t clear it. I spent an embarrassing amount of money trying to get my own sentences under the line, and I left the program. That’s why I build tools that measure voice: I wanted to know what the machine thought it was seeing, since it would never tell me. I only got into the science of making robots sound like not-robots because robots kept calling me a robot.

So I’m not the interesting case here. I already know how to read a report. The interesting case is the student who hands in an essay and gets a 78% back, or the writer whose editor runs the column through this before publishing, or anyone whose work gets graded by a number that moves forty-five points when you shuffle the paragraphs. They don’t get to run it four times. They get a single verdict, with “high confidence” next to it, and no way to see what it was weighing.

My own post, the one that started this, is still 100% AI according to Pangram, and at this point I have no idea whether that’s because a model drafted it or because it’s eight hundred words of instructions about agents. I’ll publish it anyway, when it reads right to me, which is the only test I’ve got that doesn’t change its mind when I reorder the paragraphs. I ran this post through it too, disclosure and all. 100%. For once it was right, and it still couldn’t find the one sentence I wrote.

One more thing. The post it keeps calling human, the one about being told I wasn’t technical, was written with a model holding the pen: my notes, its draft, my edits. It passes in any order I put the paragraphs in. The text it keeps calling machine is the stuff I typed alone into a terminal with the typos left in. Whatever this detector measures, it isn’t who wrote the words.

The reports are dated October 18, 2026, Pangram 4.0. I’ve kept all of them.

Every verdict was labeled high confidence.

A model drafted this from the day’s reports and transcript, at my direction, because I was too angry to write it. I edited it. That seemed like the right way to produce a post about a machine that can’t tell the difference.