I recently watched a couple interviews with Cory Doctorow discussing AI. He explained the concept of the "Reverse Centaur" from the title of his upcoming book.
Here's how he defines that term on his blog:
In automation theory, a “centaur” is a person who is assisted by a machine. You’re a human head being carried around on a tireless robot body. Driving a car makes you a centaur, and so does using autocomplete.
...a reverse centaur is a machine head on a human body, a person who is serving as a squishy meat appendage for an uncaring machine.
...
Obviously, it’s nice to be a centaur, and it’s horrible to be a reverse centaur. There are lots of AI tools that are potentially very centaur-like, but my thesis is that these tools are created and funded for the express purpose of creating reverse-centaurs, which is something none of us want to be.
I think that's such a potent framing, and it matches my experience with GenAI tools like Claude and Copilot. I actually find them pleasant to use when I'm the head of the centaur. There are a lot of tedious little tasks involved in software engineering that I’m happy to outsource to an eager assistant. I maintain my autonomy, intelligence, and dignity in this arrangement.
But I've also encountered advocates in the software industry for the reverse centaur arrangement, where AI agents are autonomously producing all the code, with human engineers essentially grading their work and scolding them. That arrangement I find depressing and alienating.
In Doctorow's interview with Democracy Now!, he mentions another concept called the "accountability sink", which means the poor human who gets blamed when the AI agents make mistakes that get past the human's red pen.
An often used example of "good AI" is software that can spot hard-to-find tumors on X-rays, even the tumors that a human radiologist missed. Doctorow warns against turning radiologists into accountability sinks:
...if the AI misses a tumor, this will be the human radiologist’s fault, because they are the "human in the loop." It’s their signature on the diagnosis.
The radiologist’s job isn’t really to oversee the AI’s work, it’s to take the blame for the AI’s mistakes.
Over many years of working in the software industry I've seen the pattern of a certain kind of shitty job where the boss wants to hire an accountability sink for a collection of inexpensive offshore developers. A kind of designated responsible party for low quality output.
Imagine if instead of being the accountability sink for contingent workers in a distant timezone, you're now responsible for what the chat bot living in a data center produces. Yikes!
I'll try to hold out being the top end of the centaur for as long as I can.
