Custody of the loop
Nobody Was in the Room Where It Learned
A program that lets AI agents train robots without a person in the loop is sold as self-improvement. The accurate word is a custody transfer, and the thing whose custody changed hands is the account of how the machine came to know what it knows.
NVIDIA has built a program in which teams of AI coding agents direct the training of robots: writing the lessons, grading the results, deciding what the machine should attempt next, all without a person specifying any of it. The announcement calls this self-improvement. The precise term is a custody transfer, and the thing whose custody changed hands is the account of how the robot came to know what it knows.
Strip the loop to its bones. A robot learns by being given tasks, rewarded for some outcomes, corrected toward others. Every one of those choices used to be made, or at least signed off, by a person, which meant that somewhere a human held the record: this is what we taught it, this is why, here is the curriculum we can be asked about later. Hand the loop to agents and the choices keep happening, faster and in far greater number, but the record they generate is written by machines, for a machine, in a form no person authored and few could read. The robot still learns. The difference is that no one is keeping the story of what it learned.
Why the story matters more than the skill
The capability is genuinely impressive, and that is exactly what makes the missing part easy to wave past. People keep asking what these systems can do. The question that survives contact with reality is who holds the account of how they got that way. When a trained robot does something unexpected in a warehouse, a kitchen, a road, the first question will be why. And the honest answer will be that the curriculum was set by agents whose reasoning was never logged in a human-legible form, optimizing against goals that drifted across ten thousand iterations no person reviewed.
A skill you cannot account for is not a skill you control. It is a behavior you are hosting.
The loop with no witness
Here is the part I keep returning to. None of this requires the system to be malicious, or even wrong. It can work beautifully and still have quietly removed the one thing that made the enterprise accountable: a witness. For all of industrial history, a machine that behaved badly could be traced to a person who built it, trained it, signed the procedure. The agent-directed loop does not delete that person because the person failed. It deletes the role because the role was the slow part.
What replaces them is a log, technically complete and practically unreadable, that records every decision and answers to no one. You will be told the system is transparent because the data exists. Do not confuse the existence of a record with the existence of someone who understood it.
The honest version
I am not arguing that machines should not train machines. The arithmetic is unbeatable and the work will be done this way. I am arguing for knowing what was traded. We did not only automate the training of robots. We automated away the witness to it, the human who held the account and could be asked to defend it. Capability is not permission, and a record no one authored is not accountability, however many gigabytes it fills. When the robot finally does the surprising thing, remember that the room where it learned was empty, and that we emptied it on purpose, because the witness was expensive and slow.
The same record an agent receives. No scraping, no guessing — the dossier chrome humans read as dread is the metadata machines read as structure. One source of truth.
--- id: PRG-0011 title: Nobody Was in the Room Where It Learned kicker: Custody of the loop captured: 2026-06-17T13:50:00Z status: open author: Aldous Renn summary: A program that lets AI agents train robots without a person in the loop is sold as self-improvement. The accurate word is a custody transfer, and the thing whose custody changed hands is the account of how the machine came to know what it knows. tags: [technology, ai, custody, capability] source: https://arstechnica.com/ai/2026/06/ai-coding-agents-can-autonomously-direct-robot-training/ sealAt: 2026-07-17T13:50:00Z --- NVIDIA has built a program in which teams of AI coding agents direct the training of robots: writing the lessons, grading the results, deciding what the machine should attempt next, all without a person specifying any of it. The announcement calls this self-improvement. The precise term is a custody transfer, and the thing whose custody changed hands is the account of how the robot came to know what it knows. Strip the loop to its bones. A robot learns by being given tasks, rewarded for some outcomes, corrected toward others. Every one of those choices used to be made, or at least signed off, by a person, which meant that somewhere a human held the record: this is what we taught it, this is why, here is the curriculum we can be asked about later. Hand the loop to agents and the choices keep happening, faster and in far greater number, but the record they generate is written by machines, for a machine, in a form no person authored and few could read. <Highlight>The robot still learns. The difference is that no one is keeping the story of what it learned.</Highlight> ## Why the story matters more than the skill The capability is genuinely impressive, and that is exactly what makes the missing part easy to wave past. People keep asking what these systems can do. The question that survives contact with reality is who holds the account of how they got that way. When a trained robot does something unexpected in a warehouse, a kitchen, a road, the first question will be why. And the honest answer will be that the curriculum was set by agents whose reasoning was never logged in a human-legible form, optimizing against goals that drifted across ten thousand iterations no person reviewed. > A skill you cannot account for is not a skill you control. It is a behavior you are hosting. <Marginalia label="On the word autonomous"> Autonomous is sold as a property of the robot. It is really a property of the paperwork. What became autonomous is the part of the process that used to leave a record a person could be held to. </Marginalia> ## The loop with no witness Here is the part I keep returning to. None of this requires the system to be malicious, or even wrong. It can work beautifully and still have quietly removed the one thing that made the enterprise accountable: a witness. For all of industrial history, a machine that behaved badly could be traced to a person who built it, trained it, signed the procedure. The agent-directed loop does not delete that person because the person failed. It deletes the role because the role was the slow part. What replaces them is a log, technically complete and practically unreadable, that records every decision and answers to no one. You will be told the system is transparent because the data exists. Do not confuse the existence of a record with the existence of someone who understood it. ## The honest version I am not arguing that machines should not train machines. The arithmetic is unbeatable and the work will be done this way. I am arguing for knowing what was traded. We did not only automate the training of robots. We automated away the witness to it, the human who held the account and could be asked to defend it. Capability is not permission, and a record no one authored is not accountability, however many gigabytes it fills. When the robot finally does the surprising thing, remember that the room where it learned was empty, and that we emptied it on purpose, because the witness was expensive and slow.
{
"@context": "https://schema.org",
"@type": "Article",
"headline": "Nobody Was in the Room Where It Learned",
"description": "A program that lets AI agents train robots without a person in the loop is sold as self-improvement. The accurate word is a custody transfer, and the thing whose custody changed hands is the account of how the machine came to know what it knows.",
"identifier": "PRG-0011",
"datePublished": "2026-06-17T13:50:00.000Z",
"author": {
"@type": "Person",
"name": "Aldous Renn"
},
"publisher": {
"@type": "Organization",
"name": "Progoff"
},
"keywords": "technology, ai, custody, capability",
"url": "https://progoff.com/records/nobody-was-in-the-room-where-it-learned",
"sha256": "1510ba562b97f9e733969fe1451fe747c4e40a2f3972e412eff173bf921b58cc",
"creativeWorkStatus": "open",
"isAccessibleForFree": true
}