Artificially Designed · Issue 19: Say What It Is

Issue 19 · Say What It Is

An agent that hands work back to a person owes them the reason, the evidence and what it could not check.

Hi friend,

Last week I asked what word in your product means two different things to two different people. Hold onto yours, because that was the first of three layers, Design for Meaning, and this issue is the second, Design for Context.

Context has two sides. For the agent, it is what travels from one step to the next. For the person, it is what is on the screen at the moment the work comes back to them, and that second side is where human-centered design lives in an agentic workflow. Most builds spend everything on the first and leave the second to whatever the log happens to say.

I spent the week on that second side, inside Orinyx. I posted one of its failures where developers could pick it apart, merged the first changes that came out of it on Sunday, and did all of it with two AI coding tools at once.

This week

The build I see most treats human-centered AI as keeping a human in the loop, which usually means an approve button at the end. Anthropic showed what that button is worth in August. Users approve 97% of permission prompts in Claude Code, and in a test with 1,053 paid professional testers, people caught 13.6% of the clearly dangerous commands slipped in among them. Those are Anthropic's figures for its own product, from testers who knew they were being evaluated, so hold them loosely. Read it

The end of the line is the last place to start. Human-centered design starts before the interface, with four questions. What does nobody want to do anymore? Automate that. What does the person do that nothing else can? Amplify that. Where does the automation run out? Design that handover on purpose. And who owns the system once it is running? Name them.

I wrote up the full approach on DEV, which starts from your role and what you want to keep doing, hands off what you don't, then points the AI at the biggest pain point in the work you kept. Designing Around What You Want to Keep

I learned how much this matters from one of Orinyx's own failures. In testing last week we sent it an order for acetaminophen at 1,500 mg every four hours, and six doses in a day is 9,000 mg. Orinyx missed the total, lost the frequency along the way, and still answered in reassuring language when its checking was incomplete. A confident output looked exactly like a verified one.

So I posted the failure in two developer communities on Reddit and asked people to help test it. The answers ranged from skeptical to precise.

Letting the people who will use a thing, and doubt it, challenge it while it is still soft is participatory design. It also gave me five moves, and none of them belongs to healthcare alone.

Decide who judges before you build. For every decision in the workflow, pick its owner, a person's expertise, a deterministic rule or a model's interpretation. Use a model where interpretation is needed and a rule where a rule already exists. In Orinyx the arithmetic and the label rules are deterministic, and no language model takes part in a verdict.

Make "I can't verify this" a real answer. On Sunday we merged the changes that stop the reassurance. The acetaminophen order now gets an answer that leads with not verified and says why. An order written in millilitres with no concentration names the missing fact. Each result from the engine carries a coverage line saying, drug by drug, whether Orinyx had a dose rule to check against, and a number the parser could not place is listed as unread instead of disappearing.

Check the work, not the presentation. The result now quotes the whole label sentence it was checked against, never a slice. And the next change, decided on Sunday, does the daily arithmetic for plain fixed schedules, says the assumption it made, round-the-clock dosing and the total it reached, and flags anything above the label's daily maximum. As-needed orders, ranges and "while awake" still abstain, because there the arithmetic would be a guess. It lands in a later release, once the as-needed handling is airtight.

Keep people in the whole build, not the last meeting. Developers testing, physicians designing and a named owner watching once it runs each catch a different failure, and an approval meeting at the end catches almost none of them.

Treat feedback as architecture. The boundary cases, the duplicate calculation and the regression set from those threads are going into the test plan I am building with Codex, which is where One Slot picks up.

My reason for the arithmetic decision was simple when I gave it. "Orinyx should never drop the ball. It should say what it is." Human-centered AI means giving the people around a system the information, the authority and the chance to make their part count. A stated assumption gives them something to push back on, and a reassuring silence gives them nothing.

The One Slot Rule is that every job gets one slot, and a new tool has to take it from whatever holds it now. This week I am holding myself to it, because I am running two AI coding tools on one product, and the only way that works is if each one has its own slot.

Claude Code, running Claude Fable 5.1, holds the engine. I have used it for a long time, it has all the context, and it does the orchestration and builds the main components. Codex, in ChatGPT and running GPT-6 Astra, holds the Arena, the part of Orinyx being built around what developers told us they want from an evaluation tool, and there is more to come on that. Codex also fills in the blanks, brainstorming with me, go-to-market research, and helping me write the test scripts. The four changes above came from tests Codex helped me script and Claude Code built into the engine.

What is the job. Two jobs, kept apart. Claude Code builds and orchestrates the verification engine. Codex builds the Arena and helps me plan what we test.

Can it do that job, not can it do more. This is the first time I have used Codex for real work rather than a quick test, and I picked it for a plain reason. The classes I teach can only use Codex or Gemini, so if I am paying for one, I want to know how it holds up on a real build. The part I did not expect to love is talking to it. I went through the feedback from my survey questions with it out loud and planned test questions around it, and it felt like collaborating with a person.

What does it touch that is hard to undo. Each other's work, if I let them. Claude Code has everything and Codex has a slice, so they work in separate places, Codex on its own branch, and nothing reaches the engine until I have read it and merged it myself. I am in the middle, carrying messages between the two.

What would have to be true to let it run unwatched. More than is true now, so I watch both. It is my own version of an AI judge, with one model helping write the test, the other changing the code, and me deciding whether either of them is right, which is the only arrangement the developers above would trust. Inside the product, no model judges anything. Would I recommend two tools on one build? Maybe, depending on the build. Right now the orchestration is doing a good job, I do not have agents set up in Codex the way I do in Claude yet, and eventually Stellar Agents will fit in there. More to come as it runs.

Three pieces from the last five months that read like a design brief for the same problem. I could not find a fresh agent-oversight story from the past seven days that I could trace to a primary source, so I would rather give you these than pad.

Run the ten-second return test on one workflow you already have. Find the last thing your agent handed back to a person, whether that is a notification, a summary, an approval request or a queue item, and open it the way the recipient sees it, with nothing else on the screen. Start a timer. In ten seconds, can you answer five questions without opening anything else?

Any question you cannot answer in the time is a missing fact in the handback, and it is a fact the agent had and did not pass along. Add the most useful one first, which is usually number five.

Read your own handback as a stranger.

WHEN TO USE IT. Before you add a second agent to a workflow that already hands work to a person, and any time a teammate says "I just click approve."

WHAT IT DOES. It turns the handback into something you can argue about, one missing fact at a time, from the seat of someone who was not in the room.

TIP. "Mark every answer you are guessing at" is the line that matters. Where the model guesses is exactly where a tired human will guess too.

Lex

My class

This week, online and across the US

Kansas City

Hackathons

Startup calls

Tool verdicts, templates and reference material live on the site at charmthirteen.com.

Next week is Design for Trust, the last of the three layers, and the one that decides whether anyone gets to rely on the other two.

Software engineering is still here (and so are Python developers, shout out to my fellow Python devs), and like everything else, the role is changing shape. One new shape is the forward deployed engineer, who can walk into any industry, understand why a business problem exists, and build the right fix, which is often not an agent. That is what Houston We Have a Build teaches, a hackathon bootcamp I am running with SOTV Creators Collective for middle schoolers, high schoolers and early college students, enrolling for 2027.

Know a young person who would want it? Email [email protected] or pass my information along.