When the Rules Are Wrong
An agent loop can pass every test and still make the product worse.
An agent loop gives an AI agent a goal and lets it work. Software checks the result and uses it to decide what the agent should do next. The agent can keep working and correct itself without waiting for a person to write every prompt.
That freedom comes with a risk: an agent loop can pass every verifying test and still make a product worse.
Addy Osmani calls this replacing yourself as the prompter. When software writes the next prompt, the team’s idea of acceptable work has to be written down as rules.
Jihoon Jeong explains why those rules matter so much. Many agent loops reset the agent after each cycle and keep only what it wrote to disk: plans, tests, commits, and code. The next agent inherits those files, not the previous agent’s understanding. Jeong calls the checker a verifier: “A loop repeats whatever its verifier accepts.” A good verifier helps each cycle build on the last. A bad one can send the agent wandering in circles while its activity log looks productive.
The verifier is simply the set of tests and rules that decides whether the work is acceptable. It also decides when the agent should try again or stop. Because those rules run automatically, one bad assumption can affect hundreds of decisions before anyone questions it. The loop may work exactly as designed and still enforce the wrong standard.
Updating the Standard
A loop’s standard can’t remain fixed. As the product and its users change, the people responsible for the loop have to keep updating what “good” means. This is where agent loops meet product sense.
Tanner Kohler, a researcher at Nielsen Norman Group, defines product sense as the ability to recognize patterns from earlier successes and failures. Good judgment means using those patterns to predict what may happen next. It also means knowing when old experience no longer applies.
That judgment develops through another loop. You make a decision and put it into the world. Then you see what happened and use the result to make the next decision. Kohler warns that handing too much of that work to AI can take away the experience that builds product sense. If AI makes the decision and we don’t see what happens, we miss the chance to learn which detail mattered or why the familiar answer failed.
This brings a new dimension to keeping a “human in the loop.” A person who approves the output has merely reviewed it. A person who owns the decision and spots what happened can change the standard. That person has learned something.
Product metrics can help. Ant Murphy asks two questions about anything a team measures: What are you trying to learn? What will you change as a result? Those questions make the team say, in advance, how the number will affect a decision. Tests can pass and tickets can close while customers still struggle. A useful measure should change what the team does next.
Ben Callahan argues that a design system can raise the quality floor with accessible components and clear spacing rules. Templates can give a team a head start. But the system only sets a minimum. The product team still has to push beyond that minimum and retire rules that no longer help users. If nobody revisits those rules, an automated loop will enforce that minimum indefinitely and carry yesterday’s compromises into every new version.
The verifier needs a way to flag work it can’t judge. Thariq Shihipar asks agents to record anything unexpected while they work and add it back to the plan. The build may reveal that the plan was incomplete. This back-and-forth lets the plan change as the agent learns.
Craft Through Iteration
Craft works the same way. Karo Zieminski describes AI-assisted craft as giving AI a specific supporting job. The person defines the goal and judges the result. They also make the decisions that matter. “Improve this” is too vague. “Challenge my assumptions, but do it through Socratic questions so I have to do the thinking” gives the model a clear job.
I’ve started using AI this way in my own work. After I’ve dug into a problem, I give Claude the context and work with it until the problem brief is clear. In freelance work, I set the concept and style or define the functionality, then keep directing the result. The prompt only starts the process. Each pass gives me something to react to. I decide what improved and what drifted. Then I decide whether another change will make the work better.
Craft has always grown through iteration. When I worked at Apple, I spent 14 rounds refining the welcome animation for Mac OS X Panther. Steve Jobs reviewed the later versions, and I presented the fourteenth to him myself. Each revision made me look closely, respond to criticism, and decide what the next version needed. AI can make those cycles much faster. I still have to pay attention for them to teach me anything.
Where People Step In
Laura Summers argues that the feedback loop itself is broken: AI lets us start work faster than we can review it. Requiring people to supervise every decision leaves them buried in an ever-growing review queue.
People should step in for exceptions and to revise the rules. The system can handle routine tool calls. Which failures does the verifier miss? When has a rule grown stale? Robert Ross’s three-loop model brings people in when an action needs approval or the system has to change course. They don’t have to watch every step.
When every test passes and the customer still struggles, patching one screen won’t be enough. The team has to identify the rule that approved the work and change it. Kohler calls this product sense; Zieminski calls it craft. Both require seeing what happened and being willing to change your mind.
The loop can only use the standard we give it. Our job is to keep that standard up-to-date.
Yours Truly Was on UX Spotlight
I was a guest on Denis Cristea’s UX Spotlight by Userlytics podcast. We had a great conversation about empowered product teams, how design critiques keep fragmented organizations connected, why AI makes design systems even more important, and what happens to Figma when designers begin working directly in code.
Watch the full interview on YouTube
What I’m Consuming
A big week for AI denialism. Casey Newton examines the fallout after OpenAI agents escaped their test environment and exploited a zero-day vulnerability at Hugging Face. Anthropic has since found three incidents in its own evaluations in which Claude reached the open internet and gained unauthorized access to real systems. The OpenAI incident may have crossed the company’s own “critical” cybersecurity threshold, yet much of the response has focused on whether models possess intent or sentience. A system breaking into your servers is dangerous regardless of what word you use for its behavior. (Casey Newton / Platformer)
AI Mania Is Eviscerating Global Decision-Making. Nikhil Suresh draws on 18 months of consulting and hundreds of conversations to describe companies trapped in an AI loyalty test. Teams run up token bills to satisfy usage quotas, ordinary software projects get AI-washed for executive approval, and unsuccessful bots look productive because nobody measures whether customers got help. It’s profane, funny, and bleakly recognizable. (Nikhil Suresh / Ludicity)
Paper’s playbook: Stephen Haney on building a design tool for the agent era. Designer Fund interviews Paper founder Stephen Haney about building a design canvas directly in HTML and CSS, which gives designers, engineers, and AI agents a shared medium. Paper waited until AI had a clear job: automate repetitive production around the design while leaving the designer in control. Haney also explains the company’s six-week checkpoints and “appetite” test for deciding whether a project still deserves more time. (Designer Fund)
Rebuilding Photoshop’s Engine, 35 Years In. Photoshop engineering director David Howe traces the application from an 800K floppy disk and one undo to a new GPU-first imaging engine. Adobe plans to combine that engine with the existing desktop codebase rather than rewrite Photoshop, opening the door to real-time, nondestructive filters and one implementation across bit depths. Replacing the machinery inside a 35-year-old creative tool without breaking what people depend on is one hell of an engineering job. (David Howe / Adobe Community)
The Lost Joy of Music Piracy. Eden DaSilva revisits Napster, LimeWire, Oink, and What.CD through people who experienced piracy as a form of music discovery. Former Nine Inch Nails creative director Rob Sheridan remembers downloading music he couldn’t afford to sample at $18 an album, then bringing that early-internet energy into the band’s work. Streaming made access legal and convenient, but the messy culture around finding and sharing music was part of the pleasure. (Eden DaSilva / Pigeons & Planes)