NerdKungFu storefront home page A licensed collection page built by the agents
Snapshot 3 (David)
Snapshot 4 (David)

Teaching AI to Check Its Own Work

Every supplier has its own idea of what a product spreadsheet should look like. Different columns, inconsistent terminology, sizes spelled several ways, and images that may or may not match the garment. Someone had to interpret all of it before a product could go into the store.

Doing that manually was slow and tedious. Worse, errors didn’t necessarily announce themselves. A mistake in a spreadsheet could become hundreds of incorrect product variations before anyone noticed.

Supplier spreadsheet (blur names)
A second, different format

I started building agents with Claude Code to do the repetitive work. But getting them to create listings was only half the problem. I needed to know that what they created was correct, so I built a system of gates, written procedures, automated checks, independent AI review, and human oversight.

One early version encountered a sweatshirt without its own product photo and helpfully borrowed the t-shirt photo instead. Dozens of listings ended up showing the wrong garment. The new rule is absolute: no artwork for that garment means no listing. Never substitute. Obvious in theory. Painfully obvious when it breaks in practice.

Mistake loop: a mistake ships, find the cause, fix it everywhere, make it a check

As the catalog grew, another problem emerged. The agents were repeatedly scanning the entire store to answer simple questions. Some lookups took around ten minutes and consumed substantial AI resources. The system was spending too much time checking things it should already know.

I built a compact text index that reduced those lookups to under a second. But a fast wrong answer isn’t much of an improvement, so the index verifies its own record count against the store. If the counts disagree, it stops and rebuilds itself. I explain the approach in my LinkedIn article, The Inviolable Index.

The Inviolable Index: up to 10 minutes before, under 1 second after