Building AI systems that fail honestly
Most writing about AI engineering describes what worked. These are notes on what broke — specifically, the failures that passing test suites did not catch, and the disciplines that would have caught them sooner.
I build AMCO, a campaign orchestrator that generates marketing material end to end: strategy briefs, channel copy, content calendars, visual direction, email sequences, and the export files a client loads into their own tools. It is a real system with real output going to real businesses, which means its failures are the useful kind — the ones that cost something.
The recurring theme in everything below: a language model will produce confident, well-formed, plausible output whether or not it has any basis for it. Almost every hard bug I have shipped traces back to trusting the shape of an answer instead of its substance.
-
One character kept my domain dark for nine days
A GitHub Pages CNAME with an extra zero. DNS resolved, the build was green, the hosting was correct — and the site 404'd, because none of those checks can see a misspelled name inside a file.
-
Design for text you didn't write
The same layout bug, shipped twice in the same component. When a model writes your copy, every length assumption in your CSS becomes a bug waiting for the right output.
-
The mutation test that proved nothing
Breaking your code to confirm a test fails is good practice — and it has three ways to lie to you. Twice now, mine did.
-
The tests passed. The feature was broken for every user.
Five tests, all green, covering a feature that was uniformly broken in production. The mocks asserted what the code did with data it was given — and the actual defect was that the model was never given anything.
-
A fabricated citation is worse than no citation
If you ask a model to cite its sources, it will. Whether or not it read them. The fix is not a better prompt — it is refusing to trust the model's URLs at all, and matching them against what you actually retrieved.
-
No email platform imports a sequence
I planned an email export for a week on an assumption that turned out to be false in the first ten minutes of reading vendor documentation. A note on checking the world before designing against it.