Build log 1: The five checks I put around an AI workflow before it touches real work

This is the first entry in a build log. I build AI workflows end to end, and I write down how, including the parts that went wrong. Today: the five checks I put around a workflow before it touches real work.

What this log is, and who it is for

I am a platform engineer. For years I ran production systems: Kubernetes, databases, cloud cost, migrations. Now I use the same discipline to build AI workflows for teams: an agent or pipeline that does a real job, such as triaging requests, drafting documents or reconciling data, with people supervising the steps that matter.

This log is for the people who will own one of these workflows: operations and product leads, engineering managers, founders and the engineers they work with. You do not need to be an AI specialist. You need to know what “good” looks like for your process, and this log shows how to turn that into something a workflow can be held to.

The problem

Most teams can get a convincing AI demo working in an afternoon. The hard part comes after: the demo meets real requests, real documents and real exceptions. Answers drift, nobody is sure who approved what, and trust quietly drops. The workflow gets switched off, or worse, kept running without anyone watching it.

The cause is rarely the model. It is the missing engineering around it: a clear job, known sources, a human in the right places and a way to notice when quality changes.

The goals

• A workflow a team can rely on, not a demo that needs its author in the room.
• Quality you can see. A change that makes answers worse shows up the same day.
• People in control. Anything that matters has an approval step, and anyone can reach a human.
• Something you can explain. To a colleague, a customer or an auditor, in plain words.
• A log you can reuse. Every entry ends with a check or a template you can take to your own workflow.

The model is the smallest part of a production AI workflow. What decides whether a team can rely on it is everything around it. I treat these checks the way I treat a deploy pipeline: boring, written down and run every time.

The five checks

1. A narrow job. One task with a clear end, for example “draft a reply to a support request” and not “handle support”. A narrow job can be tested. A broad one can only be hoped for.

2. Approved sources only. The workflow answers from documents and systems I name, and says so when it does not find the answer. If it cannot cite where something came from, I do not ship it.

3. A person for anything that matters. Drafting is automatic. Sending, paying, deleting or promising something needs a human approval step, and the workflow cannot skip it.

4. An easy handoff. The person on the other end can always reach a human in one step, and the workflow tells them when they are dealing with AI. It is also the simplest way to meet the transparency expectations in rules such as the EU AI Act, which I treat as a practical review and not legal advice.

5. A list of real questions I re-run. I collect twenty to fifty real questions, with the answer a good person would give, and run them on every change to the prompt, the sources or the model. A change that makes an answer worse shows up the same day, not three weeks later in a customer message.

Why this order

The first two checks shrink the problem. The third and fourth limit the damage when something still goes wrong. The fifth stops quality from drifting without anyone noticing. Skipping any one of them is how a demo that worked becomes a system nobody trusts.

What comes next in this log

I will build one small workflow in the open and report what each check caught: what the question list found, where the handoff was missing, and what it cost to run. Failures included.

If you have a workflow that works in a demo and not yet in production, I am happy to look at it: book a call. More on how I work is on the Work with me page.