Dave Bettner
Email Dave

How do you know an AI update worked?

A convincing proposal is a starting point. Before trusting an update, check its source, approve the change, and read the result from the application.

Try it with an inventory update.

A shop starts with 100 units, receives 30, and ships 20. The expected stock is 110 units. Try the valid proposal, then see what changes when the source is stale, the calculation is wrong, or a write response is lost.

This example uses fictional inventory data and deterministic local checks. No AI model or external application is called. State stays in this page and resets when you reload.

Source records
Source revisioninventory-7
Proposal inspectedinventory-7
Starting stock100 units
Received30 units
Shipped20 units
Proposal and application
Proposed stock110 units
Application stock100 units
Application revision3
Writes made0
Write receiptNot requested

Application values show the last read. Applying a change does not refresh them. Use “Read application” to inspect the saved result.

Enable JavaScript to try the controls. The example above shows the source and proposed update.

What happened

  1. No interactive actions have run.

Three different kinds of evidence.

  1. A proposal. An assistant suggests a new value. Check that its source is current and its calculation is right before allowing a write.
  2. A write receipt. An application reports that it accepted the change. That tells you about the request, but you still need to inspect the saved result.
  3. A checked final state. Read the application again and compare its saved value and revision with the approved update.

If a response gets lost, the write may already have succeeded. Read the application before deciding to retry. Otherwise, you risk applying the same change twice.

Make the result easy to inspect.

The same questions apply when an assistant updates a report, a customer record, or a project. Which source did it inspect? What exactly did you approve? What is saved now? A useful workflow makes those answers available to the person responsible for the result.

I use this approach in my AI workflow work. You can also inspect the Accounting Acceptance Lab or try Krowbar’s fictional reporting demo.