Don't Let Your Agent Run UPDATE: Append-Only Writes for AI Automation

The most dangerous line in most agent codebases is not the prompt. It is the tool that looks like update_record(id, fields). The moment an LLM can mutate your production state directly, you lose three things at once: you can’t explain what happened, you can’t replay it, and you can’t cleanly undo it. I have debugged enough “the agent changed 400 customer records overnight and nobody knows why” incidents to have a firm rule now: agents don’t write to my database. They write to a ledger. Something boring and deterministic does the actual writing.

This article is the pattern I use, with the schema, the committer logic and the tradeoffs.

Why direct writes break down

When an agent calls a tool that runs UPDATE deals SET stage = 'lost', the only trace left behind is the new value. Maybe an updated_at timestamp. Now ask yourself the questions you will actually be asked in production:

• Which run changed this, with which model version and which prompt?
• What did the agent see when it made the decision?
• What was the value before?
• Did it change anything else in the same run?
• Can we reverse only the bad changes and keep the good ones?

With direct writes, the honest answer to most of these is “let me grep the logs and hope”. Logs are not a source of truth. They get sampled, rotated and truncated, and they are rarely structured well enough to reconstruct a before and after state.

There is a second problem. Direct writes mix two very different concerns: deciding what should change and making the change safely. LLMs are good at the first and terrible at the second. They don’t know about concurrent edits, they retry when a tool times out, and they occasionally hallucinate an ID that happens to exist.

The pattern: agents propose, a committer applies

The fix is to split the write path in two.

1. The agent gets one write tool: propose_change. It appends a row to a proposals ledger. It never touches the business tables.
2. A separate, non-LLM process, the committer, reads pending proposals, validates them, checks policy, and applies them inside a transaction. It records the outcome back to the ledger.

The ledger is append-only. Rows are never updated in place except for a status transition, and even that I prefer to model as a new event row. Nothing is ever deleted.

The ledger schema

Here is a stripped-down version of what I run in Postgres:

create table agent_proposals (
  id              bigserial primary key,
  run_id          uuid not null,
  agent_build     text not null,      -- prompt + model + tools version
  idempotency_key text not null unique,
  entity_type     text not null,      -- 'deal', 'ticket', 'contact'
  entity_id       text not null,
  action          text not null,      -- 'set_field', 'create', 'archive'
  payload         jsonb not null,     -- what to change
  expected        jsonb,              -- what the agent believed was current
  reason          text not null,      -- agent's short justification
  evidence_ref    text,               -- pointer to the context snapshot
  created_at      timestamptz not null default now()
);

create table agent_proposal_events (
  id           bigserial primary key,
  proposal_id  bigint not null references agent_proposals(id),
  status       text not null,  -- 'approved','applied','rejected','conflict','reverted'
  detail       jsonb,
  before_state jsonb,
  after_state  jsonb,
  actor        text not null,  -- 'committer', 'human:anna', 'policy'
  created_at   timestamptz not null default now()
);

Three columns do most of the work.

expected is the agent’s view of the current state when it made the decision. If the agent read that a deal was in stage negotiation and wants to move it to lost, expected is {"stage": "negotiation"}. The committer compares it to reality before applying. If a human moved the deal to won five minutes ago, the proposal becomes a conflict instead of silently overwriting a real sale. This is optimistic concurrency, and it catches a surprising number of bad writes.

idempotency_key makes retries harmless. I derive it from the run ID, the entity and the action, so if the agent calls the tool twice after a timeout, the second insert fails on the unique constraint and the tool returns “already proposed”. The agent moves on.

evidence_ref points to a stored snapshot of what the agent saw: the retrieved documents, tool outputs and the relevant slice of conversation. Storing a pointer instead of the full blob keeps the ledger small. The snapshot lives in object storage keyed by run.

The committer

The committer is intentionally dumb. For each pending proposal it does this:

1. Load the entity inside a transaction with a row lock.
2. Validate payload against a strict schema for that entity_type and action. Unknown fields are rejected, not ignored.
3. Compare expected to current state. Mismatch means conflict.
4. Evaluate policy: auto-apply, require human approval, or reject outright.
5. Apply, capture before_state and after_state, write the event, commit.

No LLM calls here. No retries with creativity. If something is off, it stops and records why.

Policy tiers are where autonomy lives

The committer is the natural place to decide how much you trust the agent, and you can tune it per action without touching the prompt.

I use three tiers:

• Auto: low-risk, reversible changes. Tagging a ticket, filling an empty field, adding a note. Applied immediately.
• Review: anything that affects money, customers or external systems. Changing a deal stage, sending an email, issuing a refund. Lands in an approval queue with the reason and evidence one click away.
• Deny: actions the agent should never take regardless of what it proposes. Deleting records, changing ownership, touching billing plans.

The nice part is that you can graduate actions. When the review queue for “set ticket priority” shows 98 percent approval over two weeks, you move it to auto. That is a data-driven autonomy decision instead of a gut feeling, and it is fully reversible.

I also cap volume per run. If a single run proposes more than, say, 50 changes of the same type, the committer holds all of them for review. A runaway loop becomes an alert instead of an incident.

What append-only buys you

Real audit

Every change in the business tables that came from an agent has a matching ledger row: which build, which run, what it saw, why it did it, who or what approved it, the before and after. When a client asks “why did the system mark this lead as spam”, I can answer in a minute with the actual evidence.

Surgical rollback

Because before_state is captured at apply time, reverting is a new proposal with the old values, applied by the same committer with the same conflict check. You can revert one run, one agent build, or everything a given build did on a given day, without restoring a backup and losing unrelated human edits.

Replay and time-travel debugging

This is the one people underestimate. With the evidence snapshot and the agent build recorded, you can rerun the exact decision with a new prompt or a different model and diff the proposals. When a new model gets cheaper and you are tempted to switch, you don’t have to guess whether it behaves the same. Replay last week’s runs, compare proposed changes against what was actually approved, and you have an eval set that came straight from production.

Drift detection

Rejection and conflict rates per agent build are a live quality metric. If the rejection rate for a build jumps from 3 to 15 percent after a provider silently updates a model, you see it on a dashboard the same day.

Details that bite if you skip them

Make the agent’s view explicit. The agent can only fill expected correctly if its read tools return the fields it will later want to change. I design read tools to return a compact current-state object per entity for exactly this reason.

Keep reason short and required. One or two sentences. It is not for the model, it is for the human reviewing at 9am. Long reasons get skipped.

External side effects need an outbox. Sending an email or calling a third-party API can’t be rolled back. Treat them as proposals too, and have the committer write to an outbox table that a separate worker delivers. Reverting then means “send a correction”, which is at least a conscious decision.

Tell the agent the outcome. If the agent needs to act on the result in the same run, the propose_change tool should return the status: applied, pending review or conflict with current values. Agents handle “your change conflicted, current stage is won” much better than silent success.

Don’t let the ledger become a second database. Business logic reads from the business tables. The ledger is for history, review and replay. If you find yourself querying proposals to render product UI, something has gone wrong.

When this is overkill

If your agent only reads data and produces a draft a human copies manually, you don’t need a ledger. The human is the committer. Same for throwaway internal scripts where the whole dataset can be regenerated.

The pattern pays off the moment an agent writes to state that people rely on and that you can’t cheaply rebuild: CRM records, tickets, inventory, anything with money attached. In those systems I add the ledger on day one, because retrofitting it after the first incident means you already have a gap in your history.

A starting checklist

If you want to adopt this today, start small:

1. Replace every agent write tool with a single propose_change tool.
2. Create the two tables above. Skip evidence storage at first if you need to, but keep the column.
3. Write a committer with schema validation and the expected check. Start with every action in the review tier.
4. Build the simplest possible approval screen: reason, diff, approve, reject.
5. After two weeks, look at approval rates and move boring actions to auto.

It is maybe two days of work for a typical MVP. In return, you get an agent you can explain, replay and undo, which is the difference between a demo and a system a client will actually let run unattended.