I’ve been building agents for calls, messaging and software delivery, along with the tools I use to coordinate the work. Much of the engineering is about keeping track of what happened: which information an agent used, what it changed and whether another person can check the result.
I use AI coding agents to implement and review the software. Here are a few details from that work.
Compare a scoped read with loading the whole workspace
In my task-and-context plugin, an agent could request the whole workspace or ask for the work relevant to a particular query. I compared the two responses using the same evaluation fixture.
- The whole-workspace response was 113,598 bytes.
- The scoped response was 3,513 bytes.
- That was a 96.91% reduction, with the checked work, closed items and source evidence retained.
That result answers a specific question about retrieval size. To compare latency, cost or task completion, I’d measure those separately.
For a model or agent comparison, I’d keep the tasks and attempt budget the same, record the model and code versions, and check the results using the same rules. I’d include retries and human intervention when calculating what it took to complete each task.
Check what the software actually did
One bug I found during local testing was an alert marked delivered after a permission check had blocked it. The check prevented the send, but the delivery status was wrong.
I needed a test that checked both. After the denied action, the alert had to remain undelivered and the record had to show why.
I use different checks for different parts of a task:
- Read the trace to see what the agent said and attempted.
- Inspect the database or provider record to see what changed.
- Listen to a call or watch someone use the result to judge the experience.
Tests can check revisions, permissions and duplicate actions directly. A model-based grader can help judge the meaning of a conversation. I choose the check based on the question I need answered. The verification essay explains this using the alert bug and phone-agent tests.
Check again when the work changes
My Python/SQLite change tracker stores a decision revision, the affected files and the evidence used to check them. Updates have an expected revision, so a worker can’t quietly overwrite a newer update. Transactions keep related changes together, and idempotency keys prevent a repeated command from running twice.
If a relevant file or decision changes, the earlier test result may no longer apply. The tracker checks those dependencies and reopens work when the evidence needs updating.
There’s a similar issue with retries. Returning the receipt from an earlier command can be correct, but someone asking about the current state needs a fresh read. Another worker may have changed the project since that receipt was created.
I used the same approach in the evidence collector for the consulting website’s organic-growth work: check the source and reporting period, save the snapshot, compare compatible periods, then record what to do next. A repeated report doesn’t count as growth. Missing data stays marked missing. Signed-in account access is still needed for collection; the collector itself stores and checks the observations.
Give each agent a clear job
For a healthcare-project pilot, I set up a coordinator and five specialists for product, relationships, implementation, check-in and reputation. Each had its own responsibilities and references to the relevant original conversations.
A new specialist conversation doesn’t contain the full history of the original chat. I keep source references, notes about what was read and saved checkpoints so a worker can recover the context it needs.
I use parallel workers for research, separate implementation tasks and independent reviews. When they change shared code or requirements, someone has to reconcile the changes. I try to give workers separate files to edit and make the integration responsibility explicit.
After a worker reports that it finished, I still check the result against the current request. To judge whether the extra agents helped, I’d compare completed work and include the time spent coordinating, integrating and fixing it.
The code
The public Agent Infrastructure repository includes the SQLite workspace, MCP integration, versioned updates and hosted authentication components. The Voice and SMS Operations repository includes call-orchestration and handoff code. These are selected exports of the implementation; client records and live provider access stay private.
Agents inside real workflows · How I check whether an agent finished the task · Agent coordination case study