No. 15Automation that holds up3 min readOCTYN
Monitoring an AI workflow without an engineering team starts with a morning read.
Monitoring an AI workflow without an engineering team comes down to two things a person actually reads. One is a short summary each morning of what the system did and what it held back. The other is a failure alert on the phone they already check, plus a regular test alert, since a broken alert path is silent too.
The handover for a new system tends to end with a link to a dashboard, and the head of operations bookmarks it meaning to look every day. For two weeks they do. After that it sits in a folder with the others, and the first sign of trouble is a site manager ringing to ask why something the system was meant to flag never got flagged, while the dashboard had it on screen the whole time.
A company with no engineers can run an AI workflow and keep it honest. The condition is that monitoring comes to a person, in a form they already read, and that somebody checks the monitoring itself. The examples below come from systems OCTYN runs, and one of them still has a gap in it.
Build the morning read first
OCTYN runs an outreach system for an independent craft retailer, and it is looked after by one person, in one window a day. Nothing goes out in the morning, so the owner can read the day's queue and pull anything they dislike before it moves. It does not ask for approval on every item. That is on purpose: at 25 a day, approving each one becomes rubber stamping within a week.
The case study's list of what OCTYN would do differently opens with build the daily summary first, because without it the operator has nothing to read. A log is something you search once you know what went wrong, and whoever owns a warehouse network or a chain of stores is not searching anything at seven in the morning. They will read a page, if it is short and it says what the system did and what it left for a person.
For an operations workflow, that page needs little:
- what the system handled since yesterday
- what it held back for a person, and why
- anything that stopped, or slowed right down
- each failure in plain words, with where it happened
Put the alert where the person already looks
Every project OCTYN runs reports its errors to one tracker, and that tracker delivers by email and a dashboard. Both are places a person visits when they choose to. A small relay sends every error to a phone instead, into the chat app that already carries everything else, and the notification line says what broke and where, short enough to read on a lock screen. The full detail rides along as an attached file, one tap away.
All of it lands in one shared group. What matters at a glance is whether anything is on fire, and splitting it across twenty channels turns one look into twenty.
The quiet week
That relay has a hole, and its own case study says so. If it fails, the alert does not arrive, and there is no second channel and no heartbeat. A relay broken for a week looks exactly like a week with no errors. The fix it names is a periodic test alert, which, as the case study stands, has not been built.
A company without engineers carries the same gap, with fewer people placed to spot it. Before trusting a quiet phone, ask whoever runs the system how you would find out that the alerts themselves had stopped. If they built it and still run it, as an operating partner does, the answer should come easily. Working out why something failed is a separate question, and observability is where it starts.
It is a dull question to put to a new system, and OCTYN has had to put it to its own.
Bring the version of this that is about your company.
The first conversation is with the people who would build it. If there is a fit, the Consult comes next.