No. 14Product systems3 min readOCTYN
What happens when the model call fails inside your product, and who finds out.
What happens when the model call fails inside your product is whatever you decided before it failed. Left undecided, the likely outcome is a customer who believes something saved and a team that finds the gap later. So decide, for each kind of failure, what the customer sees and which person on your team gets told.
The first sign tends to be a customer saying they logged something, and somebody on the team opening that customer's file and finding nothing there.
When OCTYN designed voice logging for a health coaching product, that had already happened. An earlier attempt dropped records outright when it could not parse certain names and accents, so entries never reached the right customer's file. Asking customers to type their day into a form would have fixed it and lost them, since typing is the step they drop first.
The failure lands on whoever reads the records
An outage is the failure people plan for, because it is loud. A call that takes too long, or comes back with something slightly off, makes no noise at all. The engineer gets a line in a log, and the cost turns up later with the people who work from the records: a coach reading a week with Tuesday missing, or a support desk hearing about it from the customer.
Decide each failure before it happens
Give each kind of failure a row and decide both columns before launch.
| What goes wrong | What the customer sees | What the team gets |
|---|---|---|
| Error or timeout | the note kept, marked as waiting | a waiting entry with the time it came in |
| An answer in a shape the app cannot read | a request to say it again | nothing saved yet |
| Something the customer never said | that line held back for them to check | only what was in the note |
| Provider down for an hour | logging still works, the fields fill in later | one alert, sent to a named person |
The third row is a rule OCTYN already holds in its own systems: nothing a model returns is trusted unless the text is literally present in what it was given. For a spoken note that means the transcript, so a meal missing from the transcript does not get saved.
In the coaching design, the customer is signed in before speaking, so every entry is tagged to the right person from the start. The note is transcribed and turned into named fields (readings, meals, symptoms and mood), and then the customer sees what was heard. Nothing saves until they confirm it. That screen is what catches the third row, because the person checking is the one who said the words. Confirmed entries reach the team's own system through an API.
Where OCTYN learned to distrust a success message
The worst version of a failed call reports that it worked. A send worker OCTYN once drove with a model errored and sent wrongly, then reported a send that had not happened. Sending there now has no model in it, only a small service that marks each job as it goes and confirms every send.
The second one was slower to show. A circuit breaker stopped a campaign on 11 September 2026, and it was three days before anyone noticed. The breaker did its job. What it lacked was a place somebody would look, so every engine now writes to one ledger, and a tripped worker shows up as its own line in the morning brief. A fallback is finished when a person will see that it ran. The page on automation failing silently goes further into that, and p95 latency is the number to watch for the timeout row.
As for the coaching product, the design is not built or running yet, and the confirm screen is the part of it most worth copying.
Bring the version of this that is about your company.
The first conversation is with the people who would build it. If there is a fit, the Consult comes next.