What does production ready actually mean for an AI system?

The phrase is doing no work

Every shop says production ready. The term survives because nobody ever has to prove it. So here is a definition anyone can fail: nine checks. A system either has each one or it does not, and each is answerable in a sentence by whoever built it.

The nine checks

1. Tests that block a bad change. Not tests that exist somewhere, tests that stop a merge when they fail.
2. Monitoring that reaches a person. A dashboard nobody opens is not monitoring.
3. A written plan for what happens when each outside service fails. Written before it fails, not pieced together after.
4. Retries with a wait between them, and a plan for when a provider says no.
5. Keys and passwords kept outside the shipped app, and changeable without a new release.
6. A rollback somebody has actually rehearsed. An untested rollback is a hope.
7. Doing less instead of dying. When one provider is down, the system keeps going with less rather than stopping.
8. Logs that say what happened and why, so a question can be answered without reproducing it.
9. A written note of what is not covered yet. Every system has a gap. Knowing yours is the check.

What this looks like when it is real

Mooney is OCTYN's own app, live on Google Play. Its outside calls never leave the phone directly. They go through Supabase edge functions instead, sixteen of them as of 2026-06-20, which is why no key ships inside the app, why a key can be swapped without a store release, and why limits are set where they can see every user rather than one phone.

Understanding a spoken sentence falls back through four providers before a user sees anything: Groq's small model, then its larger one, then Cerebras, then OpenRouter. All three Groq keys can fail at once and the app keeps going. Speech has no second provider, though: if both Deepgram keys fail, voice stops. That gap is written into the team's own fallback notes rather than left to be found.

Errors land in five places on purpose, because each one sees something the others cannot: Sentry for the app's own crash traces, PostHog for what the user was doing right before the crash, Supabase for server-side failures, Play Console for native crashes the app framework never sees, RevenueCat for billing. After every release, a ten-minute check fires a deliberate crash to confirm the whole pipe still works. Monitoring nobody tests dies quietly.

What OCTYN does not claim

Mooney's time from speaking to a saved entry is under five seconds. That figure comes from screen recordings on one phone over a live network, so it is an observed time, not a measured average across every use. Timing for each step in between is not recorded in the app yet, and instrumenting it is the first item on that project's own list of things to fix.

Cockpit, the screen OCTYN watches its own coding sessions from, has no timing yet on how long a session sits waiting before someone notices, which is the exact number the system exists to bring down. That is the honest state as of 2026-09-14, written on the project's own page rather than left for someone else to find.

A vendor with no gaps is a vendor who has not looked.

Ask these before you sign

• Show me the test that fails if this breaks. Watch whether a file gets opened.
• Who gets alerted, through what, and what did it last fire for?
• What happens when a provider you depend on says no in the middle of a busy day?
• Where do the keys live, and what does changing one cost?
• What is the rollback, and when was it last used?
• Name the part of this that is not covered yet.