What does production ready actually mean for an AI system?

The phrase is doing no work

Every shop says production ready. The term survives because it is never given a definition anybody can fail. So here is one, as nine checks. A system either has each of these or it does not, and each is answerable in a sentence by whoever built it.

The nine checks

1. Tests that gate the change. Not tests that exist, tests that block a merge when they go red.
2. Monitoring that reaches a person. A dashboard nobody opens is not monitoring.
3. A documented failure path per dependency. Written before it fails, not reconstructed after.
4. Retries with backoff, and a rate-limit strategy that assumes the provider says no.
5. Secrets outside the shipped artefact, rotatable without a release.
6. A rollback somebody has rehearsed. Untested rollback is a hope.
7. Degradation instead of death. When a provider is down, the system does less rather than nothing.
8. Logs that name what happened and why, so a question can be answered without reproducing it.
9. A written statement of what is not covered. Every system has a gap. Knowing yours is the check.

What this looks like when it is real

Mooney is OCTYN's own Android app, live on Google Play. Its third-party calls do not go out from the phone. They go through Supabase edge functions, sixteen of them as of 2026-06-20, which is why no API key ships inside the APK, why keys rotate without a store release, and why rate limiting sits where it can see every user rather than one handset.

Intent parsing degrades through four layers before a user sees anything: Groq 8b, Groq 70b, Cerebras, OpenRouter. All three Groq keys can fail at once and the app carries on. Speech has no second provider, and if both Deepgram keys fail, voice stops. That gap is written into the fallback audit rather than left to be discovered.

Errors land in five places on purpose, because each one sees something the others cannot: Sentry for Dart stack traces, PostHog for what the user was doing before the throw, Supabase for server-side failures, Play Console for native crashes the Flutter engine never sees, RevenueCat for billing. After every release a ten minute smoke test fires a deliberate crash to confirm the pipe still works. Monitoring that is never tested dies quietly.

What OCTYN does not claim

Mooney's end-to-end voice time is under five seconds. That number comes from screen recordings on a Samsung S23 over a live network, so it is a demonstrated wall-clock time and not a p95. Per-stage latency is not instrumented in the app. We publish it as what it is, and instrumenting it is the first item on that project's own list of things to do differently.

WhoFits runs a crawler with no uptime monitor and no alerting on it. It is a batch system, and a stalled crawl gets noticed by a person running the status command. That is the honest state as of 2026-09-14, and it would be fixed before a client's campaign depended on it.

A vendor with no gaps is a vendor who has not looked.

Ask these before you sign

• Show me the test that fails if this breaks. Watch whether a file gets opened.
• Who gets woken up, through what, and what did it last fire for?
• What happens when the model provider rate limits you at 3pm on a Tuesday?
• Where do the API keys live, and what does rotating one cost?
• What is the rollback, and when was it last used?
• Name the part of this that is not covered yet.

The receipts under this page

Every claim above comes from a system OCTYN built and operates. Each line carries the date it was recorded and where it came from, so it can be argued with rather than taken on trust.