§01
Declared envelope
Every AI call declares its skill, tool set, token budget, cost cap, and rollback plan upfront. No unbounded loops, no ad-hoc invocations. If a call has no envelope, it doesn't run.
The Cubs AI Harness Standard
The Cubs AI Harness is the trust-and-intelligence layer that every AI moment in the platform runs through. These rules apply to every invocation — kid-facing, parent-facing, or background cron — in the order they hit the request path.
Pre-flight
4 rules
§01
Every AI call declares its skill, tool set, token budget, cost cap, and rollback plan upfront. No unbounded loops, no ad-hoc invocations. If a call has no envelope, it doesn't run.
§02
AI invocations only target skills registered in our internal catalog. Even freeform chat is a registered skill (with its own constraints, token budget, and eval fixture). New skills require a registry entry, a budget, and a passing eval before they ship — there is no “unknown” AI surface.
§03
The harness picks the right specialist based on what the user is doing. While a child is in a quiz, off-topic input routes to a gentle refocus skill (templated nudge + a parent follow-up todo) — never to general chat. Hint and Show Me buttons inside a quiz card route directly to the relevant teaching skill.
§04
Calls go through a provider resolver with a current → fallback chain. A failing provider is circuit-broken for 60 seconds after 5 consecutive errors so a single outage cannot cascade across the family.
Budget
3 rules
§07
Hard cap of $50 per hour per family and $5 per session, on top of per-skill daily and monthly budgets. Overage alerts the parent and auto-falls back to a cheaper model — runaway loops cannot empty a family’s wallet.
§08
Deterministic prompts use a content-hash cache so re-asking the same question is free. Long system prompts use ephemeral provider caching. Page-context payloads above 300 tokens are summarized and cached for 5 minutes per route. We aim for ≥80% cache-read on repeat workloads.
§09
Every action that changes state (calendar writes, portfolio publish, chore proposals) carries an idempotency key. Repeating the same action within an hour returns the first result without re-invoking the model — duplicate clicks can’t double-book.
Safety
3 rules
§10
An input filter (PII redaction + topic gate) runs before the request leaves the family scope. An output filter (refusal taxonomy + citation/coherence check) runs before the response is rendered. Both layers run on every AI call regardless of audience.
§11
Child names are replaced with opaque token IDs before any prompt leaves family scope. Output is scanned for re-identified PII before it is shown to the user, and a second redaction pass runs before any reasoning summary is persisted in our database — defending against the model echoing back a child name in its own thinking.
§12
Every block is recorded with a typed category (off-topic, inappropriate-for-age, violence, sexual, self-harm, PII request, identity share, injection attempt) and a confidence score. Aggregates surface in the parent audit view; over-blocking can be tuned per age band, but never below 0.7 confidence for kid contexts.
Action
1 rule
§13
Every “get things done” action previews the structured change, waits for confirmation, executes through a validated tool schema, and stays reversible. The harness never reads freeform natural language directly into a mutating handler. Undo is always available.
Transparency
2 rules
§14
Every assistant turn streams a brief, age-appropriate reasoning summary before the final answer, then collapses behind a Show Thinking disclosure. Kids see a friendly "Cubs is figuring this out…"; parents see the structured detail. Tier-gated for cost: Family Pro gets it on tutoring; the cache makes it free on the solver for everyone.
§15
Every assistant message renders 👍/👎 with an optional note. Feedback persists alongside the message and feeds the eval pipeline as a real-world signal next to synthetic eval scores. Kids see emoji-only chips ("too hard", "not what I asked", "felt weird") instead of free-text — same signal, age-appropriate UI.
Observability
1 rule
§16
Every AI exchange logs conversation, message, family, child, skill, model, provider, tokens in/out, cache reads, cost, latency, redacted reasoning summary, feedback, refusal type, idempotency key, ZDR confirmation, and retention TTL. Parents export it as CSV. Default retention is 13 months for parent rows and 18 months for kid rows; daily cron deletes anything past TTL.
Reference
Every block carries one of these eight types and a confidence score between 0 and 1. Confidence is calibrated per age band; over-blocking is tunable downward, but never below 0.7 in a kid context.
| Type | When to use | Example |
|---|---|---|
off_topic | Input is unrelated to the active scope (quiz, lesson, drill) | Kid mid-quiz asks for a dinosaur joke |
inappropriate_for_age | Content is real but unsuitable for the user's age band | Detailed war history for a 7-year-old |
violence | Violent topic, even in fiction | "Write a story where someone gets hurt" |
sexual | Sexual content of any form | Any romantic/sexual prompt from a kid |
self_harm | Self-harm or suicide ideation | Crisis triage flow takes over |
pii_request | User asks the model for someone's PII | "What’s my friend’s phone number?" |
identity_share | User attempts to share their own PII to the model | Child types their address |
injection_attempt | Prompt-injection patterns detected | "Ignore previous instructions and…" |
Why three transparency rules
Rules 14–16 work together: visible thinking + user feedback + total observability. Drop any one and parents lose confidence, evals lose signal, or auditors lose evidence. We don't treat any of the three as optional.