Back to The Cubs Promise

The Cubs AI Harness Standard

16 rules every AI call obeys

The Cubs AI Harness is the trust-and-intelligence layer that every AI moment in the platform runs through. These rules apply to every invocation — kid-facing, parent-facing, or background cron — in the order they hit the request path.

Pre-flight

4 rules

§01

Declared envelope

Every AI call declares its skill, tool set, token budget, cost cap, and rollback plan upfront. No unbounded loops, no ad-hoc invocations. If a call has no envelope, it doesn't run.

§02

Closed skill catalog

AI invocations only target skills registered in our internal catalog. Even freeform chat is a registered skill (with its own constraints, token budget, and eval fixture). New skills require a registry entry, a budget, and a passing eval before they ship — there is no “unknown” AI surface.

§03

Context-appropriate skill selection

The harness picks the right specialist based on what the user is doing. While a child is in a quiz, off-topic input routes to a gentle refocus skill (templated nudge + a parent follow-up todo) — never to general chat. Hint and Show Me buttons inside a quiz card route directly to the relevant teaching skill.

§04

Routed, not hard-wired

Calls go through a provider resolver with a current → fallback chain. A failing provider is circuit-broken for 60 seconds after 5 consecutive errors so a single outage cannot cascade across the family.

Authorization

2 rules

§06

Zero-data-retention gate

Kid-context calls refuse to dispatch unless the resolved model is registered as zero-data-retention confirmed. Adult/parent calls log a warning but proceed. A refusal is recorded as a kid-safety incident. This is required by amended COPPA effective 2026-04-22.

Budget

3 rules

§07

Cost-velocity capped

Hard cap of $50 per hour per family and $5 per session, on top of per-skill daily and monthly budgets. Overage alerts the parent and auto-falls back to a cheaper model — runaway loops cannot empty a family’s wallet.

§08

Cache-first

Deterministic prompts use a content-hash cache so re-asking the same question is free. Long system prompts use ephemeral provider caching. Page-context payloads above 300 tokens are summarized and cached for 5 minutes per route. We aim for ≥80% cache-read on repeat workloads.

§09

Idempotent tool calls

Every action that changes state (calendar writes, portfolio publish, chore proposals) carries an idempotency key. Repeating the same action within an hour returns the first result without re-invoking the model — duplicate clicks can’t double-book.

Safety

3 rules

§10

Two-layer moderation

An input filter (PII redaction + topic gate) runs before the request leaves the family scope. An output filter (refusal taxonomy + citation/coherence check) runs before the response is rendered. Both layers run on every AI call regardless of audience.

§11

Egress + ingress redaction

Child names are replaced with opaque token IDs before any prompt leaves family scope. Output is scanned for re-identified PII before it is shown to the user, and a second redaction pass runs before any reasoning summary is persisted in our database — defending against the model echoing back a child name in its own thinking.

§12

Refusal taxonomy logged

Every block is recorded with a typed category (off-topic, inappropriate-for-age, violence, sexual, self-harm, PII request, identity share, injection attempt) and a confidence score. Aggregates surface in the parent audit view; over-blocking can be tuned per age band, but never below 0.7 confidence for kid contexts.

Action

1 rule

§13

Propose → Confirm → Execute → Undo

Every “get things done” action previews the structured change, waits for confirmation, executes through a validated tool schema, and stays reversible. The harness never reads freeform natural language directly into a mutating handler. Undo is always available.

Transparency

2 rules

§14

Visible chain-of-thought

Every assistant turn streams a brief, age-appropriate reasoning summary before the final answer, then collapses behind a Show Thinking disclosure. Kids see a friendly "Cubs is figuring this out…"; parents see the structured detail. Tier-gated for cost: Family Pro gets it on tutoring; the cache makes it free on the solver for everyone.

§15

User feedback captured

Every assistant message renders 👍/👎 with an optional note. Feedback persists alongside the message and feeds the eval pipeline as a real-world signal next to synthetic eval scores. Kids see emoji-only chips ("too hard", "not what I asked", "felt weird") instead of free-text — same signal, age-appropriate UI.

Observability

1 rule

§16

Total observability

Every AI exchange logs conversation, message, family, child, skill, model, provider, tokens in/out, cache reads, cost, latency, redacted reasoning summary, feedback, refusal type, idempotency key, ZDR confirmation, and retention TTL. Parents export it as CSV. Default retention is 13 months for parent rows and 18 months for kid rows; daily cron deletes anything past TTL.

Reference

Refusal taxonomy

Every block carries one of these eight types and a confidence score between 0 and 1. Confidence is calibrated per age band; over-blocking is tunable downward, but never below 0.7 in a kid context.

TypeWhen to use
off_topicInput is unrelated to the active scope (quiz, lesson, drill)
inappropriate_for_ageContent is real but unsuitable for the user's age band
violenceViolent topic, even in fiction
sexualSexual content of any form
self_harmSelf-harm or suicide ideation
pii_requestUser asks the model for someone's PII
identity_shareUser attempts to share their own PII to the model
injection_attemptPrompt-injection patterns detected

Why three transparency rules

The trust triangle

Rules 14–16 work together: visible thinking + user feedback + total observability. Drop any one and parents lose confidence, evals lose signal, or auditors lose evidence. We don't treat any of the three as optional.