DRIFT — the early-warning court for AI quality

Created by team Effectonious on July 09, 2026
Unicorn Track

AI quality doesn't fail loudly — it drifts. A retrieval index goes stale, a model update shifts behavior, and every response still looks fine, so threshold monitors stay green while quality slides for days. You find out from customers. DRIFT is built on one principle: no alert reaches a human until it survives cross-examination. A SENSOR scores every production response (rubric-based LLM judge, hedging, truncation, retrieval hits, latency) into an append-only LEDGER. When a trend looks suspicious, THE COURT convenes: a Prosecutor argues the decline is real, armed with regression and changepoints the code computed — never the model; a Defense attacks with every innocent explanation (traffic-mix shift, outlier user, time-of-day, scorer noise, sample size); a Judge rules DISMISS / WATCH / ALERT on a fixed standard of proof, citing ledger rows. An alert isn't a red dot — it's a COUNTDOWN: "quality crosses your floor in 3.2–5.5 hours; probable cause: retrieval decay." Every number comes from deterministic math, and every alert's outcome is backfilled, so the dashboard shows live alert precision — a false-alarm rate no monitoring vendor publishes. CI machine-checks the court on every commit: it must convict planted drift, acquit noise, and acquit the hard case — a traffic-mix shift where the aggregate falls but nothing is broken. That third case is the alert fatigue that kills threshold monitors. Why AMD: scoring everything plus two reasoning passes per suspicion is exactly the workload per-token pricing punishes. One MI300X (192 GB HBM) holds the sensing model and the judge simultaneously at flat cost. Onboarding is one command onto any OpenAI-compatible endpoint, including vLLM/ROCm. Status: 27 tests green in public CI, 300 responses scored live with correct verdicts, scorer calibration r=0.846, alert precision 1/1. Verdict first, alert second.

Category tags: