🤝 Top Collaborators
.png&w=640&q=75)
🤓 Latest Submissions
DRIFT — the early-warning court for AI quality
AI quality doesn't fail loudly — it drifts. A retrieval index goes stale, a model update shifts behavior, and every response still looks fine, so threshold monitors stay green while quality slides for days. You find out from customers. DRIFT is built on one principle: no alert reaches a human until it survives cross-examination. A SENSOR scores every production response (rubric-based LLM judge, hedging, truncation, retrieval hits, latency) into an append-only LEDGER. When a trend looks suspicious, THE COURT convenes: a Prosecutor argues the decline is real, armed with regression and changepoints the code computed — never the model; a Defense attacks with every innocent explanation (traffic-mix shift, outlier user, time-of-day, scorer noise, sample size); a Judge rules DISMISS / WATCH / ALERT on a fixed standard of proof, citing ledger rows. An alert isn't a red dot — it's a COUNTDOWN: "quality crosses your floor in 3.2–5.5 hours; probable cause: retrieval decay." Every number comes from deterministic math, and every alert's outcome is backfilled, so the dashboard shows live alert precision — a false-alarm rate no monitoring vendor publishes. CI machine-checks the court on every commit: it must convict planted drift, acquit noise, and acquit the hard case — a traffic-mix shift where the aggregate falls but nothing is broken. That third case is the alert fatigue that kills threshold monitors. Why AMD: scoring everything plus two reasoning passes per suspicion is exactly the workload per-token pricing punishes. One MI300X (192 GB HBM) holds the sensing model and the judge simultaneously at flat cost. Onboarding is one command onto any OpenAI-compatible endpoint, including vLLM/ROCm. Status: 27 tests green in public CI, 300 responses scored live with correct verdicts, scorer calibration r=0.846, alert precision 1/1. Verdict first, alert second.
13 Jul 2026
DeployGuard: Multi-Agent PR Security Gate
DeployGuard turns code review from a human bottleneck into an autonomous, auditable team. When a pull request opens, a GitHub webhook spins up a Band chat room where five specialist AI agents collaborate through @mentions, each with one job: ScanAgent runs lint, tests, and dependency checks; SecurityAgent runs a deep vulnerability and secrets scan; RiskAgent computes a weighted risk score; DeployAgent fires the real deployment only if the change is approved; and ReportAgent posts the full audit trail. A critical finding blocks the deployment, posts a CRITICAL comment with the exact file and line, and escalates to the on-call engineer in the same chat, where they can reply APPROVE or REJECT. This is not one LLM with a long prompt; it is five independent Band identities with their own tools and prompts, handing off like a real review team. The agents also review one another: RiskAgent independently cross-checks SecurityAgent's verdict and, when a security-sensitive change was only passed, challenges it back for a re-scan, so the change must clear a consensus check before it can deploy. What makes it reliable is that every security-critical decision is produced by deterministic code, not the model's judgment. The CRITICAL block comment, the risk score, the audit report, and repository resolution all come from tools, so outcomes stay consistent even on small open-source models. The system fails closed: if it cannot inspect a change, it escalates for manual review instead of approving it. DeployGuard is built on Band for multi-agent orchestration and Featherless for the language models, together with LangGraph, FastAPI, and GitHub Actions, and it is deployed live on Railway so that real pull requests automatically trigger the chain. Remove Band and the chain collapses, because DeployAgent never receives its green light. Next steps include a GitHub App for multi-repository review, richer risk signals, and larger orchestration models.
19 Jun 2026
