Drift Harness

Created by team KEEL on June 19, 2026
Internal Enterprise WorkflowsRegulated & High-Stakes Workflows

Every AI system drifts. It softens a position under pressure, drops a constraint it was holding a moment ago, or states a guess as if it were certain. The correction is almost always manual: a human notices, pushes back, forgets, and corrects the same failure on the next turn. Nothing remembers, and nothing scales. Drift Harness makes that loop automatic — it intercepts the exchange before the user has to act, logs what failed and why, and builds a structured record that drives correction at scale. The system fans a single exchange across thirteen specialist agents, each checking one slice of behaviour: constraints, antipatterns, voice, quality, identity, alignment, gap analysis, profiling and question generation. Every agent returns the same five-field verdict — agent, status, rule, excerpt, severity — so one shape holds across every layer. Its core idea is how it represents certainty. Rather than a percentage, which is just a token prediction dressed up as a probability, each agent commits to one of three states — violation, uncertain, or clean — always tied to an exact excerpt from the reasoning that triggered it. The excerpt is what makes the label mean anything. Under the hood, the logger mints a UUID4 per exchange and classifies each turn in Python before the model runs. Findings write to a FastAPI and SQLite backend; agents communicate over a shared Band session; a C++ coordinator handles multithreaded fan-out. The full stack runs live on a Hetzner VPS under pm2, with a dashboard at dashboard.malecsystems.com. We proved it end to end: one misaligned input fanned across every live agent produced six confirmed findings, written straight to the backend. All thirteen agents are deployed and the dashboard is live. The harness is the asset. The agents are the mechanism that fills it. Every AI system drifts — this one notices, records it, and turns a manual habit into infrastructure.

Category tags:

"Application of technology: Thirteen specialist AI behavior monitoring agents (constraints, antipatterns, voice, quality, identity, alignment, gap analysis, profiling, question generation) fanned concurrently via C++ multithreaded coordinator + Band session. Uniform 5-field verdict schema (agent/status/rule/excerpt/severity). UUID4 per exchange. Python pre-classifier before model run. FastAPI + SQLite backend. Live at dashboard.malecsystems.com on Hetzner VPS under pm2. Presentation: Exceptional conceptual precision — "violation/uncertain/clean + excerpt" vs. "percentage (just a token prediction dressed as probability)" is a genuinely important epistemological point about AI certainty representation. "Six confirmed findings from one misaligned input" proves end-to-end live operation. "The harness is the asset" is a memorable positioning statement. Business value: AI drift detection infrastructure that automatically intercepts exchanges, classifies behavior against 13 dimensions, and builds structured records for correction at scale addresses a real and growing enterprise need as AI deployments proliferate. Turning manual habit into infrastructure is the right framing. Originality: C++ coordinator for multithreaded 13-agent fan-out is an unusual and performance-conscious architecture choice for an AI monitoring system. Three-state certainty representation (violation/uncertain/clean) with excerpt-anchored labels rather than probabilistic confidence scores is a principled and practically important design decision for trustworthy AI monitoring."

avatar

Sanem Avcil