
Malatang is a measured self-improvement harness for bug-fixing agents. Instead of claiming that an AI “gets better” based on vibes, Malatang defines improvement operationally: run the same frozen benchmark, accept fixes only when build and tests pass in a sandbox, record pass rate, and let the system modify its own strategy between runs. The benchmark contains 25 seeded training bugs across syntax errors, off-by-one logic, null handling, wrong API usage, and async mistakes, plus a 5-bug hold-out set used only for final evaluation. A Creator agent proposes code patches using Qwen2.5-Coder-7B served through vLLM on AMD AI Notebooks with ROCm. A deterministic Judge applies each patch in a temp sandbox and accepts it only if the project builds and the Vitest suite passes. Every attempt is logged as a trajectory, so wins and failures become evidence. Between benchmark iterations, Malatang uses Fireworks for a bounded reflection step that rewrites the Creator’s own strategy playbook from sandbox-verified results. The product is not a magical bug fixer; it is the harness, contracts, evidence trail, and measured improvement curve. Our project shows an original, reproducible way to evaluate self-improving agents using real GPU-backed inference, strict verification, frozen benchmarks, and transparent integrity reporting.
13 Jul 2026