🤓 Latest Submissions
Grafting: Installable Capabilities for Gemma 4 E4B
Fine-tuning a full model for every new domain is expensive, storage-heavy, and difficult to reverse. RAG can provide retrieval, but it does not change how the model itself processes the domain. Grafting takes a different approach: it trains domain capability as an external additive weight delta that can be installed onto a frozen base model, used for native inference, and removed without permanently modifying the original model. Our Gemma 4 implementation separates the immutable host weights, D0, from the trainable graft, D1. During training, gradients and optimizer updates are restricted to D1 while the base model remains frozen. At deployment, the graft is baked into a disposable in-memory Shadow Copy using FP32 accumulation followed by a single BF16 round. The external graft tensors are then freed, and inference continues through Gemma’s normal linear layers with zero runtime hooks and no additional graft GEMMs. Uninstallation does not subtract the delta: the Shadow Copy is destroyed and the pristine base model is reloaded. We evaluated the system on a held-out medical-domain corpus. Base Gemma 4 reached a perplexity of 14.85. A 27.5 MB Compact PLE-gate graft reduced perplexity to 10.53, while a 275 MB Standard Up-projection graft reached 10.48. That is a 29.4% perplexity reduction, with the Compact version retaining nearly the full gain at one tenth the artifact size. Both grafts baked in approximately 0.022 seconds, used zero hooks after installation, freed their external D1 tensors, and restored the exact Base result after uninstall. The Gemma 4 prototype runs on AMD GPUs through PyTorch and ROCm, with training and evaluation performed on AMD Instinct MI300X and later on AMD Radeon PRO W7900 hardware.
13 Jul 2026
