Research
We're not here to ride the AI hype bubble. We believe robust and fundamental technical breakthroughs applied to real-world systems is what really moves the needle.
BYO SWE-grep: automatically train blazing fast search sub-agents on your knowledge base (Pt. 1)
RL-trained search subagents that learn your knowledge base’s structure for fast, reliable retrieval
Research
Lumina: building self-improving evaluation through customer-in-the-loop refinement
Lumina: an adaptive evaluation engine that learns to judge like a subject matter expert.
Research
Attention-based attribution: what your model is actually looking at
Cosine similarity is cosplay. Attention is attribution.
ResearchUpweight the strategy, not the tokens: faster training with explicit reasoning through RGT (Rationale-Guided Training)
Teach the why, not just the what: Rationale-Guided Training
ResearchTraining loss predicts evaluation performance, even for non-verifiable tasks
Loss: the cheapest evaluation you’ll ever run.
ResearchRobust, sample efficient SFT with prompt mutations
Low-KL divergence prompt mutations: better performance at a fraction of the cost.
ResearchWrite small, learn forever: rank-1 LoRA for continual learning
Why rank-1 LoRA updates might be the missing link between static fine-tuning and truly continuous, live-on-GPU learning.
ResearchDo transformers notice their own mistakes? Finding a linear hallucination detector inside LLMs
A linear signal in LLMs reveals hallucinations, is detected by a frozen observer, and steered with a single vector.
ResearchResurrecting the salmon: seeing clearer inside LLMs with domain-specific SAEs
A powerful, efficient, and domain-robust strategy for safeguarding medical-text generation.
ResearchWhy mechanistic interpretability needs a paradigm inversion
The conventional scaling paradigm for language models themselves may be fundamentally misaligned with interp.
Research