New Routing Architecture for AI Models

This is interesting: A developer has reverse-engineered the routing architecture underlying Skana AI Fugu and implemented an analog for open frontier models.

tinyrouter is a compact LLM router with approximately 10,000 parameters that learns to determine which model to use and what role it should perform for each request.

The goal was simple: to outperform each individual model by directing each task to the most suitable specialized executor, rather than relying on a single LLM.

Some interesting observations:

• Routing is beneficial only when models have complementary strengths. If all models show similar results on a single benchmark, the router has little to optimize.
• On MMLU, the router outperformed each individual model. On mathematical tasks, it achieved results on par with the best model, as the set of models demonstrated very low diversity in solution quality.
• Pre-initializing the router and tuning the reward function in evolutionary training improved the learning process. However, the authors do not yet claim a real quality gain, as the variance in results during evaluation was too high. More rigorous experiments are needed to confirm the effect.