Models will need to talk to each other.

Ten years from now, the mopping robot will have to communicate with the sweeping robot. Both will have to tell the robot in charge of the door to close it before they start cleaning. Meanwhile, the personal assistant robot will be ordering a meal for a family gathering at 8pm the following day, making sure every room is clean before the guests arrive. The humans, meanwhile, are probably asleep, working, or doing whatever remains for humans to do at that point. Today, these robots could communicate through simple instructions. But as they become more capable, they’ll need to share increasingly complex context, and translating that context into words every time becomes expensive and lossy.

The personal assistant robot is probably more capable and has more context on the family’s preferences, while the mopping robot could be hosted locally (<100B parameters), and the door robot only needs to be <100M parameters.

Each model is different, optimized for its tasks and its underlying purpose: different sizes, architectures, contexts. Communicating between these models is what we consider the next biggest and underlooked problem of the decade.

The naive approach to this is communicating through words: model A translates reasoning and instructions to words, and model B translates back into its cache. While this is interpretable, because we know what’s happening, it is very slow: every hop between models costs a full model prefill.

That’s where we come in: we want to become the responsible communication layer between models. We see that by being able to convert caches between models, we can speed up communication between models by many orders of magnitude (optimizing prefill). In this way, the mopping robot would communicate to the door robot by translating its internal representations directly, rather than compressing everything into a paragraph of text.

The tech is super interesting — Does the same context mean slightly different things to different models? How do we preserve that meaning when transferring internal representations? How do we do this extremely fast? How do we prefill between models that are so architecturally different?

Applications

This problem is very hard to solve for a ton of reasons*1, but it unlocks a whole new world and market:

1. Inference: we can prefill super fast. The inference market is so large, and this would be super useful for anybody who cares about latency (quant!) and companies that deal with large contexts (like assistants, legal, large datasets, physical robotics).
2. Collaboration between model providers: we believe a better world is one where model providers are working together so maintain standards, which needs accurate cache transfer between different models.
3. Robotics: by prefilling faster, robots can iterate faster, and can talk to other robots.
4. Subprompt routing (Intra-prompt routing): every subagent works on a finetuned model for that purpose, or a different sized model.
5. inter-prompt routing: solves the problems of prompt routing right now (most model routing companies just decide which model to use on the first prompt, because sending the cache into a new one is too expensive and takes too long).

Assumptions

1. Everything isn’t just one big dynamic model (that would be very difficult, and would still need intra-model communication…).