Our Research

Prefill cost scales with the size of the model doing the reading, which means today you pay large-model rates before the large model has produced a single word. On long inputs, that's most of the bill.

We prefill on a small model instead. For the published Qwen3 4B → 14B pair, time to first token moves 4.3s → 1.7s (2.5× faster). Prefill cost on 1M requests moves $11.8k → $4.6k (61% cheaper). The method claim on the house page is a 20% cut in inference cost and time.

Per-layer K/V tensors are stripped of source rotary embeddings, projected head-wise into the target model's attention space, and re-embedded at target positions. The large model enters decode against a populated cache it never computed.

1. Strip source rotary from each layer's K and V.
2. Project head-wise into the target attention space.
3. Re-embed at the target model's positions.
4. Hand the seated cache to decode on the large model.

Transfers are gated on held-out quality and latency for every model pair. Ordinary prefill takes over whenever a pair doesn't clear, so the target model uses its usual inference path. No pair ships on the chute until both gates pass.

Held-out pair clears. KV ice rides the chute; decode starts on the seated cache.