Together AI publishes Together Compiler (TKCC) for continual AI inference optimization
Together AI announces Together Compiler (TKCC), a continual agent-driven compilation system for AI inference that uses a Tiered Persistent Engine and a cost model to balance measurement throughput and fidelity. The authors report TKCC outperforms hand-written Megakernels on Llama 1B by up to 1.7x and the best open-source engines on several recent frontier models by up to 2.6x TPS, and say TKCC is deployed in production serving trillions of tokens.