Cascadia runs resident Inkling 975B/41B MoE inference on eleven Intel Core Ultra X7 AI PCs
The paper presents Cascadia’s custom resident MoE engine that executes Inkling (975B total, 41B active parameters) across eleven Intel Core Ultra X7 358H AI PCs with Arc B390 iGPU using OpenVINO. The system fits six decoder layers per machine, coordinates FP16 expert computation with FP32 output restoration, and reports up to 60.29 aggregate decode tokens/s at 88 streams (46.87 tokens/s over complete serving phases) and a median first-token latency of 6.05 s at fifteen streams; context lengths up to 64k tokens were evaluated, with first-token time growing as aN+bN^2 and decoding time scaling roughly linearly.