RESEARCH · RESEARCH · #3207
BlockLock (arXiv:2611.06977v1): reproducible LLM inference via fixed batch geometry
The paper introduces BlockLock, a method that preserves bitwise reproducibility for a protected request by fixing the batch geometry the request sees. It describes two modes (BlockLock-Rows and BlockLock-Pack), reports preservation of target sequences and Top20-logit bits across five dense models and three single-GPU setups, and finds BlockLock-Rows cuts latency by 53.5–55.0% and raises throughput by 115.2–122.2% versus vLLM Batch-Invariant while avoiding custom kernels and retaining shared prefill compute under a stated per-deployment contract.
KEY POINTS
- The paper introduces BlockLock, a method that preserves bitwise reproducibility for a protected request by fixing the batch geometry the request sees.
- It describes two modes (BlockLock-Rows and BlockLock-Pack), reports preservation of target sequences and Top20-logit bits across five dense models and three single-GPU setups, and finds BlockLock-Rows cuts latency by 53.5–55.0% and raises throughput by 115.2–122.2% versus vLLM Batch-Invariant while avoiding custom kernels and retaining shared prefill compute under a stated per-deployment contract.
- BlockLock offers a practical way to get bitwise reproducibility for individual requests with much lower latency and higher throughput than batch-invariant approaches and without custom kernel development.
WHY IT MATTERS
BlockLock offers a practical way to get bitwise reproducibility for individual requests with much lower latency and higher throughput than batch-invariant approaches and without custom kernel development.