NEWS · MODELS · #2434
Cohere opens public beta of Confidential Computing for Model Vault on NVIDIA H100 via Microsoft Azure
Cohere announced a public beta of Confidential Computing for its managed inference product Model Vault, running on NVIDIA H100 GPUs and Microsoft Azure confidential computing infrastructure. The feature adds hardware-backed isolation, GPU memory encryption, remote attestation, and an end-to-end encrypted request path so customers can verify the runtime and protect data in use during inference.
KEY POINTS
- Cohere announced a public beta of Confidential Computing for its managed inference product Model Vault, running on NVIDIA H100 GPUs and Microsoft Azure confidential computing infrastructure.
- The feature adds hardware-backed isolation, GPU memory encryption, remote attestation, and an end-to-end encrypted request path so customers can verify the runtime and protect data in use during inference.
- This lets enterprises run sensitive inference workloads on a managed platform while cryptographically verifying the hardware/software environment and keeping data protected in use.
DECISION BRIEF
CONFIDENCE
WHAT CHANGED
Cohere announced a public beta of Confidential Computing for its managed inference product Model Vault, running on NVIDIA GPUs (H100) on Microsoft Azure. The capability adds hardware-backed isolation, GPU memory protection and encryption of CPU↔GPU traffic, end-to-end encrypted request contents, and remote attestation so customers can cryptographically verify the runtime before releasing sensitive data.
WHY NOW
This change extends protection to ‘data in use’ for managed inference workloads and provides cryptographic evidence (remote attestation) that protections are in place — allowing enterprises to run more sensitive inference on a hosted platform without exposing plaintext to infrastructure operators.
WHO IS AFFECTED
Enterprises using or evaluating Cohere Model Vault for inference of sensitive data (examples cited: financial information, clinical records, privileged documents, proprietary source code, IP). Cohere (as operator of the managed service) and Microsoft Azure (as infrastructure provider) are directly implicated in the deployment and operation.
CONFIRMED
• Cohere has launched a public beta of Confidential Computing on Model Vault. • The beta runs on NVIDIA GPUs (H100) using Microsoft Azure confidential computing infrastructure. • The feature set includes hardware-enforced isolation, GPU memory protection and encryption of CPU↔GPU traffic, end-to-end encrypted request contents, and remote attestation. • Model Vault is a managed, dedicated model inference service that can accept encrypted workloads today (public beta availability). • The confidential boundary is designed to span CPU and GPU so accelerator memory and CPU↔GPU communication are protected.
UNCERTAIN
Missing/conflicting evidence (explicit in sources is absent): • Geographic/region availability and any enrollment or invite restrictions for the public beta. • Pricing, billing model, and any additional costs for confidential execution. • Performance overhead, latency, and throughput impact of confidential execution on Model Vault. • Supported model types, frameworks, and maximum model sizes. • Exact attestation formats, how customers perform verification, and tooling/APIs exposed for attestation verification. • SLA, compliance certifications, and third‑party audit status for the confidential offering. • Whether all Azure H100 instances or only specific SKUs are supported. These items are not stated in the provided excerpts.
WHAT TO WATCH
Observable signals to watch next: • Cohere product blog and documentation updates for configuration, attestation APIs, and region/availability notices. • Cohere Model Vault sign-up or beta enrollment pages and release notes. • Microsoft Azure confidential computing product pages and H100 SKU support matrix. • Technical whitepapers, security/attestation documentation, or sample verification artifacts published by Cohere. • Performance benchmarks, customer case studies, or partner announcements referencing confidential Model Vault deployments.
WHY IT MATTERS
This lets enterprises run sensitive inference workloads on a managed platform while cryptographically verifying the hardware/software environment and keeping data protected in use.
EVIDENCE MAP
4Editorial claims linked to specific sources, with support, contradiction and context shown separately.
Cohere announced a public beta of Confidential Computing for Model Vault running on NVIDIA H100 GPUs on Microsoft Azure.
SUPPORTEDChecked 2026-10-22 · 2 supporting
Confidential Computing on Model Vault is intended to protect data in use, extending protection beyond data at rest and in transit.
SUPPORTEDChecked 2026-10-22 · 2 supporting
Explains Confidential Computing is designed to 'protect the third state: data in use' and extends protection beyond at-rest and in-transit encryption.
SUPPORTS CohereMEDIA · 2026-10-22Describes confidential computing protecting 'data in use' while an application or model processes it.
The system protects GPU memory and encrypts communication between CPU and GPU during inference.
SUPPORTEDChecked 2026-10-22 · 2 supporting
States that 'GPU memory is protected and traffic between the CPU and GPU is encrypted as the model performs inference.'
SUPPORTS CohereMEDIA · 2026-10-22Explains confidential GPU capabilities protect GPU memory and enable the confidential boundary to span the accelerator as well as the CPU.
Model Vault supports end-to-end encrypted request contents and remote attestation so customers can cryptographically verify the runtime before releasing sensitive data.
SUPPORTEDChecked 2026-10-22 · 2 supporting
Says application payloads 'remain encrypted through intermediary infrastructure and can only be decrypted inside the attested confidential environment' and 'verify before you trust'.
SUPPORTS CohereMEDIA · 2026-10-22Describes attestation producing 'cryptographically signed evidence' that another party can evaluate before releasing sensitive data, and mentions end-to-end encrypted request contents.
SOURCES & TIMELINE
1Now in public beta with NVIDIA Confidential Computing with NVIDIA GPUs on Microsoft Azure, bringing hardware-backed protection and verifiable security to sensitive enterprise AI. Protect data in use: Model Vault extends protection beyond data at rest and in transit to prompts, documents, responses, and model execution while inference is running. Encrypt sensitive request contents end-to-end: Application payloads rema…
How Model Vault combines confidential CPU and GPU execution, end-to-end encrypted request contents, and remote attestation to protect sensitive AI workloads while they are being processed. In our product launch, we introduced Confidential Computing on Cohere Model Vault , now in public beta with NVIDIA GPUs on Microsoft Azure. This article goes deeper into how the system works. The goal is not simply to encrypt infer…