Confidential Computing: The Missing Layer in Enterprise AI Security

Most enterprise security teams have become comfortable with two axes of data protection: encryption at rest and encryption in transit. Disk volumes are encrypted. TLS tunnels are standard. Firewall rules are tuned. And yet, for AI workloads specifically, a dangerous gap remains — one that neither of those controls closes. When a large language model (LLM) processes a document, answers a query, or generates a summary, the data is decrypted in memory and processed in plaintext inside the CPU or GPU. Anyone with privileged access to that host — a hypervisor administrator, a cloud provider operator, a compromised root credential — can, in principle, read everything: the prompt, the document, the model weights, the response.

Confidential computing exists to close exactly that gap. It is rapidly becoming a foundational requirement for enterprises that want to run AI workloads on sensitive data without surrendering trust to their infrastructure stack. This post explains what confidential computing is, how it works in the context of AI, why regulated industries should care, and what practical deployment choices it implies.

What Confidential Computing Actually Means

Confidential computing is a hardware-enforced security capability that protects data in use — while it is being actively processed. It achieves this through a construct called a Trusted Execution Environment (TEE): an isolated, hardware-attested enclave within the CPU or GPU where code and data are encrypted even from the operating system, the hypervisor, and the cloud provider's management plane.

The three major processor TEE implementations you will encounter in enterprise contexts are:

•Intel Trust Domain Extensions (TDX) — encrypts an entire virtual machine, not just an application process, making it well-suited for containerized AI workloads.
•AMD Secure Encrypted Virtualization-Secure Nested Paging (SEV-SNP) — provides VM-level memory encryption with strong integrity protections and is now widely available on major public cloud providers.
•NVIDIA H100 Confidential Computing mode — extends TEE principles to the GPU, protecting model weights and activations during inference, which is essential because the bulk of LLM computation happens on GPU, not CPU.

What binds these technologies together is a mechanism called remote attestation. Before any sensitive data enters an enclave, a relying party (your application, your key management server, or an auditor) can cryptographically verify that the enclave is running exactly the expected software, on unmodified hardware, with the expected security configuration. Attestation transforms a trust claim ("we are running secure software") into a cryptographic proof that can be independently verified.

Why Encryption at Rest and in Transit Are Not Enough for AI

To understand why confidential computing matters specifically for AI, it helps to trace exactly where data is exposed in a conventional LLM deployment.

•User submits a prompt — encrypted in transit via TLS. ✓
•Prompt arrives at the inference server — decrypted by the TLS termination proxy. Data is now in plaintext in RAM. ✗
•Prompt is tokenized, embedded, and passed to the model — plaintext GPU memory. ✗
•RAG pipeline retrieves context chunks — plaintext in CPU/GPU memory. ✗
•Model generates a response — plaintext. ✗
•Response is sent back to the user — encrypted in transit via TLS. ✓
•Logs and traces are written to disk — encrypted at rest. ✓

Steps 2 through 5 represent the entire span of productive AI work, and all of it happens in plaintext. In a conventional cloud deployment, the cloud operator, a compromised administrator credential, or a hypervisor-level vulnerability can access all of it. For a healthcare organization processing patient records through an LLM, or a law firm running contract analysis, or a financial institution summarizing earnings calls with proprietary trading context, this is not an abstract risk — it is a direct regulatory and competitive exposure.

The Threat Model Confidential Computing Addresses

Security professionals are trained to work from threat models. Confidential computing is specifically designed to address a class of adversaries that other controls cannot reach: privileged insiders and infrastructure-level attackers.

These include:

•Malicious or compromised cloud provider employees with access to the hypervisor or physical host
•Co-tenant attacks in multi-tenant environments where VM escape vulnerabilities might expose memory
•Compromised orchestration layers such as a Kubernetes control plane or a container runtime with elevated privileges
•Supply chain attacks targeting the host OS or virtualization software
•Legal compulsion — a cloud provider served with a government order to produce data from a running workload

Confidential computing does not replace network security, identity management, or application-layer controls. It adds a layer that is specifically effective against the above threats because the hardware itself enforces the isolation. A TEE-protected workload is opaque even to the infrastructure it runs on.

Confidential Computing in AI: Three Deployment Patterns

Pattern 1: Confidential Inference

The most immediately practical deployment pattern is confidential inference — running the LLM's forward pass (the actual token generation process) inside a TEE. In this pattern:

•The model weights are sealed to the enclave and decrypted only inside it
•Incoming prompts are encrypted by the client with a key that only the attested enclave can access
•The output is encrypted before leaving the enclave

NVIDIA's H100 confidential computing mode makes this feasible at production scale. The H100 can operate in a CC mode where all PCIe traffic between the CPU and GPU is encrypted and the GPU's internal memory is protected by hardware keys that the driver and host OS cannot access.

Pattern 2: Confidential RAG Pipelines

Retrieval-Augmented Generation (RAG) pipelines introduce additional surfaces beyond pure inference: the vector database, the embedding model, the retrieval logic, and the document store. A confidential RAG pipeline runs the entire pipeline — embedding, retrieval, re-ranking, and generation — inside a TEE-protected environment, often a Confidential VM (CVM) that encompasses the full application stack.

This is particularly relevant for enterprises where the documents being retrieved are themselves sensitive: legal case files, financial models, medical records, or classified technical specifications.

Pattern 3: Confidential Fine-Tuning

Fine-tuning an LLM on proprietary data creates a model that encodes organizational knowledge — and a fine-tuning dataset that is often the most sensitive data the organization possesses. Confidential fine-tuning runs the training loop inside a TEE, ensuring that the training data and the resulting model weights are never exposed to the infrastructure provider. The trained model can then be sealed to the enclave, preventing exfiltration even in the event of a host-level compromise.

Regulatory Alignment: Why GDPR, DORA, and Sector Regulators Care

Confidential computing has direct relevance to several regulatory frameworks that govern enterprise AI in Europe and beyond.

GDPR Article 25 — Data Protection by Design and by Default

GDPR requires that data protection be built into processing systems from the ground up. The Article 29 Working Party and its successor, the EDPB, have consistently emphasized that "appropriate technical measures" must extend to protection against unauthorized access by processors and sub-processors. Confidential computing provides a hardware-enforced, attestable mechanism for demonstrating that even infrastructure operators cannot access personal data — a substantially stronger technical safeguard than contractual assurances alone.

DORA — Digital Operational Resilience Act

DORA requires financial entities to maintain detailed control over their ICT dependencies, including cloud providers. Where AI workloads process financial data or trading logic, confidential computing provides a technical means to ensure that the cloud provider is a utility that delivers compute cycles without gaining access to the enterprise's sensitive processing. This directly supports the "concentration risk" and "third-party risk" provisions of DORA.

NIS2 and Critical Infrastructure

NIS2 mandates security measures proportionate to the risk for operators of essential services. For critical infrastructure organizations deploying AI — in energy, transport, water, or financial market infrastructure — confidential computing provides a verifiable, hardware-backed control that can be cited in risk assessments and audit evidence packages.

Practical Considerations: What Deployment Teams Need to Know

Performance Overhead

The natural first question is cost. TEE-protected workloads carry overhead, though this has decreased dramatically. AMD SEV-SNP at the VM level typically adds 5–15% CPU overhead for most workloads. NVIDIA H100 in CC mode shows less than 10% throughput reduction for most LLM inference workloads according to published benchmarks. For batch processing and asynchronous workflows, this overhead is often imperceptible at the application layer.

Key Management Integration

Confidential computing shifts but does not eliminate key management complexity. Enterprises need a Key Management Service (KMS) that integrates with attestation — releasing decryption keys to an enclave only after verifying its attestation report. HashiCorp Vault, Azure Managed HSM with attestation integration, and purpose-built solutions from Anjuna and Edgeless Systems provide this capability.

Attestation Verification

Attestation reports must be verified before trusting an enclave. This is not a manual process in production — it should be integrated into the deployment pipeline. The workload should refuse to receive sensitive data until attestation is confirmed. This verification step is also the foundation for audit evidence: a timestamped attestation log demonstrates that processing occurred in a verified, unmodified environment.

Tooling and Ecosystem Maturity

The confidential computing ecosystem has matured significantly since 2021. The Confidential Computing Consortium (a Linux Foundation project) provides open standards and reference implementations. Key projects include:

•Gramine — runs unmodified applications inside Intel SGX enclaves
•Kata Containers — lightweight VMs that can run inside TEEs
•Veraison — an open attestation verification service
•NVIDIA NVTRUST — SDK and tooling for H100 confidential computing

Combining Confidential Computing with Private Deployment

Confidential computing is most powerful when combined with a private, on-premise or sovereign cloud deployment of the AI stack. In this architecture:

•The model runs on hardware that the enterprise controls or has dedicated tenancy on
•The TEE provides runtime protection even against physical host access
•No data leaves the organization's network boundary
•Attestation logs provide a cryptographically verifiable audit trail

This combination — sovereign deployment plus confidential execution plus attestation-backed audit — represents the current state of the art for high-assurance enterprise AI. It satisfies the demands of data protection authorities, sector regulators, and internal security governance programs simultaneously.

What This Means for the CISO

For a CISO evaluating AI deployment options, confidential computing changes the risk calculus in three concrete ways:

•Reduces third-party risk exposure — The cloud or hardware provider cannot access processing data, which shrinks the attack surface and simplifies vendor risk assessments.
•Strengthens audit evidence — Attestation reports provide cryptographic proof of the processing environment, which is far stronger evidence than policy documents or contractual commitments.
•Enables regulated use cases — Workloads previously considered too sensitive for any form of shared infrastructure can now be deployed on cloud or co-located hardware with a defensible security posture.

The technology is no longer experimental. H100 confidential computing is in production at several major financial institutions and healthcare organizations. The tooling is mature enough for enterprise deployment teams to operate without deep hardware security expertise.

Conclusion

The encryption story for enterprise AI has always had a conspicuous gap: the moment data is being processed, it is exposed. Confidential computing closes that gap with hardware-enforced, cryptographically attestable runtime isolation. For enterprises operating under GDPR, DORA, NIS2, or sector-specific regulations — or simply for organizations that take data sovereignty seriously — confidential computing is no longer a forward-looking research topic. It is a deployment option that is available today, cost-effective at scale, and increasingly expected by regulators and enterprise customers alike.

The architecture of private AI is converging on a clear pattern: sovereign deployment, on-premise or dedicated hardware, runtime protection through TEEs, and attestation-backed audit trails. Each layer reinforces the others. Enterprises that build this stack now will find themselves significantly ahead of the compliance curve when regulators and customers begin asking for proof — not promises — that AI workloads are genuinely private.