Confidential containers in Kubernetes: what a real operator exclusion requires

When is data in the cloud actually encrypted?

Encryption at rest and in transit protects data in storage and on the wire. It does not protect data while it is being processed. In that gap, data sits in memory as plaintext, in principle reachable by infrastructure operators, administrators, and other parties.

Confidential computing closes it. Hardware-based Trusted Execution Environments keep memory and the data it contains encrypted even during processing, and logically shield the processing. The associated remote attestation makes these security mechanisms remotely verifiable. The result is a technical operator exclusion.

What does such an implementation actually look like?

This is about Kubernetes, the default orchestration platform for cloud-native applications. Kubernetes spreads workloads across nodes. A node is a single machine in the cluster. Each node normally runs several containers, which are grouped into so-called pods.

Using confidential computing, whole nodes can be isolated ("Confidential Cluster"), or individual pods ("Confidential Containers"):

• Confidential Cluster: the entire node runs as one encrypted VM. Every pod on that node shares the same isolation boundary as the Kubernetes management processes. They are not separated from each other or from those processes. Remote attestation can only confirm that the node as a whole is trustworthy, not what any individual pod inside it is doing.
• Confidential Containers: each pod runs in its own, independent encrypted micro-VM, separate from the node's Kubernetes processes and from other pods. Remote attestation then works for each pod individually.

The paper's verdict is clear: only pod-level isolation removes Kubernetes from the trust base entirely, works with managed Kubernetes, and makes remote attestation possible per workload. Under node-level isolation, the Kubernetes management processes stay inside the same protection boundary as the pods, and so keep effective access to them. The infrastructure operator remains part of the trust base as a result.

Workload-based isolation via confidential containers is, therefore, the only option that delivers a real operator exclusion.

Bare metal, for now

Confidential containers can be run in three ways:

• directly on bare-metal nodes
• nested inside a virtualized environment via "nested virtualization"
• through an infrastructure operator's hypervisor API

Only the bare-metal path is production-ready today, and already available on some managed Kubernetes offerings. Nested virtualization looks like the long-term winner but is still early-stage. The path via the hypervisor API does not yet offer complete remote attestation, so a full operator exclusion is not possible there for now.

In practice: frontier AI models via Privatemode AI

The paper's second half applies the preceding to Privatemode AI. In Privatemode, a local proxy initially verifies every server-side component. The proxy then negotiates a cryptographic key with the server side. On the server side, only those confidential containers that have previously proven their own integrity to a central management service via remote attestation gain access to the key. Inference itself runs entirely inside confidential containers, incorporating NVIDIA GPUs with confidential-computing mode enabled.

The result is designed so that neither the infrastructure operator nor Edgeless Systems can access prompts or responses.

The full paper

The full paper explains the architecture, the trade-offs between the three implementation paths, and the complete Privatemode AI design in detail. Find it starting on page 82 in the proceedings of the 21st German IT Security Congress.