Integrating Privatemode AI with GitLab Duo

GitLab Duo brings AI features like code suggestions, chat, code review, and agentic flows into your GitLab instance. With GitLab Duo Self-Hosted you can serve those features from a model you control. This guide enables you to back Duo with Privatemode, so prompts and source code stay end-to-end encrypted and confidential.

Introduction

Run GitLab Duo's AI without exposing your code

Benefits

Why back GitLab Duo with Privatemode

How to get started

Connect Privatemode to GitLab Duo

Introduction

Run GitLab Duo's AI without exposing your source code

GitLab Duo's AI features need a model provider

GitLab Duo brings code suggestions, chat, code review, and agentic flows into your instance, but those features send your prompts and source code to a model provider. For teams working on proprietary code, regulated software, or sensitive IP, that means your codebase leaves your control.

Privatemode keeps your code confidential

Privatemode provides an OpenAI-compatible API backed by state-of-the-art models running inside confidential computing environments. With GitLab Duo Self-Hosted you point Duo's AI Gateway at the Privatemode proxy, and your prompts, source code, and AI responses are encrypted end-to-end. No third party can see your data.

Hardware-enforced, not just policy

The Privatemode proxy encrypts all data before it leaves your network. On the server side, inference runs inside hardware-isolated confidential computing environments (AMD SEV / Intel TDX together with Nvidia Confidential Computing), and the proxy verifies server integrity through remote attestation before every session. Your code is never stored and never used for model training.

Benefits

Why back GitLab Duo with Privatemode?

End-to-end encrypted across every Duo feature

Code suggestions, chat, code review, and agentic flows all run through the Privatemode proxy, which encrypts every prompt and every file before it leaves your network. Inference runs inside confidential computing environments, and the proxy verifies the server through remote attestation before each session. Your source code stays confidential, even during inference.

A model you control, no silent fallbacks

With GitLab Duo Self-Hosted, you assign your own model to every feature, so nothing routes to a GitLab-managed gateway. The proxy exposes a standard OpenAI-compatible API, so Duo's AI Gateway connects to it like any other endpoint.

Confidential AI in your editor too

Because model selection lives on the instance, the GitLab Workflow extension for VS Code and JetBrains inherits the same setup. In-editor chat and code suggestions route through the same gateway and proxy.

Start for free

Overview

How GitLab Duo connects to Privatemode

Duo Self-Hosted talks to a self-hosted AI Gateway, which forwards inference to any OpenAI-compatible endpoint. The Privatemode proxy is that endpoint; it exposes a standard /v1 API and handles the encryption and attestation against the Privatemode service.

You run the AI Gateway and the proxy yourself. The steps below are deployment-agnostic (Linux package, Docker, Helm/Kubernetes); follow the linked docs for the specifics of your setup.

Step 0

Prerequisites

Before you start, make sure you have the following ready:

GitLab Self-Managed, Premium or Ultimate, 17.9+, with the GitLab Duo Enterprise add-on.
For the agentic flows (step 4): the GitLab Duo Agent Platform, GA on 18.8+. On offline licenses it needs the Agent Platform Self-Hosted add-on; on online licenses with 18.7/18.8 you may need to enable beta features.

Hybrid vs. fully self-hosted: any feature left on a GitLab-managed model routes to GitLab's hosted gateway, not yours. To keep everything confidential, assign your self-hosted model to every feature you use (step 3).

Step 1

Set up Privatemode

Get your API key

If you don't have a Privatemode API key yet, you can generate one for free here.

Run the Privatemode proxy

Follow the Privatemode API quickstart. Enabling the shared prompt cache (--sharedPromptCache) is recommended for this workload.

The proxy serves an OpenAI-compatible API on :8080/v1. Make sure it's reachable from the AI Gateway and note that URL (e.g. http://privatemode-proxy:8080/v1 on a shared Docker network) — you'll need it in step 3.

Test if proxy is up and reachable

Confirm the proxy is up and which models it serves before moving on. The returned model IDs are what you'll enter as the Model identifier in step 3.

Step 2

Deploy the GitLab AI Gateway

Install per Install the GitLab AI Gateway. Key points:

Image version: use the latest self-hosted-vX.Y.*-ee tag matching your GitLab major.minor(patch need not match). Re-check on each GitLab upgrade.
Hostname: deploy at a real network hostname — the docs advise against localhost for the gateway.
GitLab connection: set the gateway's GitLab base URL and API URL.
JWT keys: configure both AIGW_SELF_SIGNED_JWT__* (the gateway itself, used by e.g. Duo Chat) and DUO_WORKFLOW_SELF_SIGNED_JWT__* (the Agent Platform service).
Timeout: the AI Gateway request timeout (60–600 s) is worth raising for slower self-hosted backends.
Certificates: if GitLab or the proxy uses a custom CA, add it via REQUESTS_CA_BUNDLE or the container's CA bundle.

The gateway prefetches GitLab's JWKS keys at startup; if GitLab isn't reachable yet, the Agent Platform (gRPC) half can fail to start. Bring GitLab up first, or have your orchestration wait for it.

Step 3

Register Privatemode as a self-hosted model

In Admin → GitLab Duo → Self-hosted (configure docs):

GitLab Duo Self-Hosted configuration form

Connect the gateway

Set the Local AI Gateway URL (and the Agent Platform service URL if used). Leave TLS off only if the gateway serves plain HTTP.

If that URL is a private IP or internal hostname, add it to the outbound requests allowlist or GitLab may block it.

Add the model

Add self-hosted model, with:

Endpoint: the proxy's /v1 URL (e.g. http://privatemode-proxy:8080/v1), reachable from the gateway
Platform: API
Model family: General (for OpenAI-compatible models outside a supported family)
Model identifier: the model name the proxy serves, e.g. kimi-latest (see Privatemode models)
API key: leave empty — the proxy holds the Privatemode credentials

Generic OpenAI-compatible models are a beta feature; GitLab doesn't guarantee support for model-specific issues.

Assign to features

Select the model for every feature you use (Code Suggestions, Chat, …) so none falls back to a GitLab-managed model.

Verify

Run the health check in Admin → GitLab Duo and/or gitlab-rake gitlab:duo:verify_self_hosted_setup. Then try Duo Chat or a Code Suggestion.

Step 4

Enable Duo Agent Platform flows

Foundational flows like Code Review run as CI jobs, so they need a GitLab Runner:

An executor that runs Docker images (Docker or Kubernetes — not shell), able to reach the gateway and proxy.
The literal tag gitlab--duo (two dashes) — flow jobs won't be picked up without it.
Privileged mode when using the default flow image or a custom image with the sandbox runtime, so the execution sandbox works.
An instance runner, or one on the top-level group — subgroup/project runners aren't eligible.

Then enable Allow flow execution and Allow foundational flows at the instance/group/project level, plus the specific flow (e.g. Code Review, also under a project's Settings → Merge requests). Test by assigning @GitLabDuo as an MR reviewer, or assigning an issue to Duo and confirming it opens a merge request.

Step 5

Use it from your IDE

Since model selection lives on the instance, the GitLab Workflow extension (VS Code, JetBrains, …) inherits this setup: point it at your instance, sign in, and in-editor Chat and Code Suggestions route through the same gateway and Privatemode proxy — confidential AI directly in the editor.