Can GitLab still fall back to a GitLab-managed model?

Only for features you haven't assigned. GitLab Duo can run hybrid — any feature left on a GitLab-managed model routes to GitLab's hosted gateway rather than to your AI Gateway. Assign your self-hosted model to every feature you use, and re-check after upgrades that add new Duo features.

Which GitLab edition and version do I need?

GitLab Self-Managed, Premium or Ultimate, 17.9 or later, with the GitLab Duo Enterprise add-on. The agentic flows additionally need the GitLab Duo Agent Platform, which is generally available on 18.8 and later: on offline licenses it needs the Agent Platform Self-Hosted add-on, and on online licenses with 18.7 or 18.8 you may need to enable beta features.

How do I enable the Duo Agent Platform flows?

Flows run as CI jobs, so they need a GitLab Runner that can reach the gateway and the proxy, with a Docker-based executor (docker, docker-autoscaler, or kubernetes; the shell executor is not supported). Give it the literal tag `gitlab--duo` with two dashes, or flow jobs won't be picked up, and set `privileged = true` so the execution sandbox works. The runner has to be an instance runner or a group runner on the top-level group; project and subgroup runners are not eligible without a feature flag. Then enable flow execution for the groups or projects that should use it and turn on the flows you want, following [Configure flow execution](https://docs.gitlab.com/user/duo_agent_platform/flows/execution/).

Does this also work in my IDE?

Yes, without any extra setup. Model selection lives on the GitLab instance, so the GitLab Workflow extension for VS Code, JetBrains, and others inherits it. Point the extension at your instance and sign in, and in-editor chat and code suggestions route through the same gateway and Privatemode proxy.

Duo features fail after the setup. What do I check?

Start with the health check in Admin → GitLab Duo. Common causes: if the Agent Platform service in the gateway refuses to start, set `AIGW_CUSTOMER_PORTAL_URL` and `DUO_WORKFLOW_AUTH__OIDC_CUSTOMER_PORTAL_URL` to your GitLab URL. If GitLab or the proxy uses a custom CA, pass it to the gateway container via `SSL_CERT_FILE` and `REQUESTS_CA_BUNDLE`. For slower backends, raise the AI Gateway request timeout, which accepts 60 to 600 seconds. And confirm the gateway image tag still matches your GitLab `major.minor` after an upgrade.

Integrating Privatemode AI with GitLab Duo

GitLab Duo brings code suggestions, chat, and code review into your GitLab instance, and Duo Self-Hosted lets you serve them from a model endpoint of your choice. Backing it with Privatemode means those features run on open-weight models served by Privatemode instead of a vLLM deployment on your own GPUs, with prompts and source code encrypted in memory even during processing.

Introduction

The challenge

Your own model means your own GPUs

Duo Self-Hosted lets regulated and air-gapped organizations run GitLab Duo on a model they control. Doing that yourself means operating a GPU fleet: running vLLM, planning capacity, and scheduling upgrades.

The solution

Your own model, without the hardware

Your AI Gateway connects to the Privatemode proxy as an OpenAI-compatible endpoint. The proxy encrypts every request before it leaves your network, and the model runs in a hardware-isolated environment, so by design neither the infrastructure provider nor Privatemode can read your code.

Featured by GitLab

Read GitLab’s take on confidential AI.

Confidential AI for GitLab Self-Hosted

Learn how GitLab Duo Self-Hosted and Privatemode give regulated teams AI coding agents while keeping source code inside a hardware-encrypted boundary.

Read on GitLab

Benefits

Why use Privatemode AI with GitLab Duo?

Connects through Duo Self-Hosted

No GPUs needed for confidential AI

End-to-end confidentiality

Learn how

Free tier available

How to get started

How to back GitLab Duo with Privatemode

You need GitLab Self-Managed, Premium or Ultimate, 17.9+, with the GitLab Duo Enterprise add-on. You run both the AI Gateway and the Privatemode proxy yourself, so the steps below are deployment-agnostic (Linux package, Docker, Helm) — follow the linked GitLab docs for the specifics of your setup. The setup has three parts: run the Privatemode proxy, deploy GitLab's AI Gateway, and register the proxy as a self-hosted model in GitLab.

Get your API key

If you don't have a Privatemode API key yet, you can generate one for free here.

Run the Privatemode proxy

Follow the Privatemode API quickstart. Enabling the shared prompt cache (--sharedPromptCache) is recommended for this workload.

The proxy serves an OpenAI-compatible API on :8080/v1. Make sure it's reachable from the AI Gateway and note that URL — for example http://privatemode-proxy:8080/v1 on a shared Docker network.

Check which models the proxy serves

This confirms the proxy is up and lists the models it serves. One of the returned IDs is what you enter as the Model identifier when you add the model below.

Deploy the GitLab AI Gateway

Install it per Install the GitLab AI Gateway, at a real network hostname rather than localhost, using the self-hosted-vX.Y.*-ee tag that matches your GitLab major.minor. The snippet shows self-hosted-v18.8.0-ee as an example only. Re-check that tag on every GitLab upgrade.

For the Duo Agent Platform, set the DUO_WORKFLOW_SELF_SIGNED_JWT__* keys alongside the gateway's own.

Point GitLab at the gateway

In Admin → GitLab Duo → Change configuration (configure docs), set the Local AI Gateway URL, and the Local URL for the GitLab Duo Agent Platform service (port 50052, gRPC) if you use it. Tick Use beta models and features in GitLab Duo Self-Hosted: models outside GitLab's supported families, Privatemode's included, count as compatible models, which are in beta.

If that URL is a private IP or an internal hostname, add it to the outbound requests allowlist or GitLab may block it.

Add Privatemode as a self-hosted model

Under Admin → GitLab Duo → Configure models for GitLab Duo, choose Add self-hosted model, with:

Deployment name: anything that identifies it, e.g. Privatemode GLM-5.3
Model family: General (GitLab's family for compatible models outside the supported list)
Endpoint: the proxy's /v1 URL (e.g. http://privatemode-proxy:8080/v1), reachable from the gateway
API key: leave empty — the proxy holds the Privatemode credentials
Model identifier: custom_openai/ followed by a model ID the proxy serves, e.g. custom_openai/glm-latest (see Privatemode models)

Compatible models are a beta feature, and GitLab does not provide technical support for issues specific to the model.

Assign the model to every Duo feature

On the AI-native features tab, select the Privatemode deployment for each feature you use — Code Suggestions, Chat, Code Review, and so on.

Any feature left on a GitLab-managed model keeps routing to GitLab's hosted gateway instead of yours, so assigning all of them is what keeps the whole setup confidential.

Verify the setup

Run the health check in Admin → GitLab Duo, or gitlab-rake "gitlab:duo:verify_self_hosted_setup[<username>]", then try Duo Chat or a code suggestion.

From here on, prompts and source code leave your network encrypted, and the model runs inside a hardware-isolated environment whose memory stays encrypted during processing.

FAQ

Frequently asked questions about using Privatemode with GitLab Duo

Yes. The Privatemode proxy verifies the Privatemode service through remote attestation and encrypts every request before it leaves your network. On the server side, requests are processed inside a hardware-isolated environment whose memory stays encrypted, so by design neither the infrastructure provider nor we can read them. Privatemode never learns from your data, and no prompts or responses are retained after processing.