Information und Technik NRW (IT.NRW) is the IT service provider for the State of North Rhine-Westphalia, Germany's most populous state with 18 million citizens. IT.NRW operates, optimizes, administers, and monitors fundamental areas of the IT infrastructure of the North Rhine-Westphalia state administration.
The demographic change in North Rhine-Westphalia is leading to increasing unfilled positions and knowledge loss in the administration. To maintain operational capacity, deploying new technologies like AI and supporting staff is necessary. Together with Capgemini, the state has developed "NRW Genius," an AI assistant that serves as a building block for an efficient and user-friendly administration.
With 300,000 public sector employees, and 427 municipalities with additional staff, the administration requires highly scalable solutions. Processing large amounts of text-based documents, internal data, and personal data demands secure data handling.
The data centers in North Rhine-Westphalia have limited AI capacities, and building out scalable on-prem GPU infrastructure is time-consuming and expensive. Therefore, secure, GDPR-compliant cloud processing is being pursued. Using a public cloud is essential for scaling. However, much of the data processed by NRW Genius requires strong protection, particularly personal data, which IT.NRW is reluctant to entrust to AI services and cloud providers.
Privatemode AI, the end-to-end confidential AI service offered a perfect solution. Based on confidential computing, data remains encrypted not only at rest and in transit, but also during processing. Even privileged attackers, such as cloud provider admins, can never access any data. Therefore, Privatemode AI enables the scalable integration of cloud-based LLM inference, while keeping data, such as AI prompts, encrypted at all times.
Capgemini easily integrated Privatemode, including the Privatemode proxy for client-side encryption and the Privatemode API, into the NRW Genius AI platform. Several components of NRW Genius remain on-prem, including the frontend, authentication, and vector databases for retrieval augmented generation (RAG) to easily retrieve internal knowledge.
With Privatemode AI, IT.NRW has easily and quickly connected leading open-source AI models like Llama 3.3 70B via a standardized API to the AI assistant NRW Genius and can now use scalable cloud resources for AI inference without security concerns. In an accompanying study, the security mechanisms of Privatemode have been reviewed by an external auditor that confirmed that neither Edgeless Systems, nor the cloud provider, have access to any data in plain text. The performance of the solution meets the users' requirements and, as only standard APIs are used, there is no lock-in, further strengthening IT.NRW's digital sovereignty.
Read the Capgemini blog post on Privatemode, including the case study with IT.NRW.
