sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam.sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
Information und Technik NRW (IT.NRW) is the IT service provider for the State and the Statistical State Office of North Rhine-Westphalia, Germany's most populous state. IT.NRW operates, optimizes, administers, and monitors fundamental areas of the IT infrastructure of the North Rhine-Westphalia state administration.
The demographic change in North Rhine-Westphalia is leading to increasing unfilled positions and knowledge loss in the administration. To maintain operational capacity, deploying new technologies like AI and supporting staff is necessary. Together with Capgemini, the state has developed "NRW Genius," an AI assistant that serves as a building block for an efficient and user-friendly administration.
With approximately 18 million citizens, 300,000 public sector employees, and 427 municipalities with additional staff, the administration requires highly scalable solutions. Processing large amounts of text-based documents, internal data, and personal data demands secure data handling.
The data centers in North Rhine-Westphalia have limited AI capacities, and acquiring new GPU hardware is time-consuming and expensive. Therefore, secure, GDPR-compliant cloud processing is being pursued. Using a public cloud is essential for scaling. However, much of the data processed by NRW Genius requires high protection, particularly personal data, which IT.NRW is reluctant to entrust to AI services or cloud providers.
Read the Capgemini blog post
Privatemode AI is a confidential computing-based AI service and it offers a suitable approach for this purpose. With confidential computing, data remains encrypted not only during storage and in transit, but also during processing. The CPU and GPU hardware in the backend can selectively decrypt data within their internal structures, which are inaccessible from the outside–even for privileged attackers such as system administrators or cloud provider data center staff.
In short, Privatemode AI enables the scalable integration of a large language model (LLM) from a public cloud, while keeping data, such as AI prompts, encrypted at all times and thus protected from access by the cloud or model provider.
With Privatemode AI, IT.NRW has easily and quickly connected a leading open-source AI model (currently Llama 3.3 70B) via a standardized API to the AI assistant NRW Genius and can now use public cloud resources for AI inference without security concerns. Furthermore, the central security mechanisms are currently being reviewed by an external party as part of a study.
