GenAI Cracow #29 - Enterprise AI

​Abstract

​Keeping PII Out of External LLMs: Detection, Substitution, and the Gaps

Employees paste client data into external LLMs every day, and portal-level blocks just push the behavior into shadow IT. Keeping that data in-house means intercepting traffic before it leaves the network — detecting PII on the request side, substituting it, and restoring the original values in the response so the workflow stays intact. Detection is where it gets hard: context-dependent identifiers, inflected forms, and entities never named outright open gaps that regex and off-the-shelf NER miss. And even where detection holds, the data isn't anonymous — it's masked, which under GDPR is pseudonymization, not anonymization, changing what a company is permitted to send to a cloud model at all. The throughline is the same question that drives on-device AI: how to get useful AI without shipping people's data to someone else's servers.

​Balancing costs, reliability, and regulatory compliance in LLM Engineering

AI system architecture is about making decisions under constraints, where matching the scope of the experiment to the problem determines the feasibility of implementation. Because full integration with a regulated environment is a multi-layered process, this analysis is deliberately narrowed to a critical, measurable segment: verifying non-functional requirements. A series of PoCs for a voice/chatbot based on hundreds of thousands of queries proves that specific LLM engineering techniques achieve rigorous reliability, repeatability, and time efficiency without requiring the most expensive models. Estimating operating costs and validating pipeline stability connect the technical engine room to business realities, forming a foundation for architectures compliant with the DORA Directive and BaFin guidelines.

​Speakers

​Norbert Klockiewicz

Software engineer on the AI team at Software Mansion, working on on-device AI. Core maintainer of react-native-executorch, which runs LLMs and vision models directly on phones, and co-builder of Private Mind, a fully on-device AI app for iOS and Android where no user data leaves the device.

​Krzysztof Chruniak

Lead Architect of Data & AI Products @ Munich Re/Ergo Technology Services with 20+ years of experience in system architecture and technical leadership. A pragmatic engineer and manager focused on turning complex data into real business value while navigating strict regulatory constraints, including DORA and BaFin/UKNF. Proven success in delivering high-impact enterprise solutions, such as developing an AI Factory (a platform for building and operationalizing AI models), optimizing data architecture to reduce processing time without additional financial investments (core banking system), and launching a banking product that attracted a few million PLN in deposits. Skilled in designing scalable, cost-optimized cloud architectures and building high-performing, self-organizing teams.

​Partners

​Software Mansion

Software Mansion is a Kraków-based software consultancy best known for their React Native work, and increasingly for pushing on-device AI into mobile. They built React Native ExecuTorch, which lets you run PyTorch models directly on a device, and they’re active contributors to the React Native core. If you’re building AI-powered mobile apps and care about what happens at the framework level, these are the people doing that work.

​CEE AI Hub