Securing GenAI Applications with Zscaler AI Guard and NVIDIA NeMo Guardrails

As enterprises accelerate their adoption of generative AI and large language models, security teams face an unprecedented challenge: protecting these powerful systems from a rapidly evolving threat landscape. From prompt injection attacks and data exfiltration to malicious content generation and jailbreak attempts, the security risks surrounding AI deployments are both diverse and complex. Organizations need robust, enterprise-grade security controls that can operate at the speed of AI without compromising innovation or user experience.

Zscaler and NVIDIA are collaborating to address these security controls concerns by integration between Zscaler AI Guard and NVIDIA NeMo Guardrails, delivering a powerful defense-in-depth approach to securing generative AI applications. This integration enables security teams and AI application developers to deploy comprehensive, layered protection for LLM-powered systems with minimal configuration overhead and maximum operational efficiency.

The Growing Challenge of Securing Generative AI

The proliferation of AI is creating an entirely new category of operational, security, and compliance challenges for enterprises. Prompt injection attacks can manipulate AI systems into performing unintended actions or revealing sensitive information. Data leakage through AI interactions can expose proprietary information, customer data, and trade secrets. Jailbreak attempts seek to bypass safety controls, while toxic or off-topic responses can damage brand reputation and erode customer trust. 

Traditional security approaches were not designed for the unique characteristics of AI workloads. The dynamic, conversational nature of LLM interactions requires real-time inspection and policy enforcement that can adapt to context while maintaining low latency. Organizations need purpose-built solutions that understand the nuances of AI communication patterns, can inspect prompts and responses in real-time, and enforce security policies without introducing unacceptable latency or degrading user experience. 

NVIDIA NeMo Guardrails: The Foundation for Programmable AI Safety

NVIDIA NeMo Guardrails is an open-source toolkit that enables developers to add programmable guardrails to LLM-based applications. The framework provides a flexible, extensible architecture for defining, orchestrating, and enforcing AI guardrails between application code and language models. Using NeMo Guardrails, developers can control conversation flows, enforce topic boundaries, prevent unwanted digressions, and integrate external security services directly into the AI application pipeline.

NeMo Guardrails supports both input rails, which process user prompts before they reach the LLM, and output rails, which analyze model responses before they are returned to users. This bidirectional control enables comprehensive protection across the entire AI interaction lifecycle. The framework's extensible action system allows seamless integration with external security services through custom Python actions, creating opportunities for enterprise-grade protection that goes beyond what any single tool can provide alone.

Zscaler AI Guard: Enterprise-Grade AI Runtime Protection

Zscaler AI Guard provides comprehensive runtime protection for AI users and applications by enforcing enterprise AI policies on prompts and responses between end entities and LLMs. Built on the industry's most proven inline inspection platform, AI Guard delivers real-time threat detection, data loss prevention, and content moderation capabilities purpose-built for generative AI workloads.

AI Guard features more than 18 specialized detectors with multilingual support, covering prompt injection, jailbreak attempts, malicious URLs, secrets exposure, PII detection, toxicity, brand reputation risks, competition mentions, and more. The platform leverages AI-powered engines to achieve high accuracy with low false positive rates, ensuring legitimate AI interactions proceed without unnecessary friction while blocking genuine threats. Severity settings allow organizations to fine-tune detection thresholds to match their specific risk tolerance and use case requirements.

Bringing It Together: Defense-in-Depth for LLM Applications

The integration between Zscaler AI Guard and NVIDIA NeMo Guardrails creates a powerful layered defense strategy. NeMo Guardrails provides the orchestration framework and pattern-based defenses, while AI Guard adds dynamic, real-time threat analysis powered by Zscaler's cloud-delivered security services. Together, these solutions enable AI application builders to easily add protection against a comprehensive list of attacks and misuse scenarios.

The integration works by implementing a custom action within NeMo Guardrails that calls the Zscaler AI Guard API. When a user submits a prompt, NeMo Guardrails first evaluates the input against its configured guardrails. If the prompt passes initial checks, the AI Guard action inspects the content for threats, policy violations, and sensitive data. The same process applies to LLM responses, creating a comprehensive inspection pipeline that covers both sides of every AI interaction.

Key Use Cases and Benefits

Defense-in-Depth Security: By combining semantic guardrails with an independent threat detection service, developers gain a critical second line of defense against prompt injection, data leakage, and malicious inputs. This layered approach ensures that even if one control is bypassed, additional protections remain in place to catch threats.

Simplified Deployment: Inserting AI Guard as a custom action within NeMo Guardrails enables application and security teams to centralize enforcement of AI security policies. This approach eliminates the need to implement controls across every instance of application code, reducing complexity and ensuring consistent protection across all AI applications.

Preventing Prompt Injections: AI Guard's detection capabilities identify multiple types of prompt injection attacks across eight languages, working in tandem with NeMo Guardrails' configurable guardrail definitions to block malicious manipulation attempts before they reach the LLM.

Enforcing Data Privacy: Organizations can leverage AI Guard's powerful data loss prevention capabilities to block sensitive data leaks during AI interactions. With over 100 predefined dictionaries for identifying PII, secrets, API keys, and confidential information, enterprises can maintain compliance with regulations like GDPR and CCPA while enabling productive AI use.

Content Moderation and Brand Protection: AI Guard monitors and prevents toxic, off-topic, or harmful content that could damage brand reputation. Purpose-built detectors identify inappropriate responses, competitive mentions, and content that violates organizational policies, enabling safe deployment of customer-facing AI applications.

Getting Started

The integration requires NVIDIA NeMo Guardrails 0.18.0 or higher and a Zscaler AI Guard subscription with API access enabled. The implementation involves configuring a custom action that bridges NeMo Guardrails with the AI Guard API, enabling seamless inspection of both prompts and responses within your existing guardrails configuration. The setup can be completed in minutes using the provided documentation and sample configurations.

The integration is available now on GitHub at github.com/zscaler/aiguard-nemo-guardrails. The repository includes source code, configuration examples, and detailed setup instructions to help you deploy AI Guard protection within your NeMo Guardrails environment.

Building Confidence in AI Deployments

As organizations find new and innovative ways to harness the power of large language models, AI applications, and agentic workflows to increase efficiencies, the need for robust security controls becomes paramount. The integration of Zscaler AI Guard with NVIDIA NeMo Guardrails provides developers and security teams with the layered defense mechanisms needed to confidently deploy LLMs in production environments.

Zscaler is committed to making AI security accessible, flexible, and easy for developers to consume regardless of how they build applications. AI Guard and NeMo Guardrails together represent a highly customizable and robust security solution for protecting LLMs, agentic workflows, and a variety of other AI-powered use cases. The combination of proactive, pattern-based defenses with dynamic, real-time analysis establishes an enhanced security strategy that blocks known attacks while mitigating emerging threats targeting AI runtime behavior.

We look forward to our continued collaboration with NVIDIA to solve the AI security challenges organizations face as they embrace generative AI transformation. To learn more about Zscaler AI Guard and explore the integration, visit our product page or check out the GitHub repository to get started today.

Resources:

• Zscaler AI Guard

• GitHub Integration

• NVIDIA NeMo Guardrails