(feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo · Pull Request #17175 · BerriAI/litellm

ghost · 2025-11-27T03:34:37Z

Title

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
I have added a screenshot of my new test passing locally
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

Note

Introduces a Generic Guardrail API integration (configurable, pre/during/post hooks) plus a mock Bedrock/Generic server and documentation, enabling instant third‑party guardrail support without PRs.

Guardrails (Core Integration):
- Add generic_guardrail_api integration with GenericGuardrailAPI hook to call external POST /beta/litellm_basic_guardrail_api endpoints.
- Register in guardrail_initializer_registry and guardrail_class_registry; support modes: pre_call, post_call, during_call.
- Extend types: SupportedGuardrailIntegrations.GENERIC_GUARDRAIL_API, optional additional_provider_specific_params, and config models for UI.
- Include example config example_config.yaml with dynamic params support.
Example Server (Cookbook):
- Add mock_bedrock_guardrail_server.py FastAPI server mimicking Bedrock ApplyGuardrail and the Generic Guardrail API endpoint.
- Implements token auth, blocked words/topics, PII detection/anonymization, config get/update, and health endpoints.
Docs:
- Add [BETA] Generic Guardrail API guide describing contract, config, and usage.
- Update sidebar to surface the new guide.

^{Written by Cursor Bugbot for commit 27b0468. This will update automatically on new commits. Configure here.}

Allows guardrail providers to work with litellm for guardrails without needing to make a PR to LiteLLM

vercel · 2025-11-27T03:34:43Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
litellm	Ready	Preview	Comment	Dec 1, 2025 10:31pm

cursor

This is the final PR Bugbot will review for you during this billing cycle

Your free Bugbot reviews will reset on December 25

Details

You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

cursor · 2025-11-27T03:38:03Z

+        guardrail_name=guardrail.get("guardrail_name", ""),
+        event_hook=litellm_params.mode,
+        default_on=litellm_params.default_on,
+    )


Bug: Headers from config not passed to GenericGuardrailAPI

The initialize_guardrail function creates a GenericGuardrailAPI instance without passing the headers parameter from the configuration. The GenericGuardrailAPI.__init__ expects headers as a parameter (used for authentication), but only api_base, api_key, and additional_provider_specific_params are passed. Since headers isn't forwarded, any authentication headers (like Authorization: Bearer ...) configured in the YAML will be ignored, and API calls to the external guardrail will fail authentication. The api_key being passed is also unused since GenericGuardrailAPI doesn't have an api_key parameter—it expects authentication via headers.

@cursor can you fix this?

Perfect! I've fixed all three issues identified in the PR review:

Fixed Issues

1. Headers not passed to GenericGuardrailAPI (High Severity)

File: litellm/proxy/guardrails/guardrail_hooks/generic_guardrail_api/__init__.py

Fix: Added headers=getattr(litellm_params, "headers", None) to pass authentication headers from the config to the GenericGuardrailAPI instance

Removed: The unused api_key parameter that wasn't accepted by GenericGuardrailAPI

2. Request body always empty (Medium Severity)

File: litellm/proxy/guardrails/guardrail_hooks/generic_guardrail_api/generic_guardrail_api.py

Fix: Changed request_body={} to request_body=request_data to pass the full original request for context to the external guardrail API

3. Regular expression injection vulnerability (High Severity - Security)

File: cookbook/mock_guardrail_server/mock_bedrock_guardrail_server.py

Fix: Added try-except block to handle invalid regex patterns safely by:

Compiling the pattern first (which validates it)

Catching re.error exceptions for invalid patterns

Logging warnings and continuing instead of crashing

This prevents malicious regex patterns from being executed

All changes passed linter checks with no errors.

_{Learn more about Cursor Agents}

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

CLAassistant · 2025-12-01T06:00:56Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
0 out of 2 committers have signed the CLA.

❌ Krrish Dholakia
❌ cursoragent

Krrish Dholakia seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

cursor · 2025-12-01T06:00:58Z

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
_{Learn more about Cursor Agents}

+    for pii_type, pattern in GUARDRAIL_CONFIG.pii_patterns.items():
+        try:
+            # Compile the regex pattern with a timeout to prevent ReDoS attacks
+            compiled_pattern = re.compile(pattern)


To resolve the issue, you should sanitize any user-supplied regex patterns before compiling them. In Python, this can be accomplished using re.escape, which escapes all regex metacharacters, ensuring the input can only be interpreted literally and not as a regular expression. The best approach here is to escape all patterns supplied via GUARDRAIL_CONFIG.pii_patterns before compiling them in the Content Safety/PII detection workflow.

Edit the block where pattern is used (in line 254) to apply re.escape. If you need to support advanced configuration where some patterns are meant to be real regexes (not only literal matches), you should validate or add an explicit configuration and only escape untrusted sources, but in this context, full escaping is safest.

The only change required is to edit the region (lines 251-254) inside the anonymization logic, replacing the compile step to use an escaped version of the pattern.

No new imports are required.

Update Pillar Security documentation to use the generic_guardrail_api instead of the dedicated pillar guardrail type. This aligns with the new Generic Guardrail API pattern introduced in BerriAI#17175. Changes: - Rewrite pillar_security.md to use generic_guardrail_api configuration - Add Pillar as an example in generic_guardrail_api.md - Add Pillar example to quick_start.md guardrails list This migration leverages improvements made to the Generic Guardrail API: - BerriAI#17175: Generic Guardrail API introduction - BerriAI#18647: Generic guardrail API updates - BerriAI#18932: SerializationIterator fix and tools support - BerriAI#19023: Clean error messages for blocked requests

Update Pillar Security integration to use the generic_guardrail_api instead of the dedicated pillar guardrail type. This aligns with the Generic Guardrail API specification introduced in previous PRs. Changes: - Rewrite pillar_security.md with new generic_guardrail_api config - Add Pillar Security example to generic_guardrail_api.md - Add Pillar Security to quick_start.md guardrails examples Related PRs: BerriAI#17175, BerriAI#18647, BerriAI#18932, BerriAI#19023

Update Pillar Security integration to use the generic_guardrail_api instead of the dedicated pillar guardrail type. This aligns with the Generic Guardrail API specification introduced in previous PRs. Changes: - Rewrite pillar_security.md with new generic_guardrail_api config - Add Pillar Security example to generic_guardrail_api.md - Add Pillar Security to quick_start.md guardrails examples Related PRs: #17175, #18647, #18932, #19023

…ANT support for LiteLLM w/out PR to repo (BerriAI#17175) * feat(generic_guardrail_api.py): new generic api for guardrails Allows guardrail providers to work with litellm for guardrails without needing to make a PR to LiteLLM * docs(generic_guardrail_api.md): document new generic guardrail api * Fix: Improve PII detection and guardrail API integration Co-authored-by: krrishdholakia <krrishdholakia@gmail.com> * feat: correctly extract raw request from guardrail api * docs(generic_guardrail_api.md): document this is a beta feature --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Update Pillar Security integration to use the generic_guardrail_api instead of the dedicated pillar guardrail type. This aligns with the Generic Guardrail API specification introduced in previous PRs. Changes: - Rewrite pillar_security.md with new generic_guardrail_api config - Add Pillar Security example to generic_guardrail_api.md - Add Pillar Security to quick_start.md guardrails examples Related PRs: BerriAI#17175, BerriAI#18647, BerriAI#18932, BerriAI#19023

Krrish Dholakia added 2 commits November 26, 2025 19:29

feat(generic_guardrail_api.py): new generic api for guardrails

438e0f1

Allows guardrail providers to work with litellm for guardrails without needing to make a PR to LiteLLM

docs(generic_guardrail_api.md): document new generic guardrail api

f076e98

cursor Bot reviewed Nov 27, 2025

View reviewed changes

github-advanced-security AI found potential problems Nov 27, 2025

View reviewed changes

Comment thread cookbook/mock_guardrail_server/mock_bedrock_guardrail_server.py Fixed

Fix: Improve PII detection and guardrail API integration

c093661

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

vercel Bot deployed to Preview December 1, 2025 06:02

View deployment

github-advanced-security AI found potential problems Dec 1, 2025

View reviewed changes

Krrish Dholakia added 2 commits December 1, 2025 14:27

feat: correctly extract raw request from guardrail api

40e9d2f

docs(generic_guardrail_api.md): document this is a beta feature

2e3293c

vercel Bot deployed to Preview December 1, 2025 22:29

View deployment

Merge branch 'main' into litellm_dev_11_26_2025_p2

27b0468

ghost merged commit b6d6f83 into main Dec 1, 2025
8 of 11 checks passed

ghost deleted the litellm_dev_11_26_2025_p2 branch December 1, 2025 22:29

vercel Bot deployed to Preview December 1, 2025 22:31

View deployment

This was referenced Jan 19, 2026

docs: migrate Pillar Security to Generic Guardrail API #19362

Closed

Migrate Pillar Security guardrail to Generic Guardrail API #19363

Closed

This was referenced Jan 19, 2026

docs: Migrate Pillar Security to Generic Guardrail API #19364

Merged

docs: Migrate Pillar Security to Generic Guardrail API #19365

Closed

This pull request was closed.

@@ -250,8 +250,9 @@
                 for pii_type, pattern in GUARDRAIL_CONFIG.pii_patterns.items():
                     try:
-                        # Compile the regex pattern with a timeout to prevent ReDoS attacks
-                        compiled_pattern = re.compile(pattern)
+                        # Sanitize the regex pattern to prevent injection attacks and ReDoS
+                        safe_pattern = re.escape(pattern)
+                        compiled_pattern = re.compile(safe_pattern)
                         matches = compiled_pattern.finditer(text)
                         for match in matches:
                             matched_text = match.group()

Navigation Menu

Repository navigation

(feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo - #17175

(feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo #17175
6 commits merged into
mainfrom
litellm_dev_11_26_2025_p2

ghost commented Nov 27, 2025 •

edited by cursor Bot

Loading

Uh oh!

vercel Bot commented Nov 27, 2025 •

edited

Loading

Uh oh!

cursor Bot left a comment

Uh oh!

cursor Bot Nov 27, 2025

Uh oh!

ghost Dec 1, 2025

Uh oh!

cursor Bot Dec 1, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

CLAassistant commented Dec 1, 2025 •

edited

Loading

Uh oh!

cursor Bot commented Dec 1, 2025

Uh oh!

Check failure

Copilot Autofix

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ghost commented Nov 27, 2025 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Title

Relevant issues

Pre-Submission checklist

Type

Changes

Uh oh!

vercel Bot commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

This is the final PR Bugbot will review for you during this billing cycle

Uh oh!

cursor Bot Nov 27, 2025

Choose a reason for hiding this comment

Bug: Headers from config not passed to GenericGuardrailAPI

Uh oh!

ghost Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

cursor Bot Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Fixed Issues

1. Headers not passed to GenericGuardrailAPI (High Severity)

2. Request body always empty (Medium Severity)

3. Regular expression injection vulnerability (High Severity - Security)

Uh oh!

Uh oh!

Uh oh!

CLAassistant commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot commented Dec 1, 2025

Uh oh!

Check failure

Uh oh!

Uh oh!

Copilot Autofix

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ghost commented Nov 27, 2025 •

edited by cursor Bot

Loading

vercel Bot commented Nov 27, 2025 •

edited

Loading

cursor Bot Dec 1, 2025 •

edited

Loading

CLAassistant commented Dec 1, 2025 •

edited

Loading