feat: guardrails endpoint#939
Conversation
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one) (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
OpenAPI changes 🟢 2 non-breaking changesTip Safe to merge from an API-contract perspective. Full changelog ·
|
| Method | Path | Change | |
|---|---|---|---|
| 🟢 | POST |
/api/v1/guardrails |
endpoint added |
| 🟢 | GET |
/api/v1/guardrails/{job_id} |
endpoint added |
main ↔ b26f6d14 · generated by oasdiff
| # response_id is server-minted so callers always have a stable | ||
| # correlation handle, independent of whether the upstream guardrails | ||
| # service returns one. | ||
| response_id = str(uuid4()) |
There was a problem hiding this comment.
This uuid4 is undefined. need to define above from uuid import UUID, uuid4
| job = job_crud.create( | ||
| job_type=JobType.LLM_GUARDRAILS, | ||
| trace_id=trace_id, | ||
| project_id=project_id, | ||
| ) | ||
| span.set_attribute("guardrails.job_id", str(job.id)) | ||
|
|
||
| # Persist the inbound request for traceability before the worker runs. | ||
| # We intentionally store the raw text: this endpoint exists to inspect | ||
| # potentially-unsafe content, so the request body itself is the audit log. | ||
| job_crud.update( | ||
| job_id=job.id, | ||
| job_update=JobUpdate(meta={"request": request.model_dump(mode="json")}), |
There was a problem hiding this comment.
Right now, we are doing 2 commits here, like:
INSERT --> COMMIT
// Then again
UPDATE --> COMMIT
which is unnecessary because the row was just created.
Since JobCrud.create() now supports meta, pass it during creation itself:
job = job_crud.create(
job_type=JobType.LLM_GUARDRAILS,
trace_id=trace_id,
project_id=project_id,
meta={
"request": request.model_dump(mode="json")
},
)
Then we don't need this:
job_crud.update(
job_id=job.id,
job_update=JobUpdate(
meta={"request": request.model_dump(mode="json")}
),
)
The final code looks like this:
job = job_crud.create(
job_type=JobType.LLM_GUARDRAILS,
trace_id=trace_id,
project_id=project_id,
meta={
"request": request.model_dump(mode="json")
},
)
span.set_attribute("guardrails.job_id", str(job.id))
| @celery_app.task(bind=True, queue="high_priority", priority=9) | ||
| @gevent_timeout(settings.CELERY_TASK_SOFT_TIME_LIMIT, "run_guardrails_job") |
There was a problem hiding this comment.
One thing: currently the traffic volume is expected to be maybe low from TAP, but I agree that guardrails jobs could eventually compete with interactive LLM requests. Let’s track this as a follow-up and evaluate a dedicated queue/lower priority once usage increases.
|
|
||
| warnings: list[str] = [] | ||
| if outcome.bypassed: | ||
| warnings.append("guardrails_service_unavailable_text_returned_unchanged") |
There was a problem hiding this comment.
For these types of warning crate the one constant: like:
GUARDRAILS_SERVICE_UNAVAILABLE = (
"guardrails_service_unavailable_text_returned_unchanged"
)
and then use it here for better consistency.
| # being unreachable during the config-fetch step (it swallows | ||
| # transport errors and returns []). Surface this as a warning so | ||
| # callers do not mistake an effective no-op for a clean pass. | ||
| warnings.append("validator_configs_unresolved_text_returned_unchanged") |
| warnings = [w for w in raw_warnings if isinstance(w, str)] | ||
|
|
||
| guardrails_response: GuardrailsCallbackData | None = None | ||
| if job.status.value == JobStatus.SUCCESS: |
There was a problem hiding this comment.
Can we do like this:
if job.status == JobStatus.SUCCESS:
Co-authored-by: Ayush <80516839+Ayush8923@users.noreply.github.com>
Co-authored-by: Ayush <80516839+Ayush8923@users.noreply.github.com>
2 participants
Summary
Target issue is #PLEASE_TYPE_ISSUE_NUMBER
Explain the motivation for making this change. What existing problem does the pull request solve?
Checklist
Before submitting a pull request, please ensure that you mark these task.
fastapi run --reload app/main.pyordocker compose upin the repository root and test.Notes
Please add here if any other information is required for the reviewer.