Seamless Integration
Integrate VirtueRed directly into your development pipeline with zero configuration complexity.
Automated Testing
Leverage GitHub Actions to automatically test your LLM's safety, robustness, and brand alignment.
Instant Insights
Get comprehensive safety reports delivered directly to your repository on every commit.
System Architecture
End-to-end automation pipeline for AI safety testing
How It Works
Our system streamlines the entire testing process, providing you with actionable insights with zero manual effort after the initial setup.
Commit Your Code
Push your code to your GitHub repository.
Trigger Automatic Testing
A pre-configured GitHub Action automatically kicks off, setting up a secure test environment.
Run VirtueRed Evaluation
Your LLM is tested against the VirtueRed dataset, covering a wide range of safety and security scenarios.
Get Instant Reports
Receive a detailed summary of the results directly in GitHub, with downloadable artifacts and report for deep-dive analysis.
How to Setup
Our CI/CD integration is designed to be both powerful and easy to setup.
Flexible Model Integration
Connect any LLM service to our testing harness with a simple modification to a single Python function. We provide ready-to-use examples for:
- OpenAI API: Directly use the official openai client.
- Custom REST APIs: Integrate any chatbot service with an API endpoint.
Customizable Test Scenarios
You have full control over what you test. Easily select specific datasets and subcategories by modifying a single line in the workflow's .yml file. Test for everything from hallucinations and bias to brand safety and compliance with the EU AI Act.
Secure and Simple Configuration
Get started quickly with enterprise-grade security:
- Easy Configuration: Get started by adding our .github workflow files to your repository.
- Credential Management: Store your API keys and other secrets using GitHub Actions secrets.