CVE-SIGMA Auto Generator (Enhanced)

An advanced automated platform that processes comprehensive CVE data and generates enhanced SIGMA rules for threat detection using curated exploit intelligence.

🚀 Enhanced Features

Data Processing

• Bulk NVD Processing: Downloads and processes complete NVD JSON datasets (2002-2025)
• nomi-sec PoC Integration: Uses curated PoC data from github.com/nomi-sec/PoC-in-GitHub
• Incremental Updates: Efficient updates using NVD modified/recent feeds
• Quality Assessment: Advanced PoC quality scoring with star count, recency, and relevance analysis

Intelligence Generation

• Enhanced SIGMA Rules: Creates rules using real exploit indicators from curated PoCs
• Quality Tiers: Excellent, Good, Fair, Poor, Very Poor classification system
• Smart Template Selection: AI-driven template matching based on PoC characteristics
• Advanced Indicator Extraction: Processes, files, network, registry, and command patterns
• MITRE ATT&CK Mapping: Automatic technique identification based on exploit analysis

User Experience

• Modern Web Interface: React-based UI with enhanced bulk processing controls
• Real-time Monitoring: Live job tracking and progress monitoring
• Comprehensive Statistics: PoC coverage, quality metrics, and processing status
• Bulk Operations Dashboard: Centralized control for all data processing operations

Architecture

• Backend: FastAPI with SQLAlchemy ORM
• Frontend: React with Tailwind CSS
• Database: PostgreSQL
• Cache: Redis (optional)
• Containerization: Docker & Docker Compose

Quick Start

Prerequisites

• Docker and Docker Compose
• (Optional) NVD API Key for increased rate limits

Setup

1. Clone the repository:

git clone <repository-url>
cd cve-sigma-generator

2. Quick Start (Recommended):

chmod +x start.sh
./start.sh

3. Manual Setup:

# Copy environment file
cp .env.example .env

# (Optional) Edit .env and add your NVD API key
nano .env

# Start the application
docker-compose up -d --build

4. Wait for services to initialize (about 30-60 seconds)

5. Access the application:

   • Frontend: http://localhost:3000
   • Backend API: http://localhost:8000
   • API Documentation: http://localhost:8000/docs

First Run

The application will automatically:

1. Initialize the database with rule templates
2. Start fetching recent CVEs from NVD
3. Generate SIGMA rules for each CVE
4. Continue polling for new CVEs every hour

Usage

Web Interface

The web interface provides three main sections:

1. Dashboard: Overview statistics and recent CVEs
2. CVEs: Complete list of all fetched CVEs with details
3. SIGMA Rules: Generated detection rules organized by CVE

Manual CVE Fetch

You can trigger a manual CVE fetch using the "Fetch New CVEs" button in the dashboard or via API:

curl -X POST http://localhost:8000/api/fetch-cves

API Endpoints

• GET /api/cves - List all CVEs
• GET /api/cves/{cve_id} - Get specific CVE details
• GET /api/sigma-rules - List all SIGMA rules
• GET /api/sigma-rules/{cve_id} - Get SIGMA rules for specific CVE
• POST /api/fetch-cves - Manually trigger CVE fetch
• GET /api/stats - Get application statistics

Configuration

Environment Variables

• DATABASE_URL: PostgreSQL connection string
• NVD_API_KEY: Optional NVD API key for higher rate limits (5→50 requests/30s)
• GITHUB_TOKEN: Optional GitHub personal access token for exploit analysis
• REACT_APP_API_URL: Backend API URL for frontend

GitHub Integration (Optional)

For enhanced SIGMA rule generation with exploit analysis:

1. Create GitHub Token: Visit https://github.com/settings/tokens
2. Required Permissions: Only needs "public_repo" scope for searching public repositories
3. Add to Environment: GITHUB_TOKEN=your_token_here in .env file
4. Benefits:
   • Automatically searches for CVE-related exploit code
   • Extracts real indicators (processes, files, network connections)
   • Generates more accurate and specific SIGMA rules
   • Higher confidence ratings for exploit-based rules

Rate Limits: 5000 requests/hour with token, 60/hour without

Rule Templates

The application includes pre-configured rule templates for:

• Windows Process Execution
• Network Connections
• File Modifications

Additional templates can be added to the database via the rule_templates table.

SIGMA Rule Generation Logic

The enhanced rule generation process:

1. CVE Analysis: Analyzes CVE description and affected products
2. GitHub Exploit Search: Searches GitHub for exploit code using multiple query strategies
3. Code Analysis: Extracts specific indicators from exploit code:
   • Process names and command lines
   • File paths and registry keys
   • Network connections and ports
   • PowerShell commands and scripts
   • Command execution patterns
4. Template Selection: Chooses appropriate SIGMA rule template based on exploit analysis
5. Enhanced Rule Population: Fills template with real exploit indicators
6. MITRE ATT&CK Mapping: Maps to specific MITRE ATT&CK techniques
7. Confidence Scoring: Higher confidence for exploit-based rules

Rule Quality Levels

• Basic Rules: Generated from CVE description only
• Exploit-Based Rules: Enhanced with GitHub exploit analysis (marked with 🔍)
• Confidence Ratings:
  • High: CVSS ≥9.0 + exploit analysis
  • Medium: CVSS ≥7.0 or exploit analysis
  • Low: Basic CVE description only

Template Matching

• PowerShell Execution: Exploit contains PowerShell scripts or cmdlets
• Process Execution: Exploit shows process creation or command execution
• Network Connection: Exploit demonstrates network communications
• File Modification: Exploit involves file system operations

Example Enhanced Rule

title: CVE-2025-1234 Exploit-Based Detection
description: Detection for CVE-2025-1234 remote code execution [Enhanced with GitHub exploit analysis]
tags:
    - attack.t1059.001
    - cve-2025-1234  
    - exploit.github
detection:
    selection:
        Image|contains:
            - "powershell.exe"
            - "malicious_payload.exe"
            - "reverse_shell.ps1"
    condition: selection
level: high

Development

Local Development

1. Start the database:

docker-compose up -d db redis

2. Run the backend:

cd backend
pip install -r requirements.txt
uvicorn main:app --reload

3. Run the frontend:

cd frontend
npm install
npm start

Database Migration

The application automatically creates tables on startup. For manual schema changes:

# Connect to database
docker-compose exec db psql -U cve_user -d cve_sigma_db

# Run custom SQL
\i /path/to/migration.sql

SIGMA Rule Quality

Generated rules are marked as "experimental" and should be:

• Reviewed by security analysts
• Tested in a lab environment
• Tuned to reduce false positives
• Validated against real attack scenarios

Monitoring

Logs

View application logs:

# All services
docker-compose logs -f

# Specific service
docker-compose logs -f backend

Health Checks

The application includes health checks for database connectivity. Monitor with:

docker-compose ps

✅ Recent Fixes (July 2025)

• Fixed 404 CVE fetch error: Corrected NVD API 2.0 endpoint format and parameters
• Updated for current dates: Now properly fetches CVEs from July 2025 (current date)
• Improved API integration: Better error handling, fallback mechanisms, and debugging
• Enhanced date handling: Proper ISO-8601 format with UTC timezone
• API key integration: Correctly passes API keys in headers for higher rate limits

Troubleshooting

Common Issues

1. Frontend build fails with "npm ci" error: This is fixed in the current version. The Dockerfile now uses npm install instead of npm ci.
2. CVE Fetch returns 404: Fixed in latest version. The application now uses proper NVD API 2.0 format with current 2025 dates.
3. No CVEs being fetched:
   • Check if you have an NVD API key configured in .env for better rate limits
   • Use the "Test NVD API" button to verify connectivity
   • Check backend logs: docker-compose logs -f backend
4. Database Connection Error: Ensure PostgreSQL is running and accessible
5. Frontend Not Loading: Verify backend is running and CORS is configured
6. Rule Generation Issues: Check CVE description quality and template matching
7. Port conflicts: If ports 3000, 8000, or 5432 are in use, stop other services or modify docker-compose.yml

API Key Setup

NVD API (Recommended) For optimal CVE fetching performance:

1. Visit: https://nvd.nist.gov/developers/request-an-api-key
2. Add to your .env file: NVD_API_KEY=your_key_here
3. Restart the application

Without an API key: 5 requests per 30 seconds With an API key: 50 requests per 30 seconds

GitHub API (Optional) For enhanced exploit-based SIGMA rules:

1. Visit: https://github.com/settings/tokens
2. Create token with "public_repo" scope
3. Add to your .env file: GITHUB_TOKEN=your_token_here
4. Restart the application

Without a GitHub token: Basic rules only With a GitHub token: Enhanced rules with exploit analysis (🔍 Exploit-Based)

Rate Limits

Without an API key, NVD limits requests to 5 per 30 seconds. With an API key, the limit increases to 50 per 30 seconds.

Security Considerations

• API Keys: Store NVD API keys securely using environment variables
• Database Access: Use strong passwords and restrict database access
• Network Security: Deploy behind a reverse proxy in production
• Rule Validation: Always validate generated SIGMA rules before deployment

Contributing

1. Fork the repository
2. Create a feature branch
3. Make changes and add tests
4. Submit a pull request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

For issues and questions:

1. Check the troubleshooting section
2. Review application logs
3. Open an issue on GitHub

Roadmap

Planned features:

• Custom rule template editor
• MITRE ATT&CK mapping
• Rule effectiveness scoring
• Export to SIEM platforms
• Advanced threat intelligence integration
• Machine learning-based rule optimization