auto_sigma_rule_generator/cli/README.md

# SIGMA CLI - CVE-SIGMA Auto Generator

A command-line interface for processing CVEs and generating SIGMA detection rules in a file-based directory structure.

## Quick Start

```bash
# Make CLI executable
chmod +x cli/sigma_cli.py

# Initialize configuration
./cli/sigma_cli.py config-init

# Migrate data from existing database (if applicable)
./cli/sigma_cli.py migrate from-database

# Process CVEs for a specific year
./cli/sigma_cli.py process year 2024

# Generate rules for a specific CVE
./cli/sigma_cli.py generate cve CVE-2024-0001

# Search CVEs
./cli/sigma_cli.py search cve "buffer overflow"

# View statistics
./cli/sigma_cli.py stats overview

# Export rules
./cli/sigma_cli.py export sigma ./output/rules
```

## Directory Structure

```
auto_sigma_rule_generator/
├── cves/
│   ├── 2024/
│   │   ├── CVE-2024-0001/
│   │   │   ├── metadata.json
│   │   │   ├── rule_template.sigma
│   │   │   ├── rule_llm_openai.sigma
│   │   │   └── poc_analysis.json
│   │   └── CVE-2024-0002/...
│   └── 2023/...
├── cli/
│   ├── sigma_cli.py (main CLI)
│   ├── commands/ (command modules)
│   └── config/ (CLI configuration)
└── reports/ (generated reports)
```

## Available Commands

### Process Commands
- `process year <year>` - Process all CVEs for a year
- `process cve <cve-id>` - Process specific CVE
- `process bulk` - Bulk process multiple years
- `process incremental` - Process recent changes

### Generate Commands
- `generate cve <cve-id>` - Generate rules for CVE
- `generate regenerate` - Regenerate existing rules

### Search Commands
- `search cve <pattern>` - Search CVEs
- `search rules <pattern>` - Search SIGMA rules

### Statistics Commands
- `stats overview` - General statistics
- `stats poc` - PoC coverage statistics
- `stats rules` - Rule generation statistics

### Export Commands
- `export sigma <dir>` - Export SIGMA rules
- `export metadata <file>` - Export CVE metadata

### Migration Commands
- `migrate from-database` - Migrate from web app database
- `migrate validate` - Validate migrated data

## Configuration

Edit `~/.sigma-cli/config.yaml` to configure API keys and settings:

```yaml
api_keys:
  nvd_api_key: "your-nvd-key"
  github_token: "your-github-token"
  openai_api_key: "your-openai-key"
  anthropic_api_key: "your-anthropic-key"

llm_settings:
  default_provider: "ollama"
  default_model: "llama3.2"
  ollama_base_url: "http://localhost:11434"

processing:
  default_batch_size: 50
  default_methods: ["template"]
```

## Installation

```bash
# Install dependencies
pip install -r cli/requirements.txt

# Or if you're in a virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\\Scripts\\activate
pip install -r cli/requirements.txt
```

## Examples

### Migration from Web Application
```bash
# Migrate existing data
./cli/sigma_cli.py migrate from-database --database-url "postgresql://user:pass@localhost:5432/db"

# Validate migration
./cli/sigma_cli.py migrate validate

# Check migration statistics
./cli/sigma_cli.py stats overview
```

### Processing CVEs
```bash
# Process a specific year with multiple methods
./cli/sigma_cli.py process year 2024 --method template --method llm

# Process a specific CVE with force regeneration
./cli/sigma_cli.py process cve CVE-2024-12345 --force

# Bulk process with specific batch size
./cli/sigma_cli.py process bulk --start-year 2020 --end-year 2024 --batch-size 100
```

### Searching and Analysis
```bash
# Search for CVEs with specific patterns
./cli/sigma_cli.py search cve "remote code execution" --severity critical --has-poc

# Search SIGMA rules
./cli/sigma_cli.py search rules "powershell" --method llm

# Generate comprehensive statistics
./cli/sigma_cli.py stats overview --year 2024 --output ./reports/2024-stats.json
```

### Exporting Data
```bash
# Export all SIGMA rules as YAML
./cli/sigma_cli.py export sigma ./output/sigma-rules --format yaml

# Export CVE metadata as CSV
./cli/sigma_cli.py export metadata ./reports/cve-data.csv --format csv

# Export specific year and method
./cli/sigma_cli.py export sigma ./output/2024-llm-rules --year 2024 --method llm
```

## File Formats

### metadata.json Structure
```json
{
  "cve_info": {
    "cve_id": "CVE-2024-0001",
    "description": "...",
    "cvss_score": 9.8,
    "severity": "critical"
  },
  "poc_data": {
    "poc_count": 3,
    "poc_data": {...}
  },
  "rule_generation": {
    "template": {"generated_at": "..."},
    "llm_openai": {"generated_at": "..."}
  }
}
```

### SIGMA Rule Files
- `rule_template.sigma` - Template-based generation
- `rule_llm_openai.sigma` - OpenAI LLM generation
- `rule_llm_anthropic.sigma` - Anthropic LLM generation
- `rule_hybrid.sigma` - Hybrid generation method

## Development

The CLI is built using Click and follows a modular command structure:

- `sigma_cli.py` - Main CLI entry point
- `commands/base_command.py` - Base functionality
- `commands/process_commands.py` - CVE processing
- `commands/migrate_commands.py` - Database migration
- `commands/search_commands.py` - Search functionality
- `commands/stats_commands.py` - Statistics generation
- `commands/export_commands.py` - Data export

## Troubleshooting

### Common Issues
1. **Import errors**: Make sure you're running from the project root
2. **Permission errors**: Ensure directories are writable
3. **Database connection**: Check DATABASE_URL environment variable
4. **API limits**: Configure API keys for higher rate limits

### Debug Mode
```bash
# Enable verbose logging
./cli/sigma_cli.py --verbose <command>

# Check configuration
./cli/sigma_cli.py config-init
```