bpmcdevitt/splunk_local
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
splunk_local/CLAUDE.md
bpmcdevitt fa8fd73f1a Add security-focused log generators for SOC and SIEM testing 
- Implement 5 new security log generators: Windows events, firewall logs, DNS queries, authentication logs, and cloud service logs
- Add 'security' Docker Compose profile for easy deployment of security generators
- Windows generator creates realistic Security/System/Application events with attack patterns (failed logins, account creation, service events)
- Firewall generator supports pfSense, iptables, and Cisco ASA formats with malicious traffic blocking simulation
- DNS generator includes DGA domains, suspicious lookups, and multiple DNS server formats (BIND, Pi-hole, Windows DNS)
- Authentication generator creates LDAP, RADIUS, and SSH logs with brute force attack patterns
- Cloud generator produces AWS CloudTrail, Azure Activity, and GCP audit logs with security-relevant events
- Update documentation with comprehensive security use cases for SOC training, threat hunting, and compliance testing
- Enhance Docker Compose configuration with new security profile and service definitions

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-19 17:44:47 -05:00

141 lines
No EOL
5.7 KiB
Markdown
Raw Permalink Blame History

# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
This is a Docker Compose-based local Splunk testing environment. The setup includes:
- Splunk Enterprise instance for testing and development
- Optional Universal Forwarder for log forwarding scenarios
- Log generators for creating realistic test data
- Persistent data volumes for configuration and data retention
## Common Commands
### Docker Operations
- Start Splunk: `docker-compose up -d`
- Start with forwarder: `docker-compose --profile forwarder up -d`
- Start with log generators: `docker-compose --profile generators up -d`
- Start with security generators: `docker-compose --profile security up -d`
- Start everything: `docker-compose --profile forwarder --profile generators --profile security up -d`
- Stop services: `docker-compose down`
- Reset all data: `docker-compose down -v`
- View logs: `docker-compose logs splunk`
- Access Splunk shell: `docker-compose exec splunk bash`
### Splunk Access
- Web UI: http://localhost:8000 (admin/admin123)
- HEC endpoint: http://localhost:8088
- Management API: http://localhost:8089
## Architecture
### Service Structure
- **splunk**: Main Splunk Enterprise container
- **splunk_forwarder**: Optional Universal Forwarder (profile: forwarder)
- **log_generator_web**: Web access log generator (profile: generators)
- **log_generator_syslog**: Syslog message generator (profile: generators)
- **log_generator_json**: JSON application log generator (profile: generators)
- **log_generator_hec**: HTTP Event Collector sender (profile: generators)
- **log_generator_windows**: Windows Event Log generator (profile: security)
- **log_generator_firewall**: Firewall log generator (profile: security)
- **log_generator_dns**: DNS query log generator (profile: security)
- **log_generator_auth**: Authentication log generator (profile: security)
- **log_generator_cloud**: Cloud service log generator (profile: security)
### Key Directories
- `config/`: Splunk configuration files mounted to container
- `logs/`: Directory for sample logs (generated and forwarded)
- `generators/`: Python scripts for log generation
- Docker volumes: `splunk_etc`, `splunk_var` for data persistence
### Ports
- 8000: Splunk Web UI
- 8088: HTTP Event Collector (HEC)
- 8089: Splunk Management Port
- 9997: Splunk2Splunk forwarding
- 514/udp: Syslog input
## Configuration Files
- `docker-compose.yml`: Main service definitions
- `.env`: Environment variables (passwords, tokens)
- `config/default.yml`: Splunk default configuration
- Volume mounts allow for persistent configuration across container restarts
## Log Generators
The environment includes multiple log generators organized into two profiles:
### Basic Generators (Profile: generators)
#### Web Access Logs
- **File**: `generators/web_logs.py`
- **Output**: `/logs/web_access.log`
- **Format**: Apache Common Log Format with User-Agent
- **Content**: Realistic web server access logs with various IPs, paths, status codes
#### Syslog Messages
- **File**: `generators/syslog_generator.py`
- **Output**: `/logs/syslog.log`
- **Format**: RFC3164 compliant syslog
- **Content**: System messages from various services (sshd, httpd, systemd, etc.)
#### JSON Application Logs
- **File**: `generators/json_logs.py`
- **Output**: `/logs/application.json`
- **Format**: Structured JSON logs
- **Content**: User events, API calls, payments, errors, performance metrics
#### HTTP Event Collector (HEC)
- **File**: `generators/hec_sender.py`
- **Target**: Direct HEC endpoint
- **Format**: JSON events via HTTP
- **Content**: Security events, application metrics, business events
### Security Generators (Profile: security)
#### Windows Event Logs
- **File**: `generators/windows_events.py`
- **Output**: `/logs/windows_security.log`, `/logs/windows_system.log`, `/logs/windows_application.log`
- **Format**: Windows Event Log format
- **Content**: Security events (4624, 4625, 4720), system events, application errors with attack patterns
#### Firewall Logs
- **File**: `generators/firewall_logs.py`
- **Output**: `/logs/firewall.log`
- **Format**: pfSense, iptables, Cisco ASA formats
- **Content**: Allow/deny rules, attack blocking, suspicious traffic patterns
#### DNS Query Logs
- **File**: `generators/dns_logs.py`
- **Output**: `/logs/dns_queries.log`
- **Format**: BIND, syslog, Windows DNS, Pi-hole formats
- **Content**: Normal queries, DGA domains, suspicious lookups, malicious domain patterns
#### Authentication Logs
- **File**: `generators/auth_logs.py`
- **Output**: `/logs/ldap_auth.log`, `/logs/radius_auth.log`, `/logs/ssh_auth.log`
- **Format**: LDAP, RADIUS, SSH authentication formats
- **Content**: Login attempts, brute force attacks, account management events
#### Cloud Service Logs
- **File**: `generators/cloud_logs.py`
- **Output**: `/logs/aws_cloudtrail.json`, `/logs/azure_activity.json`, `/logs/gcp_audit.json`
- **Format**: Native cloud provider JSON formats
- **Content**: API calls, resource changes, privilege escalation, security events
## Testing Workflows
The environment is designed for:
- Testing Splunk apps and configurations
- Log ingestion and forwarding scenarios
- Search and dashboard development
- API integration testing with HEC
- Performance testing with high-volume log generation
- Different log format parsing and field extraction
- **SOC/SIEM Testing**: Detection rule validation with realistic attack patterns
- **Security Training**: Hands-on experience with security event analysis
- **Threat Hunting**: Practice identifying advanced persistent threats
- **Incident Response**: Simulated security incidents for response training
- **Compliance Testing**: Generate logs for security framework validation
Reference in a new issue View git blame Copy permalink