a localized splunk instance for development/testing purposes in a docker container
Find a file
2025-07-19 21:14:34 -05:00
config added some configs to help index the log files we generate so we can search in the splunk UI 2025-07-19 21:14:34 -05:00
generators Add security-focused log generators for SOC and SIEM testing 2025-07-19 17:44:47 -05:00
.env Initial commit: Add Docker Compose Splunk testing environment 2025-07-19 16:52:36 -05:00
.gitignore Initial commit: Add Docker Compose Splunk testing environment 2025-07-19 16:52:36 -05:00
CLAUDE.md Add security-focused log generators for SOC and SIEM testing 2025-07-19 17:44:47 -05:00
docker-compose.yml added some configs to help index the log files we generate so we can search in the splunk UI 2025-07-19 21:14:34 -05:00
README.md Add security-focused log generators for SOC and SIEM testing 2025-07-19 17:44:47 -05:00

Splunk Local Testing Environment

A Docker Compose setup for running a local Splunk instance with log generators for comprehensive testing and development.

Quick Start

  1. Start Splunk:

    docker-compose up -d
    
  2. Access Splunk Web UI:

  3. Stop Splunk:

    docker-compose down
    

Services

Splunk Enterprise

Universal Forwarder (Optional)

To enable the Universal Forwarder for testing log forwarding:

docker-compose --profile forwarder up -d

Log Generators (Optional)

To enable basic log generators for testing data ingestion:

docker-compose --profile generators up -d

Security Log Generators (Optional)

To enable security-focused log generators for SOC/SIEM testing:

docker-compose --profile security up -d

Combined Setup

To run everything together (Splunk + forwarder + all generators):

docker-compose --profile forwarder --profile generators --profile security up -d

Configuration

  • Default credentials: admin/admin123
  • HEC Token: 00000000-0000-0000-0000-000000000000
  • Configuration files: ./config/
  • Log directory: ./logs/

Data Persistence

Splunk data is persisted in Docker volumes:

  • splunk_etc: Splunk configuration
  • splunk_var: Splunk data and logs

To reset all data:

docker-compose down -v

Log Generators

The environment includes multiple log generators to create realistic test data:

Available Generators

Basic Generators (Profile: generators)

  • Web Access Logs (log_generator_web): Apache-style access logs with realistic traffic patterns
  • Syslog Messages (log_generator_syslog): RFC3164 compliant system logs from various services
  • JSON Application Logs (log_generator_json): Structured application logs with user events, API calls, and metrics
  • HTTP Event Collector (log_generator_hec): Direct event submission to Splunk HEC endpoint

Security Generators (Profile: security)

  • Windows Event Logs (log_generator_windows): Windows Security, System, and Application event logs with attack patterns
  • Firewall Logs (log_generator_firewall): Multi-format firewall logs (pfSense, iptables, Cisco ASA) with blocked attacks
  • DNS Query Logs (log_generator_dns): DNS queries with DGA domains, suspicious lookups, and malicious domain patterns
  • Authentication Logs (log_generator_auth): LDAP, RADIUS, and SSH authentication events with brute force patterns
  • Cloud Service Logs (log_generator_cloud): AWS CloudTrail, Azure Activity, and GCP audit logs with security events

Log Output Locations

Basic Generator Outputs

  • Web logs: ./logs/web_access.log
  • Syslog: ./logs/syslog.log
  • JSON logs: ./logs/application.json
  • HEC events: Sent directly to Splunk HEC

Security Generator Outputs

  • Windows events: ./logs/windows_security.log, ./logs/windows_system.log, ./logs/windows_application.log
  • Firewall logs: ./logs/firewall.log
  • DNS queries: ./logs/dns_queries.log
  • Authentication: ./logs/ldap_auth.log, ./logs/radius_auth.log, ./logs/ssh_auth.log
  • Cloud logs: ./logs/aws_cloudtrail.json, ./logs/azure_activity.json, ./logs/gcp_audit.json

Adding New Generators

  1. Create your generator script in the ./generators/ directory
  2. Add a new service to docker-compose.yml under the appropriate profile (generators or security)
  3. Mount ./generators:/app and optionally ./logs:/var/log/app volumes
  4. Update this README with your new generator's details

Security Use Cases

The security generators are designed for:

  • SOC Training: Realistic attack patterns and security events for analyst training
  • SIEM Testing: Detection rule validation and alert tuning
  • Threat Hunting: Practice identifying advanced persistent threats and anomalous behavior
  • Incident Response: Simulated security incidents for response procedure testing
  • Compliance: Generate logs for security framework compliance testing

The Universal Forwarder will automatically pick up and forward any new log files placed in the ./logs/ directory.