bpmcdevitt/splunk_local
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
splunk_local/CLAUDE.md
bpmcdevitt fa8fd73f1a Add security-focused log generators for SOC and SIEM testing 
- Implement 5 new security log generators: Windows events, firewall logs, DNS queries, authentication logs, and cloud service logs
- Add 'security' Docker Compose profile for easy deployment of security generators
- Windows generator creates realistic Security/System/Application events with attack patterns (failed logins, account creation, service events)
- Firewall generator supports pfSense, iptables, and Cisco ASA formats with malicious traffic blocking simulation
- DNS generator includes DGA domains, suspicious lookups, and multiple DNS server formats (BIND, Pi-hole, Windows DNS)
- Authentication generator creates LDAP, RADIUS, and SSH logs with brute force attack patterns
- Cloud generator produces AWS CloudTrail, Azure Activity, and GCP audit logs with security-relevant events
- Update documentation with comprehensive security use cases for SOC training, threat hunting, and compliance testing
- Enhance Docker Compose configuration with new security profile and service definitions

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-19 17:44:47 -05:00

5.7 KiB
Raw Permalink Blame History

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

This is a Docker Compose-based local Splunk testing environment. The setup includes:

  • Splunk Enterprise instance for testing and development
  • Optional Universal Forwarder for log forwarding scenarios
  • Log generators for creating realistic test data
  • Persistent data volumes for configuration and data retention

Common Commands

Docker Operations

  • Start Splunk: docker-compose up -d
  • Start with forwarder: docker-compose --profile forwarder up -d
  • Start with log generators: docker-compose --profile generators up -d
  • Start with security generators: docker-compose --profile security up -d
  • Start everything: docker-compose --profile forwarder --profile generators --profile security up -d
  • Stop services: docker-compose down
  • Reset all data: docker-compose down -v
  • View logs: docker-compose logs splunk
  • Access Splunk shell: docker-compose exec splunk bash

Splunk Access

Architecture

Service Structure

  • splunk: Main Splunk Enterprise container
  • splunk_forwarder: Optional Universal Forwarder (profile: forwarder)
  • log_generator_web: Web access log generator (profile: generators)
  • log_generator_syslog: Syslog message generator (profile: generators)
  • log_generator_json: JSON application log generator (profile: generators)
  • log_generator_hec: HTTP Event Collector sender (profile: generators)
  • log_generator_windows: Windows Event Log generator (profile: security)
  • log_generator_firewall: Firewall log generator (profile: security)
  • log_generator_dns: DNS query log generator (profile: security)
  • log_generator_auth: Authentication log generator (profile: security)
  • log_generator_cloud: Cloud service log generator (profile: security)

Key Directories

  • config/: Splunk configuration files mounted to container
  • logs/: Directory for sample logs (generated and forwarded)
  • generators/: Python scripts for log generation
  • Docker volumes: splunk_etc, splunk_var for data persistence

Ports

  • 8000: Splunk Web UI
  • 8088: HTTP Event Collector (HEC)
  • 8089: Splunk Management Port
  • 9997: Splunk2Splunk forwarding
  • 514/udp: Syslog input

Configuration Files

  • docker-compose.yml: Main service definitions
  • .env: Environment variables (passwords, tokens)
  • config/default.yml: Splunk default configuration
  • Volume mounts allow for persistent configuration across container restarts

Log Generators

The environment includes multiple log generators organized into two profiles:

Basic Generators (Profile: generators)

Web Access Logs

  • File: generators/web_logs.py
  • Output: /logs/web_access.log
  • Format: Apache Common Log Format with User-Agent
  • Content: Realistic web server access logs with various IPs, paths, status codes

Syslog Messages

  • File: generators/syslog_generator.py
  • Output: /logs/syslog.log
  • Format: RFC3164 compliant syslog
  • Content: System messages from various services (sshd, httpd, systemd, etc.)

JSON Application Logs

  • File: generators/json_logs.py
  • Output: /logs/application.json
  • Format: Structured JSON logs
  • Content: User events, API calls, payments, errors, performance metrics

HTTP Event Collector (HEC)

  • File: generators/hec_sender.py
  • Target: Direct HEC endpoint
  • Format: JSON events via HTTP
  • Content: Security events, application metrics, business events

Security Generators (Profile: security)

Windows Event Logs

  • File: generators/windows_events.py
  • Output: /logs/windows_security.log, /logs/windows_system.log, /logs/windows_application.log
  • Format: Windows Event Log format
  • Content: Security events (4624, 4625, 4720), system events, application errors with attack patterns

Firewall Logs

  • File: generators/firewall_logs.py
  • Output: /logs/firewall.log
  • Format: pfSense, iptables, Cisco ASA formats
  • Content: Allow/deny rules, attack blocking, suspicious traffic patterns

DNS Query Logs

  • File: generators/dns_logs.py
  • Output: /logs/dns_queries.log
  • Format: BIND, syslog, Windows DNS, Pi-hole formats
  • Content: Normal queries, DGA domains, suspicious lookups, malicious domain patterns

Authentication Logs

  • File: generators/auth_logs.py
  • Output: /logs/ldap_auth.log, /logs/radius_auth.log, /logs/ssh_auth.log
  • Format: LDAP, RADIUS, SSH authentication formats
  • Content: Login attempts, brute force attacks, account management events

Cloud Service Logs

  • File: generators/cloud_logs.py
  • Output: /logs/aws_cloudtrail.json, /logs/azure_activity.json, /logs/gcp_audit.json
  • Format: Native cloud provider JSON formats
  • Content: API calls, resource changes, privilege escalation, security events

Testing Workflows

The environment is designed for:

  • Testing Splunk apps and configurations
  • Log ingestion and forwarding scenarios
  • Search and dashboard development
  • API integration testing with HEC
  • Performance testing with high-volume log generation
  • Different log format parsing and field extraction
  • SOC/SIEM Testing: Detection rule validation with realistic attack patterns
  • Security Training: Hands-on experience with security event analysis
  • Threat Hunting: Practice identifying advanced persistent threats
  • Incident Response: Simulated security incidents for response training
  • Compliance Testing: Generate logs for security framework validation