detection-sigma

Sigma Detection Engineering

Overview

Sigma is to log detection what Snort is to network traffic and YARA is to files - a universal signature format for describing security-relevant log events. This skill helps create, validate, and convert Sigma rules for deployment across multiple SIEM platforms, enabling detection-as-code workflows.

Core capabilities:

• Create detection rules using Sigma format
• Convert rules to 25+ SIEM/EDR backends (Splunk, Elastic, QRadar, Sentinel, etc.)
• Validate rule syntax and logic
• Map detections to MITRE ATT&CK framework
• Build threat hunting queries
• Implement compliance-based monitoring

Quick Start

Install Dependencies

pip install pysigma pysigma-backend-splunk pysigma-backend-elasticsearch pyyaml

Create a Basic Sigma Rule

title: Suspicious PowerShell Execution
id: 7d6d30b8-5b91-4b90-a71e-4f5a3f5a3c3f
status: experimental
description: Detects suspicious PowerShell execution with encoded commands
references:
    - https://attack.mitre.org/techniques/T1059/001/
author: Your Name
date: YYYY/MM/DD
modified: YYYY/MM/DD
tags:
    - attack.execution
    - attack.t1059.001
logsource:
    category: process_creation
    product: windows
detection:
    selection:
        Image|endswith: '\powershell.exe'
        CommandLine|contains:
            - '-enc'
            - '-EncodedCommand'
            - 'FromBase64String'
    condition: selection
falsepositives:
    - Legitimate administrative scripts
level: medium

Convert Rule to Target SIEM

# Convert to Splunk
python scripts/sigma_convert.py rule.yml --backend splunk

# Convert to Elasticsearch
python scripts/sigma_convert.py rule.yml --backend elasticsearch

# Convert to Microsoft Sentinel
python scripts/sigma_convert.py rule.yml --backend sentinel

Core Workflows

Workflow 1: Detection Rule Development

Progress: [ ] 1. Identify detection requirement from threat intelligence or compliance [ ] 2. Research log sources and field mappings for target environment [ ] 3. Create Sigma rule using standard template [ ] 4. Validate rule syntax: python scripts/sigma_validate.py rule.yml [ ] 5. Test rule against sample logs or historical data [ ] 6. Convert to target SIEM format [ ] 7. Deploy and tune based on false positive rate [ ] 8. Document rule metadata and MITRE ATT&CK mapping

Work through each step systematically. Check off completed items.

Workflow 2: Threat Hunting Rule Creation

For proactive threat hunting based on TTPs:

1. Select MITRE ATT&CK Technique
2. Review threat intelligence for relevant TTPs
3. Identify technique ID (e.g., T1059.001 - PowerShell)
4. See [references/mitre-attack-mapping.md](references/mitre-attack-mapping.md) for common techniques

1. Identify Log Sources
2. Determine which logs capture the technique
3. Map log source categories (process_creation, network_connection, file_event)
4. Verify log source availability in your environment

1. Define Detection Logic
2. Create selection criteria matching suspicious patterns
3. Add filters to reduce false positives
4. Use field modifiers for robust matching (endswith, contains, re)

1. Validate and Test
2. Run validation: python scripts/sigma_validate.py hunting-rule.yml
3. Test against known-good and known-bad samples
4. Tune detection logic based on results

1. Document and Deploy
2. Add references to threat reports
3. Document false positive scenarios
4. Convert and deploy to production SIEM

Workflow 3: Bulk Rule Conversion

When migrating between SIEM platforms:

# Validate all rules first
python scripts/sigma_validate.py --directory rules/ --report validation-report.json

# Convert entire rule set
python scripts/sigma_convert.py --directory rules/ --backend splunk --output converted/

# Generate deployment report
python scripts/sigma_convert.py --directory rules/ --backend splunk --report conversion-report.md

Review conversion report for:

• Successfully converted rules
• Rules requiring manual adjustment
• Unsupported field mappings
• Backend-specific limitations

Workflow 4: Compliance-Based Detection

For implementing compliance monitoring (PCI-DSS, NIST, ISO 27001):

1. Map Requirements to Detections
2. Identify compliance control requirements
3. Determine required log monitoring
4. See [references/compliance-mappings.md](references/compliance-mappings.md)

1. Create Detection Rules
2. Use compliance rule templates from assets/compliance-rules/
3. Tag rules with compliance framework (e.g., tags: [pci-dss.10.2.5])
4. Set appropriate severity levels

1. Validate Coverage
2. Run: python scripts/compliance_coverage.py --framework pci-dss
3. Review coverage gaps
4. Create additional rules as needed

1. Generate Compliance Report
2. Document detection coverage by control
3. Include sample queries and expected alerts
4. Maintain audit trail for compliance evidence

Rule Structure Reference

Required Fields

title: Human-readable rule name
id: UUID (generate with: python -c "import uuid; print(uuid.uuid4())")
status: stable|test|experimental|deprecated
description: Detailed description of what this detects
author: Your Name
date: YYYY/MM/DD
modified: YYYY/MM/DD
logsource:
    category: process_creation|network_connection|file_event|...
    product: windows|linux|macos|azure|aws|...
detection:
    selection:
        FieldName: value
    condition: selection
level: informational|low|medium|high|critical

Optional Fields

references:
    - https://attack.mitre.org/techniques/T1059/
tags:
    - attack.execution
    - attack.t1059.001
falsepositives:
    - Legitimate use cases
fields:
    - CommandLine
    - User
    - ParentImage

Detection Conditions

# Simple selection
detection:
    selection:
        Field: value
    condition: selection

# Multiple conditions (AND)
detection:
    selection:
        Field1: value1
        Field2: value2
    condition: selection

# OR conditions
detection:
    selection1:
        Field: value1
    selection2:
        Field: value2
    condition: selection1 or selection2

# NOT conditions
detection:
    selection:
        Field: suspicious_value
    filter:
        Field: legitimate_value
    condition: selection and not filter

# Complex logic
detection:
    selection:
        EventID: 4688
    suspicious_cmd:
        CommandLine|contains:
            - 'powershell'
            - 'cmd.exe'
    filter_legitimate:
        ParentImage|endswith: '\explorer.exe'
    condition: selection and suspicious_cmd and not filter_legitimate

Field Modifiers

Common modifiers for flexible matching:

• |contains - Contains substring (case-insensitive)
• |endswith - Ends with string
• |startswith - Starts with string
• |re - Regular expression match
• |all - All values must match
• |base64 - Base64-encoded value matching
• |base64offset - Base64 with offset variations

Example:

detection:
    selection:
        CommandLine|contains|all:
            - 'powershell'
            - '-enc'
        Image|endswith: '\powershell.exe'

Security Considerations

• Sensitive Data Handling: Sigma rules may reference sensitive field names or patterns. Store rules in version control with appropriate access controls. Avoid including actual sensitive data in example values.

• Access Control: Detection rules reveal defensive capabilities to adversaries. Implement role-based access for rule repositories. Limit rule modification to authorized detection engineers.

• Audit Logging: Log all rule deployments, modifications, and deletions. Track who deployed which rules to which systems. Maintain change history for compliance auditing.

• Compliance: Sigma rules support compliance monitoring (PCI-DSS 10.2, NIST SP 800-53 AU family, ISO 27001 A.12.4). Document rule-to-control mappings for audit evidence.

• **Safe Default