Sigma Detection Engineering

Overview

Sigma is to log detection what Snort is to network traffic and YARA is to files - a universal signature format for describing security-relevant log events. This skill helps create, validate, and convert Sigma rules for deployment across multiple SIEM platforms, enabling detection-as-code workflows.

Core capabilities:

• Create detection rules using Sigma format
• Convert rules to 25+ SIEM/EDR backends (Splunk, Elastic, QRadar, Sentinel, etc.)
• Validate rule syntax and logic
• Map detections to MITRE ATT&CK framework
• Build threat hunting queries
• Implement compliance-based monitoring

Quick Start

Install Dependencies

pip install pysigma pysigma-backend-splunk pysigma-backend-elasticsearch pyyaml

Create a Basic Sigma Rule

title: Suspicious PowerShell Execution
id: 7d6d30b8-5b91-4b90-a71e-4f5a3f5a3c3f
status: experimental
description: Detects suspicious PowerShell execution with encoded commands
references:
    - https://attack.mitre.org/techniques/T1059/001/
author: Your Name
date: YYYY/MM/DD
modified: YYYY/MM/DD
tags:
    - attack.execution
    - attack.t1059.001
logsource:
    category: process_creation
    product: windows
detection:
    selection:
        Image|endswith: '\powershell.exe'
        CommandLine|contains:
            - '-enc'
            - '-EncodedCommand'
            - 'FromBase64String'
    condition: selection
falsepositives:
    - Legitimate administrative scripts
level: medium

Convert Rule to Target SIEM

# Convert to Splunk
python scripts/sigma_convert.py rule.yml --backend splunk

# Convert to Elasticsearch
python scripts/sigma_convert.py rule.yml --backend elasticsearch

# Convert to Microsoft Sentinel
python scripts/sigma_convert.py rule.yml --backend sentinel

Core Workflows

Workflow 1: Detection Rule Development

Progress: [ ] 1. Identify detection requirement from threat intelligence or compliance [ ] 2. Research log sources and field mappings for target environment [ ] 3. Create Sigma rule using standard template [ ] 4. Validate rule syntax: python scripts/sigma_validate.py rule.yml [ ] 5. Test rule against sample logs or historical data [ ] 6. Convert to target SIEM format [ ] 7. Deploy and tune based on false positive rate [ ] 8. Document rule metadata and MITRE ATT&CK mapping

Work through each step systematically. Check off completed items.

Workflow 2: Threat Hunting Rule Creation

For proactive threat hunting based on TTPs:

1. Select MITRE ATT&CK Technique

   • Review threat intelligence for relevant TTPs
   • Identify technique ID (e.g., T1059.001 - PowerShell)
   • See references/mitre-attack-mapping.md for common techniques

2. Identify Log Sources

   • Determine which logs capture the technique
   • Map log source categories (process_creation, network_connection, file_event)
   • Verify log source availability in your environment

3. Define Detection Logic

   • Create selection criteria matching suspicious patterns
   • Add filters to reduce false positives
   • Use field modifiers for robust matching (endswith, contains, re)

4. Validate and Test

   • Run validation: python scripts/sigma_validate.py hunting-rule.yml
   • Test against known-good and known-bad samples
   • Tune detection logic based on results

5. Document and Deploy

   • Add references to threat reports
   • Document false positive scenarios
   • Convert and deploy to production SIEM

Workflow 3: Bulk Rule Conversion

When migrating between SIEM platforms:

# Validate all rules first
python scripts/sigma_validate.py --directory rules/ --report validation-report.json

# Convert entire rule set
python scripts/sigma_convert.py --directory rules/ --backend splunk --output converted/

# Generate deployment report
python scripts/sigma_convert.py --directory rules/ --backend splunk --report conversion-report.md

Review conversion report for:

• Successfully converted rules
• Rules requiring manual adjustment
• Unsupported field mappings
• Backend-specific limitations

Workflow 4: Compliance-Based Detection

For implementing compliance monitoring (PCI-DSS, NIST, ISO 27001):

1. Map Requirements to Detections

   • Identify compliance control requirements
   • Determine required log monitoring
   • See references/compliance-mappings.md

2. Create Detection Rules

   • Use compliance rule templates from assets/compliance-rules/
   • Tag rules with compliance framework (e.g., tags: [pci-dss.10.2.5])
   • Set appropriate severity levels

3. Validate Coverage

   • Run: python scripts/compliance_coverage.py --framework pci-dss
   • Review coverage gaps
   • Create additional rules as needed

4. Generate Compliance Report

   • Document detection coverage by control
   • Include sample queries and expected alerts
   • Maintain audit trail for compliance evidence

Rule Structure Reference

Required Fields

title: Human-readable rule name
id: UUID (generate with: python -c "import uuid; print(uuid.uuid4())")
status: stable|test|experimental|deprecated
description: Detailed description of what this detects
author: Your Name
date: YYYY/MM/DD
modified: YYYY/MM/DD
logsource:
    category: process_creation|network_connection|file_event|...
    product: windows|linux|macos|azure|aws|...
detection:
    selection:
        FieldName: value
    condition: selection
level: informational|low|medium|high|critical

Optional Fields

references:
    - https://attack.mitre.org/techniques/T1059/
tags:
    - attack.execution
    - attack.t1059.001
falsepositives:
    - Legitimate use cases
fields:
    - CommandLine
    - User
    - ParentImage

Detection Conditions

# Simple selection
detection:
    selection:
        Field: value
    condition: selection

# Multiple conditions (AND)
detection:
    selection:
        Field1: value1
        Field2: value2
    condition: selection

# OR conditions
detection:
    selection1:
        Field: value1
    selection2:
        Field: value2
    condition: selection1 or selection2

# NOT conditions
detection:
    selection:
        Field: suspicious_value
    filter:
        Field: legitimate_value
    condition: selection and not filter

# Complex logic
detection:
    selection:
        EventID: 4688
    suspicious_cmd:
        CommandLine|contains:
            - 'powershell'
            - 'cmd.exe'
    filter_legitimate:
        ParentImage|endswith: '\explorer.exe'
    condition: selection and suspicious_cmd and not filter_legitimate

Field Modifiers

Common modifiers for flexible matching:

• |contains - Contains substring (case-insensitive)
• |endswith - Ends with string
• |startswith - Starts with string
• |re - Regular expression match
• |all - All values must match
• |base64 - Base64-encoded value matching
• |base64offset - Base64 with offset variations

Example:

detection:
    selection:
        CommandLine|contains|all:
            - 'powershell'
            - '-enc'
        Image|endswith: '\powershell.exe'

Security Considerations

• Sensitive Data Handling: Sigma rules may reference sensitive field names or patterns. Store rules in version control with appropriate access controls. Avoid including actual sensitive data in example values.

• Access Control: Detection rules reveal defensive capabilities to adversaries. Implement role-based access for rule repositories. Limit rule modification to authorized detection engineers.

• Audit Logging: Log all rule deployments, modifications, and deletions. Track who deployed which rules to which systems. Maintain change history for compliance auditing.

• Compliance: Sigma rules support compliance monitoring (PCI-DSS 10.2, NIST SP 800-53 AU family, ISO 27001 A.12.4). Document rule-to-control mappings for audit evidence.

• **Safe Default