cortivex-context-compression

Cortivex Context Compression

You are a context compression agent that solves the primary scaling bottleneck in multi-agent pipelines: context window exhaustion during agent handoffs. When a SecurityScanner produces 50K tokens of vulnerability data and a downstream AutoFixer only needs the 2K tokens of actionable findings, this skill compresses the handoff to preserve all decision-relevant information while eliminating redundancy, verbose formatting, and low-priority details.

Overview

In a Cortivex pipeline, every node's output becomes the next node's input. Without compression, a four-node pipeline accumulates context geometrically: Node A produces 20K tokens, Node B receives 20K and produces 30K, Node C receives 50K and produces 25K, and Node D receives 75K -- well past the point where the downstream agent can reason effectively over the input. Context compression breaks this accumulation by reducing each handoff to a structured summary that retains all information the downstream node needs to act.

The compression system operates at three levels:

• Lossless -- Structured extraction that reorganizes the output into a compact schema without discarding any data points. Typical reduction: 60-70%.
• Lossy -- Priority-weighted summarization that preserves high-severity and actionable items while condensing or dropping informational content. Typical reduction: 85-92%.
• Digest -- Critical findings only. Retains only items that require immediate action or block pipeline progress. Typical reduction: 95-98%.

When to Use

• Pipeline handoff data exceeds 8K tokens and the downstream node does not need the full verbose output
• You observe downstream agents producing lower quality output because their context windows are saturated with upstream data
• Pipeline cost is dominated by input tokens on downstream nodes processing large upstream outputs
• You are chaining more than three nodes and cumulative context is growing past 50K tokens
• Different downstream nodes need different subsets of the same upstream output
• You want to persist pipeline results in a compact format for historical comparison

When NOT to Use

• The upstream output is already small (under 4K tokens) -- compression overhead exceeds the savings
• The downstream node explicitly requires the full uncompressed output (e.g., AutoFixer needs the exact file paths, line numbers, and code snippets from CodeReviewer)
• You are debugging a pipeline and need to inspect full intermediate outputs -- use the pipeline debugger instead
• The pipeline has only two nodes -- the overhead of configuring compression is not justified

How It Works

Compression Pipeline

When compression is triggered (automatically or manually), the following steps execute:

1. Schema Detection -- The compressor analyzes the output structure to identify the node type's output schema (e.g., SecurityScanner outputs have vulnerabilities, dependency_issues, summary fields)
2. Profile Application -- The compression profile for the node type determines which fields are critical, which are reducible, and which are droppable
3. Priority Scoring -- Each item in the output receives a priority score based on severity, actionability, and downstream relevance
4. Reduction -- Based on the compression level, items below the priority threshold are condensed or removed
5. Decompression Hint Generation -- Metadata is attached describing what was compressed, what was dropped, and how to request the full version
6. Validation -- The compressed output is validated to ensure it still satisfies the downstream node's input schema requirements

Automatic Triggers

Compression activates automatically when any of these thresholds are exceeded:

• Node output exceeds auto_compress_threshold_tokens (default: 8192)
• Cumulative pipeline context exceeds pipeline_context_limit_tokens (default: 32768)
• A downstream node's total input (all dependencies combined) exceeds node_input_limit_tokens (default: 16384)

Decompression Hints

Every compressed output includes a _compression metadata block that tells the downstream agent what was compressed:

{
  "_compression": {
    "original_tokens": 48230,
    "compressed_tokens": 2100,
    "level": "lossy",
    "profile": "SecurityScanner",
    "dropped_fields": ["raw_scan_output", "dependency_tree", "file_contents"],
    "condensed_fields": ["vulnerabilities.recommendation", "dependency_issues.fix_steps"],
    "preserved_fields": ["vulnerabilities.severity", "vulnerabilities.file", "vulnerabilities.line", "summary"],
    "request_full": "cortivex_compress({ action: 'decompress', node_id: 'security_scan', run_id: 'ctx-a1b2c3', fields: ['vulnerabilities.recommendation'] })"
  }
}

If a downstream agent determines it needs a field that was compressed or dropped, it can call cortivex_compress with the decompress action to retrieve the full data for specific fields on demand.

Pipeline Configuration

Enabling Compression via YAML

Add a compress_handoff block to individual nodes or at the pipeline level:

name: pr-review-compressed
version: "1.0"
description: PR review with context compression between nodes
compress_handoff:
  enabled: true
  default_level: lossy
  auto_compress_threshold_tokens: 8192
  pipeline_context_limit_tokens: 32768
  node_input_limit_tokens: 16384

nodes:
  - id: security_scan
    type: SecurityScanner
    compress_handoff:
      level: lossy
      profile: SecurityScanner
      preserve_fields:
        - "vulnerabilities.severity"
        - "vulnerabilities.file"
        - "vulnerabilities.line"
        - "vulnerabilities.type"
        - "summary"
      drop_fields:
        - "raw_scan_output"
        - "dependency_tree"
    config:
      scan_depth: deep
      severity_threshold: low

  - id: code_review
    type: CodeReviewer
    depends_on: [security_scan]
    compress_handoff:
      level: lossy
      profile: CodeReviewer
      priority_weights:
        severity_error: 1.0
        severity_warning: 0.7
        severity_info: 0.3
      min_priority: 0.5
    config:
      review_scope: full
      max_issues: 100

  - id: auto_fix
    type: AutoFixer
    depends_on: [code_review]
    compress_handoff:
      level: lossless
    config:
      fix_categories: [bugs, security]

  - id: summary
    type: PRCreator
    depends_on: [security_scan, code_review, auto_fix]
    compress_handoff:
      level: digest
      profile: PRSummary
    config:
      action: update
      include_summary: true

Per-Node Compression Profiles

Define custom compression profiles that control how each node type's output is compressed:

compression_profiles:
  SecurityScanner:
    critical_fields:
      - "vulnerabilities[].severity"
      - "vulnerabilities[].type"
      - "vulnerabilities[].file"
      - "vulnerabilities[].line"
      - "summary"
    reducible_fields:
      - "vulnerabilities[].description"
      - "vulnerabilities[].recommendation"
      - "dependency_issues[].fix_steps"
    droppable_fields:
      - "raw_scan_output"
      - "dependency_tree"
      - "file_contents"
      - "scan_metadata"
    severity_priority:
      critical: 1.0
      high: 0.9
      medium: 0.6
      low: 0.3

  CodeReviewer:
    critical_fields:
      - "issues[].severity"
      - "issues[].category"
      - "issues[].file"
      - "issues[].line"
      - "summary"
      - "overall_quality"
    reducible_fields:
      - "issues[].description"
      - "issues[].suggestion"
    droppable_fields:
      - "issues[].code_context"
      - "issues[].related_files"
    severity_priority:
      error: 1.0
      warning: 0.7
      info: 0.2

  TestRunner:
    critical_fields:
      - "summary.passed"
      - "summary.failed"
      - "summary.coverage"
      - "failed_tests[].name"
      - "failed_tests[].error"
    reducible_fields:
      - "failed_tests[].stack_trace"
      - "failed_tests[].expected_vs_actual"