Cortivex Learning System
You have access to a learning system that improves pipeline performance over time. After each pipeline run, insights are automatically recorded. Before starting work, you should consult past insights to make better decisions.
How Learning Works
The Cortivex learning system operates on a feedback loop:
- Observe -- Each pipeline run produces metrics: duration per node, cost per node, success/failure, error types, and output quality signals.
- Analyze -- The system compares these metrics against historical data for the same repository, same pipeline template, and similar task types.
- Record -- Actionable insights are stored with confidence scores, scoped to the repository, pipeline, or node type.
- Apply -- On future runs, insights are surfaced to agents and the orchestrator to optimize execution.
Checking Insights Before Starting Work
Before every pipeline run, query the learning system for relevant insights:
cortivex_insights({
action: "query",
repo: "/path/to/repo",
pipeline: "pr-review",
node_types: ["SecurityScanner", "CodeReviewer", "AutoFixer"],
min_confidence: 0.6
})
Response:
{
"insights": [
{
"id": "ins-7a3b",
"type": "reorder",
"confidence": 0.87,
"description": "Running CodeReviewer before SecurityScanner reduces total duration by 22% in this repo because security scans on already-reviewed code skip false positives",
"evidence": {
"sample_size": 14,
"avg_improvement": "22% faster",
"last_observed": "2025-01-14T18:30:00Z"
},
"recommendation": {
"action": "reorder",
"move": "CodeReviewer",
"before": "SecurityScanner"
}
},
{
"id": "ins-9c4d",
"type": "substitute_model",
"confidence": 0.73,
"description": "For this repo, using claude-haiku-4-20250414 for SecurityScanner produces equivalent results to claude-sonnet-4-20250514 at 85% lower cost",
"evidence": {
"sample_size": 8,
"quality_delta": "-2% (within noise)",
"cost_savings": "85%",
"last_observed": "2025-01-13T09:15:00Z"
},
"recommendation": {
"action": "substitute_model",
"node": "SecurityScanner",
"from": "claude-sonnet-4-20250514",
"to": "claude-haiku-4-20250414"
}
},
{
"id": "ins-2e5f",
"type": "add_node",
"confidence": 0.65,
"description": "PRs in this repo that include a LintFixer step before TestRunner have a 40% lower failure rate in CI",
"evidence": {
"sample_size": 6,
"failure_rate_with": "12%",
"failure_rate_without": "52%",
"last_observed": "2025-01-12T14:00:00Z"
},
"recommendation": {
"action": "add_node",
"node_type": "LintFixer",
"insert_before": "TestRunner"
}
}
]
}
How to Apply Insights
When insights are returned, evaluate them and apply high-confidence ones:
Confidence >= 0.80: Apply automatically. Inform the user what was optimized and why.
Optimization applied: Reordered CodeReviewer before SecurityScanner
(87% confidence, based on 14 previous runs: 22% faster on average)
Confidence 0.60-0.79: Suggest to the user and apply if approved, or apply with a note.
Suggestion: Use claude-haiku-4-20250414 for SecurityScanner to save ~85% cost
(73% confidence, based on 8 runs with equivalent quality)
Apply this optimization? [Y/n]
Confidence < 0.60: Present as informational only. Do not apply automatically.
Note: Early data (3 runs) suggests adding a LintFixer step could reduce CI failures.
Not enough evidence to recommend yet. Will continue monitoring.
Insight Types
reorder
Changing the execution order of nodes for better performance.
When generated: The system detects that a different node ordering consistently produces faster runs, lower costs, or higher success rates.
Structure:
{
"type": "reorder",
"recommendation": {
"action": "reorder",
"move": "<node_id>",
"before": "<other_node_id>"
}
}
How to apply: Modify the depends_on chains in the pipeline to reflect the new order. Ensure the DAG remains valid (no cycles, no broken dependencies).
substitute_model
Using a different AI model for a node to optimize cost or quality.
When generated: The system compares outputs across model variants and finds that a cheaper or faster model produces equivalent results for a specific node type in a specific context.
Structure:
{
"type": "substitute_model",
"recommendation": {
"action": "substitute_model",
"node": "<node_id>",
"from": "<current_model>",
"to": "<recommended_model>"
}
}
How to apply: Update the node's config.model field. Monitor the next few runs to confirm quality is maintained.
skip_node
Skipping a node that provides no value in a specific context.
When generated: The system detects that a node consistently produces no actionable output for a repository or file set. For example, a SecurityScanner that never finds issues in a static documentation repo.
Structure:
{
"type": "skip_node",
"recommendation": {
"action": "skip_node",
"node": "<node_id>",
"reason": "No issues found in last 12 runs on this repo"
}
}
How to apply: Add a condition to the node that skips it, or remove it from the pipeline. Always inform the user when skipping a node.
add_node
Adding a node that would improve pipeline outcomes.
When generated: The system identifies a pattern where pipelines that include an extra step have significantly better outcomes. For example, adding a LintFixer before TestRunner reduces test failures.
Structure:
{
"type": "add_node",
"recommendation": {
"action": "add_node",
"node_type": "<NodeType>",
"insert_before": "<existing_node_id>",
"config": { /* suggested config */ }
}
}
How to apply: Add the recommended node to the pipeline and wire its dependencies appropriately.
adjust_config
Tuning a node's configuration for better results.
When generated: The system identifies that specific configuration values consistently produce better outcomes. For example, increasing scan_depth for SecurityScanner in repos with deep dependency trees.
Structure:
{
"type": "adjust_config",
"recommendation": {
"action": "adjust_config",
"node": "<node_id>",
"changes": {
"scan_depth": { "from": "shallow", "to": "deep" },
"timeout_seconds": { "from": 120, "to": 300 }
}
}
}
adjust_timeout
Modifying timeout values based on observed execution times.
When generated: Nodes consistently finish much faster than their timeout (wasted buffer) or consistently time out (timeout too low).
Structure:
{
"type": "adjust_timeout",
"recommendation": {
"action": "adjust_timeout",
"node": "<node_id>",
"current_timeout": 120,
"recommended_timeout": 300,
"p95_duration": 245
}
}
Recording Insights After Pipeline Completion
After each pipeline run completes, the system automatically records execution data. You can also manually record insights from your observations:
Automatic Recording
The orchestrator automatically records:
- Duration per node
- Cost per node
- Success/failure status and error types
- Output quality signals (e.g., number of issues found, test pass rate)
- Resource utilization
Manual Insight Recording
If you observe a pattern that the automatic system might not catch, record it manually:
cortivex_insights({
action: "record",
repo: "/path/to/repo",
insight: {
type: "reorder",
description: "Running tests before code review in this repo saves time because the test suite is fast (< 10s) and review can focus on passing code",
confidence: 0.70,
evidence: {