ML Pipeline
Single entry point for ML-related tasks. Orchestrates three roles across a data-driven analysis pipeline: Product Manager formulates the task, ML Engineer owns analysis and modeling, SRE Engineer extracts production data. Domain specifics are read from the project's CLAUDE.md.
Architecture note: This workflow is extensible — future sub-flows (model training, prompt tuning, A/B testing) can be added as numbered branches in Step 2.
1. Receive ML Task
Gather the ML task from the user:
- Objective: What do you want to learn or improve? (e.g., "tune scoring weights", "improve data collection prompts", "analyze conversion funnel")
- Data source: Where is the data? (production database, logs, external API, local files)
- Expected output: What decisions will the analysis inform?
- Constraints: Timeline, data sensitivity, compute budget
If the user provides a vague request — proceed to Step 2 where Agent(product-manager) will help formulate it.
2. Determine Pipeline Type
Select the pipeline based on the ML task:
| Pipeline Type | When to Use | Status | |---|---|---| | Data Analysis | Extract prod data → analyze → model → recommend parameter/prompt changes | ✅ Active | | Model Training | Train or retrain ML models on collected data | 🔮 Future | | Prompt Tuning | Optimize LLM prompts based on output quality analysis | 🔮 Future | | A/B Test Analysis | Analyze results of live experiments | 🔮 Future |
For MVP, proceed with Data Analysis pipeline (Steps 3–9). Future pipeline types will be added as parallel branches from this step.
3. Read Project Context
Apply Agent(product-manager).
Read and internalize the project's domain:
CLAUDE.md(root) — tech stack, domain description, business contextARCHITECTURE.md— system components, data flow, service boundariesFEATURES.md(if exists) — current feature inventory, scoring/evaluation parameters
Extract and summarize:
- Domain: What does the product do? What data does it collect?
- Evaluation parameters: What metrics/weights/scores does the product use?
- Data sources: Where is production data stored? (database, logs, analytics)
- Current prompts: If the product uses LLM prompts for data collection — identify prompt locations and purpose
Present the domain summary to the user for confirmation before proceeding.
4. Formulate ML Task
Continue with Agent(product-manager).
Transform the user's request into a structured ML task specification:
## ML Task Specification
### Objective
[What we want to learn or improve — measurable outcome]
### Business Context
[Why this matters — which product decisions depend on this analysis]
### Success Criteria
[How we know the analysis is complete and actionable]
- [ ] Data requirements defined and extracted
- [ ] Statistical analysis performed with confidence intervals
- [ ] Recommendations backed by evidence (not intuition)
- [ ] Action plan produced (parameter changes, prompt modifications, or feature work)
### Scope
- **In scope**: [specific parameters, data ranges, user segments]
- **Out of scope**: [what we explicitly skip]
### Constraints
- Data sensitivity: [PII handling, anonymization needs]
- Time range: [how far back to analyze]
- Compute budget: [any limits on processing]Present to user for approval. Adjust based on feedback.
5. Define Data Requirements
Apply Agent(ml-engineer).
Based on the ML task specification from Step 4, define exactly what data is needed:
<data_requirements>
- Data points: Which fields/columns/metrics are needed?
- Granularity: Per-record, aggregated, time-series?
- Time range: From when to when?
- Filters: Which segments, categories, statuses?
- Volume estimate: Expected row count / data size
- Format: CSV, JSON, Parquet, database query result?
- Joins: Which tables/sources need to be combined?
- Anonymization: Which fields contain PII and how to handle?
</data_requirements>
Produce a concrete data extraction plan — either SQL queries, API calls, or log extraction commands that Agent(sre-engineer) can execute.
Present the data requirements and extraction plan to the user for review.
6. Extract Production Data
Apply Agent(sre-engineer).
Execute the data extraction plan from Step 5. This step uses SRE's production access and operational expertise.
⚠️ SAFETY: All commands are READ-ONLY. No mutations to production data or systems.
6a. Establish Production Access
Follow /analyze-prod Steps 3a–3b to verify cloud platform context. Consult cloud-platforms skill for platform-specific CLI commands.
6b. Execute Data Extraction
Based on the extraction plan:
| Data Source | Method | |---|---| | Managed DB (SQL) | Platform-specific connection (see cloud-platforms skill CLI Reference → Data Extraction) + psql/mysql query | | Application logs | Platform-specific log query (see cloud-platforms skill Observability → Diagnostic Commands) | | Monitoring metrics | Platform-specific metrics API (see cloud-platforms skill Observability) | | Data warehouse | Platform-specific query tool (BigQuery / Synapse / Athena) | | Object storage | Platform-specific CLI (gsutil / az storage / aws s3) | | Kubernetes pod data | kubectl logs or kubectl exec (read-only) |
For each extraction:
- Show the exact command/query to the user before execution
- Execute after approval
- Save results to a local working directory (e.g.,
data/raw/) - Verify data completeness — row counts, date ranges, null percentages
6c. Data Handoff
Present extraction summary:
## Data Extraction Summary
- Source: [database/logs/metrics]
- Records extracted: [count]
- Time range: [from — to]
- Files: [local paths]
- Data quality notes: [nulls, anomalies, gaps]7. Analyze Data and Build Models
Apply Agent(ml-engineer).
Perform the analysis following ML Engineer's reasoning protocol:
7a. Exploratory Data Analysis (EDA)
- Profile the extracted data: distributions, correlations, outliers, missing values
- Visualize key patterns (histograms, scatter plots, time series, heatmaps)
- Identify data quality issues that could affect analysis
- Document findings and assumptions
7b. Statistical Analysis
- Hypothesis testing for differences between segments/groups
- Correlation analysis between parameters and outcomes
- Time-series analysis for trends and seasonality (if applicable)
- Confidence intervals for all key metrics — never report point estimates without uncertainty
7c. Modeling (if applicable)
- Establish a baseline (current parameter weights, simple heuristic, or mean prediction)
- Train models to identify optimal parameters: regression, classification, or optimization
- Evaluate against baseline with appropriate metrics
- Feature importance analysis — which parameters matter most?
- Cross-validation for robust estimates
7d. Document Results
Present analysis as a structured report:
## Analysis Report
### Data Summary
- Records analyzed: [count]
- Time period: [range]
- Segments: [if applicable]
### Key Findings
1. [Finding with statistical evidence]
2. [Finding with statistical evidence]
3. [Finding with statistical evidence]
### Visualizations
[Charts, plots, tables illustrating findings]
### Model Performance (if applicable)
- Baseline: [metric = value]
- Proposed: [metric = value]
- Improvement: [delta with confidence interval]8. Produce Recommendations
Continue with Agent(ml-engineer), consult Agent(product-manager) for business framing.
Translate analysis findings into actionable recommendations:
## ML Recommendations
### Summary
[One paragraph: what the analysis revealed and what should change]
### Recommended Changes
#### Parameter/Weight Changes
| Parameter | Current Value | Recommended Value | Expected Impact | Confidence |