data-substrate-analysis

Data Substrate Analysis

Analyzes the fundamental units of data and state management patterns.

Process

1. Locate type files — Find types.py, schema.py, models.py, state.py
2. Classify typing — Strict (Pydantic), structural (TypedDict), loose (dict)
3. Analyze mutation — In-place modification vs. copy-on-write
4. Document serialization — json(), dict(), pickle, custom methods

Typing Strategy Classification

Detection Patterns

| Strategy | Indicators | Files to Check | |----------|-----------|----------------| | Pydantic | BaseModel, Field(), validator | models.py, schema.py | | Dataclass | @dataclass, field() | types.py, models.py | | TypedDict | TypedDict, Required[], NotRequired[] | types.py | | NamedTuple | NamedTuple, typing.NamedTuple | types.py | | Loose | Dict[str, Any], plain dict | Throughout |

Analysis Questions

• Are boundaries validated (API ingress/egress)?
• Is nesting depth reasonable (<3 levels)?
• Are optional fields explicit or implicit None?
• Version migration path (Pydantic V1 → V2)?

Immutability Analysis

Mutable Patterns (Risk Indicators)

# In-place list modification
state.messages.append(msg)
state.history.extend(new_items)

# Direct dict mutation
state['key'] = value
state.update(new_data)

# Object attribute mutation
state.status = 'complete'

Immutable Patterns (Safer)

# Pydantic copy
new_state = state.model_copy(update={'key': value})

# Dataclass replace
new_state = replace(state, messages=[*state.messages, msg])

# Spread operator style
new_state = {**state, 'key': value}

# Frozen dataclass
@dataclass(frozen=True)
class State: ...

Serialization Strategy

Common Patterns

| Method | Code Pattern | Trade-offs | |--------|-------------|------------| | Pydantic JSON | .model_dump_json() | Type-safe, automatic | | Pydantic Dict | .model_dump() | For internal use | | Dataclass | asdict(obj) | Manual, no validation | | Custom | to_dict(), from_dict() | Full control | | Pickle | pickle.dumps() | Fast, fragile, security risk | | JSON | json.dumps(obj, default=...) | Requires encoder |

Questions to Answer

• Is serialization implicit (automatic) or explicit (manual)?
• How are nested objects handled?
• Is deserialization validated?
• What happens with unknown fields?

Output Template

## Data Substrate Analysis: [Framework Name]

### Typing Strategy
- **Primary Approach**: [Pydantic/Dataclass/TypedDict/Loose]
- **Key Files**: [List of files]
- **Nesting Depth**: [Shallow/Medium/Deep]
- **Validation**: [At boundaries/Everywhere/None]

### Core Primitives

| Type | Location | Purpose | Mutability |
|------|----------|---------|------------|
| Message | schema.py:L15 | Chat message | Immutable |
| State | state.py:L42 | Agent state | Mutable ⚠️ |
| Result | types.py:L78 | Tool output | Immutable |

### Mutation Analysis
- **Pattern**: [In-place/Copy-on-write/Mixed]
- **Risk Areas**: [List of mutable state locations]
- **Concurrency Safe**: [Yes/No/Partial]

### Serialization
- **Method**: [Pydantic/Custom/JSON]
- **Implicit/Explicit**: [Description]
- **Round-trip Tested**: [Yes/No/Unknown]

Integration

• Prerequisite: codebase-mapping to identify type files
• Feeds into: comparative-matrix for typing decisions
• Related: resilience-analysis for error handling in serialization