structured-logging

Structured Logging System Design

Core Principles

• Logs are data — Treat every log as a queryable, structured data point
• Single source of truth — One logging configuration, used everywhere
• Context is king — Every log must be traceable to its source and request
• Human + Machine readable — Structured for parsing, clear for debugging
• Progressive enhancement — Start simple, add fields as needed
• No secrets — Never log passwords, tokens, or PII without masking

System Architecture

The Golden Rule

Configure once, use everywhere. Never instantiate loggers directly in business code.

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│                     APPLICATION LAYER                           │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐        │
│  │ Service A│  │ Service B│  │ Service C│  │ Handler D│        │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └────┬─────┘        │
│       │             │             │             │               │
│       └─────────────┴──────┬──────┴─────────────┘               │
│                            ▼                                    │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │              LOGGING INFRASTRUCTURE                      │   │
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────┐  │   │
│  │  │ Logger      │  │ Context     │  │ Formatters &    │  │   │
│  │  │ Factory     │  │ Provider    │  │ Transformers    │  │   │
│  │  └─────────────┘  └─────────────┘  └─────────────────┘  │   │
│  └─────────────────────────────────────────────────────────┘   │
│                            │                                    │
└────────────────────────────┼────────────────────────────────────┘
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                      OUTPUT TARGETS                             │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐        │
│  │ Console  │  │   File   │  │ Log Agg. │  │  Metrics │        │
│  │ (Dev)    │  │ (Local)  │  │ (Prod)   │  │ (Alerts) │        │
│  └──────────┘  └──────────┘  └──────────┘  └──────────┘        │
└─────────────────────────────────────────────────────────────────┘

Project Structure

src/
├── lib/
│   └── logging/                    # Centralized logging infrastructure
│       ├── index.ts                # Public API exports
│       ├── logger.ts               # Core logger implementation
│       ├── context.ts              # Request context management
│       ├── formatters.ts           # Output formatters (JSON, pretty)
│       ├── transports.ts           # Output destinations
│       ├── middleware.ts           # Framework integrations
│       ├── decorators.ts           # Method decorators (optional)
│       └── types.ts                # Type definitions
├── modules/
│   ├── users/
│   │   └── user.service.ts         # Uses: import { logger } from '@/lib/logging'
│   └── orders/
│       └── order.service.ts        # Uses: import { logger } from '@/lib/logging'
└── main.ts                         # Initialize logging once at startup

Anti-Patterns to Avoid

// ❌ BAD: Creating loggers everywhere
class UserService {
  private logger = new Logger({ service: 'user' });  // Don't do this
}

class OrderService {
  private logger = new Logger({ service: 'order' }); // Duplicated config
}

// ❌ BAD: Inconsistent configuration
const logger1 = winston.createLogger({ format: json() });
const logger2 = winston.createLogger({ format: simple() }); // Different format!

// ❌ BAD: Direct console usage in production code
console.log('User created:', userId);  // No structure, no context

// ❌ BAD: Passing logger instances around
function processOrder(order: Order, logger: Logger) { } // Don't pass loggers

Correct Patterns

// ✅ GOOD: Single configuration, exported singleton
// lib/logging/index.ts
export const logger = createLogger(config);
export { getContextLogger } from './context';

// ✅ GOOD: Import from central location
// modules/users/user.service.ts
import { logger, getContextLogger } from '@/lib/logging';

class UserService {
  async createUser(data: CreateUserInput) {
    const log = getContextLogger();  // Gets context automatically
    log.info('Creating user', { email: data.email });
    // ...
  }
}

// ✅ GOOD: Configure once at application startup
// main.ts
import { initializeLogging } from '@/lib/logging';

initializeLogging({
  service: 'my-app',
  environment: process.env.NODE_ENV,
  level: process.env.LOG_LEVEL || 'info',
});

Log Record Schema

Tier 1: Essential Fields (Always Required)

| Field | Type | Description | Example | |-------|------|-------------|---------| | timestamp | ISO 8601 | When the event occurred | 2025-12-16T10:30:00.123Z | | level | string | Log severity | INFO, WARN, ERROR | | message | string | Human-readable description | User login successful | | service | string | Service/application name | user-auth |

{
  "timestamp": "2025-12-16T10:30:00.123Z",
  "level": "INFO",
  "message": "Payment processed successfully",
  "service": "payment-service"
}

Tier 2: Tracing Fields (Distributed Systems)

| Field | Type | Description | Example | |-------|------|-------------|---------| | trace_id | string | Request chain identifier (32 hex) | 7b2e4f1a9c3d8e5b6a1f2c3d4e5f6a7b | | span_id | string | Current operation ID (16 hex) | 1a2b3c4d5e6f7890 | | parent_span_id | string | Parent operation ID | 0987654321fedcba | | request_id | string | HTTP request identifier | req_abc123 |

{
  "timestamp": "2025-12-16T10:30:00.123Z",
  "level": "INFO",
  "message": "Order created",
  "service": "order-service",
  "trace_id": "7b2e4f1a9c3d8e5b6a1f2c3d4e5f6a7b",
  "span_id": "1a2b3c4d5e6f7890",
  "request_id": "req_abc123"
}

Tier 3: Context Fields (Debugging & Analysis)

| Field | Type | Description | Example | |-------|------|-------------|---------| | user_id | string | User identifier (hashed if needed) | usr_xyz789 | | method | string | Function/method name | OrderService.create | | duration_ms | number | Operation duration | 245 | | error_code | string | Application error code | ERR_PAYMENT_FAILED | | error_message | string | Error description | Card declined | | stack_trace | string | Exception stack (ERROR only) | ... |

Tier 4: Environment Fields (Operations)

| Field | Type | Description | Example | |-------|------|-------------|---------| | env | string | Deployment environment | production | | version | string | Service version | 2.1.0 | | host | string | Server hostname | web-01 | | region | string | Cloud region | us-east-1 | | instance_id | string | Container/instance ID | i-0abc123 |

Log Levels

When to Use Each Level

| Level | When to Use | Production Visibility | Alert | |-------|-------------|----------------------|-------| | TRACE | Ultra-fine debugging | ❌ Off | No | | DEBUG | Development diagnostics | ❌ Off | No | | INFO | Normal business operations | ✅ On | No | | WARN | Unexpected but recoverable | ✅ On | Optional | | ERROR | Failures requiring attention | ✅ On | Yes | | FATAL | Application cannot continue | ✅ On | Immediate |

Level Selection Guide

DEBUG: Variable states, method entry/exit, detailed flow
       → "Processing item 5 of 10"
       → "Cache miss for key: user:123"

INFO:  Business milestones, successful operations
       → "User registered successfully"
       → "Order #123 shipped"
       → "Scheduled job completed: 50 records processed"

WARN:  Degraded service, retries, approaching limits
       → "Database connection slow (>2s), retrying"
       → "Rate limit at 80%, throttling soon"
       → "Deprecated API called, migrate to v2"

ERROR: Operation failures, exceptions, da