Debugging Techniques

Purpose

Provides systematic debugging workflows for local, remote, container, and production environments across Python, Go, Rust, and Node.js. Covers interactive debuggers, container debugging with ephemeral containers, and production-safe techniques using correlation IDs and distributed tracing.

When to Use This Skill

Trigger this skill for:

• Setting breakpoints in Python, Go, Rust, or Node.js code
• Debugging running containers or Kubernetes pods
• Setting up remote debugging connections
• Safely debugging production issues
• Inspecting goroutines, threads, or async tasks
• Analyzing core dumps or stack traces
• Choosing the right debugging tool for a scenario

Quick Reference by Language

Python Debugging

Built-in: pdb

# Python 3.7+
def buggy_function(x, y):
    breakpoint()  # Stops execution here
    return x / y

# Older Python
import pdb
pdb.set_trace()

Essential pdb commands:

• list (l) - Show code around current line
• next (n) - Execute current line, step over functions
• step (s) - Execute current line, step into functions
• continue (c) - Continue until next breakpoint
• print var (p) - Print variable value
• where (w) - Show stack trace
• quit (q) - Exit debugger

Enhanced tools:

• ipdb - Enhanced pdb with tab completion, syntax highlighting (pip install ipdb)
• pudb - Terminal GUI debugger (pip install pudb)
• debugpy - VS Code integration (included in Python extension)

Debugging tests:

pytest --pdb  # Drop into debugger on test failure

For detailed Python debugging patterns, see references/python-debugging.md.

Go Debugging

Delve - Official Go debugger

Installation:

go install github.com/go-delve/delve/cmd/dlv@latest

Basic usage:

dlv debug main.go              # Debug main package
dlv test github.com/me/pkg     # Debug test suite
dlv attach <pid>               # Attach to running process
dlv debug -- --config prod.yaml  # Pass arguments

Essential commands:

• break main.main (b) - Set breakpoint at function
• break file.go:10 (b) - Set breakpoint at line
• continue (c) - Continue execution
• next (n) - Step over
• step (s) - Step into
• print x (p) - Print variable
• goroutine (gr) - Show current goroutine
• goroutines (grs) - List all goroutines
• goroutines -t - Show goroutine stacktraces
• stack (bt) - Show stack trace

Goroutine debugging:

(dlv) goroutines                 # List all goroutines
(dlv) goroutines -t              # Show stacktraces
(dlv) goroutines -with user      # Filter user goroutines
(dlv) goroutine 5                # Switch to goroutine 5

For detailed Go debugging patterns, see references/go-debugging.md.

Rust Debugging

LLDB - Default Rust debugger

Compilation:

cargo build  # Debug build includes symbols by default

Usage:

rust-lldb target/debug/myapp   # LLDB wrapper for Rust
rust-gdb target/debug/myapp    # GDB wrapper (alternative)

Essential LLDB commands:

• breakpoint set -f main.rs -l 10 - Set breakpoint at line
• breakpoint set -n main - Set breakpoint at function
• run (r) - Start program
• continue (c) - Continue execution
• next (n) - Step over
• step (s) - Step into
• print variable (p) - Print variable
• frame variable (fr v) - Show local variables
• backtrace (bt) - Show stack trace
• thread list - List all threads

VS Code integration:

• Install CodeLLDB extension (vadimcn.vscode-lldb)
• Configure launch.json for Rust projects

For detailed Rust debugging patterns, see references/rust-debugging.md.

Node.js Debugging

Built-in: node --inspect

Basic usage:

node --inspect-brk app.js       # Start and pause immediately
node --inspect app.js           # Start and run
node --inspect=0.0.0.0:9229 app.js  # Specify host/port

Chrome DevTools:

1. Open chrome://inspect
2. Click "Open dedicated DevTools for Node"
3. Set breakpoints, inspect variables

VS Code integration: Configure launch.json:

{
  "type": "node",
  "request": "launch",
  "name": "Launch Program",
  "program": "${workspaceFolder}/app.js"
}

Docker debugging:

EXPOSE 9229
CMD ["node", "--inspect=0.0.0.0:9229", "app.js"]

For detailed Node.js debugging patterns, see references/nodejs-debugging.md.

Container & Kubernetes Debugging

kubectl debug with Ephemeral Containers

When to use:

• Container has crashed (kubectl exec won't work)
• Using distroless/minimal image (no shell, no tools)
• Need debugging tools without rebuilding image
• Debugging network issues

Basic usage:

# Add ephemeral debugging container
kubectl debug -it <pod-name> --image=nicolaka/netshoot

# Share process namespace (see other container processes)
kubectl debug -it <pod-name> --image=busybox --share-processes

# Target specific container
kubectl debug -it <pod-name> --image=busybox --target=app

Recommended debugging images:

• nicolaka/netshoot (~380MB) - Network debugging (curl, dig, tcpdump, netstat)
• busybox (~1MB) - Minimal shell and utilities
• alpine (~5MB) - Lightweight with package manager
• ubuntu (~70MB) - Full environment

Node debugging:

kubectl debug node/<node-name> -it --image=ubuntu

Docker container debugging:

docker exec -it <container-id> sh

# If no shell available
docker run -it --pid=container:<container-id> \
           --net=container:<container-id> \
           busybox sh

For detailed container debugging patterns, see references/container-debugging.md.

Production Debugging

Production Debugging Principles

Golden rules:

1. Minimal performance impact - Profile overhead, limit scope
2. No blocking operations - Use non-breaking techniques
3. Security-aware - Avoid logging secrets, PII
4. Reversible - Can roll back quickly (feature flags, Git)
5. Observable - Structured logging, correlation IDs, tracing

Safe Production Techniques

1. Structured Logging

import logging
import json

logger = logging.getLogger(__name__)
logger.info(json.dumps({
    "event": "user_login_failed",
    "user_id": user_id,
    "error": str(e),
    "correlation_id": request_id
}))

2. Correlation IDs (Request Tracing)

func handleRequest(w http.ResponseWriter, r *http.Request) {
    correlationID := r.Header.Get("X-Correlation-ID")
    if correlationID == "" {
        correlationID = generateUUID()
    }
    ctx := context.WithValue(r.Context(), "correlationID", correlationID)
    log.Printf("[%s] Processing request", correlationID)
}

3. Distributed Tracing (OpenTelemetry)

from opentelemetry import trace

tracer = trace.get_tracer(__name__)

def process_order(order_id):
    with tracer.start_as_current_span("process_order") as span:
        span.set_attribute("order.id", order_id)
        span.add_event("Order validated")

4. Error Tracking Platforms

• Sentry - Exception tracking with context
• New Relic - APM with error tracking
• Datadog - Logs, metrics, traces
• Rollbar - Error monitoring

Production debugging workflow:

1. Detect - Error tracking alert, log spike, metric anomaly
2. Locate - Find correlation ID, search logs, view distributed trace
3. Reproduce - Try to reproduce in staging with production data (sanitized)
4. Fix - Create feature flag, deploy to canary first
5. Verify - Check error rates, review logs, monitor traces

For detailed production debugging patterns, see references/production-debugging.md.

Decision Framework

Which Debugger for Which Language?