Numerical Analysis
Calculus and numerical methods form the computational backbone of applied mathematics. Calculus provides exact analytical tools — limits, derivatives, integrals, series — while numerical methods provide approximate algorithms for problems where exact solutions are unavailable or impractical. This skill covers both the analytical foundations and the computational techniques, including the pitfalls that arise when mathematics meets finite-precision arithmetic.
Agent affinity: euler (series, differential equations, numerical methods)
Concept IDs: math-computational-fluency, math-functions, math-expected-value, math-descriptive-statistics
Part I — Limits
The Epsilon-Delta Definition
A function f(x) has limit L as x approaches a (written lim_{x->a} f(x) = L) if for every epsilon > 0, there exists delta > 0 such that whenever 0 < |x - a| < delta, we have |f(x) - L| < epsilon.
This definition — due to Weierstrass, formalizing Cauchy's intuition — is the foundation of rigorous analysis. Every subsequent concept in calculus (continuity, derivatives, integrals) is built on limits.
Worked example. Prove that lim_{x->3} (2x + 1) = 7.
Proof. Let epsilon > 0 be given. We need |f(x) - 7| < epsilon whenever 0 < |x - 3| < delta. Compute: |(2x + 1) - 7| = |2x - 6| = 2|x - 3|. Choose delta = epsilon/2. Then 0 < |x - 3| < delta implies |f(x) - 7| = 2|x - 3| < 2(epsilon/2) = epsilon.
Limit Laws
For lim_{x->a} f(x) = L and lim_{x->a} g(x) = M:
- Sum: lim(f + g) = L + M
- Product: lim(f g) = L M
- Quotient: lim(f / g) = L / M (when M != 0)
- Composition: lim f(g(x)) = f(M) if f is continuous at M
L'Hopital's Rule
If lim_{x->a} f(x)/g(x) yields an indeterminate form (0/0 or infinity/infinity), and the limit of f'(x)/g'(x) exists, then:
lim_{x->a} f(x)/g(x) = lim_{x->a} f'(x)/g'(x)
Worked example. Compute lim_{x->0} sin(x)/x.
Direct substitution gives 0/0. Apply L'Hopital: lim_{x->0} cos(x)/1 = cos(0) = 1.
When NOT to use. L'Hopital requires an indeterminate form. Applying it to a determinate form (e.g., a nonzero numerator with zero denominator) gives wrong results.
The Squeeze Theorem
If g(x) <= f(x) <= h(x) near a, and lim g(x) = lim h(x) = L, then lim f(x) = L.
Worked example. Prove lim_{x->0} x^2 sin(1/x) = 0.*
Since -1 <= sin(1/x) <= 1, we have -x^2 <= x^2 sin(1/x) <= x^2. Both -x^2 and x^2 approach 0, so by the squeeze theorem, the limit is 0.
Part II — Derivatives
Definition and Rules
The derivative of f at x = a is f'(a) = lim_{h->0} (f(a + h) - f(a)) / h, when this limit exists. It measures the instantaneous rate of change.
Differentiation rules:
| Rule | Formula | |---|---| | Power | d/dx [x^n] = n*x^(n-1) | | Sum | (f + g)' = f' + g' | | Product | (f*g)' = f'*g + f*g' | | Quotient | (f/g)' = (f'*g - f*g') / g^2 | | Chain | d/dx [f(g(x))] = f'(g(x)) * g'(x) | | Exponential | d/dx [e^x] = e^x | | Logarithm | d/dx [ln(x)] = 1/x | | Trig | d/dx [sin(x)] = cos(x), d/dx [cos(x)] = -sin(x) |
Applications of Derivatives
Optimization. Find critical points where f'(x) = 0 or f'(x) is undefined. Use the second derivative test: f''(c) > 0 means local minimum, f''(c) < 0 means local maximum, f''(c) = 0 is inconclusive.
Worked example. Find the dimensions of a rectangle with perimeter 20 that maximizes area.
Let the sides be x and y with 2x + 2y = 20, so y = 10 - x. Area A = x(10 - x) = 10x - x^2. Then A'(x) = 10 - 2x = 0 gives x = 5, y = 5. A''(x) = -2 < 0, confirming a maximum. The optimal rectangle is a 5 x 5 square with area 25.
Related rates. When two or more quantities change with time, differentiate their relationship implicitly with respect to time.
Mean Value Theorem. If f is continuous on [a, b] and differentiable on (a, b), then there exists c in (a, b) with f'(c) = (f(b) - f(a)) / (b - a). This theorem is the foundation for many proofs in analysis.
Part III — Integration
Riemann Sums and the Definite Integral
The definite integral from a to b of f(x)dx is defined as the limit of Riemann sums:
integral_a^b f(x)dx = lim_{n->infinity} sum_{i=1}^{n} f(x_i*) * Delta_x
where Delta_x = (b - a)/n and x_i* is a sample point in the i-th subinterval.
The Fundamental Theorem of Calculus
Part 1 (FTC1). If f is continuous on [a, b], then F(x) = integral_a^x f(t)dt is differentiable and F'(x) = f(x). Integration and differentiation are inverse operations.
Part 2 (FTC2). If F is any antiderivative of f on [a, b], then integral_a^b f(x)dx = F(b) - F(a).
Integration Techniques
Substitution (u-substitution). For integral f(g(x)) * g'(x) dx, let u = g(x), du = g'(x)dx.
Worked example. Compute integral 2x cos(x^2) dx.*
Let u = x^2, du = 2x dx. The integral becomes integral cos(u) du = sin(u) + C = sin(x^2) + C.
Integration by parts. integral u dv = uv - integral v du. Choose u and dv using the LIATE heuristic: Logarithmic, Inverse trig, Algebraic, Trigonometric, Exponential — u should be the type appearing earliest in this list.
Worked example. Compute integral x e^x dx.*
Let u = x (algebraic), dv = e^x dx. Then du = dx, v = e^x. By parts: x*e^x - integral e^x dx = x*e^x - e^x + C = e^x(x - 1) + C.
Partial fractions. Decompose a rational function into simpler fractions before integrating. Factor the denominator, then express as a sum of fractions with linear and irreducible quadratic denominators.
Trigonometric substitution. For integrands involving sqrt(a^2 - x^2), sqrt(a^2 + x^2), or sqrt(x^2 - a^2), substitute x = a*sin(theta), x = a*tan(theta), or x = a*sec(theta) respectively.
Part IV — Series
Convergence Tests
A series sum_{n=1}^{infinity} a_n converges if the sequence of partial sums S_N = sum_{n=1}^{N} a_n has a finite limit.
| Test | Statement | Use when | |---|---|---| | Divergence | If lim a_n != 0, the series diverges | Quick first check | | Geometric | sum r^n converges iff |r| < 1, to 1/(1-r) | Series is geometric | | p-series | sum 1/n^p converges iff p > 1 | Power-law terms | | Integral | sum a_n and integral f(x)dx converge/diverge together | f is positive, decreasing, continuous | | Comparison | 0 <= a_n <= b_n and sum b_n converges => sum a_n converges | Can bound by a known series | | Limit comparison | lim a_n/b_n = L (0 < L < infinity) => same behavior | Asymptotic comparison | | Ratio | lim |a_{n+1}/a_n| < 1 => converges, > 1 => diverges | Factorial or exponential terms | | Root | lim |a_n|^{1/n} < 1 => converges, > 1 => diverges | n-th power terms | | Alternating series | sum (-1)^n b_n converges if b_n decreases to 0 | Alternating sign |
Taylor and Maclaurin Series
The Taylor series of f about x = a is:
f(x) = sum_{n=0}^{infinity} f^(n)(a) / n! * (x - a)^n
The Maclaurin series is the Taylor series about a = 0.
Key Maclaurin series:
| Function | Series | Radius of convergence | |---|---|---| | e^x | sum x^n / n! | infinity | | sin(x) | sum (-1)^n x^(2n+1) / (2n+1)! | infinity | | cos(x) | sum (-1)^n x^(2n) / (2n)! | infinity | | 1/(1-x) | sum x^n | 1 | | ln(1+x) | sum (-1)^(n+1) x^n / n | 1 | | arctan(x) | sum (-1)^n x^(2n+1) / (2n+1) | 1 |
Taylor's remainder theorem. If f has n+1 continuous derivatives on an interval containing a and x, then:
f(x) = P_n(x) + R_n(x)
where P_n is the n-th degree Taylor polynomial and |R_n(x)| <= M * |x - a|^(n+1) / (n+1)! for M = max |f^(n+1)(t)| on the interval. This bounds the approximation error.
Part V — Differential Equations
Separable Equations
Form: dy/dx = f(x) * g(y). Separate: dy/g(y) = f(x) dx. Integrate both sides.
Worked example. Solve dy/dx = xy, y(0) = 1.
Separate: dy/y = x dx. Integrate: ln|y| = x^2/2 + C. Exponentiate: y = A * e^(x^2/2). Initial condition y(0) = 1 gives A = 1. Solution: y = e^(x^2/2).
First-Order Linear Equations
Form: dy/dx + P(x)*y = Q(x). Multiply by integrating factor mu(x) = e^(integral P(