Numerical Analysis: Taylor Series

## Pre-requisite Knowledge

### Rolle's theorem

> Let $f:[a, b] \to \mathbb{R}$ be differentiable. Assume that $f(a) = f(b)$, then there is at least one point $c \in (a, b)$ where $f'(c) = 0$

### Mean value theorem

> Let $f:[a, b] \to \mathbb{R}$ be differentiable. Then there is at least one point $c \in (a, b)$ such that $f'(c) = \frac{f(b) - f(a)}{b - a}$

### Taylor theorem

Taylor's theorem is a fundamental result in calculus that provides a way to approximate a function by its Taylor series, which is an infinite sum of terms calculated from the values of the function's derivatives at a single point.

1. **Statement of Taylor's Theorem**:

- For a function $ f(x) $ that is $ n $ times differentiable at a point $ a $, Taylor's theorem states that the function can be approximated near $ a $ by the $ n $th-order Taylor polynomial plus a remainder term.

- The Taylor polynomial $ P_n(x) $ of $ f(x) $ centered at $ a $ is given by:
     $$
     P_n(x) = f(a) + f'(a)(x - a) + \frac{f''(a)}{2!}(x - a)^2 + \cdots + \frac{f^{(n)}(a)}{n!}(x - a)^n
     $$

- The remainder term $ R_n(x) $ represents the error in approximating $ f(x) $ by $ P_n(x) $.

2. **Remainder Term**:

- There are several forms of the remainder term, including the Lagrange form and the Peano form.

- The Lagrange form of the remainder is often used and is given by:
     $$
     R_n(x) = \frac{f^{(n+1)}(\xi)}{(n+1)!}(x - a)^{n+1}
     $$
     where $ \xi$ is some point between $ a $ and $ x $.

Therefore, the error is given by:
$$
E = f(x) - P_n(x) = \frac{f^{(n+1)}(\xi)}{(n+1)!} (x - a)^{n+1}
$$
with the upper bound
$$
|E| \leq \max_{\xi \in [a,x]} \frac{|f^{(n+1)}(\xi)|}{(n+1)!} (x-a)^{n+1}
$$
We can note that：
$$
f(x) = P_n(x) + O(h^{n+1}), h = (x-a)
$$
This is also called the **truncation error**.

## Approximating Continuous Functions

> Let $f:[a, b] \to \mathbb{R}$ be differentiable. The Stone-Weierstrass Theorem states that for any small number $\epsilon > 0: \exists P(x)$ a polynomial function such that：
> $$
> \max_{x \in [a, b]} |f(x) - P(x)| \leq \epsilon
> $$