Linear AlgebraDeeper Theory3 of 6
Deeper Theory · 3

Inner Product Spaces: Orthogonality for Functions

We already know how to add and scale functions like vectors, and now we need a dot product for functions so we can check when two curves are perpendicular and project one function onto another.

1

Inner Products and Orthogonal Functions

For regular vectors, the dot product multiplies matching entries and adds them up, and two vectors are perpendicular when that sum is zero. How do we do the same thing for two continuous functions f(x)\first{f(x)} and g(x)\second{g(x)} on an interval?

Inner Products and Orthogonal Functions
[34]\htmlClass{lc-green}{\begin{bmatrix} 3 \\ 4 \end{bmatrix}}
[−43]\htmlClass{lc-pink}{\begin{bmatrix} -4 \\ 3 \end{bmatrix}}
x\htmlClass{lc-pink}{x}
x2\htmlClass{lc-violet}{x^2}
orthogonal?\text{orthogonal?}

On an interval [a,b][a, b], the standard inner product and the condition for orthogonality are:

⟨f,g⟩⏟inner product\displaystyle \underbrace{\langle \first{f}, \second{g} \rangle}_{\text{inner product}} =∫abf(x)g(x) dx,⟨f,g⟩=0⏟orthogonal on [a,b]\displaystyle = \int_a^b \first{f(x)}\second{g(x)}\,dx, \qquad \underbrace{\langle \first{f}, \second{g} \rangle = 0}_{\text{orthogonal on } [a, b]}

Here is how we check which pairs of 1\first{1}, x\second{x}, and x2\third{x^2} are orthogonal on the interval [−1,1][-1, 1].

Text solution
Multiply 1\first{1} and x\second{x} and integrate over [−1,1][-1, 1]The negative triangle on the left half of [−1,1][-1, 1] cancels the identical positive triangle on the right half, so 1\first{1} and x\second{x} are orthogonal.
⟨1,x⟩\displaystyle \langle \first{1}, \second{x} \rangle =∫−111⋅x dx\displaystyle = \int_{-1}^1 \first{1} \cdot \second{x}\,dx =−12+12\displaystyle = \below{-\frac{1}{2}} + \integrand{\frac{1}{2}} =0\displaystyle = 0
Multiply x\second{x} and x2\third{x^2} and integrate over [−1,1][-1, 1]The product x3\integrand{x^3} is negative on the left and an exact positive mirror image on the right, so x\second{x} and x2\third{x^2} are also orthogonal.
⟨x,x2⟩\displaystyle \langle \second{x}, \third{x^2} \rangle =∫−11x⋅x2 dx\displaystyle = \int_{-1}^1 \second{x} \cdot \third{x^2}\,dx =∫−11x3 dx\displaystyle = \int_{-1}^1 \integrand{x^3}\,dx =0\displaystyle = 0
Multiply 1\first{1} and x2\third{x^2} and integrate over [−1,1][-1, 1]The parabola x2\third{x^2} never goes below the xx-axis, leaving no negative area to cancel it, so 1\first{1} and x2\third{x^2} are not orthogonal.
⟨1,x2⟩\displaystyle \langle \first{1}, \third{x^2} \rangle =∫−111⋅x2 dx\displaystyle = \int_{-1}^1 \first{1} \cdot \third{x^2}\,dx =[x33]−11\displaystyle = \left[\frac{x^3}{3}\right]_{-1}^1 =23≠0\displaystyle = \frac{2}{3} \neq 0

The functions 1\first{1} and x\second{x} are orthogonal only when the interval sits symmetrically around 00 so the negative area below the xx-axis cancels the positive area above it; when the interval shifts to one side, such as [0,1][0, 1], the areas no longer cancel and ⟨1,x⟩\langle \first{1}, \second{x} \rangle becomes nonzero.

These two functions are orthogonal only on intervals where the negative and positive areas of their product cancel to zero.
2

Norm and Distance for Functions

Once we have an inner product, how do we measure the size of a single function and how far apart two functions are across an interval?

Norm and Distance for Functions
[34]\htmlClass{lc-green}{\begin{bmatrix} 3 \\ 4 \end{bmatrix}}
f(x)\htmlClass{lc-green}{f(x)}
−1-1
11
∥f∥=?\|\htmlClass{lc-green}{f}\| = {?}
f(x)\htmlClass{lc-green}{f(x)}
g(x)\htmlClass{lc-pink}{g(x)}
f(x)−g(x)\htmlClass{lc-orange}{f(x) - g(x)}
−1-1
xx
11

Squaring inside the integral makes the value at every xx positive or zero so positive and negative heights cannot cancel out:

∥f∥⏟norm (length)=⟨f,f⟩=∫−11f(x)2 dx,∥f−g∥⏟distance=⟨f−g,f−g⟩=∫−11(f(x)−g(x))2⏟squared vertical gap dx\displaystyle \begin{aligned} \underbrace{\lVert \first{f} \rVert}_{\text{norm (length)}} &= \sqrt{\langle \first{f}, \first{f} \rangle} \\ &= \sqrt{\int_{-1}^1 \first{f(x)}^2\,dx}, \\[14pt] \underbrace{\lVert \gap{f - g} \rVert}_{\text{distance}} &= \sqrt{\langle \gap{f - g}, \gap{f - g} \rangle} \\ &= \sqrt{\int_{-1}^1 \underbrace{(\gap{f(x) - g(x)})^2}_{\text{squared vertical gap}}\,dx} \end{aligned}

We can now find the sizes of our first two basis polynomials, 1\first{1} and x\second{x}, on [−1,1][-1, 1].

Text solution
Square 1\first{1} and integrate over [−1,1][-1, 1]The curve 12=1\first{1}^2 = \first{1} forms a rectangle of height 11 and width 22, giving a squared size of 22 and a norm of 2\sqrt{2}.
⟨1,1⟩\displaystyle \langle \first{1}, \first{1} \rangle =∫−1112 dx\displaystyle = \int_{-1}^1 \first{1}^2\,dx =2  ⟹  ∥1∥\displaystyle = 2 \implies \lVert \first{1} \rVert =⟨1,1⟩\displaystyle = \sqrt{\langle \first{1}, \first{1} \rangle} =2\displaystyle = \sqrt{2}
Square x\second{x} and integrate over [−1,1][-1, 1]Squaring x\second{x} gives x2\integrand{x^2}, which has area 23\frac{2}{3} on [−1,1][-1, 1], so x\second{x} has a norm of 23\sqrt{\frac{2}{3}}.
⟨x,x⟩\displaystyle \langle \second{x}, \second{x} \rangle =∫−11x2 dx\displaystyle = \int_{-1}^1 \integrand{x^2}\,dx =23  ⟹  ∥x∥\displaystyle = \frac{2}{3} \implies \lVert \second{x} \rVert =⟨x,x⟩\displaystyle = \sqrt{\langle \second{x}, \second{x} \rangle} =23\displaystyle = \sqrt{\frac{2}{3}}
3

What Counts as an Inner Product?

We can change the interval of integration or multiply by a weight function when we care more about accuracy in some parts of the interval than others. Before we use a new rule ⟨f,g⟩\langle \first{f}, \second{g} \rangle, we have to check whether it obeys the same three rules as the dot product so lengths and 90∘90^\circ projections still work the same way.

What Counts as an Inner Product?
⟨f,g⟩=∫−11f(x) g(x) dx⟨f,g⟩=?\displaystyle \begin{aligned} \langle \first{f}, \second{g} \rangle &= \int_{-1}^{1} \first{f(x)}\,\second{g(x)}\,dx \\ \langle \first{f}, \second{g} \rangle &= {?} \end{aligned}
does any formula count?\displaystyle \text{does any formula count?}

Any valid inner product must pass three checks for all functions f,g,h\first{f}, \second{g}, \third{h} and any constant cc:

Symmetry:⟨f,g⟩=⟨g,f⟩Linearity:⟨cf,g⟩=c⟨f,g⟩and⟨f+h,g⟩=⟨f,g⟩+⟨h,g⟩Positivity:⟨f,f⟩>0 for every nonzero f, and⟨f,f⟩=0 only if f=0\displaystyle \begin{aligned} &\textbf{Symmetry:} \\ &\quad \langle \first{f}, \second{g} \rangle = \langle \second{g}, \first{f} \rangle \\[8pt] &\textbf{Linearity:} \\ &\quad \langle c\first{f}, \second{g} \rangle = c\langle \first{f}, \second{g} \rangle \quad \text{and} \\ &\quad \langle \first{f} + \third{h}, \second{g} \rangle = \langle \first{f}, \second{g} \rangle + \langle \third{h}, \second{g} \rangle \\[8pt] &\textbf{Positivity:} \\ &\quad \langle \first{f}, \first{f} \rangle > 0 \text{ for every nonzero } \first{f}, \text{ and} \\ &\quad \langle \first{f}, \first{f} \rangle = 0 \text{ only if } \first{f} = 0 \end{aligned}

Here is the deciding check for four candidate rules on [−1,1][-1, 1].

Text solution
Check Positivity for the positive weight x2+1\weight{x^2 + 1}Symmetry and Linearity hold, and because x2+1>0\weight{x^2 + 1} > 0 everywhere on [−1,1][-1, 1], multiplying by f(x)2≥0\first{f(x)}^2 \ge 0 guarantees a strictly positive area for every nonzero f\first{f}.
⟨f,f⟩=∫−11(x2+1)⏟>0f(x)2⏟≥0 dx>0for all f≠0(Valid inner product)\displaystyle \begin{aligned} \langle \first{f}, \first{f} \rangle &= \int_{-1}^1 \underbrace{\weight{(x^2 + 1)}}_{> 0}\underbrace{\first{f(x)}^2}_{\ge 0}\,dx \\ &> 0 \quad \text{for all } \first{f} \neq 0 \\ &\quad (\text{Valid inner product}) \end{aligned}
Test Positivity for the weight x\weight{x} using f(x)=1\first{f(x)} = \first{1}Because the weight x\weight{x} goes negative on the left half of [−1,1][-1, 1], the nonzero function f(x)=1\first{f(x)} = \first{1} cancels to zero length (and functions large near −1-1 get negative ⟨f,f⟩\langle \first{f}, \first{f} \rangle).
⟨1,1⟩\displaystyle \langle \first{1}, \first{1} \rangle =∫−11x⋅12 dx\displaystyle = \int_{-1}^1 \weight{x} \cdot \first{1}^2\,dx =0(Fails Positivity)\displaystyle = 0 \quad (\text{Fails Positivity})
Test Positivity for the derivative rule ∫−11f′(x)g′(x) dx\int_{-1}^1 \first{f'(x)}\second{g'(x)}\,dx using f(x)=5\first{f(x)} = \first{5}Any nonzero constant function has derivative 00, giving a nonzero function a length of zero.
⟨5,5⟩\displaystyle \langle \first{5}, \first{5} \rangle =∫−11(5′)2 dx\displaystyle = \int_{-1}^1 (\first{5'})^2\,dx =∫−110 dx\displaystyle = \int_{-1}^1 0\,dx =0(Fails Positivity)\displaystyle = 0 \quad (\text{Fails Positivity})
Test Linearity for ∫−11f(x)2g(x)2 dx\int_{-1}^1 \first{f(x)}^2 \second{g(x)}^2\,dx by scaling f\first{f} by 22Squaring 2f(x)2\first{f(x)} inside the integral pulls out a factor of 44 instead of 22.
⟨2f,g⟩=∫−11(2f(x))2g(x)2 dx=4∫−11f(x)2g(x)2 dx=4⟨f,g⟩≠2⟨f,g⟩(Fails Linearity)\displaystyle \begin{aligned} \langle 2\first{f}, \second{g} \rangle &= \int_{-1}^1 (2\first{f(x)})^2 \second{g(x)}^2\,dx \\ &= 4\int_{-1}^1 \first{f(x)}^2 \second{g(x)}^2\,dx \\ &= 4\langle \first{f}, \second{g} \rangle \neq 2\langle \first{f}, \second{g} \rangle \\ &\quad (\text{Fails Linearity}) \end{aligned}
4

Gram-Schmidt on Polynomials

Back on [−1,1][-1, 1] with the standard inner product ∫−11f(x)g(x) dx\int_{-1}^1 \first{f(x)}\second{g(x)}\,dx, we know 1\first{1} and x\second{x} are orthogonal, and x\second{x} and x2\third{x^2} are orthogonal, but 1\first{1} and x2\third{x^2} have an overlap of 23\frac{2}{3}. We want to adjust x2\third{x^2} so that all three polynomials are mutually orthogonal on [−1,1][-1, 1].

Gram-Schmidt on Polynomials
1\htmlClass{lc-green}{1}
x2\htmlClass{lc-violet}{x^2}
x\htmlClass{lc-pink}{x}
−1-1
11
⟨1,x⟩=0⟨x,x2⟩=0⟨x2,1⟩=23⟨x2−? ,1⟩=0\begin{aligned}\langle \htmlClass{lc-green}{1}, \htmlClass{lc-pink}{x} \rangle &= 0 \\ \langle \htmlClass{lc-pink}{x}, \htmlClass{lc-violet}{x^2} \rangle &= 0 \\ \langle \htmlClass{lc-violet}{x^2}, \htmlClass{lc-green}{1} \rangle &= \tfrac{2}{3} \\[10pt] \langle \htmlClass{lc-violet}{x^2} - {?}\,, \htmlClass{lc-green}{1} \rangle &= 0 \end{aligned}

Just as with vectors, the projection of a function f\first{f} onto a basis function g\second{g} is Overlap divided by Size:

Projection of f onto g\displaystyle \text{Projection of } \first{f} \text{ onto } \second{g} =⟨f,g⟩⏞Overlap⟨g,g⟩⏟Size g\displaystyle = \frac{\overbrace{\langle \first{f}, \second{g} \rangle}^{\text{Overlap}}}{\underbrace{\langle \second{g}, \second{g} \rangle}_{\text{Size}}}\,\second{g}

Subtracting the projections of x2\third{x^2} onto 1\first{1} and x\second{x} removes the overlap so all three basis functions are orthogonal.

Text solution
Divide the overlap of x2\third{x^2} and 1\first{1} by the size of 1\first{1}Using ⟨x2,1⟩=23\langle \third{x^2}, \first{1} \rangle = \frac{2}{3} and ⟨1,1⟩=2\langle \first{1}, \first{1} \rangle = 2 gives the projection of x2\third{x^2} onto 1\first{1}.
⟨x2,1⟩⟨1,1⟩⋅1\displaystyle \frac{\langle \third{x^2}, \first{1} \rangle}{\langle \first{1}, \first{1} \rangle} \cdot \first{1} =232⋅1\displaystyle = \frac{\tfrac{2}{3}}{2} \cdot \first{1} =13\displaystyle = \frac{1}{3}
Project x2\third{x^2} onto x\second{x}Because x2\third{x^2} and x\second{x} are already orthogonal on [−1,1][-1, 1], the projection onto x\second{x} is zero.
⟨x2,x⟩⟨x,x⟩ x\displaystyle \frac{\langle \third{x^2}, \second{x} \rangle}{\langle \second{x}, \second{x} \rangle}\,\second{x} =023 x\displaystyle = \frac{0}{\tfrac{2}{3}}\,\second{x} =0\displaystyle = 0
Subtract the projections from x2\third{x^2}Subtracting 13\frac{1}{3} from x2\third{x^2} gives the third orthogonal polynomial in the Legendre basis 1,  x,  x2−13\first{1}, \; \second{x}, \; \third{x^2 - \frac{1}{3}}.
x2−⟨x2,1⟩⟨1,1⟩⋅1−⟨x2,x⟩⟨x,x⟩ x\displaystyle \third{x^2} - \frac{\langle \third{x^2}, \first{1} \rangle}{\langle \first{1}, \first{1} \rangle} \cdot \first{1} - \frac{\langle \third{x^2}, \second{x} \rangle}{\langle \second{x}, \second{x} \rangle}\,\second{x} =x2−13\displaystyle = \third{x^2 - \frac{1}{3}}

While the parabola sits entirely above the xx-axis, its area with 1\first{1} cannot cancel to zero; shifting it down to x2−13\third{x^2 - \frac{1}{3}} dips the middle below the axis just enough for the negative middle area to cancel the two positive outer areas.

Shifting the parabola down moves its middle below the x-axis until the negative area in the middle cancels the two positive areas on the ends.
5

Best Polynomial Approximations by Projection

Now we want to find the straight line a+bx\first{a} + \second{bx} that stays as close as possible to f(x)=x3\target{f(x)} = \target{x^3} across the entire interval [−1,1][-1, 1], minimizing the least-squares distance ∥x3−(a+bx)∥\lVert \gap{x^3 - (a + bx)} \rVert. Because 1\first{1} and x\second{x} are already orthogonal on [−1,1][-1, 1], we can find the closest line by projecting x3\target{x^3} onto 1\first{1} and x\second{x} separately.

Best Polynomial Approximations by Projection
1,  x,  x2−13\htmlClass{lc-green}{1},\,\, \htmlClass{lc-pink}{x},\,\, \htmlClass{lc-violet}{x^2 - \tfrac{1}{3}}
f(x)=x3\htmlClass{lc-blue}{f(x) = x^3}
a+bx\htmlClass{lc-green}{a} + \htmlClass{lc-pink}{bx}
−1-1
00
11
best line=?\text{best line} = {?}
Text solution
Project x3\target{x^3} onto 1\first{1}Because x3\target{x^3} is negative on the left half of [−1,1][-1, 1] and an identical positive mirror image on the right, its overlap with 1\first{1} cancels to zero.
⟨x3,1⟩\displaystyle \langle \target{x^3}, \first{1} \rangle =∫−11x3⋅1 dx\displaystyle = \int_{-1}^1 \target{x^3} \cdot \first{1}\,dx =0  ⟹  ⟨x3,1⟩⟨1,1⟩⋅1\displaystyle = 0 \implies \frac{\langle \target{x^3}, \first{1} \rangle}{\langle \first{1}, \first{1} \rangle} \cdot \first{1} =0\displaystyle = 0
Compute the overlap of x3\target{x^3} with x\second{x}Multiplying x3\target{x^3} by x\second{x} gives x4\integrand{x^4}, which stays positive across [−1,1][-1, 1] and integrates to 25\frac{2}{5}.
⟨x3,x⟩=∫−11x3⋅x dx=∫−11x4 dx=[x55]−11=25\displaystyle \begin{aligned} \langle \target{x^3}, \second{x} \rangle &= \int_{-1}^1 \target{x^3} \cdot \second{x}\,dx = \int_{-1}^1 \integrand{x^4}\,dx \\ &= \left[\frac{x^5}{5}\right]_{-1}^1 = \frac{2}{5} \end{aligned}
Divide the overlap by the size of x\second{x}Dividing ⟨x3,x⟩=25\langle \target{x^3}, \second{x} \rangle = \frac{2}{5} by ⟨x,x⟩=23\langle \second{x}, \second{x} \rangle = \frac{2}{3} gives the best straight-line approximation on [−1,1][-1, 1].
y\displaystyle y =⟨x3,x⟩⟨x,x⟩ x\displaystyle = \frac{\langle \target{x^3}, \second{x} \rangle}{\langle \second{x}, \second{x} \rangle}\,\second{x} =2523 x\displaystyle = \frac{\tfrac{2}{5}}{\tfrac{2}{3}}\,\second{x} =35x\displaystyle = \frac{3}{5}\second{x}

Subtracting this projection from x3\target{x^3} leaves the error x3−35x\gap{x^3 - \frac{3}{5}x}, which is orthogonal to both 1\first{1} and x\second{x}; since it is odd, it is also orthogonal to x2−13\third{x^2 - \frac{1}{3}}, so it is the next orthogonal polynomial in our Gram-Schmidt list.

The flat tangent line y=0y = 0 matches x3\target{x^3} right at the origin but leaves large vertical gaps near −1-1 and 11, whereas the projection line y=35xy = \frac{3}{5}\second{x} cuts through x3\target{x^3} to minimize the total squared gap across the whole interval.

A line through the origin gets closest overall to the curve when it cuts right through it to trim the big gaps at the ends.
6

Orthogonal Sine and Cosine Waves

Polynomials work well on a short interval, but when we want to approximate a function that repeats itself like a sound wave or an electrical signal, we need basis functions that oscillate. On the full period [−π,π][-\pi, \pi] with the inner product ⟨f,g⟩\langle \first{f}, \second{g} \rangle =∫−ππf(x)g(x) dx{}= \int_{-\pi}^\pi \first{f(x)}\second{g(x)}\,dx, the family of waves 1,cos⁡(x),sin⁡(x),cos⁡(2x),sin⁡(2x),\first{1}, \third{\cos(x)}, \second{\sin(x)}, \third{\cos(2x)}, \second{\sin(2x)}, cos⁡(3x),sin⁡(3x),…\third{\cos(3x)}, \second{\sin(3x)}, \dots is already mutually orthogonal without running Gram-Schmidt.

Orthogonal Sine and Cosine Waves
⟨sin⁡(x),cos⁡(x)⟩=?\langle \htmlClass{lc-pink}{\sin(x)}, \htmlClass{lc-violet}{\cos(x)} \rangle = {?}
1\htmlClass{lc-green}{1}
sin⁡(x)\htmlClass{lc-pink}{\sin(x)}
cos⁡(x)\htmlClass{lc-violet}{\cos(x)}
−π-\pi
00
π\pi
sin⁡2(x)\htmlClass{lc-blue}{\sin^2(x)}
12\tfrac{1}{2}
−π-\pi
00
π\pi

Because sin⁡2(nx)\integrand{\sin^2(nx)} and cos⁡2(nx)\integrand{\cos^2(nx)} oscillate symmetrically between 00 and 11, their average height over [−π,π][-\pi, \pi] is 12\frac{1}{2}, which multiplies by the interval width 2π2\pi to give a squared size of π\pi for every wave:

⟨1,1⟩=∫−ππ12 dx=2π,⟨sin⁡(nx),sin⁡(nx)⟩=∫−ππsin⁡2(nx) dx=π,⟨cos⁡(nx),cos⁡(nx)⟩=∫−ππcos⁡2(nx) dx=π\displaystyle \begin{aligned} \langle \first{1}, \first{1} \rangle &= \int_{-\pi}^\pi \first{1}^2\,dx = 2\pi, \\[6pt] \langle \second{\sin(nx)}, \second{\sin(nx)} \rangle &= \int_{-\pi}^\pi \integrand{\sin^2(nx)}\,dx \\ &= \pi, \\[6pt] \langle \third{\cos(nx)}, \third{\cos(nx)} \rangle &= \int_{-\pi}^\pi \integrand{\cos^2(nx)}\,dx \\ &= \pi \end{aligned}
7

Approximating a Square Wave with a Fourier Series

Now we want to approximate the square wave f(x)\target{f(x)} on [−π,π][-\pi, \pi], which equals −1-1 on [−π,0)[-\pi, 0) and jumps to +1+1 on (0,π](0, \pi], using a sum of smooth sines and cosines called its Fourier series. Because our sine and cosine waves are already orthogonal on [−π,π][-\pi, \pi], we project f(x)\target{f(x)} onto each wave using Overlap over Size.

Approximating a Square Wave with a Fourier Series
f(x)≈? 1+? cos⁡(x)+? sin⁡(x)+⋯\htmlClass{lc-blue}{f(x)} \approx {?}\,\htmlClass{lc-green}{1} + {?}\,\htmlClass{lc-violet}{\cos(x)} + {?}\,\htmlClass{lc-pink}{\sin(x)} + \cdots
f(x)\htmlClass{lc-blue}{f(x)}
11
−1-1
−π-\pi
π\pi
Text solution
Eliminate 1\first{1} and cos⁡(nx)\third{\cos(nx)} by symmetryBecause f(x)\target{f(x)} flips sign from −1-1 on the left half to +1+1 on the right half while 1\first{1} and cos⁡(nx)\third{\cos(nx)} are symmetric, their products cancel across [−π,π][-\pi, \pi].
⟨f,1⟩\displaystyle \langle \target{f}, \first{1} \rangle =0,⟨f,cos⁡(nx)⟩\displaystyle = 0, \qquad \langle \target{f}, \third{\cos(nx)} \rangle =0\displaystyle = 0
Project f(x)\target{f(x)} onto sin⁡(x)\second{\sin(x)}Multiplying by f(x)=−1\target{f(x)} = -1 on [−π,0)[-\pi, 0) flips the negative arch of sin⁡(x)\second{\sin(x)} up into a second copy of the positive arch on (0,π](0, \pi], so we double the integral on [0,π][0, \pi] and divide by the squared size π\pi.
⟨f,sin⁡(x)⟩=2∫0πsin⁡(x) dx=2[−cos⁡(x)]0π=4  ⟹  ⟨f,sin⁡(x)⟩⟨sin⁡(x),sin⁡(x)⟩sin⁡(x)=4πsin⁡(x)\displaystyle \begin{aligned} &\langle \target{f}, \second{\sin(x)} \rangle \\ &\quad = 2\int_0^\pi \second{\sin(x)}\,dx \\ &\quad = 2\big[-\cos(x)\big]_0^\pi = 4 \\[6pt] &\implies \frac{\langle \target{f}, \second{\sin(x)} \rangle}{\langle \second{\sin(x)}, \second{\sin(x)} \rangle}\second{\sin(x)} \\ &\quad = \frac{4}{\pi}\second{\sin(x)} \end{aligned}
Project f(x)\target{f(x)} onto sin⁡(2x)\second{\sin(2x)} and sin⁡(3x)\second{\sin(3x)}On [0,π][0, \pi], the positive and negative arches of sin⁡(2x)\second{\sin(2x)} cancel to zero, while two of the three arches of sin⁡(3x)\second{\sin(3x)} cancel and leave one positive arch of area 23\frac{2}{3} on each side.
⟨f,sin⁡(2x)⟩=0,⟨f,sin⁡(3x)⟩=2∫0πsin⁡(3x) dx=2[−cos⁡(3x)3]0π=43  ⟹  43πsin⁡(3x)\displaystyle \begin{aligned} \langle \target{f}, \second{\sin(2x)} \rangle &= 0, \\ \langle \target{f}, \second{\sin(3x)} \rangle &= 2\int_0^\pi \second{\sin(3x)}\,dx \\ &= 2\left[-\frac{\cos(3x)}{3}\right]_0^\pi = \frac{4}{3} \\[6pt] &\implies \frac{4}{3\pi}\second{\sin(3x)} \end{aligned}
Add the projections to form the Fourier seriesSumming the nonzero wave projections builds the square wave out of odd-frequency sine waves.
f(x)≈4πsin⁡(x)+43πsin⁡(3x)+45πsin⁡(5x)+…\displaystyle \target{f(x)} \approx \frac{4}{\pi}\second{\sin(x)} + \frac{4}{3\pi}\second{\sin(3x)} + \frac{4}{5\pi}\second{\sin(5x)} + \dots
4πsin⁡(x)\tfrac{4}{\pi}\htmlClass{lc-pink}{\sin(x)}
f(x)\htmlClass{lc-blue}{f(x)}
11
−1-1
−π-\pi
π\pi
4πsin⁡(x)+ 43πsin⁡(3x)\begin{aligned}&\tfrac{4}{\pi}\htmlClass{lc-pink}{\sin(x)}\\[-2pt] +\,&\tfrac{4}{3\pi}\htmlClass{lc-pink}{\sin(3x)}\end{aligned}
f(x)\htmlClass{lc-blue}{f(x)}
11
−1-1
−π-\pi
π\pi

Key concepts

Everything in inner product spaces for functions comes from replacing the vector dot product with an integral over a chosen interval [a,b][a, b]. Under the standard inner product ⟨f,g⟩=∫abf(x)g(x) dx\langle \first{f}, \second{g} \rangle = \int_a^b \first{f(x)}\second{g(x)}\,dx, two functions are orthogonal when the positive and negative areas of their product cancel to ⟨f,g⟩=0\langle \first{f}, \second{g} \rangle = 0, which always depends on the interval: ⟨1,x⟩=∫−11x dx=0\langle \first{1}, \second{x} \rangle = \int_{-1}^1 \second{x}\,dx = 0 on [−1,1][-1, 1], but ⟨1,x⟩=∫01x dx=12\langle \first{1}, \second{x} \rangle = \int_0^1 \second{x}\,dx = \frac{1}{2} on [0,1][0, 1]. Squaring keeps positive and negative parts from canceling, giving the norm ∥f∥=⟨f,f⟩\lVert \first{f} \rVert = \sqrt{\langle \first{f}, \first{f} \rangle} and the least-squares distance ∥f−g∥=∫ab(f(x)−g(x))2 dx\lVert \gap{f - g} \rVert = \sqrt{\int_a^b (\gap{f(x) - g(x)})^2\,dx}. If you are given a new candidate rule for ⟨f,g⟩\langle \first{f}, \second{g} \rangle, test Symmetry, Linearity, and Positivity (⟨f,f⟩>0\langle \first{f}, \first{f} \rangle > 0 for every nonzero f\first{f}): a weighted integral ∫abw(x)f(x)g(x) dx\int_a^b \weight{w(x)}\first{f(x)}\second{g(x)}\,dx is valid whenever w(x)>0\weight{w(x)} > 0 across the interval (like x2+1\weight{x^2 + 1}), and fails Positivity if w(x)\weight{w(x)} is negative anywhere on the interval (like w(x)=x\weight{w(x)} = \weight{x} on [−1,1][-1, 1]) or if derivatives turn a nonzero constant like f(x)=5\first{f(x)} = \first{5} into zero.

Every projection uses Overlap over Size:

Projection of f onto g\displaystyle \text{Projection of } \first{f} \text{ onto } \second{g} =⟨f,g⟩⟨g,g⟩ g\displaystyle = \frac{\langle \first{f}, \second{g} \rangle}{\langle \second{g}, \second{g} \rangle}\,\second{g}

When your basis is not yet orthogonal (such as 1,x,x2\first{1}, \second{x}, \third{x^2} on [−1,1][-1, 1] or [0,1][0, 1]), run Gram-Schmidt by subtracting the projections of each new function onto the earlier orthogonal ones (giving x2−⟨x2,1⟩⟨1,1⟩⋅1=x2−13\third{x^2} - \frac{\langle \third{x^2}, \first{1} \rangle}{\langle \first{1}, \first{1} \rangle}\cdot \first{1} = \third{x^2 - \frac{1}{3}} on [−1,1][-1, 1]). When your basis is already mutually orthogonal (such as the Legendre polynomials 1,x,x2−13\first{1}, \second{x}, \third{x^2 - \frac{1}{3}} on [−1,1][-1, 1] or the waves 1,cos⁡(nx),sin⁡(nx)\first{1}, \third{\cos(nx)}, \second{\sin(nx)} on [−π,π][-\pi, \pi], where ⟨1,1⟩=2π\langle \first{1}, \first{1} \rangle = 2\pi and ⟨sin⁡(nx),sin⁡(nx)⟩\langle \second{\sin(nx)}, \second{\sin(nx)} \rangle =⟨cos⁡(nx),cos⁡(nx)⟩=π{}= \langle \third{\cos(nx)}, \third{\cos(nx)} \rangle = \pi), do not run Gram-Schmidt again: project f\target{f} onto each basis function separately and add the projections to get the best polynomial or Fourier approximation. On any symmetric interval like [−1,1][-1, 1] or [−π,π][-\pi, \pi], check even and odd symmetry before integrating so you can spot zero overlaps like ⟨x3,1⟩=0\langle \target{x^3}, \first{1} \rangle = 0 or ⟨f,cos⁡(nx)⟩=0\langle \target{f}, \third{\cos(nx)} \rangle = 0 immediately.

8

Practice

1

Using the standard inner product ⟨f,g⟩=∫abf(x)g(x) dx\langle \first{f}, \second{g} \rangle = \int_a^b \first{f(x)}\second{g(x)}\,dx, compute ⟨x,3x−2⟩\langle \first{x}, \second{3x - 2} \rangle on the interval [−1,1][-1, 1] and on the interval [0,1][0, 1]. On which interval are f(x)=x\first{f(x)} = \first{x} and g(x)=3x−2\second{g(x)} = \second{3x - 2} orthogonal?

Multiply x(3x−2)\first{x}(\second{3x - 2}) first, then integrate over each interval to see where the positive and negative areas cancel to 00.

Multiply the two functions to get f(x)g(x)=3x2−2x\first{f(x)}\second{g(x)} = \integrand{3x^2 - 2x}. On [−1,1][-1, 1]:
⟨x,3x−2⟩=∫−11(3x2−2x) dx=[x3−x2]−11=(1−1)−(−1−1)=2\displaystyle \begin{aligned} & \langle \first{x}, \second{3x - 2} \rangle = \int_{-1}^1 \integrand{(3x^2 - 2x)}\,dx \\ &\quad = \big[x^3 - x^2\big]_{-1}^1 \\ &\quad = (1 - 1) - (-1 - 1) = 2 \end{aligned}
(Watch the signs at the lower bound x=−1x = -1: (−1)3=−1(-1)^3 = -1 while (−1)2=+1(-1)^2 = +1, so the −2x-2x term cancels on [−1,1][-1, 1] and leaves 22, not 00.) Since 2≠02 \neq 0, they are not orthogonal on [−1,1][-1, 1].
On [0,1][0, 1]:
⟨x,3x−2⟩\displaystyle \langle \first{x}, \second{3x - 2} \rangle =∫01(3x2−2x) dx\displaystyle = \int_0^1 \integrand{(3x^2 - 2x)}\,dx =[x3−x2]01\displaystyle = \big[x^3 - x^2\big]_0^1 =(1−1)−0\displaystyle = (1 - 1) - 0 =0\displaystyle = 0
They are orthogonal on [0,1][0, 1].
2

On the interval [−1,1][-1, 1] with the standard inner product ⟨f,g⟩=∫−11f(x)g(x) dx\langle \first{f}, \second{g} \rangle = \int_{-1}^1 \first{f(x)}\second{g(x)}\,dx, find the norm ∥x2∥\lVert \third{x^2} \rVert and the distance ∥x2−1∥\lVert \gap{x^2 - 1} \rVert between f(x)=x2\third{f(x)} = \third{x^2} and g(x)=1\first{g(x)} = \first{1}.

For the distance, expand the squared vertical gap (x2−1)2(\gap{x^2 - 1})^2 before integrating, and remember to take the square root at the end of both calculations.

For the norm ∥x2∥\lVert \third{x^2} \rVert:
∥x2∥=∫−11(x2)2 dx=∫−11x4 dx=[x55]−11=25\displaystyle \begin{aligned} \lVert \third{x^2} \rVert &= \sqrt{\int_{-1}^1 (\third{x^2})^2\,dx} = \sqrt{\int_{-1}^1 \integrand{x^4}\,dx} \\ &= \sqrt{\left[\frac{x^5}{5}\right]_{-1}^1} = \sqrt{\frac{2}{5}} \end{aligned}
For the distance ∥x2−1∥\lVert \gap{x^2 - 1} \rVert, expand (x2−1)2=x4−2x2+1(\gap{x^2 - 1})^2 = x^4 - 2x^2 + 1 (do not square the terms separately as x4+1x^4 + 1, which drops the middle term −2x2-2x^2 and gives 125\sqrt{\frac{12}{5}}):
∥x2−1∥=∫−11(x2−1)2 dx=∫−11(x4−2x2+1) dx=25−43+2=1615=415\displaystyle \begin{aligned} & \lVert \gap{x^2 - 1} \rVert = \sqrt{\int_{-1}^1 (\gap{x^2 - 1})^2\,dx} \\ &\quad = \sqrt{\int_{-1}^1 (x^4 - 2x^2 + 1)\,dx} \\ &\quad = \sqrt{\frac{2}{5} - \frac{4}{3} + 2} = \sqrt{\frac{16}{15}} \\ &\quad = \frac{4}{\sqrt{15}} \end{aligned}
3

Test whether each rule on the interval [−1,1][-1, 1] is a valid inner product. If a rule fails, name which check (Symmetry, Linearity, or Positivity) it breaks and give a counterexample: (a) ⟨f,g⟩=∫−11(x+2)f(x)g(x) dx\langle \first{f}, \second{g} \rangle = \int_{-1}^1 \weight{(x + 2)}\first{f(x)}\second{g(x)}\,dx (b) ⟨f,g⟩=∫−11(x2−1)f(x)g(x) dx\langle \first{f}, \second{g} \rangle = \int_{-1}^1 \weight{(x^2 - 1)}\first{f(x)}\second{g(x)}\,dx (c) ⟨f,g⟩=∫−11f(x)g′(x) dx\langle \first{f}, \second{g} \rangle = \int_{-1}^1 \first{f(x)}\second{g'(x)}\,dx (d) ⟨f,g⟩=∫−11∣f(x)g(x)∣ dx\langle \first{f}, \second{g} \rangle = \int_{-1}^1 |\first{f(x)}\second{g(x)}|\,dx

For weighted integrals (a) and (b), check whether the weight stays strictly positive across [−1,1][-1, 1]. For (c), swap f\first{f} and g\second{g} or test a nonzero constant function. For (d), test scaling f\first{f} by a negative number.

(a)Valid inner product: it is symmetric and linear, and because the weight x+2≥1>0\weight{x + 2} \ge 1 > 0 across the entire interval [−1,1][-1, 1] (even though xx can be negative on [−1,1][-1, 1], adding 22 keeps x+2\weight{x + 2} strictly positive), ⟨f,f⟩=∫−11(x+2)f(x)2 dx>0\langle \first{f}, \first{f} \rangle = \int_{-1}^1 \weight{(x + 2)}\first{f(x)}^2\,dx > 0 for every nonzero f\first{f}.
(b)Fails Positivity: on [−1,1][-1, 1], the weight x2−1≤0\weight{x^2 - 1} \le 0, so plugging in f(x)=1\first{f(x)} = \first{1} gives ⟨1,1⟩=∫−11(x2−1) dx=−43<0\langle \first{1}, \first{1} \rangle = \int_{-1}^1 \weight{(x^2 - 1)}\,dx = -\frac{4}{3} < 0.
(c)Fails Symmetry and Positivity: for Symmetry, ⟨1,x⟩=∫−111⋅1 dx=2\langle \first{1}, \second{x} \rangle = \int_{-1}^1 \first{1} \cdot \second{1}\,dx = 2 while ⟨x,1⟩=∫−11x⋅0 dx=0\langle \second{x}, \first{1} \rangle = \int_{-1}^1 \second{x} \cdot \first{0}\,dx = 0; for Positivity, the nonzero function f(x)=1\first{f(x)} = \first{1} gives ⟨1,1⟩=∫−111⋅0 dx=0\langle \first{1}, \first{1} \rangle = \int_{-1}^1 \first{1} \cdot \first{0}\,dx = 0.
(d)Fails Linearity: scaling f\first{f} by c=−1c = -1 gives ⟨−f,g⟩=∫−11∣−f(x)g(x)∣ dx=⟨f,g⟩\langle -\first{f}, \second{g} \rangle = \int_{-1}^1 |-\first{f(x)}\second{g(x)}|\,dx = \langle \first{f}, \second{g} \rangle instead of −⟨f,g⟩-\langle \first{f}, \second{g} \rangle (testing only a positive number like c=2c = 2 misses this because ∣2f(x)∣=2∣f(x)∣|2\first{f(x)}| = 2|\first{f(x)}|).
4

On the interval [0,1][0, 1] with the standard inner product ⟨f,g⟩=∫01f(x)g(x) dx\langle \first{f}, \second{g} \rangle = \int_0^1 \first{f(x)}\second{g(x)}\,dx, run the Gram-Schmidt process on the basis 1,x,x2\first{1}, \second{x}, \third{x^2} to find three mutually orthogonal polynomials.

Start with p1(x)=1\first{p_1(x)} = \first{1}. On [0,1][0, 1], odd powers no longer cancel to zero, so for the third polynomial you must subtract the projections of x2\third{x^2} onto both p1(x)\first{p_1(x)} and p2(x)\second{p_2(x)}.

Keep the first basis function p1(x)=1\first{p_1(x)} = \first{1}, which has squared size ⟨1,1⟩=∫011 dx=1\langle \first{1}, \first{1} \rangle = \int_0^1 \first{1}\,dx = 1.
Subtract the projection of x\second{x} onto 1\first{1}: since ⟨x,1⟩=∫01x dx=12\langle \second{x}, \first{1} \rangle = \int_0^1 \second{x}\,dx = \frac{1}{2},
p2(x)\displaystyle \second{p_2(x)} =x−⟨x,1⟩⟨1,1⟩⋅1\displaystyle = \second{x} - \frac{\langle \second{x}, \first{1} \rangle}{\langle \first{1}, \first{1} \rangle} \cdot \first{1} =x−12\displaystyle = \second{x - \frac{1}{2}}
Find the squared size of p2(x)\second{p_2(x)}:
⟨x−12,x−12⟩\displaystyle \left\langle \second{x - \frac{1}{2}}, \second{x - \frac{1}{2}} \right\rangle =∫01(x2−x+14)dx\displaystyle = \int_0^1 \left(x^2 - x + \frac{1}{4}\right)dx =13−12+14\displaystyle = \frac{1}{3} - \frac{1}{2} + \frac{1}{4} =112\displaystyle = \frac{1}{12}
For p3(x)\third{p_3(x)}, project x2\third{x^2} onto both 1\first{1} and p2(x)=x−12\second{p_2(x)} = \second{x - \frac{1}{2}} (do not assume ⟨x2,p2⟩=0\langle \third{x^2}, \second{p_2} \rangle = 0 as on [−1,1][-1, 1], and be sure to project onto the orthogonalized x−12\second{x - \frac{1}{2}} rather than x\second{x}, which would give x2−34x−13x^2 - \frac{3}{4}x - \frac{1}{3}):
⟨x2,1⟩=∫01x2 dx=13,⟨x2,x−12⟩=∫01(x3−x22)dx=14−16=112\displaystyle \begin{aligned} & \langle \third{x^2}, \first{1} \rangle = \int_0^1 \third{x^2}\,dx = \frac{1}{3}, \\ & \left\langle \third{x^2}, \second{x - \frac{1}{2}} \right\rangle \\ &\quad = \int_0^1 \left(x^3 - \frac{x^2}{2}\right)dx \\ &\quad = \frac{1}{4} - \frac{1}{6}= \frac{1}{12} \end{aligned}
Subtract both projections from x2\third{x^2}:
p3(x)=x2−131⋅1−112112(x−12)=x2−13−x+12=x2−x+16\displaystyle \begin{aligned} \third{p_3(x)} &= \third{x^2} - \frac{\tfrac{1}{3}}{1} \cdot \first{1} - \frac{\tfrac{1}{12}}{\tfrac{1}{12}}\left(\second{x - \frac{1}{2}}\right) \\ &= x^2 - \frac{1}{3} - x + \frac{1}{2} = \third{x^2 - x + \frac{1}{6}} \end{aligned}
The orthogonal basis is 1,  x−12,  x2−x+16\first{1}, \; \second{x - \frac{1}{2}}, \; \third{x^2 - x + \frac{1}{6}}.
5

Using the orthogonal Legendre basis 1,x,x2−13\first{1}, \second{x}, \third{x^2 - \frac{1}{3}} on [−1,1][-1, 1] with ⟨f,g⟩=∫−11f(x)g(x) dx\langle \first{f}, \second{g} \rangle = \int_{-1}^1 \first{f(x)}\second{g(x)}\,dx, find the best least-squares quadratic approximation p(x)=a+bx+cx2p(x) = \first{a} + \second{bx} + \third{cx^2} to f(x)=x4\target{f(x)} = \target{x^4}.

Project x4\target{x^4} onto each of the three orthogonal basis functions 1\first{1}, x\second{x}, and x2−13\third{x^2 - \frac{1}{3}} using Overlap over Size, then add the three projections together.

Project x4\target{x^4} onto 1\first{1}:
⟨x4,1⟩=∫−11x4 dx=25,⟨1,1⟩=2  ⟹  252⋅1=15\displaystyle \begin{aligned} \langle \target{x^4}, \first{1} \rangle &= \int_{-1}^1 \target{x^4}\,dx = \frac{2}{5}, \\ \langle \first{1}, \first{1} \rangle &= 2 \implies \frac{\tfrac{2}{5}}{2} \cdot \first{1} = \frac{1}{5} \end{aligned}
Project x4\target{x^4} onto x\second{x}: by symmetry on [−1,1][-1, 1], ⟨x4,x⟩=∫−11x5 dx=0\langle \target{x^4}, \second{x} \rangle = \int_{-1}^1 \integrand{x^5}\,dx = 0.
Project x4\target{x^4} onto x2−13\third{x^2 - \frac{1}{3}}:
Overlap:⟨x4,x2−13⟩\displaystyle \text{Overlap:} \quad \left\langle \target{x^4}, \third{x^2 - \frac{1}{3}} \right\rangle =∫−11(x6−x43)dx\displaystyle = \int_{-1}^1 \left(x^6 - \frac{x^4}{3}\right)dx =27−215\displaystyle = \frac{2}{7} - \frac{2}{15} =16105\displaystyle = \frac{16}{105}
Size:⟨x2−13,x2−13⟩\displaystyle \text{Size:} \quad \left\langle \third{x^2 - \frac{1}{3}}, \third{x^2 - \frac{1}{3}} \right\rangle =∫−11(x4−23x2+19)dx\displaystyle = \int_{-1}^1 \left(x^4 - \frac{2}{3}x^2 + \frac{1}{9}\right)dx =25−49+29\displaystyle = \frac{2}{5} - \frac{4}{9} + \frac{2}{9} =845\displaystyle = \frac{8}{45}
Projection onto x2−13:16105845(x2−13)\displaystyle \text{Projection onto } \third{x^2 - \frac{1}{3}}: \quad \frac{\tfrac{16}{105}}{\tfrac{8}{45}}\left(\third{x^2 - \frac{1}{3}}\right) =67(x2−13)\displaystyle = \frac{6}{7}\left(\third{x^2 - \frac{1}{3}}\right)
Add the projections together (remember to multiply 67\frac{6}{7} by the entire basis polynomial x2−13\third{x^2 - \frac{1}{3}}, not just x2\third{x^2}, which would miss −27-\frac{2}{7} and give 67x2+15\frac{6}{7}x^2 + \frac{1}{5}):
p(x)\displaystyle p(x) =15+67(x2−13)\displaystyle = \frac{1}{5} + \frac{6}{7}\left(\third{x^2 - \frac{1}{3}}\right) =67x2−335\displaystyle = \frac{6}{7}x^2 - \frac{3}{35}
6

On [−π,π][-\pi, \pi] with ⟨f,g⟩\langle \first{f}, \second{g} \rangle =∫−ππf(x)g(x) dx{}= \int_{-\pi}^\pi \first{f(x)}\second{g(x)}\,dx, let f(x)=1\target{f(x)} = 1 for −π2≤x≤π2-\frac{\pi}{2} \le x \le \frac{\pi}{2} and f(x)=0\target{f(x)} = 0 elsewhere on [−π,π][-\pi, \pi]. Find the Fourier series approximation of f(x)\target{f(x)} through frequency 3x3x by projecting onto 1,cos⁡(x),sin⁡(x),\first{1}, \third{\cos(x)}, \second{\sin(x)}, cos⁡(2x),sin⁡(2x),\third{\cos(2x)}, \second{\sin(2x)}, cos⁡(3x),sin⁡(3x)\third{\cos(3x)}, \second{\sin(3x)}.

Use symmetry first to eliminate the sine waves. When computing the overlaps, integrate from −π2-\frac{\pi}{2} to π2\frac{\pi}{2} where f(x)=1\target{f(x)} = 1, but keep the basis wave sizes over the full interval [−π,π][-\pi, \pi].

Because f(x)\target{f(x)} is symmetric (even) around x=0x = 0 and sin⁡(nx)\second{\sin(nx)} is odd, ⟨f,sin⁡(nx)⟩=0\langle \target{f}, \second{\sin(nx)} \rangle = 0 for all nn.
Project f(x)\target{f(x)} onto 1\first{1} (remember that on [−π,π][-\pi, \pi], the squared size of 1\first{1} is ⟨1,1⟩=2π\langle \first{1}, \first{1} \rangle = 2\pi, not π\pi, and do not shrink the basis sizes to [−π2,π2][-\frac{\pi}{2}, \frac{\pi}{2}], which would double the coefficients):
⟨f,1⟩=∫−π/2π/21 dx=π,⟨1,1⟩=2π  ⟹  π2π⋅1=12\displaystyle \begin{aligned} \langle \target{f}, \first{1} \rangle &= \int_{-\pi/2}^{\pi/2} \first{1}\,dx = \pi, \\ \langle \first{1}, \first{1} \rangle &= 2\pi \implies \frac{\pi}{2\pi} \cdot \first{1} = \frac{1}{2} \end{aligned}
Project f(x)\target{f(x)} onto cos⁡(nx)\third{\cos(nx)}, which has squared size ⟨cos⁡(nx),cos⁡(nx)⟩=π\langle \third{\cos(nx)}, \third{\cos(nx)} \rangle = \pi:
⟨f,cos⁡(nx)⟩=∫−π/2π/2cos⁡(nx) dx=[sin⁡(nx)n]−π/2π/2=2nsin⁡(nπ2)\displaystyle \begin{aligned} \langle \target{f}, \third{\cos(nx)} \rangle &= \int_{-\pi/2}^{\pi/2} \third{\cos(nx)}\,dx \\ &= \left[\frac{\sin(nx)}{n}\right]_{-\pi/2}^{\pi/2} \\ &= \frac{2}{n}\sin\left(\frac{n\pi}{2}\right) \end{aligned}
Dividing by the squared size π\pi for n=1,2,3n = 1, 2, 3 gives 2π\frac{2}{\pi} for n=1n = 1, 00 for n=2n = 2, and −23π-\frac{2}{3\pi} for n=3n = 3:
f(x)≈12+2πcos⁡(x)−23πcos⁡(3x)\displaystyle \target{f(x)} \approx \frac{1}{2} + \frac{2}{\pi}\third{\cos(x)} - \frac{2}{3\pi}\third{\cos(3x)}