Linear AlgebraOrthogonality and Projections2 of 9
Orthogonality and Projections · 2

Projection onto a Line

When a vector b\given{b} does not lie on the line through a\lineVec{a}, its projection p\proj{p} is the point on the line that comes as close to the tip of b\given{b} as possible.

1

Finding the Projection onto a Line

To find the closest point p\proj{p} on the line through a\lineVec{a}, we look for the multiple of a\lineVec{a} where the line segment to the tip of b\given{b} meets the line at a 90∘90^\circ angle.

Finding the Projection onto a Line
a\lineVec{a}
b\given{b}
p=  ?\proj{p} = \;?
a\lineVec{a}
b\given{b}
p=ca\proj{p} = \lineVec{ca}
e=b−ca\err{e} = \given{b} - \lineVec{ca}

Every point on the line is a scaled copy of a\lineVec{a}, so p=ca\proj{p} = \lineVec{ca} for some number c\lineVec{c}. At the closest point, the leftover error vector e=b−ca\err{e} = \given{b} - \lineVec{ca} is perpendicular to a\lineVec{a}, meaning their dot product is zero:

a⋅(b−ca⏟error e)=0Error vector is perpendicularto the linea⋅b−c(a⋅a)=0Distribute the dot productc=a⋅ba⋅aSolve for the multiplier c\displaystyle \begin{aligned} & \lineVec{a} \cdot (\underbrace{\given{b} - \lineVec{ca}}_{\text{error } \err{e}}) = 0 \\ &\quad \text{Error vector is perpendicular} \\ &\quad \text{to the line} \\[6pt] & \lineVec{a} \cdot \given{b} - \lineVec{c}(\lineVec{a} \cdot \lineVec{a}) = 0 \\ &\quad \text{Distribute the dot product} \\[6pt] & \lineVec{c} = \frac{\lineVec{a} \cdot \given{b}}{\lineVec{a} \cdot \lineVec{a}} \text{Solve for the multiplier } \lineVec{c} \end{aligned}

Multiplying the direction vector a\lineVec{a} by that number c\lineVec{c} gives the projection vector p\proj{p}:

p\displaystyle \proj{p} =a⋅ba⋅a⏟multiplier ca\displaystyle = \underbrace{\frac{\lineVec{a} \cdot \given{b}}{\lineVec{a} \cdot \lineVec{a}}}_{\text{multiplier } \lineVec{c}} \lineVec{a}

Both a⋅b\lineVec{a} \cdot \given{b} and a⋅a\lineVec{a} \cdot \lineVec{a} are single numbers, so c=a⋅ba⋅a\lineVec{c} = \frac{\lineVec{a} \cdot \given{b}}{\lineVec{a} \cdot \lineVec{a}} is just a number that stretches or shrinks the vector a\lineVec{a}. You cannot cancel a\lineVec{a} across the dot products.

As the point ca\lineVec{ca} moves along the line, the vector b−ca\given{b} - \lineVec{ca} to the tip of b\given{b} is shortest at the exact spot where it meets the line at 90∘90^\circ and a⋅(b−ca)=0\lineVec{a} \cdot (\given{b} - \lineVec{ca}) = 0.

The arrow from a point on the line to the tip of the vector off the line is shortest right where it meets the line at a right angle.
2

Why We Divide by Length Squared

In the multiplier c=a⋅ba⋅a\lineVec{c} = \frac{\lineVec{a} \cdot \given{b}}{\lineVec{a} \cdot \lineVec{a}}, the bottom is a⋅a=∥a∥2\lineVec{a} \cdot \lineVec{a} = \|\lineVec{a}\|^2. We divide by the length of a\lineVec{a}, squared, so that the projection point on the line stays in the same place no matter how long a\lineVec{a} is.

Why We Divide by Length Squared
c=a⋅ba⋅aa⋅a=∥a∥2\lineVec{c} = \dfrac{\lineVec{a} \cdot \given{b}}{\lineVec{a} \cdot \lineVec{a}} \qquad \lineVec{a} \cdot \lineVec{a} = \|\lineVec{a}\|^2
a\lineVec{a}
b\given{b}
p\proj{p}
e\err{e}

Vector a\lineVec{a} only sets the direction of the line. If we replace a\lineVec{a} with 2a2\lineVec{a}, the line and the closest point p\proj{p} do not change, so we need half as many copies of 2a2\lineVec{a} to reach the same spot:

cnew\displaystyle \lineVec{c_{\text{new}}} =(2a)⋅b(2a)⋅(2a)\displaystyle = \frac{(2\lineVec{a}) \cdot \given{b}}{(2\lineVec{a}) \cdot (2\lineVec{a})} =2(a⋅b)4(a⋅a)\displaystyle = \frac{2(\lineVec{a} \cdot \given{b})}{4(\lineVec{a} \cdot \lineVec{a})} =12c\displaystyle = \frac{1}{2}\lineVec{c}

When we multiply 12c\frac{1}{2}\lineVec{c} by 2a2\lineVec{a}, the factors of 22 cancel and give the exact same projection vector p=ca\proj{p} = \lineVec{ca}. And if a\lineVec{a} is already a unit vector (∥a∥=1\|\lineVec{a}\| = 1), then a⋅a=1\lineVec{a} \cdot \lineVec{a} = 1 and the denominator is 11:

p\displaystyle \proj{p} =(a⋅b)awhen ∥a∥\displaystyle = (\lineVec{a} \cdot \given{b})\lineVec{a} \quad \text{when } \|\lineVec{a}\| =1\displaystyle = 1

Stretching, shrinking, or flipping a\lineVec{a} along the same line rescales the multiplier c\lineVec{c} (and makes the denominator 11 when ∥a∥=1\|\lineVec{a}\| = 1), while the projection point p=ca\proj{p} = \lineVec{ca} stays in the exact same spot.

Stretching, shrinking, or flipping a\lineVec{a} along the line changes the multiplier c\lineVec{c}, while the projection p\proj{p} never moves.
3

Calculating a Projection Vector

Now let's find the projection p\proj{p} of b=[34]\given{b} = \given{\begin{bmatrix} 3 \\ 4 \end{bmatrix}} onto the line through a=[21]\lineVec{a} = \lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}}.

Calculating a Projection Vector
1\htmlClass{lc-mute}{1}
2\htmlClass{lc-mute}{2}
3\htmlClass{lc-mute}{3}
4\htmlClass{lc-mute}{4}
5\htmlClass{lc-mute}{5}
1\htmlClass{lc-mute}{1}
2\htmlClass{lc-mute}{2}
3\htmlClass{lc-mute}{3}
4\htmlClass{lc-mute}{4}
x\htmlClass{lc-mute}{x}
y\htmlClass{lc-mute}{y}
a=[21]\lineVec{a} = \lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}}
b=[34]\given{b} = \given{\begin{bmatrix} 3 \\ 4 \end{bmatrix}}
p=  ?\proj{p} = \;?
Text solution
Find the multiplier c\lineVec{c} from the dot productsCompute a⋅b\lineVec{a} \cdot \given{b} on top and a⋅a\lineVec{a} \cdot \lineVec{a} on the bottom to find how many copies of a\lineVec{a} reach the closest point.
a⋅b=(2)(3)+(1)(4)=6+4=10a⋅a=(2)(2)+(1)(1)=4+1=5c=a⋅ba⋅a=105=2\displaystyle \begin{aligned} \lineVec{a} \cdot \given{b} &= (\lineVec{2})(\given{3}) + (\lineVec{1})(\given{4}) \\ &= 6 + 4 = 10 \\[4pt] \lineVec{a} \cdot \lineVec{a} &= (\lineVec{2})(\lineVec{2}) + (\lineVec{1})(\lineVec{1}) \\ &= 4 + 1 = 5 \\[4pt] \lineVec{c} &= \frac{\lineVec{a} \cdot \given{b}}{\lineVec{a} \cdot \lineVec{a}} = \frac{10}{5} = 2 \end{aligned}
Scale the line vector a\lineVec{a} by c\lineVec{c}Multiply a\lineVec{a} by c=2\lineVec{c} = 2 to get the projection vector on the line.
p=ca=2[21]=[42]\displaystyle \proj{p} = \lineVec{ca} = 2\lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}} = \proj{\begin{bmatrix} 4 \\ 2 \end{bmatrix}}

So the point on the line through [21]\lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}} that is closest to [34]\given{\begin{bmatrix} 3 \\ 4 \end{bmatrix}} is p=[42]\proj{p} = \proj{\begin{bmatrix} 4 \\ 2 \end{bmatrix}}.

4

Error Vector and Distance to a Line

Once we have the projection p\proj{p}, the leftover vector from p\proj{p} to b\given{b} is the error vector e=b−p\err{e} = \given{b} - \proj{p}, which splits b\given{b} into perpendicular pieces and gives the shortest distance from b\given{b} to the line.

Error Vector and Distance to a Line
e=  ?distance=  ?\err{e} = \;? \qquad \text{distance} = \;?
1\htmlClass{lc-mute}{1}
2\htmlClass{lc-mute}{2}
3\htmlClass{lc-mute}{3}
4\htmlClass{lc-mute}{4}
5\htmlClass{lc-mute}{5}
1\htmlClass{lc-mute}{1}
2\htmlClass{lc-mute}{2}
3\htmlClass{lc-mute}{3}
4\htmlClass{lc-mute}{4}
x\htmlClass{lc-mute}{x}
y\htmlClass{lc-mute}{y}
a=[21]\lineVec{a} = \lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}}
b=[34]\given{b} = \given{\begin{bmatrix} 3 \\ 4 \end{bmatrix}}
p=[42]\proj{p} = \proj{\begin{bmatrix} 4 \\ 2 \end{bmatrix}}
1\htmlClass{lc-mute}{1}
2\htmlClass{lc-mute}{2}
3\htmlClass{lc-mute}{3}
4\htmlClass{lc-mute}{4}
5\htmlClass{lc-mute}{5}
1\htmlClass{lc-mute}{1}
2\htmlClass{lc-mute}{2}
3\htmlClass{lc-mute}{3}
4\htmlClass{lc-mute}{4}
x\htmlClass{lc-mute}{x}
y\htmlClass{lc-mute}{y}
a=[21]\lineVec{a} = \lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}}
b=[34]\given{b} = \given{\begin{bmatrix} 3 \\ 4 \end{bmatrix}}
p=[42]\proj{p} = \proj{\begin{bmatrix} 4 \\ 2 \end{bmatrix}}
e=[−12]\err{e} = \err{\begin{bmatrix} -1 \\ 2 \end{bmatrix}}
Text solution
Subtract p\proj{p} from b\given{b} to get the error vector e\err{e}This splits b=p+e\given{b} = \proj{p} + \err{e} into a piece along the line and a piece perpendicular to the line.
e=b−p=[34]−[42]=[−12]\displaystyle \err{e} = \given{b} - \proj{p} = \given{\begin{bmatrix} 3 \\ 4 \end{bmatrix}} - \proj{\begin{bmatrix} 4 \\ 2 \end{bmatrix}} = \err{\begin{bmatrix} -1 \\ 2 \end{bmatrix}}
Take the length of e\err{e} to find the distance to the lineBecause e\err{e} meets the line at a 90∘90^\circ angle (a⋅e=(2)(−1)+(1)(2)=0\lineVec{a} \cdot \err{e} = (\lineVec{2})(\err{-1}) + (\lineVec{1})(\err{2}) = 0), its length is the perpendicular distance from the tip of b\given{b} to the line.
distance\displaystyle \text{distance} =∥e∥\displaystyle = \|\err{e}\| =∥b−p∥\displaystyle = \|\given{b} - \proj{p}\| =(−1)2+22\displaystyle = \sqrt{(\err{-1})^2 + \err{2}^2} =5\displaystyle = \sqrt{5}

So the tip of b=[34]\given{b} = \given{\begin{bmatrix} 3 \\ 4 \end{bmatrix}} is at a distance of 5\sqrt{5} from the line.

Because p\proj{p} and e\err{e} meet at a right angle with b\given{b} as the hypotenuse, their squared lengths always satisfy the Pythagorean theorem:

∥p∥2+∥e∥2\displaystyle \|\proj{p}\|^2 + \|\err{e}\|^2 =∥b∥2\displaystyle = \|\given{b}\|^2

For every position of b\given{b} around the fixed line, p\proj{p} and e=b−p\err{e} = \given{b} - \proj{p} form a right triangle with hypotenuse b\given{b}, so ∥p∥2+∥e∥2=∥b∥2\|\proj{p}\|^2 + \|\err{e}\|^2 = \|\given{b}\|^2 and ∥e∥\|\err{e}\| is the perpendicular distance to the line.

Every vector b\given{b} splits into a projection p\proj{p} along the line and a perpendicular error e\err{e} to the line, and their squared lengths always add up to the squared length of b\given{b}.
5

The Projection Matrix

When we want to project many different vectors b\given{b} onto the same line through a\lineVec{a}, we can write the projection as a single matrix P\projMat{P} that multiplies any b\given{b} to give p=Pb\proj{p} = \projMat{P}\given{b}.

The Projection Matrix
p=□ b\proj{p} = \projMat{\square}\,\given{b}
x\htmlClass{lc-mute}{x}
y\htmlClass{lc-mute}{y}
a\lineVec{a}
b\given{b}
p\proj{p}

In p=aa⋅ba⋅a\proj{p} = \lineVec{a}\frac{\lineVec{a} \cdot \given{b}}{\lineVec{a} \cdot \lineVec{a}}, the vector b\given{b} is inside the dot product on top. Writing each dot product of column vectors as a row times a column (a⋅b=aTb\lineVec{a} \cdot \given{b} = \lineVec{a^T} \given{b}) lets us pull b\given{b} out to the right:

p=aaTbaTaWrite the dot products as aTb and aTap=(aaTaTa)⏟projection matrix PbGroup the a’s together on the left\displaystyle \begin{aligned} \proj{p} &= \lineVec{a}\frac{\lineVec{a^T} \given{b}}{\lineVec{a^T} \lineVec{a}} \\ & \text{Write the dot products as } \lineVec{a^T} \given{b} \text{ and } \lineVec{a^T} \lineVec{a} \\[6pt] \proj{p} &= \underbrace{\left(\frac{\lineVec{a} \lineVec{a^T}}{\lineVec{a^T} \lineVec{a}}\right)}_{\text{projection matrix } \projMat{P}} \given{b} \\ & \text{Group the } \lineVec{a}\text{'s together on the left} \end{aligned}

Watch the order of multiplication in the two parts of P\projMat{P}: on the bottom, aTa\lineVec{a^T} \lineVec{a} is a row times a column, which gives a single number; on the top, aaT\lineVec{a} \lineVec{a^T} is a column times a row, which produces a full matrix:

P\displaystyle \projMat{P} =aaTaTa\displaystyle = \frac{\lineVec{a} \lineVec{a^T}}{\lineVec{a^T} \lineVec{a}}
Text solution
Multiply row by column for the number aTa\lineVec{a^T} \lineVec{a} on the bottomFor a=[21]\lineVec{a} = \lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}}, this is the familiar dot product a⋅a\lineVec{a} \cdot \lineVec{a}.
aTa=[21][21]=22+12=5\displaystyle \lineVec{a^T} \lineVec{a} = \lineVec{\begin{bmatrix} 2 & 1 \end{bmatrix}}\lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}} = \lineVec{2}^2 + \lineVec{1}^2 = 5
Multiply column by row for the matrix aaT\lineVec{a} \lineVec{a^T} on top and divide by aTa\lineVec{a^T} \lineVec{a}Multiplying the column a\lineVec{a} by the row aT\lineVec{a^T} builds a 2×22 \times 2 matrix.
aaT=[21][21]=[4221]P=aaTaTa=15[4221]\displaystyle \begin{aligned} \lineVec{a} \lineVec{a^T} &= \lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}}\lineVec{\begin{bmatrix} 2 & 1 \end{bmatrix}} = \begin{bmatrix} 4 & 2 \\ 2 & 1 \end{bmatrix} \\ \projMat{P} &= \frac{\lineVec{a} \lineVec{a^T}}{\lineVec{a^T} \lineVec{a}} = \frac{1}{5}\begin{bmatrix} 4 & 2 \\ 2 & 1 \end{bmatrix} \end{aligned}
So multiplying P=15[4221]\projMat{P} = \frac{1}{5}\begin{bmatrix} 4 & 2 \\ 2 & 1 \end{bmatrix} by any vector b\given{b} projects b\given{b} onto the line through [21]\lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}}.
Because every output Pb\projMat{P}\given{b} lands on the line through a\lineVec{a}, we get two shortcuts right away:
Every column of P\projMat{P} is a multiple of a\lineVec{a}In 15[4221]\frac{1}{5}\begin{bmatrix} 4 & 2 \\ 2 & 1 \end{bmatrix}, both columns are multiples of a=[21]\lineVec{a} = \lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}}, so you can read a direction vector for the line straight from any nonzero column of P\projMat{P}.
Projecting twice is the same as projecting once (P2=P\projMat{P}^2 = \projMat{P})Once Pb\projMat{P}\given{b} is on the line, projecting it onto the same line a second time leaves it in place, so P2b=Pb\projMat{P}^2 \given{b} = \projMat{P}\given{b}.

Key concepts

To project a single vector b\given{b} onto the line through a\lineVec{a}, first check whether b\given{b} is already on the line (if b\given{b} is a multiple of a\lineVec{a}, such as [42]=2[21]\given{\begin{bmatrix} 4 \\ 2 \end{bmatrix}} = 2\lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}}, then p=b\proj{p} = \given{b} with no calculation needed) or perpendicular to the line (a⋅b=0\lineVec{a} \cdot \given{b} = 0, which immediately gives p=0\proj{p} = 0). Otherwise, find how many copies of a\lineVec{a} reach the closest point and scale a\lineVec{a}:

c=a⋅ba⋅a,p=ca=a⋅ba⋅aa(for a=[21], b=[34]:c=105=2,p=2[21]=[42])\displaystyle \begin{gathered} \lineVec{c} = \frac{\lineVec{a} \cdot \given{b}}{\lineVec{a} \cdot \lineVec{a}}, \\ \proj{p} = \lineVec{ca} = \frac{\lineVec{a} \cdot \given{b}}{\lineVec{a} \cdot \lineVec{a}}\lineVec{a} \\[6pt] \Bigg(\text{for } \lineVec{a} = \lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}}, \, \given{b} = \given{\begin{bmatrix} 3 \\ 4 \end{bmatrix}}: \\ \lineVec{c} = \frac{10}{5} = 2, \quad \proj{p} = 2\lineVec{\begin{bmatrix} 2 \\ 1 \end{bmatrix}} = \proj{\begin{bmatrix} 4 \\ 2 \end{bmatrix}}\Bigg) \end{gathered}

If a\lineVec{a} is already a unit vector (∥a∥=1\|\lineVec{a}\| = 1), then a⋅a=1\lineVec{a} \cdot \lineVec{a} = 1 and the denominator is 11: p=(a⋅b)a\proj{p} = (\lineVec{a} \cdot \given{b})\lineVec{a}. Replacing a\lineVec{a} with a multiple like 3a3\lineVec{a} divides the multiplier c\lineVec{c} by 33 but never changes the projection vector p\proj{p}.

Once you have p\proj{p}, subtracting it from b\given{b} gives the perpendicular error vector e=b−p\err{e} = \given{b} - \proj{p}, which splits b=p+e\given{b} = \proj{p} + \err{e} into perpendicular pieces ([34]=[42]+[−12]\given{\begin{bmatrix} 3 \\ 4 \end{bmatrix}} = \proj{\begin{bmatrix} 4 \\ 2 \end{bmatrix}} + \err{\begin{bmatrix} -1 \\ 2 \end{bmatrix}}) and gives the distance from b\given{b} to the line as ∥e∥=5\|\err{e}\| = \sqrt{5}. If a problem only gives the lengths ∥b∥\|\given{b}\| and ∥p∥\|\proj{p}\|, don't look for the vectors; use the right triangle ∥p∥2+∥e∥2=∥b∥2\|\proj{p}\|^2 + \|\err{e}\|^2 = \|\given{b}\|^2 directly.

When projecting many vectors onto the same line, build the projection matrix P=aaTaTa\projMat{P} = \frac{\lineVec{a} \lineVec{a^T}}{\lineVec{a^T} \lineVec{a}} so that p=Pb\proj{p} = \projMat{P}\given{b}, keeping the two products straight: row-times-column aTa=5\lineVec{a^T} \lineVec{a} = 5 is a number on the bottom, while column-times-row aaT=[4221]\lineVec{a} \lineVec{a^T} = \begin{bmatrix} 4 & 2 \\ 2 & 1 \end{bmatrix} is a matrix on top whose columns are multiples of a\lineVec{a}. Never compute the entries of P\projMat{P} just to find P2b\projMat{P}^2 \given{b} or P3b\projMat{P}^3 \given{b}: projecting a vector that is already on the line leaves it where it is (P2=P\projMat{P}^2 = \projMat{P}), so P3b=Pb\projMat{P}^3 \given{b} = \projMat{P}\given{b}.

6

Practice

1

Find the multiplier c\lineVec{c} and the projection vector p\proj{p} of b=[1−3]\given{b} = \given{\begin{bmatrix} 1 \\ -3 \end{bmatrix}} onto the line through a=[12]\lineVec{a} = \lineVec{\begin{bmatrix} 1 \\ 2 \end{bmatrix}}.

Which dot product belongs in the denominator of c\lineVec{c}: a⋅a\lineVec{a} \cdot \lineVec{a} or b⋅b\given{b} \cdot \given{b}?

Compute the dot products a⋅b\lineVec{a} \cdot \given{b} and a⋅a\lineVec{a} \cdot \lineVec{a}:
a⋅b=(1)(1)+(2)(−3)=−5a⋅a=12+22=5\displaystyle \begin{aligned} \lineVec{a} \cdot \given{b} &= (\lineVec{1})(\given{1}) + (\lineVec{2})(\given{-3}) = -5 \\ \lineVec{a} \cdot \lineVec{a} &= \lineVec{1}^2 + \lineVec{2}^2 = 5 \end{aligned}
Divide a⋅b\lineVec{a} \cdot \given{b} by a⋅a\lineVec{a} \cdot \lineVec{a} (don't divide by b⋅b=10\given{b} \cdot \given{b} = 10, which gives c=−12\lineVec{c} = -\tfrac{1}{2}), and multiply c\lineVec{c} by the line vector a\lineVec{a} (not by b\given{b}, which gives [−13]\begin{bmatrix} -1 \\ 3 \end{bmatrix}):
c=−55=−1,p=−1[12]=[−1−2]\displaystyle \lineVec{c} = \frac{-5}{5} = -1, \qquad \proj{p} = -1\lineVec{\begin{bmatrix} 1 \\ 2 \end{bmatrix}} = \proj{\begin{bmatrix} -1 \\ -2 \end{bmatrix}}
2

Find the projection p\proj{p} of b=[510]\given{b} = \given{\begin{bmatrix} 5 \\ 10 \end{bmatrix}} onto the line through the unit vector a=[3545]\lineVec{a} = \lineVec{\begin{bmatrix} \tfrac{3}{5} \\ \tfrac{4}{5} \end{bmatrix}}.

Check the length of a\lineVec{a} first: what is a⋅a\lineVec{a} \cdot \lineVec{a} when a\lineVec{a} is a unit vector?

Because ∥a∥=1\|\lineVec{a}\| = 1, we have a⋅a=1\lineVec{a} \cdot \lineVec{a} = 1, so the multiplier c\lineVec{c} is simply the dot product a⋅b\lineVec{a} \cdot \given{b}:
c\displaystyle \lineVec{c} =a⋅b\displaystyle = \lineVec{a} \cdot \given{b} =(35)(5)+(45)(10)\displaystyle = \left(\lineVec{\frac{3}{5}}\right)(\given{5}) + \left(\lineVec{\frac{4}{5}}\right)(\given{10}) =3+8\displaystyle = 3 + 8 =11\displaystyle = 11
Multiply c=11\lineVec{c} = 11 by the unit vector a\lineVec{a} (don't stop at the number 1111, and don't drop the /5/5 to multiply by [34]\begin{bmatrix} 3 \\ 4 \end{bmatrix}, which would give [3344]\begin{bmatrix} 33 \\ 44 \end{bmatrix}):
p=11a=11[3545]=[335445]\displaystyle \proj{p} = 11\lineVec{a} = 11\lineVec{\begin{bmatrix} \tfrac{3}{5} \\ \tfrac{4}{5} \end{bmatrix}} = \proj{\begin{bmatrix} \tfrac{33}{5} \\ \tfrac{44}{5} \end{bmatrix}}
3

Split b=[231]\given{b} = \given{\begin{bmatrix} 2 \\ 3 \\ 1 \end{bmatrix}} into a projection p\proj{p} along the line through a=[111]\lineVec{a} = \lineVec{\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}} and an error vector e\err{e} perpendicular to the line, then find the distance from b\given{b} to the line.

After finding p=ca\proj{p} = \lineVec{ca}, how do you get the leftover vector e\err{e} from p\proj{p} to b\given{b}, and which vector's length is the distance to the line?

Find the multiplier c\lineVec{c} and the projection vector p\proj{p} (remember to divide by a⋅a=12+12+12=3\lineVec{a} \cdot \lineVec{a} = \lineVec{1}^2 + \lineVec{1}^2 + \lineVec{1}^2 = 3, not ∥a∥=3\|\lineVec{a}\| = \sqrt{3}):
c=a⋅ba⋅a=2+3+112+12+12=63=2,p=2[111]=[222]\displaystyle \begin{aligned} \lineVec{c} &= \frac{\lineVec{a} \cdot \given{b}}{\lineVec{a} \cdot \lineVec{a}} = \frac{2 + 3 + 1}{\lineVec{1}^2 + \lineVec{1}^2 + \lineVec{1}^2} = \frac{6}{3} = 2, \\ \proj{p} &= 2\lineVec{\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}} = \proj{\begin{bmatrix} 2 \\ 2 \\ 2 \end{bmatrix}} \end{aligned}
Subtract p\proj{p} from b\given{b} to get the error vector e\err{e} (subtracting p−b\proj{p} - \given{b} instead reverses the signs to [0−11]\begin{bmatrix} 0 \\ -1 \\ 1 \end{bmatrix}):
e=b−p=[231]−[222]=[01−1]\displaystyle \err{e} = \given{b} - \proj{p} = \given{\begin{bmatrix} 2 \\ 3 \\ 1 \end{bmatrix}} - \proj{\begin{bmatrix} 2 \\ 2 \\ 2 \end{bmatrix}} = \err{\begin{bmatrix} 0 \\ 1 \\ -1 \end{bmatrix}}
Take the length of e\err{e} to find the distance (don't forget the square root and answer 22, and don't take ∥p∥=12\|\proj{p}\| = \sqrt{12}):
distance\displaystyle \text{distance} =∥e∥\displaystyle = \|\err{e}\| =02+12+(−1)2\displaystyle = \sqrt{\err{0}^2 + \err{1}^2 + (\err{-1})^2} =2\displaystyle = \sqrt{2}
4

Projecting a vector b\given{b} onto the line through a\lineVec{a} gives multiplier c=6\lineVec{c} = 6 and projection p=[12−6]\proj{p} = \proj{\begin{bmatrix} 12 \\ -6 \end{bmatrix}}. If the same line is defined using 3a3\lineVec{a} instead of a\lineVec{a}, what are the new multiplier and the new projection vector?

If the direction vector is 33 times as long, how many copies of it do you need to reach the same point on the line?

Making the line vector 33 times as long means we need one-third as many copies of it to reach the same spot (watch out for multiplying c\lineVec{c} by 33 to get 1818, or dividing by 32=93^2 = 9 to get 23\tfrac{2}{3} by forgetting that the numerator (3a)⋅b(3\lineVec{a}) \cdot \given{b} also triples). Because 3a3\lineVec{a} defines the exact same line, the projection vector p\proj{p} does not change (don't multiply p\proj{p} by 33 to get [36−18]\begin{bmatrix} 36 \\ -18 \end{bmatrix}):
cnew=63=2,p=[12−6]\displaystyle \lineVec{c_{\text{new}}} = \frac{6}{3} = 2, \qquad \proj{p} = \proj{\begin{bmatrix} 12 \\ -6 \end{bmatrix}}
5

Find the projection matrix P\projMat{P} that projects vectors in R2\mathbb{R}^2 onto the line through a=[1−3]\lineVec{a} = \lineVec{\begin{bmatrix} 1 \\ -3 \end{bmatrix}}.

In P=aaTaTa\projMat{P} = \frac{\lineVec{a} \lineVec{a^T}}{\lineVec{a^T} \lineVec{a}}, which part is a single number and which part is a 2×22 \times 2 matrix?

Compute the number aTa\lineVec{a^T} \lineVec{a} on the bottom (be sure to square −3\lineVec{-3} to get +9+9, not −9-9, which would give aTa=−8\lineVec{a^T} \lineVec{a} = -8) and the column-times-row matrix aaT\lineVec{a} \lineVec{a^T} on top (keep the negative signs on the entries −3-3 rather than writing [1339]\begin{bmatrix} 1 & 3 \\ 3 & 9 \end{bmatrix}):
aTa=12+(−3)2=10,aaT=[1−3][1−3]=[1−3−39]\displaystyle \lineVec{a^T} \lineVec{a} = \lineVec{1}^2 + (\lineVec{-3})^2 = 10, \qquad \lineVec{a} \lineVec{a^T} = \lineVec{\begin{bmatrix} 1 \\ -3 \end{bmatrix}}\lineVec{\begin{bmatrix} 1 & -3 \end{bmatrix}} = \begin{bmatrix} 1 & -3 \\ -3 & 9 \end{bmatrix}
Divide by aTa=10\lineVec{a^T} \lineVec{a} = 10:
P=110[1−3−39]\displaystyle \projMat{P} = \frac{1}{10}\begin{bmatrix} 1 & -3 \\ -3 & 9 \end{bmatrix}
6

A vector b\given{b} has length ∥b∥=5\|\given{b}\| = 5, and its projection onto a line has length ∥p∥=4\|\proj{p}\| = 4. What is the distance from the tip of b\given{b} to the line?

Which vector is the hypotenuse of the right triangle formed by p\proj{p}, e\err{e}, and b\given{b}?

By the Pythagorean theorem ∥p∥2+∥e∥2=∥b∥2\|\proj{p}\|^2 + \|\err{e}\|^2 = \|\given{b}\|^2 (don't subtract the lengths directly to get 5−4=15 - 4 = 1, or treat p\proj{p} as the hypotenuse to get 42+52=41\sqrt{4^2 + 5^2} = \sqrt{41}):
42+∥e∥2\displaystyle 4^2 + \|\err{e}\|^2 =52  ⟹  ∥e∥2\displaystyle = 5^2 \implies \|\err{e}\|^2 =9\displaystyle = 9
Take the square root to find the distance (don't stop at ∥e∥2=9\|\err{e}\|^2 = 9):
distance\displaystyle \text{distance} =∥e∥\displaystyle = \|\err{e}\| =3\displaystyle = 3
7

Let P\projMat{P} be the projection matrix onto the line through a=[3−12]\lineVec{a} = \lineVec{\begin{bmatrix} 3 \\ -1 \\ 2 \end{bmatrix}}. Without calculating the entries of P\projMat{P}, find P[6−24]\projMat{P}\given{\begin{bmatrix} 6 \\ -2 \\ 4 \end{bmatrix}} and simplify P3b\projMat{P}^3 \given{b} in terms of Pb\projMat{P}\given{b}.

How is [6−24]\given{\begin{bmatrix} 6 \\ -2 \\ 4 \end{bmatrix}} related to a\lineVec{a}, and what happens when you project a vector that is already on the line?

Since [6−24]=2a\given{\begin{bmatrix} 6 \\ -2 \\ 4 \end{bmatrix}} = 2\lineVec{a} already lies on the line, projecting it leaves it unchanged (don't shrink it back to a=[3−12]\lineVec{a} = \lineVec{\begin{bmatrix} 3 \\ -1 \\ 2 \end{bmatrix}} or scale it by 22 again to get [12−48]\begin{bmatrix} 12 \\ -4 \\ 8 \end{bmatrix}). And since projecting twice is the same as projecting once (P2=P\projMat{P}^2 = \projMat{P}), multiplying by P\projMat{P} three times is the same as multiplying by P\projMat{P} once (not 3Pb3\projMat{P}\given{b}):
P[6−24]=[6−24],P3b=Pb\displaystyle \projMat{P}\given{\begin{bmatrix} 6 \\ -2 \\ 4 \end{bmatrix}} = \proj{\begin{bmatrix} 6 \\ -2 \\ 4 \end{bmatrix}}, \qquad \projMat{P}^3 \given{b} = \projMat{P}\given{b}
8

The matrix P=12[1111]\projMat{P} = \frac{1}{2}\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix} projects vectors onto a line in R2\mathbb{R}^2. Give a direction vector a\lineVec{a} for the line, and describe all vectors b=[b1b2]\given{b} = \given{\begin{bmatrix} b_1 \\ b_2 \end{bmatrix}} whose projection onto this line is [00]\begin{bmatrix} 0 \\ 0 \end{bmatrix}.

How are the columns of P\projMat{P} related to the line, and what angle must b\given{b} make with a\lineVec{a} for its projection to be zero?

The columns of P\projMat{P} are multiples of a=[11]\lineVec{a} = \lineVec{\begin{bmatrix} 1 \\ 1 \end{bmatrix}}, which gives the direction of the line. The projection of b\given{b} is [00]\begin{bmatrix} 0 \\ 0 \end{bmatrix} when a⋅b=b1+b2=0\lineVec{a} \cdot \given{b} = \given{b_1} + \given{b_2} = 0, so b2=−b1\given{b_2} = -\given{b_1} (don't drop the negative sign and give multiples of [11]\begin{bmatrix} 1 \\ 1 \end{bmatrix}, which lie on the line and project to themselves):
a=[11],all multiples of [1−1] (the vectors perpendicularto the line)\displaystyle \begin{aligned} & \lineVec{a} = \lineVec{\begin{bmatrix} 1 \\ 1 \end{bmatrix}}, \text{all multiples of } \err{\begin{bmatrix} 1 \\ -1 \end{bmatrix}} \\ &\quad \text{ (the vectors perpendicular} \\ &\quad \text{to the line)} \end{aligned}