When a vector b does not lie on the line through a, its projection p is the point on the line that comes as close to the tip of b as possible.
1
Finding the Projection onto a Line
To find the closest point p on the line through a, we look for the multiple of a where the line segment to the tip of b meets the line at a 90∘ angle.
Finding the Projection onto a Line
Every point on the line is a scaled copy of a, so p=ca for some number c. At the closest point, the leftover error vector e=b−ca is perpendicular to a, meaning their dot product is zero:
a⋅(error eb−ca)=0Error vector is perpendicularto the linea⋅b−c(a⋅a)=0Distribute the dot productc=a⋅aa⋅bSolve for the multiplier c
Multiplying the direction vector a by that number c gives the projection vector p:
p=multiplier ca⋅aa⋅ba
Both a⋅b and a⋅a are single numbers, so c=a⋅aa⋅b is just a number that stretches or shrinks the vector a. You cannot cancel a across the dot products.
As the point ca moves along the line, the vector b−ca to the tip of b is shortest at the exact spot where it meets the line at 90∘ and a⋅(b−ca)=0.
The arrow from a point on the line to the tip of the vector off the line is shortest right where it meets the line at a right angle.
2
Why We Divide by Length Squared
In the multiplier c=a⋅aa⋅b, the bottom is a⋅a=∥a∥2. We divide by the length of a, squared, so that the projection point on the line stays in the same place no matter how long a is.
Why We Divide by Length Squared
Vector a only sets the direction of the line. If we replace a with 2a, the line and the closest point p do not change, so we need half as many copies of 2a to reach the same spot:
cnew=(2a)⋅(2a)(2a)⋅b=4(a⋅a)2(a⋅b)=21c
When we multiply 21c by 2a, the factors of 2 cancel and give the exact same projection vector p=ca. And if a is already a unit vector (∥a∥=1), then a⋅a=1 and the denominator is 1:
p=(a⋅b)awhen ∥a∥=1
Stretching, shrinking, or flipping a along the same line rescales the multiplier c (and makes the denominator 1 when ∥a∥=1), while the projection point p=ca stays in the exact same spot.
Stretching, shrinking, or flipping a along the line changes the multiplier c, while the projection p never moves.
3
Calculating a Projection Vector
Now let's find the projection p of b=[34] onto the line through a=[21].
Calculating a Projection Vector
Text solution
Find the multiplier c from the dot productsCompute a⋅b on top and a⋅a on the bottom to find how many copies of a reach the closest point.
Scale the line vector a by cMultiply a by c=2 to get the projection vector on the line.
p=ca=2[21]=[42]
So the point on the line through [21] that is closest to [34] is p=[42].
4
Error Vector and Distance to a Line
Once we have the projection p, the leftover vector from p to b is the error vector e=b−p, which splits b into perpendicular pieces and gives the shortest distance from b to the line.
Error Vector and Distance to a Line
Text solution
Subtract p from b to get the error vector eThis splits b=p+e into a piece along the line and a piece perpendicular to the line.
e=b−p=[34]−[42]=[−12]
Take the length of e to find the distance to the lineBecause e meets the line at a 90∘ angle (a⋅e=(2)(−1)+(1)(2)=0), its length is the perpendicular distance from the tip of b to the line.
distance=∥e∥=∥b−p∥=(−1)2+22=5
So the tip of b=[34] is at a distance of 5 from the line.
Because p and e meet at a right angle with b as the hypotenuse, their squared lengths always satisfy the Pythagorean theorem:
∥p∥2+∥e∥2=∥b∥2
For every position of b around the fixed line, p and e=b−p form a right triangle with hypotenuse b, so ∥p∥2+∥e∥2=∥b∥2 and ∥e∥ is the perpendicular distance to the line.
Every vector b splits into a projection p along the line and a perpendicular error e to the line, and their squared lengths always add up to the squared length of b.
5
The Projection Matrix
When we want to project many different vectors b onto the same line through a, we can write the projection as a single matrix P that multiplies any b to give p=Pb.
The Projection Matrix
In p=aa⋅aa⋅b, the vector b is inside the dot product on top. Writing each dot product of column vectors as a row times a column (a⋅b=aTb) lets us pull b out to the right:
pp=aaTaaTbWrite the dot products as aTb and aTa=projection matrix P(aTaaaT)bGroup the a’s together on the left
Watch the order of multiplication in the two parts of P: on the bottom, aTa is a row times a column, which gives a single number; on the top, aaT is a column times a row, which produces a full matrix:
P=aTaaaT
Text solution
Multiply row by column for the number aTa on the bottomFor a=[21], this is the familiar dot product a⋅a.
aTa=[21][21]=22+12=5
Multiply column by row for the matrix aaT on top and divide by aTaMultiplying the column a by the row aT builds a 2×2 matrix.
aaTP=[21][21]=[4221]=aTaaaT=51[4221]
So multiplying P=51[4221] by any vector b projects b onto the line through [21].
Because every output Pb lands on the line through a, we get two shortcuts right away:
Every column of P is a multiple of aIn 51[4221], both columns are multiples of a=[21], so you can read a direction vector for the line straight from any nonzero column of P.
Projecting twice is the same as projecting once (P2=P)Once Pb is on the line, projecting it onto the same line a second time leaves it in place, so P2b=Pb.
Key concepts
To project a single vector b onto the line through a, first check whether b is already on the line (if b is a multiple of a, such as [42]=2[21], then p=b with no calculation needed) or perpendicular to the line (a⋅b=0, which immediately gives p=0). Otherwise, find how many copies of a reach the closest point and scale a:
If a is already a unit vector (∥a∥=1), then a⋅a=1 and the denominator is 1:p=(a⋅b)a. Replacing a with a multiple like 3a divides the multiplier c by 3 but never changes the projection vector p.
Once you have p, subtracting it from b gives the perpendicular error vector e=b−p, which splits b=p+e into perpendicular pieces ([34]=[42]+[−12]) and gives the distance from b to the line as ∥e∥=5. If a problem only gives the lengths ∥b∥ and ∥p∥, don't look for the vectors; use the right triangle ∥p∥2+∥e∥2=∥b∥2 directly.
When projecting many vectors onto the same line, build the projection matrix P=aTaaaT so that p=Pb, keeping the two products straight: row-times-column aTa=5 is a number on the bottom, while column-times-row aaT=[4221] is a matrix on top whose columns are multiples of a. Never compute the entries of P just to find P2b or P3b: projecting a vector that is already on the line leaves it where it is (P2=P), so P3b=Pb.
6
Practice
1
Find the multiplier c and the projection vector p of b=[1−3] onto the line through a=[12].
Divide a⋅b by a⋅a (don't divide by b⋅b=10, which gives c=−21), and multiply c by the line vector a (not by b, which gives [−13]).
Which dot product belongs in the denominator of c:a⋅a or b⋅b?
Compute the dot products a⋅b and a⋅a:
a⋅ba⋅a=(1)(1)+(2)(−3)=−5=12+22=5
Divide a⋅b by a⋅a (don't divide by b⋅b=10, which gives c=−21), and multiply c by the line vector a (not by b, which gives [−13]):
c=5−5=−1,p=−1[12]=[−1−2]
2
Find the projection p of b=[510] onto the line through the unit vector a=[5354].
Multiply c=11 by the unit vector a (don't stop at the number 11, and don't drop the /5 to multiply by [34], which would give [3344]).
Check the length of a first: what is a⋅a when a is a unit vector?
Because ∥a∥=1, we have a⋅a=1, so the multiplier c is simply the dot product a⋅b:
c=a⋅b=(53)(5)+(54)(10)=3+8=11
Multiply c=11 by the unit vector a (don't stop at the number 11, and don't drop the /5 to multiply by [34], which would give [3344]):
p=11a=11[5354]=[533544]
3
Split b=231 into a projection p along the line through a=111 and an error vector e perpendicular to the line, then find the distance from b to the line.
Find the multiplier c and the projection vector p (remember to divide by a⋅a=12+12+12=3, not ∥a∥=3).
After finding p=ca, how do you get the leftover vector e from p to b, and which vector's length is the distance to the line?
Find the multiplier c and the projection vector p (remember to divide by a⋅a=12+12+12=3, not ∥a∥=3):
cp=a⋅aa⋅b=12+12+122+3+1=36=2,=2111=222
Subtract p from b to get the error vector e (subtracting p−b instead reverses the signs to 0−11):
e=b−p=231−222=01−1
Take the length of e to find the distance (don't forget the square root and answer 2, and don't take ∥p∥=12):
distance=∥e∥=02+12+(−1)2=2
4
Projecting a vector b onto the line through a gives multiplier c=6 and projection p=[12−6]. If the same line is defined using 3a instead of a, what are the new multiplier and the new projection vector?
Making the line vector 3 times as long means we need one-third as many copies of it to reach the same spot (watch out for multiplying c by 3 to get 18, or dividing by 32=9 to get 32 by forgetting that the numerator (3a)⋅b also triples).
If the direction vector is 3 times as long, how many copies of it do you need to reach the same point on the line?
Making the line vector 3 times as long means we need one-third as many copies of it to reach the same spot (watch out for multiplying c by 3 to get 18, or dividing by 32=9 to get 32 by forgetting that the numerator (3a)⋅b also triples). Because 3a defines the exact same line, the projection vector p does not change (don't multiply p by 3 to get [36−18]):
cnew=36=2,p=[12−6]
5
Find the projection matrix P that projects vectors in R2 onto the line through a=[1−3].
Compute the number aTa on the bottom (be sure to square −3 to get +9, not −9, which would give aTa=−8) and the column-times-row matrix aaT on top (keep the negative signs on the entries −3 rather than writing [1339]).
In P=aTaaaT, which part is a single number and which part is a 2×2 matrix?
Compute the number aTa on the bottom (be sure to square −3 to get +9, not −9, which would give aTa=−8) and the column-times-row matrix aaT on top (keep the negative signs on the entries −3 rather than writing [1339]):
aTa=12+(−3)2=10,aaT=[1−3][1−3]=[1−3−39]
Divide by aTa=10:
P=101[1−3−39]
6
A vector b has length ∥b∥=5, and its projection onto a line has length ∥p∥=4. What is the distance from the tip of b to the line?
By the Pythagorean theorem ∥p∥2+∥e∥2=∥b∥2 (don't subtract the lengths directly to get 5−4=1, or treat p as the hypotenuse to get 42+52=41).Take the square root to find the distance (don't stop at ∥e∥2=9).
Which vector is the hypotenuse of the right triangle formed by p,e, and b?
By the Pythagorean theorem ∥p∥2+∥e∥2=∥b∥2 (don't subtract the lengths directly to get 5−4=1, or treat p as the hypotenuse to get 42+52=41):
42+∥e∥2=52⟹∥e∥2=9
Take the square root to find the distance (don't stop at ∥e∥2=9):
distance=∥e∥=3
7
Let P be the projection matrix onto the line through a=3−12. Without calculating the entries of P, find P6−24 and simplify P3b in terms of Pb.
Since 6−24=2a already lies on the line, projecting it leaves it unchanged (don't shrink it back to a=3−12 or scale it by 2 again to get 12−48).
How is 6−24 related to a, and what happens when you project a vector that is already on the line?
Since 6−24=2a already lies on the line, projecting it leaves it unchanged (don't shrink it back to a=3−12 or scale it by 2 again to get 12−48). And since projecting twice is the same as projecting once (P2=P), multiplying by P three times is the same as multiplying by P once (not 3Pb):
P6−24=6−24,P3b=Pb
8
The matrix P=21[1111] projects vectors onto a line in R2. Give a direction vector a for the line, and describe all vectors b=[b1b2] whose projection onto this line is [00].
How are the columns of P related to the line, and what angle must b make with a for its projection to be zero?
The columns of P are multiples of a=[11], which gives the direction of the line. The projection of b is [00] when a⋅b=b1+b2=0, so b2=−b1 (don't drop the negative sign and give multiples of [11], which lie on the line and project to themselves):
a=[11],all multiples of [1−1] (the vectors perpendicularto the line)