Linear AlgebraOrthogonality and Projections1 of 9
Orthogonality and Projections · 1

Orthogonal and Orthonormal Sets

Finding the coordinates of a vector in a basis usually means setting up a system of equations and row-reducing a matrix, and we want to know when we can find each coordinate on its own without that algebra.

1

Why 90∘90^\circ angles stop basis vectors from interfering

To write a target vector v=[79]\target{v} = \target{\begin{bmatrix} 7 \\ 9 \end{bmatrix}} as a combination of two tilted basis vectors b1=[12]\first{b_1} = \first{\begin{bmatrix} 1 \\ 2 \end{bmatrix}} and b2=[31]\second{b_2} = \second{\begin{bmatrix} 3 \\ 1 \end{bmatrix}}, we look for the coordinates c1c_1 and c2c_2 in

c1[12]+c2[31]=[79]\displaystyle c_1 \first{\begin{bmatrix} 1 \\ 2 \end{bmatrix}} + c_2 \second{\begin{bmatrix} 3 \\ 1 \end{bmatrix}} = \target{\begin{bmatrix} 7 \\ 9 \end{bmatrix}}

In the standard basis [10]\first{\begin{bmatrix} 1 \\ 0 \end{bmatrix}} and [01]\second{\begin{bmatrix} 0 \\ 1 \end{bmatrix}}, we can read off the coordinates of [34]\target{\begin{bmatrix} 3 \\ 4 \end{bmatrix}} right away because the axes sit at 90∘90^\circ to each other and each basis vector has length 11.

Why 90-degree angles stop cross-talk
2\htmlClass{lc-mute}{2}
2\htmlClass{lc-mute}{2}
3\htmlClass{lc-mute}{3}
3\htmlClass{lc-mute}{3}
4\htmlClass{lc-mute}{4}
4\htmlClass{lc-mute}{4}
5\htmlClass{lc-mute}{5}
5\htmlClass{lc-mute}{5}
[10]\htmlClass{lc-green}{\begin{bmatrix} 1 \\ 0 \end{bmatrix}}
[01]\htmlClass{lc-pink}{\begin{bmatrix} 0 \\ 1 \end{bmatrix}}
[34]\htmlClass{lc-blue}{\begin{bmatrix} 3 \\ 4 \end{bmatrix}}
? [10]+  ? [01]=[34]?\,\htmlClass{lc-green}{\begin{bmatrix} 1 \\ 0 \end{bmatrix}} + \;?\,\htmlClass{lc-pink}{\begin{bmatrix} 0 \\ 1 \end{bmatrix}} = \htmlClass{lc-blue}{\begin{bmatrix} 3 \\ 4 \end{bmatrix}}

Try tilting b2\second{b_2} below to see when c1b1c_1 \first{b_1} matches the perpendicular shadow of v\target{v} on b1\first{b_1}.

Tilting the second basis vector changes how much of the first one you need to reach v, and only at a right angle does the first piece match v's perpendicular shadow.
2

Orthogonal sets and orthogonal bases

An orthogonal set is a group of vectors in which every vector is at a 90∘90^\circ angle to every other vector in the set.

Two vectors meet at 90∘90^\circ when their dot product is 00. To test whether u1=[110]\first{u_1} = \first{\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}}, u2=[−110]\second{u_2} = \second{\begin{bmatrix} -1 \\ 1 \\ 0 \end{bmatrix}}, and u3=[005]\third{u_3} = \third{\begin{bmatrix} 0 \\ 0 \\ 5 \end{bmatrix}} form an orthogonal set in R3\mathbb{R}^3, we check the dot product of every pair.

Orthogonal sets and orthogonal bases
u1⋅u2=  ?u1⋅u3=  ?u2⋅u3=  ?\begin{aligned} \first{u_1}\cdot\second{u_2} &= \;? \\ \first{u_1}\cdot\third{u_3} &= \;? \\ \second{u_2}\cdot\third{u_3} &= \;? \end{aligned}
Text solution
Dot the first pair (u1\first{u_1} and u2\second{u_2})Multiply matching entries and add them up to check that u1\first{u_1} and u2\second{u_2} meet at 90∘90^\circ.
u1⋅u2\displaystyle \first{u_1} \cdot \second{u_2} =(1)(−1)+(1)(1)+(0)(0)\displaystyle = (\first{1})(\second{-1}) + (\first{1})(\second{1}) + (\first{0})(\second{0}) =−1+1+0\displaystyle = -1 + 1 + 0 =0\displaystyle = 0
Dot the remaining two pairs (u1,u3\first{u_1}, \third{u_3} and u2,u3\second{u_2}, \third{u_3})Check every other pair to make sure those meet at 90∘90^\circ too.
u1⋅u3=(1)(0)+(1)(0)+(0)(5)=0,u2⋅u3=(−1)(0)+(1)(0)+(0)(5)=0\displaystyle \begin{aligned} \first{u_1} \cdot \third{u_3} &= (\first{1})(\third{0}) + (\first{1})(\third{0}) + (\first{0})(\third{5}) \\ &= 0, \\ \second{u_2} \cdot \third{u_3} &= (\second{-1})(\third{0}) + (\second{1})(\third{0}) + (\second{0})(\third{5}) \\ &= 0 \end{aligned}
Count non-zero orthogonal vectors in R3\mathbb{R}^3Because each new vector is at 90∘90^\circ to the previous ones, it points into a new dimension and cannot lie in their span, making the set linearly independent.
{[110],[−110],[005]}⏟orthogonal set of 3 non-zero vectors in R3  ⟹  orthogonal basis for R3\displaystyle \begin{aligned} &\underbrace{\left\{ \first{\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}}, \second{\begin{bmatrix} -1 \\ 1 \\ 0 \end{bmatrix}}, \third{\begin{bmatrix} 0 \\ 0 \\ 5 \end{bmatrix}} \right\}}_{\text{orthogonal set of 3 non-zero vectors in } \mathbb{R}^3} \\ &\implies \text{orthogonal basis for } \mathbb{R}^3 \end{aligned}

So {u1,u2,u3}\{\first{u_1}, \second{u_2}, \third{u_3}\} is an orthogonal basis for R3\mathbb{R}^3, which guarantees that any vector v\target{v} in R3\mathbb{R}^3 can be written as a unique combination v=c1u1+c2u2+c3u3\target{v} = c_1 \first{u_1} + c_2 \second{u_2} + c_3 \third{u_3}.

For three vectors, make sure to check all three pairs (u1⋅u2\first{u_1} \cdot \second{u_2}, u1⋅u3\first{u_1} \cdot \third{u_3}, and u2⋅u3\second{u_2} \cdot \third{u_3}). Checking only adjacent pairs like u1⋅u2\first{u_1} \cdot \second{u_2} and u2⋅u3\second{u_2} \cdot \third{u_3} is a common mistake, because u1\first{u_1} and u3\third{u_3} still might not be at 90∘90^\circ to each other.

3

Finding coordinates in an orthogonal basis

Given our orthogonal basis u1=[110]\first{u_1} = \first{\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}}, u2=[−110]\second{u_2} = \second{\begin{bmatrix} -1 \\ 1 \\ 0 \end{bmatrix}}, u3=[005]\third{u_3} = \third{\begin{bmatrix} 0 \\ 0 \\ 5 \end{bmatrix}} and a target vector v=[2610]\target{v} = \target{\begin{bmatrix} 2 \\ 6 \\ 10 \end{bmatrix}}, we want to find the coordinates c1,c2,c3c_1, c_2, c_3 in

v\displaystyle \target{v} =c1u1+c2u2+c3u3\displaystyle = c_1 \first{u_1} + c_2 \second{u_2} + c_3 \third{u_3}

without row reduction.

Finding coordinates in an orthogonal basis
v=c1u1+c2u2+c3u3c1,  c2,  c3=  ?\begin{aligned} \target{v} &= c_1\first{u_1} + c_2\second{u_2} + c_3\third{u_3} \\ c_1,\; c_2,\; c_3 &= \;? \end{aligned}

Taking the dot product of both sides with u1\first{u_1} zeroes out the terms with the other basis vectors because their dot products with u1\first{u_1} are zero:

v⋅u1\displaystyle \target{v} \cdot \first{u_1} =c1(u1⋅u1)+c2(u2⋅u1)⏟0+c3(u3⋅u1)⏟0\displaystyle = c_1(\first{u_1} \cdot \first{u_1}) + c_2\underbrace{(\second{u_2} \cdot \first{u_1})}_{0} + c_3\underbrace{(\third{u_3} \cdot \first{u_1})}_{0} =c1(u1⋅u1)\displaystyle = c_1(\first{u_1} \cdot \first{u_1})

Dividing by the number u1⋅u1\first{u_1} \cdot \first{u_1} gives c1c_1, and dotting with any other ui\first{u_i} the same way gives each coordinate cic_i on its own:

ci\displaystyle c_i =v⋅uiui⋅ui\displaystyle = \frac{\target{v} \cdot \first{u_i}}{\first{u_i} \cdot \first{u_i}}
Text solution
Find c1c_1 along u1\first{u_1}Dot v=[2610]\target{v} = \target{\begin{bmatrix} 2 \\ 6 \\ 10 \end{bmatrix}} with u1=[110]\first{u_1} = \first{\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}} on top and dot u1\first{u_1} with itself on the bottom.
c1=v⋅u1u1⋅u1=(2)(1)+(6)(1)+(10)(0)12+12+02=82=4\displaystyle \begin{aligned} c_1 &= \frac{\target{v} \cdot \first{u_1}}{\first{u_1} \cdot \first{u_1}} \\ &= \frac{(\target{2})(\first{1}) + (\target{6})(\first{1}) + (\target{10})(\first{0})}{\first{1}^2 + \first{1}^2 + \first{0}^2} \\ &= \frac{8}{2} = 4 \end{aligned}
Find c2c_2 along u2\second{u_2}Use the same formula with u2=[−110]\second{u_2} = \second{\begin{bmatrix} -1 \\ 1 \\ 0 \end{bmatrix}}.
c2=v⋅u2u2⋅u2=(2)(−1)+(6)(1)+(10)(0)(−1)2+12+02=42=2\displaystyle \begin{aligned} c_2 &= \frac{\target{v} \cdot \second{u_2}}{\second{u_2} \cdot \second{u_2}} \\ &= \frac{(\target{2})(\second{-1}) + (\target{6})(\second{1}) + (\target{10})(\second{0})}{(\second{-1})^2 + \second{1}^2 + \second{0}^2} \\ &= \frac{4}{2} = 2 \end{aligned}
Find c3c_3 along u3\third{u_3}Use the same formula with u3=[005]\third{u_3} = \third{\begin{bmatrix} 0 \\ 0 \\ 5 \end{bmatrix}}.
c3=v⋅u3u3⋅u3=(2)(0)+(6)(0)+(10)(5)02+02+52=5025=2\displaystyle \begin{aligned} c_3 &= \frac{\target{v} \cdot \third{u_3}}{\third{u_3} \cdot \third{u_3}} \\ &= \frac{(\target{2})(\third{0}) + (\target{6})(\third{0}) + (\target{10})(\third{5})}{\third{0}^2 + \third{0}^2 + \third{5}^2} \\ &= \frac{50}{25} = 2 \end{aligned}
Add the projected vector piecesMultiply each coordinate by its basis vector to build v\target{v} from its perpendicular projections along the three axes.
v=4[110]+2[−110]+2[005]=[440]⏟projection on u1+[−220]⏟projection on u2+[0010]⏟projection on u3=[2610]\displaystyle \begin{aligned} \target{v} &= 4\first{\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}} + 2\second{\begin{bmatrix} -1 \\ 1 \\ 0 \end{bmatrix}} + 2\third{\begin{bmatrix} 0 \\ 0 \\ 5 \end{bmatrix}} \\ &= \underbrace{\first{\begin{bmatrix} 4 \\ 4 \\ 0 \end{bmatrix}}}_{\text{projection on } \first{u_1}} + \underbrace{\second{\begin{bmatrix} -2 \\ 2 \\ 0 \end{bmatrix}}}_{\text{projection on } \second{u_2}} \\ &\qquad + \underbrace{\third{\begin{bmatrix} 0 \\ 0 \\ 10 \end{bmatrix}}}_{\text{projection on } \third{u_3}} \\ &= \target{\begin{bmatrix} 2 \\ 6 \\ 10 \end{bmatrix}} \end{aligned}

So the coordinates of v\target{v} in {u1,u2,u3}\{\first{u_1}, \second{u_2}, \third{u_3}\} are c1=4c_1 = 4, c2=2c_2 = 2, and c3=2c_3 = 2, and each piece ciuic_i \first{u_i} is the perpendicular projection of v\target{v} onto that basis line.

Try dragging v\target{v} parallel to u2\second{u_2} below to see that changing c2c_2 leaves the projection c1u1c_1 \first{u_1} and the coordinate c1c_1 completely alone.

Because the two axes meet at a right angle, your vector splits into two perpendicular pieces along the axes that add up to the vector, and sliding parallel to one axis changes only that axis's coordinate.
4

Normalizing to an orthonormal basis

An orthonormal set is an orthogonal set whose vectors all have length 11 (unit vectors), and an orthonormal basis is a basis that is an orthonormal set.

In ci=v⋅uiui⋅uic_i = \frac{\target{v} \cdot \first{u_i}}{\first{u_i} \cdot \first{u_i}}, we had to divide by ui⋅ui\first{u_i} \cdot \first{u_i} only because our basis vectors were not 11 unit long. Since a vector's length is ∥ui∥=ui⋅ui\|\first{u_i}\| = \sqrt{\first{u_i} \cdot \first{u_i}}, dividing each vector in {u1,u2,u3}\{\first{u_1}, \second{u_2}, \third{u_3}\} by its own length shrinks or stretches it to length 11 without changing its direction.

Normalizing to an orthonormal basis
c3=v⋅u3u3⋅u3=5025why 25 ?\begin{gathered} c_3 = \frac{\target{v}\cdot\third{u_3}}{\third{u_3}\cdot\third{u_3}} = \frac{50}{25} \\[4pt] \text{why } 25\,? \end{gathered}
∥u1∥=∥u2∥=2,∥u3∥=5\|\first{u_1}\| = \|\second{u_2}\| = \sqrt{2}, \quad \|\third{u_3}\| = 5
∥q1∥=∥q2∥=∥q3∥=1\|\first{q_1}\| = \|\second{q_2}\| = \|\third{q_3}\| = 1
Text solution
Divide u1\first{u_1} by its length ∥u1∥\|\first{u_1}\|Compute ∥u1∥=12+12+02=2\|\first{u_1}\| = \sqrt{\first{1}^2 + \first{1}^2 + \first{0}^2} = \sqrt{2} and divide each entry of u1\first{u_1} by 2\sqrt{2}.
q1=u1∥u1∥=12[110]=[12120]\displaystyle \first{q_1} = \frac{\first{u_1}}{\|\first{u_1}\|} = \frac{1}{\sqrt{2}}\first{\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}} = \first{\begin{bmatrix} \tfrac{1}{\sqrt{2}} \\ \tfrac{1}{\sqrt{2}} \\ 0 \end{bmatrix}}
Divide u2\second{u_2} by its length ∥u2∥\|\second{u_2}\|Compute ∥u2∥=(−1)2+12+02=2\|\second{u_2}\| = \sqrt{(\second{-1})^2 + \second{1}^2 + \second{0}^2} = \sqrt{2} and divide each entry of u2\second{u_2} by 2\sqrt{2}.
q2=u2∥u2∥=12[−110]=[−12120]\displaystyle \second{q_2} = \frac{\second{u_2}}{\|\second{u_2}\|} = \frac{1}{\sqrt{2}}\second{\begin{bmatrix} -1 \\ 1 \\ 0 \end{bmatrix}} = \second{\begin{bmatrix} -\tfrac{1}{\sqrt{2}} \\ \tfrac{1}{\sqrt{2}} \\ 0 \end{bmatrix}}
Divide u3\third{u_3} by its length ∥u3∥\|\third{u_3}\|Compute ∥u3∥=02+02+52=5\|\third{u_3}\| = \sqrt{\third{0}^2 + \third{0}^2 + \third{5}^2} = 5 and divide each entry of u3\third{u_3} by 55.
q3=u3∥u3∥=15[005]=[001]\displaystyle \third{q_3} = \frac{\third{u_3}}{\|\third{u_3}\|} = \frac{1}{5}\third{\begin{bmatrix} 0 \\ 0 \\ 5 \end{bmatrix}} = \third{\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}}

So {q1,q2,q3}\{\first{q_1}, \second{q_2}, \third{q_3}\} is an orthonormal basis for R3\mathbb{R}^3, keeping the exact same 90∘90^\circ directions as {u1,u2,u3}\{\first{u_1}, \second{u_2}, \third{u_3}\} with every vector now of length 11.

5

Finding coordinates in an orthonormal basis

Now we want to find the coordinates c1,c2,c3c_1, c_2, c_3 of the same target vector v=[2610]\target{v} = \target{\begin{bmatrix} 2 \\ 6 \\ 10 \end{bmatrix}} in our orthonormal basis {q1,q2,q3}\{\first{q_1}, \second{q_2}, \third{q_3}\}:

v\displaystyle \target{v} =c1q1+c2q2+c3q3\displaystyle = c_1 \first{q_1} + c_2 \second{q_2} + c_3 \third{q_3}
Finding coordinates in an orthonormal basis
c1=v⋅q1q1⋅q1q1⋅q1=  ?\displaystyle \begin{aligned} c_1 &= \frac{\target{v}\cdot\first{q_1}}{\first{q_1}\cdot\first{q_1}} \\[6pt] \first{q_1}\cdot\first{q_1} &= \;? \end{aligned}

Because every qi\first{q_i} is a unit vector of length 11, its dot product with itself is qi⋅qi=12=1\first{q_i} \cdot \first{q_i} = 1^2 = 1, and the denominator in the coordinate formula disappears:

ci\displaystyle c_i =v⋅qiqi⋅qi⏟1\displaystyle = \frac{\target{v} \cdot \first{q_i}}{\underbrace{\first{q_i} \cdot \first{q_i}}_{1}} =v⋅qi\displaystyle = \target{v} \cdot \first{q_i}

Every vector v\target{v} is built directly from these dot products:

v\displaystyle \target{v} =(v⋅q1)q1+(v⋅q2)q2+(v⋅q3)q3\displaystyle = (\target{v} \cdot \first{q_1})\first{q_1} + (\target{v} \cdot \second{q_2})\second{q_2} + (\target{v} \cdot \third{q_3})\third{q_3}
Text solution
Dot v\target{v} with q1\first{q_1} to get c1c_1Multiply matching entries of v=[2610]\target{v} = \target{\begin{bmatrix} 2 \\ 6 \\ 10 \end{bmatrix}} and q1=[12120]\first{q_1} = \first{\begin{bmatrix} \tfrac{1}{\sqrt{2}} \\ \tfrac{1}{\sqrt{2}} \\ 0 \end{bmatrix}}.
c1=v⋅q1=(2)(12)+(6)(12)+(10)(0)=82=42\displaystyle \begin{aligned} c_1 &= \target{v} \cdot \first{q_1} \\ &= (\target{2})\left(\first{\tfrac{1}{\sqrt{2}}}\right) + (\target{6})\left(\first{\tfrac{1}{\sqrt{2}}}\right) \\ &\qquad + (\target{10})(\first{0}) \\ &= \frac{8}{\sqrt{2}} = 4\sqrt{2} \end{aligned}
Dot v\target{v} with q2\second{q_2} to get c2c_2Multiply matching entries of v\target{v} and q2=[−12120]\second{q_2} = \second{\begin{bmatrix} -\tfrac{1}{\sqrt{2}} \\ \tfrac{1}{\sqrt{2}} \\ 0 \end{bmatrix}}.
c2=v⋅q2=(2)(−12)+(6)(12)+(10)(0)=42=22\displaystyle \begin{aligned} c_2 &= \target{v} \cdot \second{q_2} \\ &= (\target{2})\left(\second{-\tfrac{1}{\sqrt{2}}}\right) + (\target{6})\left(\second{\tfrac{1}{\sqrt{2}}}\right) \\ &\qquad + (\target{10})(\second{0}) \\ &= \frac{4}{\sqrt{2}} = 2\sqrt{2} \end{aligned}
Dot v\target{v} with q3\third{q_3} to get c3c_3Multiply matching entries of v\target{v} and q3=[001]\third{q_3} = \third{\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}}.
c3\displaystyle c_3 =v⋅q3\displaystyle = \target{v} \cdot \third{q_3} =(2)(0)+(6)(0)+(10)(1)\displaystyle = (\target{2})(\third{0}) + (\target{6})(\third{0}) + (\target{10})(\third{1}) =10\displaystyle = 10
Multiply the new coordinates by the unit vectors to check the piecesThe 2\sqrt{2} factors cancel out, giving the exact same projected vectors along each axis as before.
v=42[12120]+22[−12120]+10[001]=[440]+[−220]+[0010]=[2610]\displaystyle \begin{aligned} \target{v} &= 4\sqrt{2}\first{\begin{bmatrix} \tfrac{1}{\sqrt{2}} \\ \tfrac{1}{\sqrt{2}} \\ 0 \end{bmatrix}} + 2\sqrt{2}\second{\begin{bmatrix} -\tfrac{1}{\sqrt{2}} \\ \tfrac{1}{\sqrt{2}} \\ 0 \end{bmatrix}} \\ &\qquad + 10\third{\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}} \\ &= \first{\begin{bmatrix} 4 \\ 4 \\ 0 \end{bmatrix}} + \second{\begin{bmatrix} -2 \\ 2 \\ 0 \end{bmatrix}} + \third{\begin{bmatrix} 0 \\ 0 \\ 10 \end{bmatrix}} \\ &= \target{\begin{bmatrix} 2 \\ 6 \\ 10 \end{bmatrix}} \end{aligned}

So the coordinates of v\target{v} in {q1,q2,q3}\{\first{q_1}, \second{q_2}, \third{q_3}\} are 424\sqrt{2}, 222\sqrt{2}, and 1010: dividing the length of each basis vector ui\first{u_i} by ∥ui∥\|\first{u_i}\| means we need ∥ui∥\|\first{u_i}\| times as many copies of qi\first{q_i} to get the same projected piece.

Try changing the length of u1\first{u_1} below to see how the coordinate c1c_1 adjusts so the vector piece c1u1c_1 \first{u_1} stays the same, and what happens right when ∥u1∥=1\|\first{u_1}\| = 1.

Changing the basis vector's length changes how many copies it takes to reach the projection, while the projected piece itself never moves.
6

Matrices with orthonormal columns

When we put an orthonormal set of vectors q1,q2,q3\first{q_1}, \second{q_2}, \third{q_3} into the columns of a matrix QQ, we want to see what happens when we multiply QQ on the left by its transpose QTQ^T.

Taking the transpose turns the columns of QQ into rows, so multiplying rows by columns in QTQQ^T Q takes the dot product of every pair of vectors.

Matrices with orthonormal columns
Q=Q =
q1\htmlClass{lc-green}{q_1}
q2\htmlClass{lc-pink}{q_2}
q3\htmlClass{lc-violet}{q_3}
QTQ=  ?Q^TQ = \;?

Down the main diagonal, each unit vector dots with itself (qi⋅qi=1\first{q_i} \cdot \first{q_i} = 1), and off the diagonal, orthogonal vectors dot with each other (qi⋅qj=0\first{q_i} \cdot \second{q_j} = 0):

QTQ=[q1⋅q1q1⋅q2q1⋅q3q2⋅q1q2⋅q2q2⋅q3q3⋅q1q3⋅q2q3⋅q3]=[100010001]=I\displaystyle \begin{aligned} Q^T Q &= \begin{bmatrix} \first{q_1} \cdot \first{q_1} & \first{q_1} \cdot \second{q_2} & \first{q_1} \cdot \third{q_3} \\ \second{q_2} \cdot \first{q_1} & \second{q_2} \cdot \second{q_2} & \second{q_2} \cdot \third{q_3} \\ \third{q_3} \cdot \first{q_1} & \third{q_3} \cdot \second{q_2} & \third{q_3} \cdot \third{q_3} \end{bmatrix} \\ &= \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} = I \end{aligned}

When QQ is a square matrix (meaning its columns form an orthonormal basis), its inverse is simply its transpose:

Q−1\displaystyle Q^{-1} =QT\displaystyle = Q^T

Key concepts

Everything here starts from checking whether vectors sit at 90∘90^\circ to each other. A set of vectors is orthogonal when every pair has dot product 00 (ui⋅uj=0\first{u_i} \cdot \second{u_j} = 0 for i≠ji \neq j); for three vectors, make sure to check all three pairs u1⋅u2\first{u_1} \cdot \second{u_2}, u1⋅u3\first{u_1} \cdot \third{u_3}, and u2⋅u3\second{u_2} \cdot \third{u_3}. Because each perpendicular vector points into a brand-new dimension, non-zero orthogonal vectors are automatically linearly independent, so nn of them in Rn\mathbb{R}^n automatically form an orthogonal basis (such as u1=[110]\first{u_1} = \first{\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}}, u2=[−110]\second{u_2} = \second{\begin{bmatrix} -1 \\ 1 \\ 0 \end{bmatrix}}, u3=[005]\third{u_3} = \third{\begin{bmatrix} 0 \\ 0 \\ 5 \end{bmatrix}} in R3\mathbb{R}^3).

When finding coordinates in a basis, first check whether the basis is orthogonal, and then whether its vectors have length 11:

Key concept

If the basis vectors are not orthogonal (like b1=[12]\first{b_1} = \first{\begin{bmatrix} 1 \\ 2 \end{bmatrix}} and b2=[31]\second{b_2} = \second{\begin{bmatrix} 3 \\ 1 \end{bmatrix}}), moving along one changes your position along the other; the dot-product coordinate formulas do not apply, and you must solve the system by row reduction.

Key concept

In an orthogonal basis {u1,…,un}\{u_1, \dots, u_n\} where lengths are not 11, find any coordinate on its own by dividing by ui⋅ui\first{u_i} \cdot \first{u_i}:

ci=v⋅uiui⋅ui,projected piece=ciui\displaystyle \begin{aligned} c_i &= \frac{\target{v} \cdot \first{u_i}}{\first{u_i} \cdot \first{u_i}}, \\ \text{projected piece} &= c_i \first{u_i} \end{aligned}
For v=[2610]\target{v} = \target{\begin{bmatrix} 2 \\ 6 \\ 10 \end{bmatrix}} along u3=[005]\third{u_3} = \third{\begin{bmatrix} 0 \\ 0 \\ 5 \end{bmatrix}}, c3=5025=2c_3 = \frac{50}{25} = 2 and the vector piece is 2[005]=[0010]2\third{\begin{bmatrix} 0 \\ 0 \\ 5 \end{bmatrix}} = \third{\begin{bmatrix} 0 \\ 0 \\ 10 \end{bmatrix}}. If a problem only asks for coordinates in an orthogonal basis {u1,u2,u3}\{\first{u_1}, \second{u_2}, \third{u_3}\}, use v⋅uiui⋅ui\frac{\target{v} \cdot \first{u_i}}{\first{u_i} \cdot \first{u_i}} directly, and don't normalize the basis into square roots first.

Key concept

In an orthonormal basis {q1,…,qn}\{q_1, \dots, q_n\} where each vector has already been normalized to length 11 (qi=ui∥ui∥\first{q_i} = \frac{\first{u_i}}{\|\first{u_i}\|}, so qi⋅qi=1\first{q_i} \cdot \first{q_i} = 1), the denominator disappears:

ci\displaystyle c_i =v⋅qi\displaystyle = \target{v} \cdot \first{q_i}
For v=[2610]\target{v} = \target{\begin{bmatrix} 2 \\ 6 \\ 10 \end{bmatrix}} along q3=[001]\third{q_3} = \third{\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}}, c3=v⋅q3=10c_3 = \target{v} \cdot \third{q_3} = 10. Dividing the length of ui\first{u_i} by ∥ui∥\|\first{u_i}\| multiplies its coordinate by ∥ui∥\|\first{u_i}\|, while the vector piece ciui=(v⋅qi)qic_i \first{u_i} = (\target{v} \cdot \first{q_i})\first{q_i} stays the same.

Key concept

When orthonormal vectors are put into the columns of a matrix QQ, the dot products of rows and columns give QTQ=IQ^T Q = I. If QQ is square, Q−1=QTQ^{-1} = Q^T and Qx=bQx = \target{b} has the immediate solution x=QTbx = Q^T \target{b}.

7

Practice

1

Let u1=[12−1]\first{u_1} = \first{\begin{bmatrix} 1 \\ 2 \\ -1 \end{bmatrix}}, u2=[2−10]\second{u_2} = \second{\begin{bmatrix} 2 \\ -1 \\ 0 \end{bmatrix}}, and u3=[012]\third{u_3} = \third{\begin{bmatrix} 0 \\ 1 \\ 2 \end{bmatrix}}. Notice that u1⋅u2=0\first{u_1} \cdot \second{u_2} = 0 and u1⋅u3=0\first{u_1} \cdot \third{u_3} = 0. Is {u1,u2,u3}\{\first{u_1}, \second{u_2}, \third{u_3}\} an orthogonal set?

How many pairs do three vectors make in total, and which pair hasn't been tested yet?

Stopping after the two given dot products u1⋅u2=0\first{u_1} \cdot \second{u_2} = 0 and u1⋅u3=0\first{u_1} \cdot \third{u_3} = 0 gives the wrong answer "Yes"; in an orthogonal set, every pair must have a dot product of 00, so we must also check u2⋅u3\second{u_2} \cdot \third{u_3}:
u2⋅u3\displaystyle \second{u_2} \cdot \third{u_3} =(2)(0)+(−1)(1)+(0)(2)\displaystyle = (\second{2})(\third{0}) + (\second{-1})(\third{1}) + (\second{0})(\third{2}) =−1≠0\displaystyle = \fails{-1 \neq 0}
No
2

Let u1=[120]\first{u_1} = \first{\begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix}}, u2=[−215]\second{u_2} = \second{\begin{bmatrix} -2 \\ 1 \\ 5 \end{bmatrix}}, and u3=[2ab]\third{u_3} = \third{\begin{bmatrix} 2 \\ a \\ b \end{bmatrix}}. Find the values of aa and bb that make {u1,u2,u3}\{\first{u_1}, \second{u_2}, \third{u_3}\} an orthogonal set, and explain why {u1,u2,u3}\{\first{u_1}, \second{u_2}, \third{u_3}\} is then guaranteed to be a basis for R3\mathbb{R}^3.

Between u1⋅u3=0\first{u_1} \cdot \third{u_3} = 0 and u2⋅u3=0\second{u_2} \cdot \third{u_3} = 0, which equation contains only one unknown so you can solve it first?

First note that u1⋅u2\first{u_1} \cdot \second{u_2} =(1)(−2)+(2)(1)+(0)(5){}= (\first{1})(\second{-2}) + (\first{2})(\second{1}) + (\first{0})(\second{5}) =0{}= 0. Setting u1⋅u3=0\first{u_1} \cdot \third{u_3} = 0 gives
u1⋅u3\displaystyle \first{u_1} \cdot \third{u_3} =(1)(2)+(2)(a)+(0)(b)\displaystyle = (\first{1})(\third{2}) + (\first{2})(\third{a}) + (\first{0})(\third{b}) =2+2a\displaystyle = 2 + 2a =0  ⟹  a\displaystyle = 0 \implies a =−1\displaystyle = -1
(Watch the sign when moving 22 across the equals sign: writing a=1a = \fails{1} here carries into the second equation and gives the wrong value b=35b = \fails{\tfrac{3}{5}}.) Substituting a=−1a = -1 into u2⋅u3=0\second{u_2} \cdot \third{u_3} = 0 gives
u2⋅u3\displaystyle \second{u_2} \cdot \third{u_3} =(−2)(2)+(1)(−1)+(5)(b)\displaystyle = (\second{-2})(\third{2}) + (\second{1})(\third{-1}) + (\second{5})(\third{b}) =−4−1+5b\displaystyle = -4 - 1 + 5b =0  ⟹  b\displaystyle = 0 \implies b =1\displaystyle = 1
Because any orthogonal set of non-zero vectors is automatically linearly independent, 33 non-zero orthogonal vectors in R3\mathbb{R}^3 automatically form a basis for R3\mathbb{R}^3, with a=−1a = -1 and b=1b = 1.
3

The vectors u1=[1−10]\first{u_1} = \first{\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}}, u2=[111]\second{u_2} = \second{\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}}, and u3=[11−2]\third{u_3} = \third{\begin{bmatrix} 1 \\ 1 \\ -2 \end{bmatrix}} form an orthogonal basis for R3\mathbb{R}^3. Find the coordinates c1,c2,c3c_1, c_2, c_3 of v=[4−24]\target{v} = \target{\begin{bmatrix} 4 \\ -2 \\ 4 \end{bmatrix}} in this basis.

Check the lengths of u1,u2,u3\first{u_1}, \second{u_2}, \third{u_3}: are they unit vectors, or do you need the denominator ui⋅ui\first{u_i} \cdot \first{u_i} for each coordinate?

Use ci=v⋅uiui⋅uic_i = \frac{\target{v} \cdot \first{u_i}}{\first{u_i} \cdot \first{u_i}}. Do not leave off the denominator ui⋅ui\first{u_i} \cdot \first{u_i} as if the basis were orthonormal (which gives 6,6,−6\fails{6, 6, -6}), and divide by ui⋅ui\first{u_i} \cdot \first{u_i} rather than the square-root length ∥ui∥\|\first{u_i}\| (which gives 32,23,−6\fails{3\sqrt{2}, 2\sqrt{3}, -\sqrt{6}}):
c1\displaystyle c_1 =v⋅u1u1⋅u1\displaystyle = \frac{\target{v} \cdot \first{u_1}}{\first{u_1} \cdot \first{u_1}} =(4)(1)+(−2)(−1)+(4)(0)12+(−1)2+02\displaystyle = \frac{(\target{4})(\first{1}) + (\target{-2})(\first{-1}) + (\target{4})(\first{0})}{\first{1}^2 + (\first{-1})^2 + \first{0}^2} =62\displaystyle = \frac{6}{2} =3\displaystyle = 3
c2\displaystyle c_2 =v⋅u2u2⋅u2\displaystyle = \frac{\target{v} \cdot \second{u_2}}{\second{u_2} \cdot \second{u_2}} =(4)(1)+(−2)(1)+(4)(1)12+12+12\displaystyle = \frac{(\target{4})(\second{1}) + (\target{-2})(\second{1}) + (\target{4})(\second{1})}{\second{1}^2 + \second{1}^2 + \second{1}^2} =63\displaystyle = \frac{6}{3} =2\displaystyle = 2
c3\displaystyle c_3 =v⋅u3u3⋅u3\displaystyle = \frac{\target{v} \cdot \third{u_3}}{\third{u_3} \cdot \third{u_3}} =(4)(1)+(−2)(1)+(4)(−2)12+12+(−2)2\displaystyle = \frac{(\target{4})(\third{1}) + (\target{-2})(\third{1}) + (\target{4})(\third{-2})}{\third{1}^2 + \third{1}^2 + (\third{-2})^2} =−66\displaystyle = \frac{-6}{6} =−1\displaystyle = -1
4

Suppose {u1,u2,u3}\{\first{u_1}, \second{u_2}, \third{u_3}\} is an orthogonal basis for R3\mathbb{R}^3 with u1=[2−12]\first{u_1} = \first{\begin{bmatrix} 2 \\ -1 \\ 2 \end{bmatrix}}, and u2\second{u_2} and u3\third{u_3} are not given. If v=[7−21]\target{v} = \target{\begin{bmatrix} 7 \\ -2 \\ 1 \end{bmatrix}} is written as v=c1u1+c2u2+c3u3\target{v} = c_1 \first{u_1} + c_2 \second{u_2} + c_3 \third{u_3}, find the coordinate c1c_1 and the individual vector piece c1u1c_1 \first{u_1} along u1\first{u_1}.

What happens to the c2u2c_2 \second{u_2} and c3u3c_3 \third{u_3} terms when you dot both sides of v=c1u1+c2u2+c3u3\target{v} = c_1 \first{u_1} + c_2 \second{u_2} + c_3 \third{u_3} with u1\first{u_1}?

Dotting both sides with u1\first{u_1} zeroes out the u2\second{u_2} and u3\third{u_3} terms. Make sure to divide v⋅u1=18\target{v} \cdot \first{u_1} = 18 by u1⋅u1=9\first{u_1} \cdot \first{u_1} = 9 rather than by the length ∥u1∥=3\|\first{u_1}\| = 3 (which gives the wrong coordinate c1=6c_1 = \fails{6} and the wrong vector piece [12−612]\fails{\begin{bmatrix} 12 \\ -6 \\ 12 \end{bmatrix}}):
c1\displaystyle c_1 =v⋅u1u1⋅u1\displaystyle = \frac{\target{v} \cdot \first{u_1}}{\first{u_1} \cdot \first{u_1}} =(7)(2)+(−2)(−1)+(1)(2)22+(−1)2+22\displaystyle = \frac{(\target{7})(\first{2}) + (\target{-2})(\first{-1}) + (\target{1})(\first{2})}{\first{2}^2 + (\first{-1})^2 + \first{2}^2} =189\displaystyle = \frac{18}{9} =2\displaystyle = 2
Multiply c1=2c_1 = 2 by the basis vector u1\first{u_1} (not by v\target{v}) to get the projected vector piece:
c1u1=2[2−12]=[4−24]\displaystyle c_1 \first{u_1} = 2\first{\begin{bmatrix} 2 \\ -1 \\ 2 \end{bmatrix}} = \first{\begin{bmatrix} 4 \\ -2 \\ 4 \end{bmatrix}}
5

Turn the orthogonal basis u1=[340]\first{u_1} = \first{\begin{bmatrix} 3 \\ 4 \\ 0 \end{bmatrix}}, u2=[−430]\second{u_2} = \second{\begin{bmatrix} -4 \\ 3 \\ 0 \end{bmatrix}}, u3=[002]\third{u_3} = \third{\begin{bmatrix} 0 \\ 0 \\ 2 \end{bmatrix}} into an orthonormal basis {q1,q2,q3}\{\first{q_1}, \second{q_2}, \third{q_3}\} for R3\mathbb{R}^3, and find the coordinates of v=[105−6]\target{v} = \target{\begin{bmatrix} 10 \\ 5 \\ -6 \end{bmatrix}} in {q1,q2,q3}\{\first{q_1}, \second{q_2}, \third{q_3}\}.

Once you divide each ui\first{u_i} by its length ∥ui∥\|\first{u_i}\| to get qi\first{q_i}, what is the denominator qi⋅qi\first{q_i} \cdot \first{q_i} when finding the coordinates in {q1,q2,q3}\{\first{q_1}, \second{q_2}, \third{q_3}\}?

Divide each vector by its length (∥u1∥=32+42=5\|\first{u_1}\| = \sqrt{\first{3}^2 + \first{4}^2} = 5, ∥u2∥=(−4)2+32=5\|\second{u_2}\| = \sqrt{(\second{-4})^2 + \second{3}^2} = 5, ∥u3∥=2\|\third{u_3}\| = 2):
q1=[35450],q2=[−45350],q3=[001]\displaystyle \first{q_1} = \first{\begin{bmatrix} \tfrac{3}{5} \\ \tfrac{4}{5} \\ 0 \end{bmatrix}}, \qquad \second{q_2} = \second{\begin{bmatrix} -\tfrac{4}{5} \\ \tfrac{3}{5} \\ 0 \end{bmatrix}}, \qquad \third{q_3} = \third{\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}}
Because {q1,q2,q3}\{\first{q_1}, \second{q_2}, \third{q_3}\} is now orthonormal, each coordinate is simply ci=v⋅qic_i = \target{v} \cdot \first{q_i} (do not divide by ∥ui∥\|\first{u_i}\| a second time, which gives the old {u1,u2,u3}\{\first{u_1}, \second{u_2}, \third{u_3}\} coordinates 2,−1,−3\fails{2, -1, -3}):
c1\displaystyle c_1 =v⋅q1\displaystyle = \target{v} \cdot \first{q_1} =(10)(35)+(5)(45)+(−6)(0)\displaystyle = (\target{10})\left(\first{\frac{3}{5}}\right) + (\target{5})\left(\first{\frac{4}{5}}\right) + (\target{-6})(\first{0}) =10\displaystyle = 10
c2\displaystyle c_2 =v⋅q2\displaystyle = \target{v} \cdot \second{q_2} =(10)(−45)+(5)(35)+(−6)(0)\displaystyle = (\target{10})\left(\second{-\frac{4}{5}}\right) + (\target{5})\left(\second{\frac{3}{5}}\right) + (\target{-6})(\second{0}) =−5\displaystyle = -5
c3\displaystyle c_3 =v⋅q3\displaystyle = \target{v} \cdot \third{q_3} =(10)(0)+(5)(0)+(−6)(1)\displaystyle = (\target{10})(\third{0}) + (\target{5})(\third{0}) + (\target{-6})(\third{1}) =−6\displaystyle = -6
6

A vector v\target{v} in R3\mathbb{R}^3 has coordinates (3,−2,5)(3, -2, 5) in an orthogonal basis {u1,u2,u3}\{\first{u_1}, \second{u_2}, \third{u_3}\} where ∥u1∥=2\|\first{u_1}\| = 2, ∥u2∥=6\|\second{u_2}\| = 6, and ∥u3∥=3\|\third{u_3}\| = 3. When {u1,u2,u3}\{\first{u_1}, \second{u_2}, \third{u_3}\} is normalized into the orthonormal basis {q1,q2,q3}\{\first{q_1}, \second{q_2}, \third{q_3}\} with qi=ui∥ui∥\first{q_i} = \tfrac{\first{u_i}}{\|\first{u_i}\|}, what are the coordinates of v\target{v} in {q1,q2,q3}\{\first{q_1}, \second{q_2}, \third{q_3}\}?

If a basis vector u1\first{u_1} of length 22 is replaced by a unit vector q1\first{q_1} of length 11, do you need twice as many copies or half as many copies to build the same vector piece?

Because each unit vector qi=ui∥ui∥\first{q_i} = \tfrac{\first{u_i}}{\|\first{u_i}\|} is 1∥ui∥\frac{1}{\|\first{u_i}\|} times as long as ui\first{u_i}, reaching the exact same vector pieces takes ∥ui∥\|\first{u_i}\| times as many copies (do not divide the coordinates by the lengths ∥ui∥\|\first{u_i}\|, which gives (32,−13,53)\fails{(\tfrac{3}{2}, -\tfrac{1}{3}, \tfrac{5}{3})}):
c1=3×2=6,c2=−2×6=−12,c3=5×3=15\displaystyle \begin{aligned} c_1 &= 3 \times 2 = 6, \\ c_2 &= -2 \times 6 = -12, \\ c_3 &= 5 \times 3 = 15 \end{aligned}
(6,−12,15)(6, -12, 15)
7

The columns of Q=[13121613−1216130−26]Q = \begin{bmatrix} \first{\tfrac{1}{\sqrt{3}}} & \second{\tfrac{1}{\sqrt{2}}} & \third{\tfrac{1}{\sqrt{6}}} \\ \first{\tfrac{1}{\sqrt{3}}} & \second{-\tfrac{1}{\sqrt{2}}} & \third{\tfrac{1}{\sqrt{6}}} \\ \first{\tfrac{1}{\sqrt{3}}} & \second{0} & \third{-\tfrac{2}{\sqrt{6}}} \end{bmatrix} form an orthonormal basis for R3\mathbb{R}^3. Without row reduction, find QTQQ^T Q, Q−1Q^{-1}, and the solution xx to Qx=[600]Qx = \target{\begin{bmatrix} 6 \\ 0 \\ 0 \end{bmatrix}}.

For a square matrix QQ with orthonormal columns, how is Q−1Q^{-1} related to QTQ^T, and what happens when you multiply Qx=[600]Qx = \target{\begin{bmatrix} 6 \\ 0 \\ 0 \end{bmatrix}} on the left by QTQ^T?

Because the columns of QQ are orthonormal and QQ is square:
QTQ=I=[100010001],Q−1=QT=[13131312−1201616−26]\displaystyle \begin{aligned} Q^T Q &= I = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}, \\ Q^{-1} &= Q^T = \begin{bmatrix} \first{\tfrac{1}{\sqrt{3}}} & \first{\tfrac{1}{\sqrt{3}}} & \first{\tfrac{1}{\sqrt{3}}} \\ \second{\tfrac{1}{\sqrt{2}}} & \second{-\tfrac{1}{\sqrt{2}}} & \second{0} \\ \third{\tfrac{1}{\sqrt{6}}} & \third{\tfrac{1}{\sqrt{6}}} & \third{-\tfrac{2}{\sqrt{6}}} \end{bmatrix} \end{aligned}
To solve Qx=[600]Qx = \target{\begin{bmatrix} 6 \\ 0 \\ 0 \end{bmatrix}}, multiply on the left by QTQ^T (make sure to multiply QT[600]Q^T\target{\begin{bmatrix} 6 \\ 0 \\ 0 \end{bmatrix}} rather than Q[600]Q\target{\begin{bmatrix} 6 \\ 0 \\ 0 \end{bmatrix}}, which picks the first column of QQ instead of the first column of QTQ^T and gives [232323]\fails{\begin{bmatrix} 2\sqrt{3} \\ 2\sqrt{3} \\ 2\sqrt{3} \end{bmatrix}}):
x=QT[600]=[13131312−1201616−26][600]=[636266]=[23326]\displaystyle \begin{aligned} x &= Q^T \target{\begin{bmatrix} 6 \\ 0 \\ 0 \end{bmatrix}} \\ &= \begin{bmatrix} \first{\tfrac{1}{\sqrt{3}}} & \first{\tfrac{1}{\sqrt{3}}} & \first{\tfrac{1}{\sqrt{3}}} \\ \second{\tfrac{1}{\sqrt{2}}} & \second{-\tfrac{1}{\sqrt{2}}} & \second{0} \\ \third{\tfrac{1}{\sqrt{6}}} & \third{\tfrac{1}{\sqrt{6}}} & \third{-\tfrac{2}{\sqrt{6}}} \end{bmatrix} \target{\begin{bmatrix} 6 \\ 0 \\ 0 \end{bmatrix}} \\ &= \begin{bmatrix} \tfrac{6}{\sqrt{3}} \\ \tfrac{6}{\sqrt{2}} \\ \tfrac{6}{\sqrt{6}} \end{bmatrix} = \begin{bmatrix} 2\sqrt{3} \\ 3\sqrt{2} \\ \sqrt{6} \end{bmatrix} \end{aligned}