Linear AlgebraOrthogonality and Projections7 of 9
Orthogonality and Projections · 7

Gram-Schmidt: Building an Orthonormal Basis

When basis vectors come out of real data or a system of equations, they are usually at awkward angles and have uneven lengths, so we need a way to replace them with perpendicular unit vectors that span the exact same space.

Why We Want an Orthonormal Basis

An orthonormal basis is a set of basis vectors that are orthogonal (at 90∘90^\circ angles to each other, so moving along one axis has zero effect on your position along the others) and normalized (each scaled to a unit vector of length 11). Gram-Schmidt builds an orthonormal basis one vector at a time.

1

Building an Orthonormal Basis in 2D

Given two vectors a\first{a} and b\second{b} that span a flat 2D plane, we want to build two perpendicular unit vectors q1\first{q_1} and q2\second{q_2} that span that same plane.

How Gram-Schmidt Works in 2D
a\first{a}
b\second{b}
q1, q2=  ?\first{q_1},\ \second{q_2} = \;?

The first direction has nothing to be perpendicular to yet, so we get our first axis q1\first{q_1} simply by dividing a\first{a} by its length:

q1\displaystyle \first{q_1} =a∥a∥\displaystyle = \frac{\first{a}}{\|\first{a}\|}

The second vector b\second{b} is tilted toward q1\first{q_1}. Because q1\first{q_1} is already a unit vector, the shadow of b\second{b} along the line of q1\first{q_1} is (b⋅q1)q1\shadow{(b \cdot q_1)q_1}: it has length ∣b⋅q1∣|\second{b} \cdot \first{q_1}|, and it points in the direction of q1\first{q_1} when b⋅q1\second{b} \cdot \first{q_1} is positive and the opposite way when b⋅q1\second{b} \cdot \first{q_1} is negative. Subtracting this projection removes all of b\second{b}'s movement along the first axis, leaving only the piece perpendicular to q1\first{q_1}:

bperp\displaystyle \perpPiece{b_{\text{perp}}} =b−(b⋅q1)q1⏟projection onto q1\displaystyle = \second{b} - \underbrace{\shadow{(b \cdot q_1)q_1}}_{\text{projection onto } \first{q_1}}
q1\first{q_1}
b\second{b}
(b⋅q1)q1\shadow{(b \cdot q_1)q_1}
bperp\perpPiece{b_{\text{perp}}}

Dividing that perpendicular piece by its length gives the second unit vector:

q2\displaystyle \second{q_2} =bperp∥bperp∥\displaystyle = \frac{\perpPiece{b_{\text{perp}}}}{\|\perpPiece{b_{\text{perp}}}\|}

Try dragging b\second{b} around the plane to see how bperp\perpPiece{b_{\text{perp}}} and q2\second{q_2} stay at a 90∘90^\circ angle to q1\first{q_1}, and what happens when b\second{b} lines up directly with q1\first{q_1}.

Removing the shadow of the second vector along the first axis leaves a piece that always sits at a right angle to it, giving us our second axis.

Let's use these steps to find an orthonormal basis q1,q2\first{q_1}, \second{q_2} for the plane spanned by a=⟨3,4⟩\first{a} = \first{\langle 3, 4 \rangle} and b=⟨−1,7⟩\second{b} = \second{\langle -1, 7 \rangle}.

2D Example: Orthonormalizing Two Vectors
−2\htmlClass{lc-mute}{-2}
2\htmlClass{lc-mute}{2}
3\htmlClass{lc-mute}{3}
4\htmlClass{lc-mute}{4}
2\htmlClass{lc-mute}{2}
3\htmlClass{lc-mute}{3}
4\htmlClass{lc-mute}{4}
5\htmlClass{lc-mute}{5}
6\htmlClass{lc-mute}{6}
7\htmlClass{lc-mute}{7}
a\first{a}
b\second{b}
q1, q2=  ?\first{q_1},\ \second{q_2} = \;?
Text solution
Normalize a\first{a} to get q1\first{q_1}Find the length of a=⟨3,4⟩\first{a} = \first{\langle 3, 4 \rangle} and divide a\first{a} by that length to make our first unit vector.
∥a∥\displaystyle \|\first{a}\| =32+42\displaystyle = \sqrt{\first{3}^2 + \first{4}^2} =9+16\displaystyle = \sqrt{9 + 16} =25\displaystyle = \sqrt{25} =5\displaystyle = 5
q1\displaystyle \first{q_1} =a∥a∥\displaystyle = \frac{\first{a}}{\|\first{a}\|} =15⟨3,4⟩\displaystyle = \frac{1}{5}\first{\langle 3, 4 \rangle} =⟨35,45⟩\displaystyle = \first{\left\langle \tfrac{3}{5}, \tfrac{4}{5} \right\rangle}
Find the projection of b\second{b} onto q1\first{q_1}Take the dot product of b=⟨−1,7⟩\second{b} = \second{\langle -1, 7 \rangle} with the unit vector q1\first{q_1} to get the projection's length, then multiply that number by q1\first{q_1}.
b⋅q1\displaystyle \second{b} \cdot \first{q_1} =⟨−1,7⟩⋅⟨35,45⟩\displaystyle = \second{\langle -1, 7 \rangle} \cdot \first{\left\langle \tfrac{3}{5}, \tfrac{4}{5} \right\rangle} =−35+285\displaystyle = -\frac{3}{5} + \frac{28}{5} =255\displaystyle = \frac{25}{5} =5\displaystyle = 5
(b⋅q1)q1\displaystyle \shadow{(b \cdot q_1)q_1} =5⟨35,45⟩\displaystyle = 5\first{\left\langle \tfrac{3}{5}, \tfrac{4}{5} \right\rangle} =⟨3,4⟩\displaystyle = \shadow{\langle 3, 4 \rangle}
Subtract the projection from b\second{b}Subtract the projection vector from b\second{b} to keep only the piece perpendicular to q1\first{q_1}.
bperp\displaystyle \perpPiece{b_{\text{perp}}} =b−(b⋅q1)q1\displaystyle = \second{b} - \shadow{(b \cdot q_1)q_1} =⟨−1,7⟩−⟨3,4⟩\displaystyle = \second{\langle -1, 7 \rangle} - \shadow{\langle 3, 4 \rangle} =⟨−4,3⟩\displaystyle = \perpPiece{\langle -4, 3 \rangle}
Normalize bperp\perpPiece{b_{\text{perp}}} to get q2\second{q_2}Divide the perpendicular vector bperp\perpPiece{b_{\text{perp}}} by its length to turn it into a unit vector.
∥bperp∥\displaystyle \|\perpPiece{b_{\text{perp}}}\| =(−4)2+32\displaystyle = \sqrt{(\perpPiece{-4})^2 + \perpPiece{3}^2} =16+9\displaystyle = \sqrt{16 + 9} =25\displaystyle = \sqrt{25} =5\displaystyle = 5
q2\displaystyle \second{q_2} =bperp∥bperp∥\displaystyle = \frac{\perpPiece{b_{\text{perp}}}}{\|\perpPiece{b_{\text{perp}}}\|} =15⟨−4,3⟩\displaystyle = \frac{1}{5}\perpPiece{\langle -4, 3 \rangle} =⟨−45,35⟩\displaystyle = \second{\left\langle -\tfrac{4}{5}, \tfrac{3}{5} \right\rangle}

So q1=⟨35,45⟩\first{q_1} = \first{\left\langle \tfrac{3}{5}, \tfrac{4}{5} \right\rangle} and q2=⟨−45,35⟩\second{q_2} = \second{\left\langle -\tfrac{4}{5}, \tfrac{3}{5} \right\rangle} are perpendicular unit vectors that span the exact same plane as a\first{a} and b\second{b}.

2

Extending Gram-Schmidt to 3D

When we have three vectors a\first{a}, b\second{b}, and c\third{c} in 3D space, the first two vectors give us two perpendicular unit axes q1\first{q_1} and q2\second{q_2}, and we now want a third unit vector q3\third{q_3} that is perpendicular to both q1\first{q_1} and q2\second{q_2}, and therefore perpendicular to the entire plane they span.

Extending Gram-Schmidt to 3D

Because c\third{c} is tilted toward both q1\first{q_1} and q2\second{q_2}, we subtract its projections onto both unit vectors to remove its entire shadow in the q1–q2\first{q_1}\text{--}\second{q_2} plane:

cperp=c−((c⋅q1)q1+(c⋅q2)q2)⏟projection onto the q1–q2 plane=c−(c⋅q1)q1−(c⋅q2)q2\displaystyle \begin{aligned} \perpPiece{c_{\text{perp}}} &= \third{c} - \underbrace{\big(\shadow{(c \cdot q_1)q_1} + \shadow{(c \cdot q_2)q_2}\big)}_{\text{projection onto the } \first{q_1}\text{--}\second{q_2} \text{ plane}} \\ &= \third{c} - \shadow{(c \cdot q_1)q_1} - \shadow{(c \cdot q_2)q_2} \end{aligned}

Dividing the remaining vertical piece by its length gives the third unit vector:

q3\displaystyle \third{q_3} =cperp∥cperp∥\displaystyle = \frac{\perpPiece{c_{\text{perp}}}}{\|\perpPiece{c_{\text{perp}}}\|}

Move c\third{c} parallel to the plane at a fixed height to see how its two axis projections add up to its shadow in the plane, leaving cperp\perpPiece{c_{\text{perp}}} and q3\third{q_3} at a 90∘90^\circ angle to the plane.

Because the two axes are at right angles, a vector's projections along each axis change on their own and add up to its full shadow in the plane, so subtracting both leaves only the part perpendicular to the plane.

Now let's find an orthonormal basis q1,q2,q3\first{q_1}, \second{q_2}, \third{q_3} for the 3D space spanned by a=⟨1,2,2⟩\first{a} = \first{\langle 1, 2, 2 \rangle}, b=⟨3,3,0⟩\second{b} = \second{\langle 3, 3, 0 \rangle}, and c=⟨6,3,3⟩\third{c} = \third{\langle 6, 3, 3 \rangle}.

3D Example: Orthonormalizing Three Vectors
a=⟨1,2,2⟩b=⟨3,3,0⟩c=⟨6,3,3⟩q1, q2, q3=  ?\displaystyle \begin{aligned} \first{a} &= \first{\langle 1, 2, 2 \rangle} \\ \second{b} &= \second{\langle 3, 3, 0 \rangle} \\ \third{c} &= \third{\langle 6, 3, 3 \rangle} \\[8pt] \first{q_1},\ \second{q_2},\ \third{q_3} &= \;? \end{aligned}
Text solution
Normalize a\first{a} to get q1\first{q_1}Divide a=⟨1,2,2⟩\first{a} = \first{\langle 1, 2, 2 \rangle} by its length to get the first unit axis.
∥a∥\displaystyle \|\first{a}\| =12+22+22\displaystyle = \sqrt{\first{1}^2 + \first{2}^2 + \first{2}^2} =9\displaystyle = \sqrt{9} =3\displaystyle = 3
q1\displaystyle \first{q_1} =a∥a∥\displaystyle = \frac{\first{a}}{\|\first{a}\|} =13⟨1,2,2⟩\displaystyle = \frac{1}{3}\first{\langle 1, 2, 2 \rangle} =⟨13,23,23⟩\displaystyle = \first{\left\langle \tfrac{1}{3}, \tfrac{2}{3}, \tfrac{2}{3} \right\rangle}
Subtract the projection of b\second{b} onto q1\first{q_1} and normalize to get q2\second{q_2}Subtract the projection of b=⟨3,3,0⟩\second{b} = \second{\langle 3, 3, 0 \rangle} onto q1\first{q_1}, then divide the remaining perpendicular piece by its length.
b⋅q1\displaystyle \second{b} \cdot \first{q_1} =⟨3,3,0⟩⋅⟨13,23,23⟩\displaystyle = \second{\langle 3, 3, 0 \rangle} \cdot \first{\left\langle \tfrac{1}{3}, \tfrac{2}{3}, \tfrac{2}{3} \right\rangle} =1+2+0\displaystyle = 1 + 2 + 0 =3\displaystyle = 3
(b⋅q1)q1\displaystyle \shadow{(b \cdot q_1)q_1} =3⟨13,23,23⟩\displaystyle = 3\first{\left\langle \tfrac{1}{3}, \tfrac{2}{3}, \tfrac{2}{3} \right\rangle} =⟨1,2,2⟩\displaystyle = \shadow{\langle 1, 2, 2 \rangle}
bperp\displaystyle \perpPiece{b_{\text{perp}}} =⟨3,3,0⟩−⟨1,2,2⟩\displaystyle = \second{\langle 3, 3, 0 \rangle} - \shadow{\langle 1, 2, 2 \rangle} =⟨2,1,−2⟩\displaystyle = \perpPiece{\langle 2, 1, -2 \rangle}
∥bperp∥\displaystyle \|\perpPiece{b_{\text{perp}}}\| =22+12+(−2)2\displaystyle = \sqrt{\perpPiece{2}^2 + \perpPiece{1}^2 + (\perpPiece{-2})^2} =9\displaystyle = \sqrt{9} =3\displaystyle = 3
q2\displaystyle \second{q_2} =bperp∥bperp∥\displaystyle = \frac{\perpPiece{b_{\text{perp}}}}{\|\perpPiece{b_{\text{perp}}}\|} =13⟨2,1,−2⟩\displaystyle = \frac{1}{3}\perpPiece{\langle 2, 1, -2 \rangle} =⟨23,13,−23⟩\displaystyle = \second{\left\langle \tfrac{2}{3}, \tfrac{1}{3}, -\tfrac{2}{3} \right\rangle}
Subtract the projections of c\third{c} onto both q1\first{q_1} and q2\second{q_2}Project c=⟨6,3,3⟩\third{c} = \third{\langle 6, 3, 3 \rangle} onto each of the first two unit axes and subtract both projections from c\third{c}.
c⋅q1\displaystyle \third{c} \cdot \first{q_1} =⟨6,3,3⟩⋅⟨13,23,23⟩\displaystyle = \third{\langle 6, 3, 3 \rangle} \cdot \first{\left\langle \tfrac{1}{3}, \tfrac{2}{3}, \tfrac{2}{3} \right\rangle} =2+2+2\displaystyle = 2 + 2 + 2 =6\displaystyle = 6
(c⋅q1)q1\displaystyle \shadow{(c \cdot q_1)q_1} =6⟨13,23,23⟩\displaystyle = 6\first{\left\langle \tfrac{1}{3}, \tfrac{2}{3}, \tfrac{2}{3} \right\rangle} =⟨2,4,4⟩\displaystyle = \shadow{\langle 2, 4, 4 \rangle}
c⋅q2\displaystyle \third{c} \cdot \second{q_2} =⟨6,3,3⟩⋅⟨23,13,−23⟩\displaystyle = \third{\langle 6, 3, 3 \rangle} \cdot \second{\left\langle \tfrac{2}{3}, \tfrac{1}{3}, -\tfrac{2}{3} \right\rangle} =4+1−2\displaystyle = 4 + 1 - 2 =3\displaystyle = 3
(c⋅q2)q2\displaystyle \shadow{(c \cdot q_2)q_2} =3⟨23,13,−23⟩\displaystyle = 3\second{\left\langle \tfrac{2}{3}, \tfrac{1}{3}, -\tfrac{2}{3} \right\rangle} =⟨2,1,−2⟩\displaystyle = \shadow{\langle 2, 1, -2 \rangle}
cperp\displaystyle \perpPiece{c_{\text{perp}}} =⟨6,3,3⟩−⟨2,4,4⟩−⟨2,1,−2⟩\displaystyle = \third{\langle 6, 3, 3 \rangle} - \shadow{\langle 2, 4, 4 \rangle} - \shadow{\langle 2, 1, -2 \rangle} =⟨2,−2,1⟩\displaystyle = \perpPiece{\langle 2, -2, 1 \rangle}
Normalize cperp\perpPiece{c_{\text{perp}}} to get q3\third{q_3}Divide cperp\perpPiece{c_{\text{perp}}} by its length to get the third unit vector.
∥cperp∥\displaystyle \|\perpPiece{c_{\text{perp}}}\| =22+(−2)2+12\displaystyle = \sqrt{\perpPiece{2}^2 + (\perpPiece{-2})^2 + \perpPiece{1}^2} =9\displaystyle = \sqrt{9} =3\displaystyle = 3
q3\displaystyle \third{q_3} =cperp∥cperp∥\displaystyle = \frac{\perpPiece{c_{\text{perp}}}}{\|\perpPiece{c_{\text{perp}}}\|} =13⟨2,−2,1⟩\displaystyle = \frac{1}{3}\perpPiece{\langle 2, -2, 1 \rangle} =⟨23,−23,13⟩\displaystyle = \third{\left\langle \tfrac{2}{3}, -\tfrac{2}{3}, \tfrac{1}{3} \right\rangle}

So q1=⟨13,23,23⟩\first{q_1} = \first{\left\langle \tfrac{1}{3}, \tfrac{2}{3}, \tfrac{2}{3} \right\rangle}, q2=⟨23,13,−23⟩\second{q_2} = \second{\left\langle \tfrac{2}{3}, \tfrac{1}{3}, -\tfrac{2}{3} \right\rangle}, and q3=⟨23,−23,13⟩\third{q_3} = \third{\left\langle \tfrac{2}{3}, -\tfrac{2}{3}, \tfrac{1}{3} \right\rangle} form an orthonormal basis for 3D space, where every vector has length 11 and any pair has dot product 00.

3

Avoiding Square Roots by Normalizing at the End

In the first two examples, the lengths happened to be whole numbers (55 and 33). Most of the time, the very first vector has a length like 3\sqrt{3}, and normalizing right away means carrying square roots through every dot product. Or sometimes a question only asks for an orthogonal basis without needing unit vectors at all. Either way, we can find unnormalized orthogonal vectors u1,u2,u3\first{u_1}, \second{u_2}, \third{u_3} first and save normalizing for the very end.

When an axis vector uu is not a unit vector, projecting b\second{b} onto uu divides by u⋅u=∥u∥2u \cdot u = \|u\|^2 to adjust for its length:

Projection of b onto u\displaystyle \text{Projection of } \second{b} \text{ onto } u =(b⋅uu⋅u⏟∥u∥2)u\displaystyle = \shadow{\left(\frac{b \cdot u}{\underbrace{u \cdot u}_{\|u\|^2}}\right)u}

Because u⋅uu \cdot u is the length squared, the square root disappears completely during the projection steps. Let's use this to find orthogonal vectors from a=⟨1,1,1⟩\first{a} = \first{\langle 1, 1, 1 \rangle}, b=⟨1,2,3⟩\second{b} = \second{\langle 1, 2, 3 \rangle}, and c=⟨1,5,3⟩\third{c} = \third{\langle 1, 5, 3 \rangle}.

Avoiding Square Roots by Normalizing at the End
a=⟨1,1,1⟩b=⟨1,2,3⟩c=⟨1,5,3⟩u1, u2, u3=  ?\displaystyle \begin{aligned} \first{a} &= \first{\langle 1, 1, 1 \rangle} \\ \second{b} &= \second{\langle 1, 2, 3 \rangle} \\ \third{c} &= \third{\langle 1, 5, 3 \rangle} \\[8pt] \first{u_1},\ \second{u_2},\ \third{u_3} &= \;? \end{aligned}
Text solution
Keep a\first{a} as the first orthogonal vector u1\first{u_1}Leave a=⟨1,1,1⟩\first{a} = \first{\langle 1, 1, 1 \rangle} at its original length so we do not bring 3\sqrt{3} into the dot products.
u1\displaystyle \first{u_1} =⟨1,1,1⟩\displaystyle = \first{\langle 1, 1, 1 \rangle}
Subtract the projection of b\second{b} onto u1\first{u_1} to get u2\second{u_2}Project b=⟨1,2,3⟩\second{b} = \second{\langle 1, 2, 3 \rangle} onto the unnormalized vector u1\first{u_1} by dividing b⋅u1\second{b} \cdot \first{u_1} by u1⋅u1\first{u_1} \cdot \first{u_1}, then subtract that projection from b\second{b}.
b⋅u1\displaystyle \second{b} \cdot \first{u_1} =⟨1,2,3⟩⋅⟨1,1,1⟩\displaystyle = \second{\langle 1, 2, 3 \rangle} \cdot \first{\langle 1, 1, 1 \rangle} =1+2+3\displaystyle = 1 + 2 + 3 =6\displaystyle = 6
u1⋅u1\displaystyle \first{u_1} \cdot \first{u_1} =⟨1,1,1⟩⋅⟨1,1,1⟩\displaystyle = \first{\langle 1, 1, 1 \rangle} \cdot \first{\langle 1, 1, 1 \rangle} =12+12+12\displaystyle = \first{1}^2 + \first{1}^2 + \first{1}^2 =3\displaystyle = 3
u2=b−(b⋅u1u1⋅u1)u1=⟨1,2,3⟩−63⟨1,1,1⟩=⟨1,2,3⟩−⟨2,2,2⟩=⟨−1,0,1⟩\displaystyle \begin{aligned} \second{u_2} &= \second{b} - \shadow{\left(\frac{b \cdot u_1}{u_1 \cdot u_1}\right)u_1} \\ &= \second{\langle 1, 2, 3 \rangle} - \frac{6}{3}\first{\langle 1, 1, 1 \rangle} \\ &= \second{\langle 1, 2, 3 \rangle} - \shadow{\langle 2, 2, 2 \rangle} \\ &= \second{\langle -1, 0, 1 \rangle} \end{aligned}
Subtract the projections of c\third{c} onto u1\first{u_1} and u2\second{u_2} to get u3\third{u_3}Project c=⟨1,5,3⟩\third{c} = \third{\langle 1, 5, 3 \rangle} onto both u1\first{u_1} and u2\second{u_2} using their squared lengths u1⋅u1\first{u_1} \cdot \first{u_1} and u2⋅u2\second{u_2} \cdot \second{u_2}, then subtract both projections from c\third{c}.
c⋅u1\displaystyle \third{c} \cdot \first{u_1} =⟨1,5,3⟩⋅⟨1,1,1⟩\displaystyle = \third{\langle 1, 5, 3 \rangle} \cdot \first{\langle 1, 1, 1 \rangle} =1+5+3\displaystyle = 1 + 5 + 3 =9\displaystyle = 9
c⋅u2\displaystyle \third{c} \cdot \second{u_2} =⟨1,5,3⟩⋅⟨−1,0,1⟩\displaystyle = \third{\langle 1, 5, 3 \rangle} \cdot \second{\langle -1, 0, 1 \rangle} =−1+0+3\displaystyle = -1 + 0 + 3 =2\displaystyle = 2
u2⋅u2\displaystyle \second{u_2} \cdot \second{u_2} =⟨−1,0,1⟩⋅⟨−1,0,1⟩\displaystyle = \second{\langle -1, 0, 1 \rangle} \cdot \second{\langle -1, 0, 1 \rangle} =(−1)2+02+12\displaystyle = (\second{-1})^2 + \second{0}^2 + \second{1}^2 =2\displaystyle = 2
u3=c−(c⋅u1u1⋅u1)u1−(c⋅u2u2⋅u2)u2=⟨1,5,3⟩−93⟨1,1,1⟩−22⟨−1,0,1⟩\displaystyle \begin{aligned} \third{u_3} &= \third{c} - \shadow{\left(\frac{c \cdot u_1}{u_1 \cdot u_1}\right)u_1} \\ &\quad - \shadow{\left(\frac{c \cdot u_2}{u_2 \cdot u_2}\right)u_2} \\ &= \third{\langle 1, 5, 3 \rangle} - \frac{9}{3}\first{\langle 1, 1, 1 \rangle} \\ &\quad - \frac{2}{2}\second{\langle -1, 0, 1 \rangle} \end{aligned}
u3\displaystyle \third{u_3} =⟨1,5,3⟩−⟨3,3,3⟩−⟨−1,0,1⟩\displaystyle = \third{\langle 1, 5, 3 \rangle} - \shadow{\langle 3, 3, 3 \rangle} - \shadow{\langle -1, 0, 1 \rangle} =⟨−1,2,−1⟩\displaystyle = \third{\langle -1, 2, -1 \rangle}
Normalize at the end to get q1,q2,q3\first{q_1}, \second{q_2}, \third{q_3}If an orthonormal basis is needed, divide each orthogonal vector by its length once all projections are done.
∥u1∥=3,∥u2∥=2,∥u3∥=(−1)2+22+(−1)2=6\displaystyle \begin{aligned} & \|\first{u_1}\| = \sqrt{3}, \\ & \|\second{u_2}\| = \sqrt{2}, \\ & \|\third{u_3}\| \\ &\quad = \sqrt{(\third{-1})^2 + \third{2}^2 + (\third{-1})^2} \\ &\quad = \sqrt{6} \end{aligned}
q1=13⟨1,1,1⟩,q2=12⟨−1,0,1⟩,q3=16⟨−1,2,−1⟩\displaystyle \begin{aligned} \first{q_1} &= \frac{1}{\sqrt{3}}\first{\langle 1, 1, 1 \rangle}, \\ \second{q_2} &= \frac{1}{\sqrt{2}}\second{\langle -1, 0, 1 \rangle}, \\ \third{q_3} &= \frac{1}{\sqrt{6}}\third{\langle -1, 2, -1 \rangle} \end{aligned}

The vectors u1=⟨1,1,1⟩\first{u_1} = \first{\langle 1, 1, 1 \rangle}, u2=⟨−1,0,1⟩\second{u_2} = \second{\langle -1, 0, 1 \rangle}, and u3=⟨−1,2,−1⟩\third{u_3} = \third{\langle -1, 2, -1 \rangle} give an orthogonal basis using only whole-number arithmetic, and dividing by their lengths gives the orthonormal basis q1,q2,q3\first{q_1}, \second{q_2}, \third{q_3}. (On a test, check carefully whether the question asks for an orthogonal basis or an orthonormal basis so you don't do the final square-root step when it isn't asked for.)

4

When a Vector Collapses to Zero

Sometimes subtracting a vector's projections onto the earlier axes cancels the vector out completely, leaving the zero vector ⟨0,0,0⟩\langle 0, 0, 0 \rangle. Let's see what that zero result tells us about the vectors and their span.

Keeping a=⟨1,1,1⟩\first{a} = \first{\langle 1, 1, 1 \rangle} and b=⟨1,2,3⟩\second{b} = \second{\langle 1, 2, 3 \rangle} from the last problem, so we already have u1=⟨1,1,1⟩\first{u_1} = \first{\langle 1, 1, 1 \rangle} and u2=⟨−1,0,1⟩\second{u_2} = \second{\langle -1, 0, 1 \rangle}, let's replace the third vector with c=⟨0,3,6⟩\third{c} = \third{\langle 0, 3, 6 \rangle}.

When a Vector Collapses to Zero
u1=⟨1,1,1⟩u2=⟨−1,0,1⟩c=⟨0,3,6⟩u3=  ?\displaystyle \begin{aligned} \first{u_1} &= \first{\langle 1, 1, 1 \rangle} \\ \second{u_2} &= \second{\langle -1, 0, 1 \rangle} \\ \third{c} &= \third{\langle 0, 3, 6 \rangle} \\[8pt] \third{u_3} &= \;? \end{aligned}
c=⟨1,5,3⟩u3=c−⟨3,3,3⟩−⟨−1,0,1⟩=⟨−1,2,−1⟩\begin{aligned} \third{c} &= \third{\langle 1, 5, 3 \rangle} \\ \third{u_3} &= \third{c} - \shadow{\langle 3, 3, 3 \rangle} - \shadow{\langle -1, 0, 1 \rangle} \\ &= \third{\langle -1, 2, -1 \rangle} \end{aligned}
c=⟨0,3,6⟩u3=c−⟨3,3,3⟩−⟨−3,0,3⟩=⟨0,0,0⟩\begin{aligned} \third{c} &= \third{\langle 0, 3, 6 \rangle} \\ \third{u_3} &= \third{c} - \shadow{\langle 3, 3, 3 \rangle} - \shadow{\langle -3, 0, 3 \rangle} \\ &= \langle 0, 0, 0 \rangle \end{aligned}
Text solution
Keep u1\first{u_1} and u2\second{u_2} from the first two vectorsSince a=⟨1,1,1⟩\first{a} = \first{\langle 1, 1, 1 \rangle} and b=⟨1,2,3⟩\second{b} = \second{\langle 1, 2, 3 \rangle} have not changed, the first two perpendicular axes are still u1=⟨1,1,1⟩\first{u_1} = \first{\langle 1, 1, 1 \rangle} (with u1⋅u1=3\first{u_1} \cdot \first{u_1} = 3) and u2=⟨−1,0,1⟩\second{u_2} = \second{\langle -1, 0, 1 \rangle} (with u2⋅u2=2\second{u_2} \cdot \second{u_2} = 2).
Subtract the projections of c\third{c} onto u1\first{u_1} and u2\second{u_2}Project c=⟨0,3,6⟩\third{c} = \third{\langle 0, 3, 6 \rangle} onto both u1\first{u_1} and u2\second{u_2} and subtract those projections from c\third{c}.
c⋅u1\displaystyle \third{c} \cdot \first{u_1} =⟨0,3,6⟩⋅⟨1,1,1⟩\displaystyle = \third{\langle 0, 3, 6 \rangle} \cdot \first{\langle 1, 1, 1 \rangle} =0+3+6\displaystyle = 0 + 3 + 6 =9\displaystyle = 9
(c⋅u1u1⋅u1)u1\displaystyle \shadow{\left(\frac{c \cdot u_1}{u_1 \cdot u_1}\right)u_1} =93⟨1,1,1⟩\displaystyle = \frac{9}{3}\first{\langle 1, 1, 1 \rangle} =⟨3,3,3⟩\displaystyle = \shadow{\langle 3, 3, 3 \rangle}
c⋅u2\displaystyle \third{c} \cdot \second{u_2} =⟨0,3,6⟩⋅⟨−1,0,1⟩\displaystyle = \third{\langle 0, 3, 6 \rangle} \cdot \second{\langle -1, 0, 1 \rangle} =0+0+6\displaystyle = 0 + 0 + 6 =6\displaystyle = 6
(c⋅u2u2⋅u2)u2\displaystyle \shadow{\left(\frac{c \cdot u_2}{u_2 \cdot u_2}\right)u_2} =62⟨−1,0,1⟩\displaystyle = \frac{6}{2}\second{\langle -1, 0, 1 \rangle} =⟨−3,0,3⟩\displaystyle = \shadow{\langle -3, 0, 3 \rangle}
u3\displaystyle \third{u_3} =⟨0,3,6⟩−⟨3,3,3⟩−⟨−3,0,3⟩\displaystyle = \third{\langle 0, 3, 6 \rangle} - \shadow{\langle 3, 3, 3 \rangle} - \shadow{\langle -3, 0, 3 \rangle} =⟨0,0,0⟩\displaystyle = \langle 0, 0, 0 \rangle
Discard the zero vectorBecause ⟨0,0,0⟩\langle 0, 0, 0 \rangle has length 00 and adds no new direction, throw it away and keep only u1\first{u_1} and u2\second{u_2}.

Getting ⟨0,0,0⟩\langle 0, 0, 0 \rangle means c\third{c} equals its own shadow in the u1–u2\first{u_1}\text{--}\second{u_2} plane, so c=3b−3a\third{c} = 3\second{b} - 3\first{a} was already sitting inside the plane of a\first{a} and b\second{b}. All three vectors span only that 2D plane in 3D space, with orthogonal basis u1=⟨1,1,1⟩\first{u_1} = \first{\langle 1, 1, 1 \rangle} and u2=⟨−1,0,1⟩\second{u_2} = \second{\langle -1, 0, 1 \rangle}.

Key concepts

Gram-Schmidt builds perpendicular axes one vector at a time by taking the next vector and subtracting its projections onto every perpendicular axis built so far.

Choosing whether to normalize as you go or at the end

Check the length ∥a∥\|\first{a}\| of the first vector before you start.

  • Whole-number lengths (normalize as you go): When ∥a∥\|\first{a}\| is a whole number (such as a=⟨3,4⟩\first{a} = \first{\langle 3, 4 \rangle} with ∥a∥=5\|\first{a}\| = 5), divide by the length right away to get q1=⟨35,45⟩\first{q_1} = \first{\left\langle \tfrac{3}{5}, \tfrac{4}{5} \right\rangle} and project using unit-vector dot products:
  • When not to normalize as you go (square-root lengths or orthogonal-only questions): When ∥a∥\|\first{a}\| is a square root (such as a=⟨1,1,1⟩\first{a} = \first{\langle 1, 1, 1 \rangle} with ∥a∥=3\|\first{a}\| = \sqrt{3}) or the problem only asks for an orthogonal basis, leave u1=a\first{u_1} = \first{a} unnormalized and, in each projection, divide the dot product by u⋅uu \cdot u so the squared length gets rid of the square root:
When not to subtract a projection, or when to discard a vector

If a dot product is already 00 (for example, b⋅u1=0\second{b} \cdot \first{u_1} = 0), the vector is already at 90∘90^\circ to that axis and its projection term is zero. If subtracting all projections cancels a vector down to ⟨0,0,0⟩\langle 0, 0, 0 \rangle (as with c=⟨0,3,6⟩\third{c} = \third{\langle 0, 3, 6 \rangle} giving u3=⟨0,0,0⟩\third{u_3} = \langle 0, 0, 0 \rangle), that vector is just a combination of the earlier ones, so throw it away and never include ⟨0,0,0⟩\langle 0, 0, 0 \rangle in a basis.

5

Practice

1

Use Gram-Schmidt to find an orthonormal basis q1,q2\first{q_1}, \second{q_2} for the plane spanned by a=⟨4,−3⟩\first{a} = \first{\langle 4, -3 \rangle} and b=⟨2,11⟩\second{b} = \second{\langle 2, 11 \rangle}.

Check the length of a\first{a} first: is ∥a∥\|\first{a}\| a whole number so that you can normalize a\first{a} into q1\first{q_1} right away?

Find the length of a\first{a} and normalize to get q1\first{q_1}:
∥a∥\displaystyle \|\first{a}\| =42+(−3)2\displaystyle = \sqrt{\first{4}^2 + (\first{-3})^2} =25\displaystyle = \sqrt{25} =5  ⟹  q1\displaystyle = 5 \implies \first{q_1} =⟨45,−35⟩\displaystyle = \first{\left\langle \tfrac{4}{5}, -\tfrac{3}{5} \right\rangle}
Take the dot product of b\second{b} with q1\first{q_1} to find the projection onto q1\first{q_1}:
b⋅q1\displaystyle \second{b} \cdot \first{q_1} =⟨2,11⟩⋅⟨45,−35⟩\displaystyle = \second{\langle 2, 11 \rangle} \cdot \first{\left\langle \tfrac{4}{5}, -\tfrac{3}{5} \right\rangle} =8−335\displaystyle = \frac{8 - 33}{5} =−5\displaystyle = -5
(b⋅q1)q1\displaystyle \shadow{(b \cdot q_1)q_1} =−5⟨45,−35⟩\displaystyle = -5\first{\left\langle \tfrac{4}{5}, -\tfrac{3}{5} \right\rangle} =⟨−4,3⟩\displaystyle = \shadow{\langle -4, 3 \rangle}
Subtract the projection from b\second{b} (watch the negative sign on b⋅q1=−5\second{b} \cdot \first{q_1} = -5: dropping the minus sign and subtracting ⟨4,−3⟩\langle 4, -3 \rangle instead of ⟨−4,3⟩\shadow{\langle -4, 3 \rangle} is the most common mistake, which leads to the wrong vector ⟨−2,14⟩\langle -2, 14 \rangle):
bperp\displaystyle \perpPiece{b_{\text{perp}}} =⟨2,11⟩−⟨−4,3⟩\displaystyle = \second{\langle 2, 11 \rangle} - \shadow{\langle -4, 3 \rangle} =⟨6,8⟩\displaystyle = \perpPiece{\langle 6, 8 \rangle}
Normalize bperp\perpPiece{b_{\text{perp}}} (which has length 62+82=10\sqrt{\perpPiece{6}^2 + \perpPiece{8}^2} = 10):
q1=⟨45,−35⟩,q2=⟨35,45⟩\displaystyle \begin{aligned} \first{q_1} &= \first{\left\langle \tfrac{4}{5}, -\tfrac{3}{5} \right\rangle}, \\ \second{q_2} &= \second{\left\langle \tfrac{3}{5}, \tfrac{4}{5} \right\rangle} \end{aligned}
2

Find an orthonormal basis q1,q2,q3\first{q_1}, \second{q_2}, \third{q_3} for the space spanned by a=⟨2,2,1⟩\first{a} = \first{\langle 2, 2, 1 \rangle}, b=⟨3,0,3⟩\second{b} = \second{\langle 3, 0, 3 \rangle}, and c=⟨7,1,2⟩\third{c} = \third{\langle 7, 1, 2 \rangle}.

Normalize a\first{a} to get q1\first{q_1} and build q2\second{q_2} from b\second{b} first, then remember that c\third{c} is tilted toward both q1\first{q_1} and q2\second{q_2}.

Normalize a\first{a} (length 22+22+12=3\sqrt{\first{2}^2 + \first{2}^2 + \first{1}^2} = 3):
q1\displaystyle \first{q_1} =⟨23,23,13⟩\displaystyle = \first{\left\langle \tfrac{2}{3}, \tfrac{2}{3}, \tfrac{1}{3} \right\rangle}
Subtract the projection of b\second{b} onto q1\first{q_1}, using b⋅q1=⟨3,0,3⟩⋅⟨23,23,13⟩=3\second{b} \cdot \first{q_1} = \second{\langle 3, 0, 3 \rangle} \cdot \first{\left\langle \tfrac{2}{3}, \tfrac{2}{3}, \tfrac{1}{3} \right\rangle} = 3:
bperp=⟨3,0,3⟩−3⟨23,23,13⟩=⟨3,0,3⟩−⟨2,2,1⟩=⟨1,−2,2⟩\displaystyle \begin{aligned} \perpPiece{b_{\text{perp}}} &= \second{\langle 3, 0, 3 \rangle} - 3\first{\left\langle \tfrac{2}{3}, \tfrac{2}{3}, \tfrac{1}{3} \right\rangle} \\ &= \second{\langle 3, 0, 3 \rangle} - \shadow{\langle 2, 2, 1 \rangle} \\ &= \perpPiece{\langle 1, -2, 2 \rangle} \end{aligned}
Normalize bperp\perpPiece{b_{\text{perp}}} (length 12+(−2)2+22=3\sqrt{\perpPiece{1}^2 + (\perpPiece{-2})^2 + \perpPiece{2}^2} = 3):
q2\displaystyle \second{q_2} =⟨13,−23,23⟩\displaystyle = \second{\left\langle \tfrac{1}{3}, -\tfrac{2}{3}, \tfrac{2}{3} \right\rangle}
Project c\third{c} onto both q1\first{q_1} and q2\second{q_2}, with c⋅q1=6\third{c} \cdot \first{q_1} = 6 and c⋅q2=3\third{c} \cdot \second{q_2} = 3 (be careful to project c\third{c} onto the new perpendicular unit vector q2\second{q_2} rather than onto the original vector b\second{b}, and do not forget to subtract the q2\second{q_2} projection ⟨1,−2,2⟩\shadow{\langle 1, -2, 2 \rangle}, or you will get ⟨3,−3,0⟩\langle 3, -3, 0 \rangle and the wrong third vector ⟨12,−12,0⟩\left\langle \tfrac{1}{\sqrt{2}}, -\tfrac{1}{\sqrt{2}}, 0 \right\rangle):
cperp=⟨7,1,2⟩−6⟨23,23,13⟩−3⟨13,−23,23⟩=⟨7,1,2⟩−⟨4,4,2⟩−⟨1,−2,2⟩=⟨2,−1,−2⟩\displaystyle \begin{aligned} \perpPiece{c_{\text{perp}}} &= \third{\langle 7, 1, 2 \rangle} - 6\first{\left\langle \tfrac{2}{3}, \tfrac{2}{3}, \tfrac{1}{3} \right\rangle} \\ &\quad - 3\second{\left\langle \tfrac{1}{3}, -\tfrac{2}{3}, \tfrac{2}{3} \right\rangle} \\ &= \third{\langle 7, 1, 2 \rangle} - \shadow{\langle 4, 4, 2 \rangle} \\ &\quad - \shadow{\langle 1, -2, 2 \rangle} \\ &= \perpPiece{\langle 2, -1, -2 \rangle} \end{aligned}
Normalize cperp\perpPiece{c_{\text{perp}}} (length 22+(−1)2+(−2)2=3\sqrt{\perpPiece{2}^2 + (\perpPiece{-1})^2 + (\perpPiece{-2})^2} = 3):
q1=⟨23,23,13⟩,q2=⟨13,−23,23⟩,q3=⟨23,−13,−23⟩\displaystyle \begin{aligned} \first{q_1} &= \first{\left\langle \tfrac{2}{3}, \tfrac{2}{3}, \tfrac{1}{3} \right\rangle}, \\ \second{q_2} &= \second{\left\langle \tfrac{1}{3}, -\tfrac{2}{3}, \tfrac{2}{3} \right\rangle}, \\ \third{q_3} &= \third{\left\langle \tfrac{2}{3}, -\tfrac{1}{3}, -\tfrac{2}{3} \right\rangle} \end{aligned}
3

Given a=⟨1,0,1⟩\first{a} = \first{\langle 1, 0, 1 \rangle}, b=⟨2,1,0⟩\second{b} = \second{\langle 2, 1, 0 \rangle}, and c=⟨2,3,2⟩\third{c} = \third{\langle 2, 3, 2 \rangle}, first find an orthogonal basis u1,u2,u3\first{u_1}, \second{u_2}, \third{u_3} without normalizing as you go, then normalize at the end to get an orthonormal basis q1,q2,q3\first{q_1}, \second{q_2}, \third{q_3}.

Set u1=a\first{u_1} = \first{a}, and when projecting onto an unnormalized vector uku_k, what do you divide the dot product by so no square roots appear?

Keep u1=⟨1,0,1⟩\first{u_1} = \first{\langle 1, 0, 1 \rangle}, which has u1⋅u1=12+02+12=2\first{u_1} \cdot \first{u_1} = \first{1}^2 + \first{0}^2 + \first{1}^2 = 2.
For b=⟨2,1,0⟩\second{b} = \second{\langle 2, 1, 0 \rangle}, we have b⋅u1=2\second{b} \cdot \first{u_1} = 2. Divide by u1⋅u1=2\first{u_1} \cdot \first{u_1} = 2 (don't forget to divide by u1⋅u1\first{u_1} \cdot \first{u_1}: treating u1\first{u_1} as a unit vector and subtracting 2u1=⟨2,0,2⟩2\first{u_1} = \langle 2, 0, 2 \rangle is a common mistake here, giving the wrong vector ⟨0,1,−2⟩\langle 0, 1, -2 \rangle):
u2\displaystyle \second{u_2} =⟨2,1,0⟩−22⟨1,0,1⟩\displaystyle = \second{\langle 2, 1, 0 \rangle} - \frac{2}{2}\first{\langle 1, 0, 1 \rangle} =⟨1,1,−1⟩(with u2⋅u2=3)\displaystyle = \second{\langle 1, 1, -1 \rangle} \quad (\text{with } \second{u_2} \cdot \second{u_2} = 3)
For c=⟨2,3,2⟩\third{c} = \third{\langle 2, 3, 2 \rangle}, we have c⋅u1=4\third{c} \cdot \first{u_1} = 4 and c⋅u2=3\third{c} \cdot \second{u_2} = 3:
u3=⟨2,3,2⟩−42⟨1,0,1⟩−33⟨1,1,−1⟩=⟨2,3,2⟩−⟨2,0,2⟩−⟨1,1,−1⟩=⟨−1,2,1⟩(with u3⋅u3=6)\displaystyle \begin{aligned} \third{u_3} &= \third{\langle 2, 3, 2 \rangle} - \frac{4}{2}\first{\langle 1, 0, 1 \rangle} \\ &\qquad - \frac{3}{3}\second{\langle 1, 1, -1 \rangle} \\ &= \third{\langle 2, 3, 2 \rangle} - \shadow{\langle 2, 0, 2 \rangle} - \shadow{\langle 1, 1, -1 \rangle} \\ &= \third{\langle -1, 2, 1 \rangle} \\ &\qquad (\text{with } \third{u_3} \cdot \third{u_3} = 6) \end{aligned}
Divide each orthogonal vector by its length (∥u1∥=2\|\first{u_1}\| = \sqrt{2}, ∥u2∥=3\|\second{u_2}\| = \sqrt{3}, ∥u3∥=6\|\third{u_3}\| = \sqrt{6}):
u1=⟨1,0,1⟩,u2=⟨1,1,−1⟩,u3=⟨−1,2,1⟩\displaystyle \begin{aligned} \first{u_1} &= \first{\langle 1, 0, 1 \rangle}, \\ \second{u_2} &= \second{\langle 1, 1, -1 \rangle}, \\ \third{u_3} &= \third{\langle -1, 2, 1 \rangle} \end{aligned}
q1=12⟨1,0,1⟩,q2=13⟨1,1,−1⟩,q3=16⟨−1,2,1⟩\displaystyle \begin{aligned} \first{q_1} &= \frac{1}{\sqrt{2}}\first{\langle 1, 0, 1 \rangle}, \\ \second{q_2} &= \frac{1}{\sqrt{3}}\second{\langle 1, 1, -1 \rangle}, \\ \third{q_3} &= \frac{1}{\sqrt{6}}\third{\langle -1, 2, 1 \rangle} \end{aligned}
4

Find an orthogonal basis for the space spanned by a=⟨1,1,2⟩\first{a} = \first{\langle 1, 1, 2 \rangle}, b=⟨2,0,2⟩\second{b} = \second{\langle 2, 0, 2 \rangle}, and c=⟨−1,5,4⟩\third{c} = \third{\langle -1, 5, 4 \rangle}, and describe the space they span.

Use the unnormalized projection formula starting with u1=a\first{u_1} = \first{a}, and see what remains of c\third{c} once you subtract its projections onto u1\first{u_1} and u2\second{u_2}.

Set u1=⟨1,1,2⟩\first{u_1} = \first{\langle 1, 1, 2 \rangle}, which has u1⋅u1=12+12+22=6\first{u_1} \cdot \first{u_1} = \first{1}^2 + \first{1}^2 + \first{2}^2 = 6.
Since b⋅u1=2+0+4=6\second{b} \cdot \first{u_1} = 2 + 0 + 4 = 6, subtract the projection of b\second{b} onto u1\first{u_1}:
u2\displaystyle \second{u_2} =⟨2,0,2⟩−66⟨1,1,2⟩\displaystyle = \second{\langle 2, 0, 2 \rangle} - \frac{6}{6}\first{\langle 1, 1, 2 \rangle} =⟨1,−1,0⟩(with u2⋅u2=2)\displaystyle = \second{\langle 1, -1, 0 \rangle} \quad (\text{with } \second{u_2} \cdot \second{u_2} = 2)
For c=⟨−1,5,4⟩\third{c} = \third{\langle -1, 5, 4 \rangle}, we have c⋅u1=−1+5+8=12\third{c} \cdot \first{u_1} = -1 + 5 + 8 = 12 and c⋅u2=−1−5+0=−6\third{c} \cdot \second{u_2} = -1 - 5 + 0 = -6. Subtract both projections (watch the minus sign on −62u2=⟨−3,3,0⟩\frac{-6}{2}\second{u_2} = \shadow{\langle -3, 3, 0 \rangle}: adding ⟨−3,3,0⟩\shadow{\langle -3, 3, 0 \rangle} instead of subtracting it gives ⟨−6,6,0⟩\langle -6, 6, 0 \rangle and misses the collapse to zero, and never keep ⟨0,0,0⟩\langle 0, 0, 0 \rangle as a third basis vector):
u3=⟨−1,5,4⟩−126⟨1,1,2⟩−−62⟨1,−1,0⟩=⟨−1,5,4⟩−⟨2,2,4⟩−⟨−3,3,0⟩=⟨0,0,0⟩\displaystyle \begin{aligned} \third{u_3} &= \third{\langle -1, 5, 4 \rangle} - \frac{12}{6}\first{\langle 1, 1, 2 \rangle} \\ &\quad - \frac{-6}{2}\second{\langle 1, -1, 0 \rangle} \\ &= \third{\langle -1, 5, 4 \rangle} - \shadow{\langle 2, 2, 4 \rangle} \\ &\quad - \shadow{\langle -3, 3, 0 \rangle} \\ &= \langle 0, 0, 0 \rangle \end{aligned}
Because u3\third{u_3} collapses to ⟨0,0,0⟩\langle 0, 0, 0 \rangle, c\third{c} lies in the plane of a\first{a} and b\second{b} and is discarded; the space spanned is a 2D plane in 3D space with orthogonal basis u1=⟨1,1,2⟩\first{u_1} = \first{\langle 1, 1, 2 \rangle} and u2=⟨1,−1,0⟩\second{u_2} = \second{\langle 1, -1, 0 \rangle}.