Linear AlgebraOrthogonality and Projections3 of 9
Orthogonality and Projections · 3

Orthogonal Complements: Row Space Meets Null Space

We know that the Row Space and the Null Space of a matrix both live in the input space, and we want to see how they fit together geometrically.

1

Why the Row Space and Null Space Are Orthogonal Complements

Two subspaces are orthogonal complements when every vector in one is perpendicular (90∘90^\circ) to every vector in the other, and their dimensions add up to the full space.

To see why the Row Space and the Null Space of any matrix always form orthogonal complements, let's look at what Ax=0A\vecx{x} = 0 says about the rows of A=[11−212−3]A = \begin{bmatrix} \firstvec{1} & \firstvec{1} & \firstvec{-2} \\ \secondvec{1} & \secondvec{2} & \secondvec{-3} \end{bmatrix} and an input vector x=[x1x2x3]\vecx{x} = \vecx{\begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}}.

Why the Row Space and Null Space Are Orthogonal Complements
Text solution
Write Ax=0A\vecx{x} = 0 as dot products with the rowsMultiplying AA by x\vecx{x} takes the dot product of each row with x\vecx{x}, so being in the Null Space means x\vecx{x} is perpendicular to every row of AA at once:
Ax=[(Row 1)⋅x(Row 2)⋅x]=[00]\displaystyle A\vecx{x} = \begin{bmatrix} (\firstvec{\text{Row } 1}) \cdot \vecx{x} \\ (\secondvec{\text{Row } 2}) \cdot \vecx{x} \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}
Row reduce AA to get cleaner vectors on the same Row Space planeSubtracting Row 1 from Row 2, and then subtracting the new Row 2 from Row 1, gives us simpler combinations of the rows on the exact same plane:
[11−212−3]→[11−201−1]→[10−101−1]\displaystyle \begin{aligned} &\begin{bmatrix} \firstvec{1} & \firstvec{1} & \firstvec{-2} \\ \secondvec{1} & \secondvec{2} & \secondvec{-3} \end{bmatrix} \rightarrow \begin{bmatrix} \firstvec{1} & \firstvec{1} & \firstvec{-2} \\ \secondvec{0} & \secondvec{1} & \secondvec{-1} \end{bmatrix} \\ &\rightarrow \begin{bmatrix} \firstvec{1} & \firstvec{0} & \firstvec{-1} \\ \secondvec{0} & \secondvec{1} & \secondvec{-1} \end{bmatrix} \end{aligned}
Solve for the Null Space line n\nullvec{n}Setting the dot products of the reduced rows with x\vecx{x} to zero gives x1−x3=0\vecx{x_1} - \vecx{x_3} = 0 and x2−x3=0\vecx{x_2} - \vecx{x_3} = 0, so choosing the free variable x3=1\vecx{x_3} = 1 gives x1=1\vecx{x_1} = 1 and x2=1\vecx{x_2} = 1:
(Row Space)⊥⏟orthogonal complement=Null Space=span{[111]}\displaystyle \underbrace{(\keptsp{\text{Row Space}})^\perp}_{\text{orthogonal complement}} = \lostsp{\text{Null Space}} = \text{span}\left\{\nullvec{\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}}\right\}

The perpendicular symbol ⊥\perp (read "perp") stands for the orthogonal complement: all vectors perpendicular to that subspace. Because perpendicularity goes both ways and dim⁡(Row Space)+dim⁡(Null Space)\dim(\keptsp{\text{Row Space}}) + \dim(\lostsp{\text{Null Space}}) =2+1=3{}= \keptsp{2} + \lostsp{1} = 3, the two subspaces split the 3D input space completely, with (Null Space)⊥=Row Space(\lostsp{\text{Null Space}})^\perp = \keptsp{\text{Row Space}} as well.

Every linear combination of Row 1 and Row 2, including the new rows formed during row reduction, lies in the same flat Row Space plane at a 90∘90^\circ angle to the Null Space line, so its dot product with n=[111]\nullvec{n} = \nullvec{\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}} is still zero.

Every combination of Row 1 and Row 2, including the replacement rows from row reduction, lies in the same flat plane at 90° to the Null Space line.
2

Finding the Orthogonal Complement of a Subspace

Even when a problem doesn't mention matrices, we can use a matrix to find the orthogonal complement W⊥W^\perp of a subspace WW. Because the Row Space and Null Space are orthogonal complements, we just set up a matrix so that WW is one of those two spaces, and the other one gives us W⊥W^\perp.

Finding the Orthogonal Complement of a Subspace
W\displaystyle W =span{v1,v2}\displaystyle = \text{span}\{\firstvec{v_1}, \secondvec{v_2}\}
v1=[11−2],v2=[12−3]\displaystyle \firstvec{v_1} = \firstvec{\begin{bmatrix} 1 \\ 1 \\ -2 \end{bmatrix}}, \quad \secondvec{v_2} = \secondvec{\begin{bmatrix} 1 \\ 2 \\ -3 \end{bmatrix}}
basis for W⊥\displaystyle \text{basis for } W^{\perp} =  ?\displaystyle = \;?
Text solution
When WW is given by spanning vectors, write them as rows and solve Ax=0A\vecx{x} = 0For W=span{[11−2],[12−3]}W = \text{span}\left\{\firstvec{\begin{bmatrix} 1 \\ 1 \\ -2 \end{bmatrix}}, \secondvec{\begin{bmatrix} 1 \\ 2 \\ -3 \end{bmatrix}}\right\}, making the spanning vectors the rows of AA makes WW the Row Space, so the Null Space gives W⊥W^\perp:
A=[11−212−3]→[10−101−1]  ⟹  W⊥=Null Space of A=span{[111]}\displaystyle \begin{aligned} A &= \begin{bmatrix} \firstvec{1} & \firstvec{1} & \firstvec{-2} \\ \secondvec{1} & \secondvec{2} & \secondvec{-3} \end{bmatrix} \\ &\rightarrow \begin{bmatrix} \firstvec{1} & \firstvec{0} & \firstvec{-1} \\ \secondvec{0} & \secondvec{1} & \secondvec{-1} \end{bmatrix} \\ &\implies W^\perp = \lostsp{\text{Null Space of } A} \\ &= \text{span}\left\{\nullvec{\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}}\right\} \end{aligned}
When WW is given by homogeneous equations, read the coefficients off as rowsFor the subspace WW where x1+x2−2x3=0\vecx{x_1} + \vecx{x_2} - 2\vecx{x_3} = 0 and x1+2x2−3x3=0\vecx{x_1} + 2\vecx{x_2} - 3\vecx{x_3} = 0, the equations already have the form Ax=0A\vecx{x} = 0, so WW is the Null Space and the coefficient vectors immediately span W⊥W^\perp:
[11−212−3][x1x2x3]=[00]⏟W=Null Space of A  ⟹  W⊥=Row Space of A=span{[11−2],[12−3]}\displaystyle \begin{aligned} &\underbrace{\begin{bmatrix} \firstvec{1} & \firstvec{1} & \firstvec{-2} \\ \secondvec{1} & \secondvec{2} & \secondvec{-3} \end{bmatrix}\vecx{\begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}}_{W = \lostsp{\text{Null Space of } A}} \\ &\implies W^\perp = \keptsp{\text{Row Space of } A} \\ &= \text{span}\left\{\firstvec{\begin{bmatrix} 1 \\ 1 \\ -2 \end{bmatrix}}, \secondvec{\begin{bmatrix} 1 \\ 2 \\ -3 \end{bmatrix}}\right\} \end{aligned}

When a problem gives WW as homogeneous equations, don't jump in and solve them: solving Ax=0A\vecx{x} = 0 finds a basis for WW itself, while the vectors for W⊥W^\perp are already right there in the equation coefficients.

3

Splitting Input Vectors and the Shortest Solution

Now we want to split any input vector x\vecx{x} into a Row Space piece xrow\xrow{x_{\text{row}}} and a perpendicular Null Space piece xnull\xnull{x_{\text{null}}}, and find the shortest vector that solves Ax=bA\vecx{x} = \target{b}.

Because projecting onto a 1D line is simpler than projecting onto a 2D plane, we find xnull\xnull{x_{\text{null}}} first by projecting x\vecx{x} onto the Null Space direction vector n\nullvec{n}:

xnull\displaystyle \xnull{x_{\text{null}}} =x⋅nn⋅nn\displaystyle = \frac{\vecx{x} \cdot \nullvec{n}}{\nullvec{n} \cdot \nullvec{n}} \nullvec{n}

Let's split x=[312]\vecx{x} = \vecx{\begin{bmatrix} 3 \\ 1 \\ 2 \end{bmatrix}} for A=[11−212−3]A = \begin{bmatrix} \firstvec{1} & \firstvec{1} & \firstvec{-2} \\ \secondvec{1} & \secondvec{2} & \secondvec{-3} \end{bmatrix} and see why xrow\xrow{x_{\text{row}}} is the shortest solution to Ax=[0−1]A\vecx{x} = \target{\begin{bmatrix} 0 \\ -1 \end{bmatrix}}.

Splitting Input Vectors and the Shortest Solution
Text solution
Project x\vecx{x} onto the Null Space line n=[111]\nullvec{n} = \nullvec{\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}}Using the line projection formula gives the part of x\vecx{x} along the Null Space:
xnull=x⋅nn⋅nn=3(1)+1(1)+2(1)12+12+12[111]=63[111]=[222]\displaystyle \begin{aligned} \xnull{x_{\text{null}}} &= \frac{\vecx{x} \cdot \nullvec{n}}{\nullvec{n} \cdot \nullvec{n}} \nullvec{n} \\ &= \frac{\vecx{3}(\nullvec{1}) + \vecx{1}(\nullvec{1}) + \vecx{2}(\nullvec{1})}{\nullvec{1}^2 + \nullvec{1}^2 + \nullvec{1}^2}\nullvec{\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}} \\ &= \frac{6}{3}\nullvec{\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}} = \xnull{\begin{bmatrix} 2 \\ 2 \\ 2 \end{bmatrix}} \end{aligned}
Subtract xnull\xnull{x_{\text{null}}} from x\vecx{x} to get the Row Space pieceSince the two pieces add up to x\vecx{x}, whatever is left over lies flat on the Row Space plane:
xrow=x−xnull=[312]−[222]=[1−10]\displaystyle \begin{aligned} \xrow{x_{\text{row}}} &= \vecx{x} - \xnull{x_{\text{null}}} \\ &= \vecx{\begin{bmatrix} 3 \\ 1 \\ 2 \end{bmatrix}} - \xnull{\begin{bmatrix} 2 \\ 2 \\ 2 \end{bmatrix}} = \xrow{\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}} \end{aligned}
Compare outputs and squared lengthsMultiplying by AA sends xnull\xnull{x_{\text{null}}} to zero, while the right triangle between xrow\xrow{x_{\text{row}}} and xnull\xnull{x_{\text{null}}} makes x\vecx{x} longer by the Pythagorean theorem:
Ax=Axrow+Axnull⏟0=[0−1],∥x∥2=∥xrow∥2⏟2+∥xnull∥2⏟12=14\displaystyle \begin{aligned} A\vecx{x} &= A\xrow{x_{\text{row}}} + \underbrace{A\xnull{x_{\text{null}}}}_{0} = \target{\begin{bmatrix} 0 \\ -1 \end{bmatrix}}, \\ \|\vecx{x}\|^2 &= \underbrace{\|\xrow{x_{\text{row}}}\|^2}_{2} + \underbrace{\|\xnull{x_{\text{null}}}\|^2}_{12} = 14 \end{aligned}

So xrow=[1−10]\xrow{x_{\text{row}}} = \xrow{\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}} is the unique shortest solution to Ax=[0−1]A\vecx{x} = \target{\begin{bmatrix} 0 \\ -1 \end{bmatrix}}.

Sliding x\vecx{x} parallel to the Null Space line changes xnull\xnull{x_{\text{null}}} and the length of the hypotenuse x\vecx{x}, which is longer than xrow\xrow{x_{\text{row}}} whenever xnull≠0\xnull{x_{\text{null}}} \neq 0, while the Row Space leg xrow\xrow{x_{\text{row}}} and the output Ax=AxrowA\vecx{x} = A\xrow{x_{\text{row}}} stay the same.

Sliding an input along the Null Space never changes its output, and the piece in the Row Space is the shortest input that gets you there.
4

Column Space, Left Null Space, and When Ax=bAx = b Is Solvable

Over in the output space, we want a quick test for whether a right-hand side vector b\target{b} lies in the Column Space so that Ax=bA\vecx{x} = \target{b} has a solution.

When we transpose AA, its columns become the rows of ATA^T. That means the Left Null Space (all output vectors y\nullvec{y} that solve ATy=0A^T \nullvec{y} = 0) is perpendicular to every column of AA, making the Column Space and the Left Null Space orthogonal complements in the output space:

Left Null Space⏟Null Space of AT\displaystyle \underbrace{\lostsp{\text{Left Null Space}}}_{\lostsp{\text{Null Space of } A^T}} =(Column Space)⊥\displaystyle = (\keptsp{\text{Column Space}})^\perp

Let's find the Left Null Space for the 3×23 \times 2 matrix A=[101121]A = \begin{bmatrix} \firstvec{1} & \secondvec{0} \\ \firstvec{1} & \secondvec{1} \\ \firstvec{2} & \secondvec{1} \end{bmatrix} and use it to test which vectors b=[b1b2b3]\target{b} = \target{\begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix}} make Ax=bA\vecx{x} = \target{b} solvable.

Column Space, Left Null Space, and When Ax = b Is Solvable
A=[101121]\displaystyle A = \begin{bmatrix} \firstvec{1} & \secondvec{0} \\ \firstvec{1} & \secondvec{1} \\ \firstvec{2} & \secondvec{1} \end{bmatrix}
(Column Space)⊥\displaystyle (\keptsp{\text{Column Space}})^{\perp} =  ?\displaystyle = \;?
Ax\displaystyle A\vecx{x} =b solvable for which b ?\displaystyle = \target{b} \text{ solvable for which } \target{b}\,?
Text solution
Transpose AA and row reduce ATy=0A^T \nullvec{y} = 0Writing the columns of AA as the rows of ATA^T and subtracting Row 2 from Row 1 simplifies the system for the Left Null Space:
AT=[112011]→[101011]\displaystyle A^T = \begin{bmatrix} \firstvec{1} & \firstvec{1} & \firstvec{2} \\ \secondvec{0} & \secondvec{1} & \secondvec{1} \end{bmatrix} \rightarrow \begin{bmatrix} \firstvec{1} & \firstvec{0} & \firstvec{1} \\ \secondvec{0} & \secondvec{1} & \secondvec{1} \end{bmatrix}
Find the normal vector y\nullvec{y} spanning the Left Null SpaceSetting the dot products with y=[y1y2y3]\nullvec{y} = \nullvec{\begin{bmatrix} y_1 \\ y_2 \\ y_3 \end{bmatrix}} to zero gives y1=−y3\nullvec{y_1} = -\nullvec{y_3} and y2=−y3\nullvec{y_2} = -\nullvec{y_3}, so picking y3=−1\nullvec{y_3} = -1 gives the line perpendicular to the Column Space plane:
y=[11−1]\displaystyle \nullvec{y} = \nullvec{\begin{bmatrix} 1 \\ 1 \\ -1 \end{bmatrix}}
Set y⋅b=0\nullvec{y} \cdot \target{b} = 0 to get the solvability equationA target vector b=[b1b2b3]\target{b} = \target{\begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix}} sits on the Column Space plane if and only if it is perpendicular to the normal vector y\nullvec{y}:
y⋅b=[11−1]⋅[b1b2b3]=b1+b2−b3=0\displaystyle \nullvec{y} \cdot \target{b} = \nullvec{\begin{bmatrix} 1 \\ 1 \\ -1 \end{bmatrix}} \cdot \target{\begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix}} = \target{b_1} + \target{b_2} - \target{b_3} = 0
Test vectors b\target{b} with a dot productChecking b=[347]\target{b} = \target{\begin{bmatrix} 3 \\ 4 \\ 7 \end{bmatrix}} and b=[126]\target{b} = \target{\begin{bmatrix} 1 \\ 2 \\ 6 \end{bmatrix}} with y\nullvec{y} tells us right away whether Ax=bA\vecx{x} = \target{b} has a solution:
y⋅[347]=3+4−7=0 (solvable),y⋅[126]=1+2−6=−3≠0 (no exact solution)\displaystyle \begin{aligned} \nullvec{y} \cdot \target{\begin{bmatrix} 3 \\ 4 \\ 7 \end{bmatrix}} &= 3 + 4 - 7 \\ &= 0 \text{ (solvable)}, \\ \nullvec{y} \cdot \target{\begin{bmatrix} 1 \\ 2 \\ 6 \end{bmatrix}} &= 1 + 2 - 6 \\ &= -3 \neq 0 \\ &\quad \text{ (no exact solution)} \end{aligned}

The system Ax=bA\vecx{x} = \target{b} has an exact solution only when b\target{b} sits flat on the Column Space plane where y⋅b=0\nullvec{y} \cdot \target{b} = 0; moving b\target{b} off the plane gives it a Left Null Space piece that no combination of the columns can reach.

You can solve the system exactly only when your output vector lies flat in the Column Space plane, at a right angle to the Left Null Space.

Key concepts

Every matrix AA splits its input space into the perpendicular pair (Row Space)⊥=Null Space(\keptsp{\text{Row Space}})^\perp = \lostsp{\text{Null Space}} and its output space into the perpendicular pair (Column Space)⊥=Left Null Space(\keptsp{\text{Column Space}})^\perp = \lostsp{\text{Left Null Space}}, where the Left Null Space is all vectors y\nullvec{y} that solve ATy=0A^T \nullvec{y} = 0.

Finding W⊥W^\perp from how WW is described

If WW is given by spanning vectors, such as W=span{[11−2],[12−3]}W = \text{span}\left\{\firstvec{\begin{bmatrix} 1 \\ 1 \\ -2 \end{bmatrix}}, \secondvec{\begin{bmatrix} 1 \\ 2 \\ -3 \end{bmatrix}}\right\}, write them as the rows of AA and solve Ax=0A\vecx{x} = 0 so the Null Space gives W⊥=span{[111]}W^\perp = \text{span}\left\{\nullvec{\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}}\right\}. If WW is already given by homogeneous equations, such as 3x1−x2+5x3=03\vecx{x_1} - \vecx{x_2} + 5\vecx{x_3} = 0, do not row reduce or solve: WW is already the Null Space, so read the coefficients straight off as W⊥=span{[3−15]}W^\perp = \text{span}\left\{\firstvec{\begin{bmatrix} 3 \\ -1 \\ 5 \end{bmatrix}}\right\}.

Splitting x=xrow+xnull\vecx{x} = \xrow{x_{\text{row}}} + \xnull{x_{\text{null}}} and the shortest solution

When the Null Space is a 1D line spanned by n\nullvec{n}, project x\vecx{x} onto n\nullvec{n} first and subtract to get the Row Space piece:

xnull=x⋅nn⋅nn,xrow=x−xnull\displaystyle \begin{aligned} \xnull{x_{\text{null}}} &= \frac{\vecx{x} \cdot \nullvec{n}}{\nullvec{n} \cdot \nullvec{n}} \nullvec{n}, \\ \xrow{x_{\text{row}}} &= \vecx{x} - \xnull{x_{\text{null}}} \end{aligned}

Because Axnull=0A\xnull{x_{\text{null}}} = 0 and ∥x∥2=∥xrow∥2+∥xnull∥2\|\vecx{x}\|^2 = \|\xrow{x_{\text{row}}}\|^2 + \|\xnull{x_{\text{null}}}\|^2, xrow\xrow{x_{\text{row}}} is always the shortest solution to Ax=bA\vecx{x} = \target{b} (for x=[312]\vecx{x} = \vecx{\begin{bmatrix} 3 \\ 1 \\ 2 \end{bmatrix}} and n=[111]\nullvec{n} = \nullvec{\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}}, xnull=[222]\xnull{x_{\text{null}}} = \xnull{\begin{bmatrix} 2 \\ 2 \\ 2 \end{bmatrix}} and the shortest solution is xrow=[1−10]\xrow{x_{\text{row}}} = \xrow{\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}} with ∥xrow∥2=2\|\xrow{x_{\text{row}}}\|^2 = 2).

Testing when Ax=bA\vecx{x} = \target{b} is solvable

Solve ATy=0A^T \nullvec{y} = 0 for the Left Null Space, then check whether y⋅b=0\nullvec{y} \cdot \target{b} = 0 (for y=[11−1]\nullvec{y} = \nullvec{\begin{bmatrix} 1 \\ 1 \\ -1 \end{bmatrix}}, b=[347]\target{b} = \target{\begin{bmatrix} 3 \\ 4 \\ 7 \end{bmatrix}} gives y⋅b=3+4−7=0\nullvec{y} \cdot \target{b} = 3 + 4 - 7 = 0, so Ax=bA\vecx{x} = \target{b} is solvable).

5

Practice

1

Let WW be the subspace of R4\mathbb{R}^4 made of all vectors [x1x2x3x4]\vecx{\begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{bmatrix}} that satisfy the equations 2x1−x2+3x3=02\vecx{x_1} - \vecx{x_2} + 3\vecx{x_3} = 0 and x2+4x3−x4=0\vecx{x_2} + 4\vecx{x_3} - \vecx{x_4} = 0. Without solving the equations, find a basis for W⊥W^\perp and give the dimensions of W⊥W^\perp and WW.

Each equation has the form (Row)⋅x=0(\text{Row}) \cdot \vecx{x} = 0. Which subspace of the coefficient matrix is WW, and which subspace is W⊥W^\perp?

Because WW is given by homogeneous equations, WW is the Null Space of the coefficient matrix [2−130014−1]\begin{bmatrix} \firstvec{2} & \firstvec{-1} & \firstvec{3} & \firstvec{0} \\ \secondvec{0} & \secondvec{1} & \secondvec{4} & \secondvec{-1} \end{bmatrix}, so W⊥W^\perp is its Row Space. (Watch out: solving the equations for x1\vecx{x_1} and x2\vecx{x_2} in terms of x3\vecx{x_3} and x4\vecx{x_4} finds a basis for WW itself, {[−72−410],[12101]}\left\{\begin{bmatrix} -\tfrac{7}{2} \\ -4 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} \tfrac{1}{2} \\ 1 \\ 0 \\ 1 \end{bmatrix}\right\}, and leaving out the 00 coefficients for missing variables gives 3-entry vectors instead of vectors in R4\mathbb{R}^4.) Reading off the two independent rows gives a basis for W⊥W^\perp of {[2−130],[014−1]}\left\{\firstvec{\begin{bmatrix} 2 \\ -1 \\ 3 \\ 0 \end{bmatrix}}, \secondvec{\begin{bmatrix} 0 \\ 1 \\ 4 \\ -1 \end{bmatrix}}\right\}, with dim⁡(W⊥)=2\dim(W^\perp) = 2 and dim⁡(W)=4−2=2\dim(W) = 4 - 2 = 2.
2

Let WW be the subspace of R3\mathbb{R}^3 spanned by v1=[12−1]\firstvec{v_1} = \firstvec{\begin{bmatrix} 1 \\ 2 \\ -1 \end{bmatrix}} and v2=[25−4]\secondvec{v_2} = \secondvec{\begin{bmatrix} 2 \\ 5 \\ -4 \end{bmatrix}}. Find a basis for W⊥W^\perp.

You want all vectors x\vecx{x} whose dot product with both v1\firstvec{v_1} and v2\secondvec{v_2} is zero. How should you put v1\firstvec{v_1} and v2\secondvec{v_2} into a matrix AA so that Ax=0A\vecx{x} = 0 takes those dot products?

Write v1\firstvec{v_1} and v2\secondvec{v_2} as the rows of A=[12−125−4]A = \begin{bmatrix} \firstvec{1} & \firstvec{2} & \firstvec{-1} \\ \secondvec{2} & \secondvec{5} & \secondvec{-4} \end{bmatrix} so that WW is the Row Space and W⊥W^\perp is the Null Space. Row reducing AA gives [10301−2]\begin{bmatrix} \firstvec{1} & \firstvec{0} & \firstvec{3} \\ \secondvec{0} & \secondvec{1} & \secondvec{-2} \end{bmatrix}, which means x1+3x3=0\vecx{x_1} + 3\vecx{x_3} = 0 and x2−2x3=0\vecx{x_2} - 2\vecx{x_3} = 0, or x1=−3x3\vecx{x_1} = -3\vecx{x_3} and x2=2x3\vecx{x_2} = 2\vecx{x_3}. (Be careful to flip the signs when moving 3x33\vecx{x_3} and −2x3-2\vecx{x_3} to the other side of the equals sign; copying the third column directly without flipping signs gives [3−21]\begin{bmatrix} 3 \\ -2 \\ 1 \end{bmatrix}.) Setting x3=1\vecx{x_3} = 1 gives the basis {[−321]}\left\{\nullvec{\begin{bmatrix} -3 \\ 2 \\ 1 \end{bmatrix}}\right\}.
3

For the matrix A=[1−122−13]A = \begin{bmatrix} 1 & -1 & 2 \\ 2 & -1 & 3 \end{bmatrix}, split the input vector x=[152]\vecx{x} = \vecx{\begin{bmatrix} 1 \\ 5 \\ 2 \end{bmatrix}} into its Row Space piece xrow\xrow{x_{\text{row}}} and its Null Space piece xnull\xnull{x_{\text{null}}}.

Compare the dimensions of the Row Space and the Null Space. Which subspace is a 1D line that you can project x\vecx{x} onto first?

Row reducing AA gives [10101−1]\begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & -1 \end{bmatrix}, so the Null Space is the line spanned by n=[−111]\nullvec{n} = \nullvec{\begin{bmatrix} -1 \\ 1 \\ 1 \end{bmatrix}}. Projecting x\vecx{x} onto n\nullvec{n} gives xnull=x⋅nn⋅nn=63[−111]\xnull{x_{\text{null}}} = \frac{\vecx{x} \cdot \nullvec{n}}{\nullvec{n} \cdot \nullvec{n}} \nullvec{n} = \frac{6}{3}\nullvec{\begin{bmatrix} -1 \\ 1 \\ 1 \end{bmatrix}} =[−222]{}= \xnull{\begin{bmatrix} -2 \\ 2 \\ 2 \end{bmatrix}} (make sure to divide by n⋅n=3\nullvec{n} \cdot \nullvec{n} = 3 rather than x⋅x=30\vecx{x} \cdot \vecx{x} = 30, which would multiply n\nullvec{n} by 630=15\frac{6}{30} = \frac{1}{5} and give [−151515]\begin{bmatrix} -\tfrac{1}{5} \\ \tfrac{1}{5} \\ \tfrac{1}{5} \end{bmatrix}). Subtracting from x\vecx{x} gives xrow=x−xnull\xrow{x_{\text{row}}} = \vecx{x} - \xnull{x_{\text{null}}} =[152]−[−222]=[330]{}= \vecx{\begin{bmatrix} 1 \\ 5 \\ 2 \end{bmatrix}} - \xnull{\begin{bmatrix} -2 \\ 2 \\ 2 \end{bmatrix}} = \xrow{\begin{bmatrix} 3 \\ 3 \\ 0 \end{bmatrix}}.
4

Let A=[122311]A = \begin{bmatrix} 1 & 2 \\ 2 & 3 \\ 1 & 1 \end{bmatrix} and b=[b1b2b3]\target{b} = \target{\begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix}}. Find a basis for the Left Null Space of AA, write the equation that b1,b2,b3\target{b_1}, \target{b_2}, \target{b_3} must satisfy for Ax=bA\vecx{x} = \target{b} to be solvable, and check whether [451]\begin{bmatrix} 4 \\ 5 \\ 1 \end{bmatrix} and [232]\begin{bmatrix} 2 \\ 3 \\ 2 \end{bmatrix} lie in the Column Space of AA.

A vector y\nullvec{y} in the Left Null Space must have a zero dot product with every column of AA. Which matrix has the columns of AA as its rows?

Solving ATy=0A^T \nullvec{y} = 0 for AT=[121231]A^T = \begin{bmatrix} \firstvec{1} & \firstvec{2} & \firstvec{1} \\ \secondvec{2} & \secondvec{3} & \secondvec{1} \end{bmatrix} gives the reduced form [10−1011]\begin{bmatrix} \firstvec{1} & \firstvec{0} & \firstvec{-1} \\ \secondvec{0} & \secondvec{1} & \secondvec{1} \end{bmatrix}, so y1=y3\nullvec{y_1} = \nullvec{y_3} and y2=−y3\nullvec{y_2} = -\nullvec{y_3}. (Don't forget to flip the signs on the third-column entries when solving for y1\nullvec{y_1} and y2\nullvec{y_2}: reading off [−111]\begin{bmatrix} -1 \\ 1 \\ 1 \end{bmatrix} without flipping signs gives the wrong condition −b1+b2+b3=0-b_1 + b_2 + b_3 = 0, which makes [451]\begin{bmatrix} 4 \\ 5 \\ 1 \end{bmatrix} give 2≠02 \neq 0.) Setting y3=1\nullvec{y_3} = 1 gives the Left Null Space basis {[1−11]}\left\{\nullvec{\begin{bmatrix} 1 \\ -1 \\ 1 \end{bmatrix}}\right\}, and Ax=bA\vecx{x} = \target{b} is solvable when y⋅b=b1−b2+b3=0\nullvec{y} \cdot \target{b} = \target{b_1} - \target{b_2} + \target{b_3} = 0. For [451]\target{\begin{bmatrix} 4 \\ 5 \\ 1 \end{bmatrix}}, 4−5+1=04 - 5 + 1 = 0 (in the Column Space); for [232]\target{\begin{bmatrix} 2 \\ 3 \\ 2 \end{bmatrix}}, 2−3+2=1≠02 - 3 + 2 = 1 \neq 0 (not in the Column Space).
5

For a 2×32 \times 3 matrix AA, the Null Space is the line spanned by n=[212]\nullvec{n} = \nullvec{\begin{bmatrix} 2 \\ 1 \\ 2 \end{bmatrix}}, and x=[523]\vecx{x} = \vecx{\begin{bmatrix} 5 \\ 2 \\ 3 \end{bmatrix}} is one solution to Ax=[3−1]A\vecx{x} = \target{\begin{bmatrix} 3 \\ -1 \end{bmatrix}}. Without knowing the entries of AA, find the shortest solution to Ax=[3−1]A\vecx{x} = \target{\begin{bmatrix} 3 \\ -1 \end{bmatrix}} and its squared length.

Changing x\vecx{x} along the Null Space line doesn't change AxA\vecx{x}. How can you remove the Null Space piece xnull\xnull{x_{\text{null}}} from x=[523]\vecx{x} = \vecx{\begin{bmatrix} 5 \\ 2 \\ 3 \end{bmatrix}}?

Since Ax=AxrowA\vecx{x} = A\xrow{x_{\text{row}}} and ∥x∥2=∥xrow∥2+∥xnull∥2\|\vecx{x}\|^2 = \|\xrow{x_{\text{row}}}\|^2 + \|\xnull{x_{\text{null}}}\|^2, the shortest solution is the Row Space piece xrow\xrow{x_{\text{row}}} of x\vecx{x}. Projecting x\vecx{x} onto n\nullvec{n} gives xnull=x⋅nn⋅nn=189[212]=[424]\xnull{x_{\text{null}}} = \frac{\vecx{x} \cdot \nullvec{n}}{\nullvec{n} \cdot \nullvec{n}} \nullvec{n} = \frac{18}{9}\nullvec{\begin{bmatrix} 2 \\ 1 \\ 2 \end{bmatrix}} = \xnull{\begin{bmatrix} 4 \\ 2 \\ 4 \end{bmatrix}}. (Don't stop at [424]\begin{bmatrix} 4 \\ 2 \\ 4 \end{bmatrix} and give its squared length 3636: that projection is xnull\xnull{x_{\text{null}}}, which AA sends to [00]\begin{bmatrix} 0 \\ 0 \end{bmatrix} instead of [3−1]\target{\begin{bmatrix} 3 \\ -1 \end{bmatrix}}.) Subtracting gives the shortest solution xrow=x−xnull=[523]−[424]\xrow{x_{\text{row}}} = \vecx{x} - \xnull{x_{\text{null}}} = \vecx{\begin{bmatrix} 5 \\ 2 \\ 3 \end{bmatrix}} - \xnull{\begin{bmatrix} 4 \\ 2 \\ 4 \end{bmatrix}} =[10−1]{}= \xrow{\begin{bmatrix} 1 \\ 0 \\ -1 \end{bmatrix}}, with squared length ∥xrow∥2=12+02+(−1)2=2\|\xrow{x_{\text{row}}}\|^2 = \xrow{1}^2 + \xrow{0}^2 + (\xrow{-1})^2 = 2.