Orthogonal Complements: Row Space Meets Null Space
We know that the Row Space and the Null Space of a matrix both live in the input space, and we want to see how they fit together geometrically.
1
Why the Row Space and Null Space Are Orthogonal Complements
Two subspaces are orthogonal complements when every vector in one is perpendicular (90∘) to every vector in the other, and their dimensions add up to the full space.
To see why the Row Space and the Null Space of any matrix always form orthogonal complements, let's look at what Ax=0 says about the rows of A=[1112−2−3] and an input vector x=x1x2x3.
Why the Row Space and Null Space Are Orthogonal Complements
Text solution
Write Ax=0 as dot products with the rowsMultiplying A by x takes the dot product of each row with x, so being in the Null Space means x is perpendicular to every row of A at once:
Ax=[(Row 1)⋅x(Row 2)⋅x]=[00]
Row reduce A to get cleaner vectors on the same Row Space planeSubtracting Row 1 from Row 2, and then subtracting the new Row 2 from Row 1, gives us simpler combinations of the rows on the exact same plane:
[1112−2−3]→[1011−2−1]→[1001−1−1]
Solve for the Null Space line nSetting the dot products of the reduced rows with x to zero gives x1−x3=0 and x2−x3=0, so choosing the free variable x3=1 gives x1=1 and x2=1:
The perpendicular symbol ⊥ (read "perp") stands for the orthogonal complement: all vectors perpendicular to that subspace. Because perpendicularity goes both ways and dim(Row Space)+dim(Null Space)=2+1=3, the two subspaces split the 3D input space completely, with (Null Space)⊥=Row Space as well.
Every linear combination of Row 1 and Row 2, including the new rows formed during row reduction, lies in the same flat Row Space plane at a 90∘ angle to the Null Space line, so its dot product with n=111 is still zero.
Every combination of Row 1 and Row 2, including the replacement rows from row reduction, lies in the same flat plane at 90° to the Null Space line.
2
Finding the Orthogonal Complement of a Subspace
Even when a problem doesn't mention matrices, we can use a matrix to find the orthogonal complement W⊥ of a subspace W. Because the Row Space and Null Space are orthogonal complements, we just set up a matrix so that W is one of those two spaces, and the other one gives us W⊥.
Finding the Orthogonal Complement of a Subspace
W=span{v1,v2}
v1=11−2,v2=12−3
basis for W⊥=?
Text solution
When W is given by spanning vectors, write them as rows and solve Ax=0For W=span⎩⎨⎧11−2,12−3⎭⎬⎫, making the spanning vectors the rows of A makes W the Row Space, so the Null Space gives W⊥:
A=[1112−2−3]→[1001−1−1]⟹W⊥=Null Space of A=span⎩⎨⎧111⎭⎬⎫
When W is given by homogeneous equations, read the coefficients off as rowsFor the subspace W where x1+x2−2x3=0 and x1+2x2−3x3=0, the equations already have the form Ax=0, so W is the Null Space and the coefficient vectors immediately span W⊥:
W=Null Space of A[1112−2−3]x1x2x3=[00]⟹W⊥=Row Space of A=span⎩⎨⎧11−2,12−3⎭⎬⎫
When a problem gives W as homogeneous equations, don't jump in and solve them: solving Ax=0 finds a basis for W itself, while the vectors for W⊥ are already right there in the equation coefficients.
3
Splitting Input Vectors and the Shortest Solution
Now we want to split any input vector x into a Row Space piece xrow and a perpendicular Null Space piece xnull, and find the shortest vector that solves Ax=b.
Because projecting onto a 1D line is simpler than projecting onto a 2D plane, we find xnull first by projecting x onto the Null Space direction vector n:
xnull=n⋅nx⋅nn
Let's split x=312 for A=[1112−2−3] and see why xrow is the shortest solution to Ax=[0−1].
Splitting Input Vectors and the Shortest Solution
Text solution
Project x onto the Null Space line n=111Using the line projection formula gives the part of x along the Null Space:
Subtract xnull from x to get the Row Space pieceSince the two pieces add up to x, whatever is left over lies flat on the Row Space plane:
xrow=x−xnull=312−222=1−10
Compare outputs and squared lengthsMultiplying by A sends xnull to zero, while the right triangle between xrow and xnull makes x longer by the Pythagorean theorem:
So xrow=1−10 is the unique shortest solution to Ax=[0−1].
Sliding x parallel to the Null Space line changes xnull and the length of the hypotenuse x, which is longer than xrow whenever xnull=0, while the Row Space leg xrow and the output Ax=Axrow stay the same.
Sliding an input along the Null Space never changes its output, and the piece in the Row Space is the shortest input that gets you there.
4
Column Space, Left Null Space, and When Ax=b Is Solvable
Over in the output space, we want a quick test for whether a right-hand side vector b lies in the Column Space so that Ax=b has a solution.
When we transpose A, its columns become the rows of AT. That means the Left Null Space (all output vectors y that solve ATy=0) is perpendicular to every column of A, making the Column Space and the Left Null Space orthogonal complements in the output space:
Null Space of ATLeft Null Space=(Column Space)⊥
Let's find the Left Null Space for the 3×2 matrix A=112011 and use it to test which vectors b=b1b2b3 make Ax=b solvable.
Column Space, Left Null Space, and When Ax = b Is Solvable
A=112011
(Column Space)⊥=?
Ax=b solvable for which b?
Text solution
Transpose A and row reduce ATy=0Writing the columns of A as the rows of AT and subtracting Row 2 from Row 1 simplifies the system for the Left Null Space:
AT=[101121]→[100111]
Find the normal vector y spanning the Left Null SpaceSetting the dot products with y=y1y2y3 to zero gives y1=−y3 and y2=−y3, so picking y3=−1 gives the line perpendicular to the Column Space plane:
y=11−1
Set y⋅b=0 to get the solvability equationA target vector b=b1b2b3 sits on the Column Space plane if and only if it is perpendicular to the normal vector y:
y⋅b=11−1⋅b1b2b3=b1+b2−b3=0
Test vectors b with a dot productChecking b=347 and b=126 with y tells us right away whether Ax=b has a solution:
y⋅347y⋅126=3+4−7=0 (solvable),=1+2−6=−3=0 (no exact solution)
The system Ax=b has an exact solution only when b sits flat on the Column Space plane where y⋅b=0; moving b off the plane gives it a Left Null Space piece that no combination of the columns can reach.
You can solve the system exactly only when your output vector lies flat in the Column Space plane, at a right angle to the Left Null Space.
Key concepts
Every matrix A splits its input space into the perpendicular pair (Row Space)⊥=Null Space and its output space into the perpendicular pair (Column Space)⊥=Left Null Space, where the Left Null Space is all vectors y that solve ATy=0.
Finding W⊥ from how W is described
If W is given by spanning vectors, such as W=span⎩⎨⎧11−2,12−3⎭⎬⎫, write them as the rows of A and solve Ax=0 so the Null Space gives W⊥=span⎩⎨⎧111⎭⎬⎫. If W is already given by homogeneous equations, such as 3x1−x2+5x3=0, do not row reduce or solve: W is already the Null Space, so read the coefficients straight off as W⊥=span⎩⎨⎧3−15⎭⎬⎫.
Splitting x=xrow+xnull and the shortest solution
When the Null Space is a 1D line spanned by n, project x onto n first and subtract to get the Row Space piece:
xnullxrow=n⋅nx⋅nn,=x−xnull
Because Axnull=0 and ∥x∥2=∥xrow∥2+∥xnull∥2,xrow is always the shortest solution to Ax=b (for x=312 and n=111,xnull=222 and the shortest solution is xrow=1−10 with ∥xrow∥2=2).
Testing when Ax=b is solvable
Solve ATy=0 for the Left Null Space, then check whether y⋅b=0 (for y=11−1,b=347 gives y⋅b=3+4−7=0, so Ax=b is solvable).
5
Practice
1
Let W be the subspace of R4 made of all vectors x1x2x3x4 that satisfy the equations 2x1−x2+3x3=0 and x2+4x3−x4=0. Without solving the equations, find a basis for W⊥ and give the dimensions of W⊥ and W.
Each equation has the form (Row)⋅x=0. Which subspace of the coefficient matrix is W, and which subspace is W⊥?
Because W is given by homogeneous equations, W is the Null Space of the coefficient matrix [20−11340−1], so W⊥ is its Row Space. (Watch out: solving the equations for x1 and x2 in terms of x3 and x4 finds a basis for W itself, ⎩⎨⎧−27−410,21101⎭⎬⎫, and leaving out the 0 coefficients for missing variables gives 3-entry vectors instead of vectors in R4.) Reading off the two independent rows gives a basis for W⊥ of ⎩⎨⎧2−130,014−1⎭⎬⎫, with dim(W⊥)=2 and dim(W)=4−2=2.
2
Let W be the subspace of R3 spanned by v1=12−1 and v2=25−4. Find a basis for W⊥.
You want all vectors x whose dot product with both v1 and v2 is zero. How should you put v1 and v2 into a matrix A so that Ax=0 takes those dot products?
Write v1 and v2 as the rows of A=[1225−1−4] so that W is the Row Space and W⊥ is the Null Space. Row reducing A gives [10013−2], which means x1+3x3=0 and x2−2x3=0, or x1=−3x3 and x2=2x3. (Be careful to flip the signs when moving 3x3 and −2x3 to the other side of the equals sign; copying the third column directly without flipping signs gives 3−21.) Setting x3=1 gives the basis ⎩⎨⎧−321⎭⎬⎫.
3
For the matrix A=[12−1−123], split the input vector x=152 into its Row Space piece xrow and its Null Space piece xnull.
Compare the dimensions of the Row Space and the Null Space. Which subspace is a 1D line that you can project x onto first?
Row reducing A gives [10011−1], so the Null Space is the line spanned by n=−111. Projecting x onto n gives xnull=n⋅nx⋅nn=36−111=−222 (make sure to divide by n⋅n=3 rather than x⋅x=30, which would multiply n by 306=51 and give −515151). Subtracting from x gives xrow=x−xnull=152−−222=330.
4
Let A=121231 and b=b1b2b3. Find a basis for the Left Null Space of A, write the equation that b1,b2,b3 must satisfy for Ax=b to be solvable, and check whether 451 and 232 lie in the Column Space of A.
A vector y in the Left Null Space must have a zero dot product with every column of A. Which matrix has the columns of A as its rows?
Solving ATy=0 for AT=[122311] gives the reduced form [1001−11], so y1=y3 and y2=−y3. (Don't forget to flip the signs on the third-column entries when solving for y1 and y2: reading off −111 without flipping signs gives the wrong condition −b1+b2+b3=0, which makes 451 give 2=0.) Setting y3=1 gives the Left Null Space basis ⎩⎨⎧1−11⎭⎬⎫, and Ax=b is solvable when y⋅b=b1−b2+b3=0. For 451,4−5+1=0 (in the Column Space); for 232,2−3+2=1=0 (not in the Column Space).
5
For a 2×3 matrix A, the Null Space is the line spanned by n=212, and x=523 is one solution to Ax=[3−1]. Without knowing the entries of A, find the shortest solution to Ax=[3−1] and its squared length.
Don't stop at 424 and give its squared length 36: that projection is xnull, which A sends to [00] instead of [3−1].
Changing x along the Null Space line doesn't change Ax. How can you remove the Null Space piece xnull from x=523?
Since Ax=Axrow and ∥x∥2=∥xrow∥2+∥xnull∥2, the shortest solution is the Row Space piece xrow of x. Projecting x onto n gives xnull=n⋅nx⋅nn=918212=424. (Don't stop at 424 and give its squared length 36: that projection is xnull, which A sends to [00] instead of [3−1].) Subtracting gives the shortest solution xrow=x−xnull=523−424=10−1, with squared length ∥xrow∥2=12+02+(−1)2=2.