Linear AlgebraVector Spaces and Subspaces4 of 9
Vector Spaces and Subspaces · 4

The Four Fundamental Subspaces

When a matrix A\act{A} multiplies an input vector xx to produce an output vector b\out{b}, we want to know which outputs AA can actually reach and which inputs produce them.

1

The Column Space

The Column Space C(A)\colsp{C(A)} is the space of all outputs b\out{b} that a matrix AA can reach by combining its columns. For the 3×43 \times 4 matrix

A=[1213243736410]\displaystyle A = \begin{bmatrix} 1 & 2 & 1 & 3 \\ 2 & 4 & 3 & 7 \\ 3 & 6 & 4 & 10 \end{bmatrix}

inputs have 44 components (in R4\mathbb{R}^4) and outputs have 33 components (in R3\mathbb{R}^3). To find a basis for C(A)\colsp{C(A)}, we keep only the columns that point in new directions and drop the redundant ones.

The Column Space
A\htmlClass{lc-violet}{A}
[1213243736410]\begin{bmatrix} 1 & 2 & 1 & 3 \\ 2 & 4 & 3 & 7 \\ 3 & 6 & 4 & 10 \end{bmatrix}
input space R4\text{input space } \mathbb{R}^4
x=[x1x2x3x4]x = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{bmatrix}
output space R3\text{output space } \mathbb{R}^3
b=[b1b2b3]\htmlClass{lc-blue}{b} = \htmlClass{lc-blue}{\begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix}}
C(A)=  ?\htmlClass{lc-blue}{C(A)} = \;?
Text solution
Combine the columns of AA to form an outputMultiplying AA by an input xx uses the four entries of xx as weights on the columns of AA to build an output b\out{b} in R3\mathbb{R}^3:
x1[123]+x2[246]+x3[134]+x4[3710]=[b1b2b3]⏟reachable output in C(A)\displaystyle x_1 \pivone{\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}} + x_2 \colmult{\begin{bmatrix} 2 \\ 4 \\ 6 \end{bmatrix}} + x_3 \pivtwo{\begin{bmatrix} 1 \\ 3 \\ 4 \end{bmatrix}} + x_4 \colbuilt{\begin{bmatrix} 3 \\ 7 \\ 10 \end{bmatrix}} = \underbrace{\out{\begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix}}}_{\text{reachable output in } \colsp{C(A)}}
Spot the redundant columnsColumn 2 is double Column 1, and Column 4 is built from Columns 1 and 3, so neither one lets us reach a new direction in R3\mathbb{R}^3:
[246]⏟Column 2=2[123],[3710]⏟Column 4=2[123]+1[134]\displaystyle \underbrace{\colmult{\begin{bmatrix} 2 \\ 4 \\ 6 \end{bmatrix}}}_{\colmult{\text{Column 2}}} = 2 \pivone{\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}}, \qquad \underbrace{\colbuilt{\begin{bmatrix} 3 \\ 7 \\ 10 \end{bmatrix}}}_{\colbuilt{\text{Column 4}}} = 2 \pivone{\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}} + 1 \pivtwo{\begin{bmatrix} 1 \\ 3 \\ 4 \end{bmatrix}}
Keep the independent columns for the basisOnly Columns 1 and 3 point in new directions, spanning a 2D plane inside R3\mathbb{R}^3:
Basis for C(A)={[123],[134]}\displaystyle \text{Basis for } \colsp{C(A)} = \left\{ \pivone{\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}}, \pivtwo{\begin{bmatrix} 1 \\ 3 \\ 4 \end{bmatrix}} \right\}

In 3D output space, every multiple x4[3710]x_4 \colbuilt{\begin{bmatrix} 3 \\ 7 \\ 10 \end{bmatrix}} of the redundant fourth column reaches the exact same point as 2x4[123]+x4[134]2x_4 \pivone{\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}} + x_4 \pivtwo{\begin{bmatrix} 1 \\ 3 \\ 4 \end{bmatrix}}, so adding the fourth column never moves the output off the 2D Column Space plane.

Every multiple of the fourth column stays inside the 2D plane, reaching the exact same point that a matching combination of the first and third columns already reaches on its own.
2

Finding the Column Space and Null Space from RR

When a matrix is too complicated to spot redundant columns by eye, we use elimination to get the reduced row echelon form RR. From RR, we can find both the Column Space C(A)\colsp{C(A)} in the output space and the Null Space N(A)\nullsp{N(A)}: the space of all inputs xx in R4\mathbb{R}^4 that the matrix sends to zero (Ax=0\act{A}x = 0).

Finding the Column Space and Null Space from R
A=[1213243736410]\displaystyle A = \begin{bmatrix} \pivone{1} & \colmult{2} & \pivtwo{1} & \colbuilt{3} \\ \pivone{2} & \colmult{4} & \pivtwo{3} & \colbuilt{7} \\ \pivone{3} & \colmult{6} & \pivtwo{4} & \colbuilt{10} \end{bmatrix}
C(A)\displaystyle \colsp{C(A)} =  ?N(A)\displaystyle = \;? \qquad \nullsp{N(A)} =  ?\displaystyle = \;?
Text solution
Subtract multiples of Row 1 to clear the first columnSubtracting 2×Row 12 \times \text{Row 1} from Row 2 and 3×Row 13 \times \text{Row 1} from Row 3 zeroes out the entries below the top-left pivot without changing how the columns relate to one another:
[1213243736410]→R2←R2−2R1R3←R3−3R1[121300110011]\displaystyle \begin{bmatrix} \pivone{1} & \colmult{2} & \pivtwo{1} & \colbuilt{3} \\ \pivone{2} & \colmult{4} & \pivtwo{3} & \colbuilt{7} \\ \pivone{3} & \colmult{6} & \pivtwo{4} & \colbuilt{10} \end{bmatrix} \xrightarrow{\substack{R_2 \leftarrow R_2 - 2R_1 \\ R_3 \leftarrow R_3 - 3R_1}} \begin{bmatrix} \pivone{1} & \colmult{2} & \pivtwo{1} & \colbuilt{3} \\ \pivone{0} & \colmult{0} & \pivtwo{1} & \colbuilt{1} \\ \pivone{0} & \colmult{0} & \pivtwo{1} & \colbuilt{1} \end{bmatrix}
Use Row 2 to clear the third column and get RRSubtracting Row 2 from Row 3 and from Row 1 isolates the pivots in Columns 1 and 3, showing r=2\rank{r} = 2 independent columns:
[121300110011]→R3←R3−R2R1←R1−R2[120200110000]⏟R (pivots in Columns 1 and 3)\displaystyle \begin{bmatrix} \pivone{1} & \colmult{2} & \pivtwo{1} & \colbuilt{3} \\ \pivone{0} & \colmult{0} & \pivtwo{1} & \colbuilt{1} \\ \pivone{0} & \colmult{0} & \pivtwo{1} & \colbuilt{1} \end{bmatrix} \xrightarrow{\substack{R_3 \leftarrow R_3 - R_2 \\ R_1 \leftarrow R_1 - R_2}} \underbrace{\begin{bmatrix} \pivone{\mathbf{1}} & \colmult{2} & \pivtwo{0} & \colbuilt{2} \\ \pivone{0} & \colmult{0} & \pivtwo{\mathbf{1}} & \colbuilt{1} \\ 0 & 0 & 0 & 0 \end{bmatrix}}_{R \text{ (pivots in Columns 1 and 3)}}
Take the pivot columns from the original matrix AA for C(A)\colsp{C(A)}Row operations change the columns so the columns of RR lie in the plane z=0z = 0. We use RR only to see that Columns 1 and 3 have the pivots, then go back to AA for the vectors:
Basis for C(A)={[123],[134]}\displaystyle \text{Basis for } \colsp{C(A)} = \left\{ \pivone{\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}}, \pivtwo{\begin{bmatrix} 1 \\ 3 \\ 4 \end{bmatrix}} \right\}
Read the special solutions for N(A)\nullsp{N(A)} from the free columns of RRSetting one free variable (x2x_2 or x4x_4) to 11 and the other to 00 gives the input weights that make the columns cancel out to Ax=0\act{A}x = 0:
Basis for N(A)={[−2100]⏟x2=1, x4=0,[−20−11]⏟x2=0, x4=1}\displaystyle \text{Basis for } \nullsp{N(A)} = \left\{ \underbrace{\nullsp{\begin{bmatrix} -2 \\ 1 \\ 0 \\ 0 \end{bmatrix}}}_{x_2 = 1,\, x_4 = 0}, \underbrace{\nullsp{\begin{bmatrix} -2 \\ 0 \\ -1 \\ 1 \end{bmatrix}}}_{x_2 = 0,\, x_4 = 1} \right\}
3

The Row Space

Looking at the columns of AA gave us the Column Space in R3\mathbb{R}^3. Looking at the rows of AA gives us the Row Space: all combinations of the rows of AA, which have 44 numbers each and live in the 4D input space R4\mathbb{R}^4. To write rows as standard column vectors, we take the transpose ATA^T so the rows of AA become the columns of ATA^T, and write the Row Space as C(AT)\rowsp{C(A^T)}.

The Row Space
input space R4\text{input space } \mathbb{R}^4
output space R3\text{output space } \mathbb{R}^3
C(A)\htmlClass{lc-blue}{C(A)}
N(A)\htmlClass{lc-orange}{N(A)}
?\htmlClass{lc-blue}{?}
?\htmlClass{lc-orange}{?}
A\htmlClass{lc-violet}{A}
A\htmlClass{lc-violet}{A}
00
Text solution
Turn the rows of AA into columns by taking the transpose ATA^TTaking the transpose puts the three 4-component rows of AA into column form in R4\mathbb{R}^4:
AT=[1232461343710]\displaystyle A^T = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \\ 1 & 3 & 4 \\ 3 & 7 & 10 \end{bmatrix}
Take the non-zero rows of RR for the basis of C(AT)\rowsp{C(A^T)}Because row operations only combine rows with one another, they never leave the Row Space, so we can write the non-zero rows of RR as column vectors without going back to AA:
Basis for C(AT)={[1202],[0011]}\displaystyle \text{Basis for } \rowsp{C(A^T)} = \left\{ \rowsp{\begin{bmatrix} 1 \\ 2 \\ 0 \\ 2 \end{bmatrix}}, \rowsp{\begin{bmatrix} 0 \\ 0 \\ 1 \\ 1 \end{bmatrix}} \right\}
Check that the Row Space and Null Space are perpendicularMultiplying AA by xx takes the dot product of each row with xx, so Ax=0\act{A}x = 0 means every vector in N(A)\nullsp{N(A)} has a dot product of zero with every vector in C(AT)\rowsp{C(A^T)}:
[1202]⋅[−2100]=(1)(−2)+(2)(1)+(0)(0)+(2)(0)=0\displaystyle \rowsp{\begin{bmatrix} 1 \\ 2 \\ 0 \\ 2 \end{bmatrix}} \cdot \nullsp{\begin{bmatrix} -2 \\ 1 \\ 0 \\ 0 \end{bmatrix}} = (\rowsp{1})(\nullsp{-2}) + (\rowsp{2})(\nullsp{1}) + (\rowsp{0})(\nullsp{0}) + (\rowsp{2})(\nullsp{0}) = 0
[1202]⋅[−20−11]=(1)(−2)+(2)(0)+(0)(−1)+(2)(1)=0\displaystyle \rowsp{\begin{bmatrix} 1 \\ 2 \\ 0 \\ 2 \end{bmatrix}} \cdot \nullsp{\begin{bmatrix} -2 \\ 0 \\ -1 \\ 1 \end{bmatrix}} = (\rowsp{1})(\nullsp{-2}) + (\rowsp{2})(\nullsp{0}) + (\rowsp{0})(\nullsp{-1}) + (\rowsp{2})(\nullsp{1}) = 0
4

The Left Null Space

In the 4D input space, the Null Space N(A)\nullsp{N(A)} is perpendicular to the Row Space C(AT)\rowsp{C(A^T)}. In the 3D output space, the matching space that is perpendicular to the 2D Column Space plane C(A)\colsp{C(A)} is the Left Null Space N(AT)\leftnull{N(A^T)}: all vectors y\leftnull{y} whose weights combine the rows of AA into a zero row (yTA=0T\leftnull{y^T} \act{A} = 0^T, or ATy=0\act{A^T} \leftnull{y} = 0).

The Left Null Space
Text solution
Write the row combination that produced the zero row of RRIn AA, Row 3 is Row 1 plus Row 2, so subtracting Row 1 and Row 2 from Row 3 cancels all entries to zero:
−1(Row 1)−1(Row 2)+1(Row 3)=[0000]\displaystyle \leftnull{-1}(\text{Row 1}) - \leftnull{1}(\text{Row 2}) + \leftnull{1}(\text{Row 3}) = \begin{bmatrix} 0 & 0 & 0 & 0 \end{bmatrix}
Collect the row weights into a basis vector y\leftnull{y} for N(AT)\leftnull{N(A^T)}Writing those weights as a column vector y\leftnull{y} satisfies yTA=0T\leftnull{y^T} \act{A} = 0^T, or ATy=0\act{A^T} \leftnull{y} = 0:
Basis for N(AT)={[−1−11]}\displaystyle \text{Basis for } \leftnull{N(A^T)} = \left\{ \leftnull{\begin{bmatrix} -1 \\ -1 \\ 1 \end{bmatrix}} \right\}
Check that N(AT)\leftnull{N(A^T)} is perpendicular to C(A)\colsp{C(A)}Because −1(top)−1(middle)\leftnull{-1}(\text{top}) - \leftnull{1}(\text{middle}) +1(bottom)=0{}+ \leftnull{1}(\text{bottom}) = 0 in every column of AA, any output b\out{b} built from those columns has a dot product of zero with y\leftnull{y}:
[−1−11]⋅[123]=−1−2+3=0,[−1−11]⋅[134]=−1−3+4=0\displaystyle \leftnull{\begin{bmatrix} -1 \\ -1 \\ 1 \end{bmatrix}} \cdot \pivone{\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}} = -1 - 2 + 3 = 0, \qquad \leftnull{\begin{bmatrix} -1 \\ -1 \\ 1 \end{bmatrix}} \cdot \pivtwo{\begin{bmatrix} 1 \\ 3 \\ 4 \end{bmatrix}} = -1 - 3 + 4 = 0

In 3D output space, the dot product y⋅b=−b1−b2+b3\leftnull{y} \cdot \out{b} = -\out{b_1} - \out{b_2} + \out{b_3} equals 00 and the angle between b\out{b} and y=[−1−11]\leftnull{y} = \leftnull{\begin{bmatrix} -1 \\ -1 \\ 1 \end{bmatrix}} is 90∘90^\circ precisely when b\out{b} lies in the Column Space plane C(A)\colsp{C(A)}.

A vector in 3D output space lies on the column space plane right when it sits at a right angle to the left null space vector, giving a dot product of zero.
5

Finding All Four Subspaces with [A∣I][A \mid I]

When you cannot spot by eye which combination of rows made the zero row in RR, attach an identity matrix II on the right of AA before row-reducing. Doing the same row steps across [A∣I][A \mid I] turns the left side into RR while the right side automatically records the row combinations, letting you read off bases for all four subspaces from one matrix.

Finding All Four Subspaces with [A | I]
[1213100243701036410001]\displaystyle \left[\begin{array}{cccc|ccc} 1 & 2 & 1 & 3 & 1 & 0 & 0 \\ 2 & 4 & 3 & 7 & 0 & 1 & 0 \\ 3 & 6 & 4 & 10 & 0 & 0 & 1 \end{array}\right]
C(AT)=  ?C(A)=  ?N(A)=  ?N(AT)=  ?\displaystyle \begin{alignedat}{2} \rowsp{C(A^T)} &= \;? &\qquad \colsp{C(A)} &= \;? \\ \nullsp{N(A)} &= \;? &\qquad \leftnull{N(A^T)} &= \;? \end{alignedat}
Text solution
Attach the 3×33 \times 3 identity matrix II on the right of AAThe ones down the diagonal of II keep track of how many of Row 1, Row 2, and Row 3 we use:
[A∣I]=[1213100243701036410001]\displaystyle [A \mid I] = \left[\begin{array}{cccc|ccc} 1 & 2 & 1 & 3 & 1 & 0 & 0 \\ 2 & 4 & 3 & 7 & 0 & 1 & 0 \\ 3 & 6 & 4 & 10 & 0 & 0 & 1 \end{array}\right]
Clear the first column across both sidesSubtracting 2×Row 12 \times \text{Row 1} from Row 2 and 3×Row 13 \times \text{Row 1} from Row 3 records −2-2 and −3-3 on the right side:
[1213100243701036410001]→R2←R2−2R1R3←R3−3R1[12131000011−2100011−301]\displaystyle \begin{gathered} \left[\begin{array}{cccc|ccc} 1 & 2 & 1 & 3 & 1 & 0 & 0 \\ 2 & 4 & 3 & 7 & 0 & 1 & 0 \\ 3 & 6 & 4 & 10 & 0 & 0 & 1 \end{array}\right] \\ \xrightarrow{\substack{R_2 \leftarrow R_2 - 2R_1 \\ R_3 \leftarrow R_3 - 3R_1}} \\ \left[\begin{array}{cccc|ccc} 1 & 2 & 1 & 3 & 1 & 0 & 0 \\ 0 & 0 & 1 & 1 & -2 & 1 & 0 \\ 0 & 0 & 1 & 1 & -3 & 0 & 1 \end{array}\right] \end{gathered}
Clear the third column across both sides to reach RR on the leftSubtracting the new Row 2 from Row 3 and from Row 1 finishes RR on the left and leaves the row combination for the zero row at the bottom right:
[12131000011−2100011−301]→R3←R3−R2R1←R1−R2[12023−100011−2100000−1−11]\displaystyle \begin{gathered} \left[\begin{array}{cccc|ccc} 1 & 2 & 1 & 3 & 1 & 0 & 0 \\ 0 & 0 & 1 & 1 & -2 & 1 & 0 \\ 0 & 0 & 1 & 1 & -3 & 0 & 1 \end{array}\right] \\ \xrightarrow{\substack{R_3 \leftarrow R_3 - R_2 \\ R_1 \leftarrow R_1 - R_2}} \\ \left[\begin{array}{cccc|ccc} \pivone{\mathbf{1}} & 2 & 0 & 2 & 3 & -1 & 0 \\ 0 & 0 & \pivtwo{\mathbf{1}} & 1 & -2 & 1 & 0 \\ 0 & 0 & 0 & 0 & \leftnull{\mathbf{-1}} & \leftnull{\mathbf{-1}} & \leftnull{\mathbf{1}} \end{array}\right] \end{gathered}
Read all four bases from the reduced matrix and AATake the non-zero left rows for C(AT)\rowsp{C(A^T)}, the free left columns for N(A)\nullsp{N(A)}, the pivot columns from the original matrix AA for C(A)\colsp{C(A)}, and the right-hand row opposite the zero row for N(AT)\leftnull{N(A^T)}:
Basis for C(AT)={[1202],[0011]},Basis for N(A)={[−2100],[−20−11]}\displaystyle \begin{aligned} &\text{Basis for } \rowsp{C(A^T)} = \\ &\qquad \left\{ \rowsp{\begin{bmatrix} 1 \\ 2 \\ 0 \\ 2 \end{bmatrix}}, \rowsp{\begin{bmatrix} 0 \\ 0 \\ 1 \\ 1 \end{bmatrix}} \right\}, \\ &\text{Basis for } \nullsp{N(A)} = \\ &\qquad \left\{ \nullsp{\begin{bmatrix} -2 \\ 1 \\ 0 \\ 0 \end{bmatrix}}, \nullsp{\begin{bmatrix} -2 \\ 0 \\ -1 \\ 1 \end{bmatrix}} \right\} \end{aligned}
Basis for C(A)={[123],[134]},Basis for N(AT)={[−1−11]}\displaystyle \begin{aligned} \text{Basis for } \colsp{C(A)} &= \left\{ \pivone{\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}}, \pivtwo{\begin{bmatrix} 1 \\ 3 \\ 4 \end{bmatrix}} \right\}, \\ \text{Basis for } \leftnull{N(A^T)} &= \left\{ \leftnull{\begin{bmatrix} -1 \\ -1 \\ 1 \end{bmatrix}} \right\} \end{aligned}
6

Dimensions of the Four Subspaces

Even if you are not given the numbers inside an m×nm \times n matrix AA, knowing its size and its rank r\rank{r} (the number of pivots in RR) tells you the dimensions of all four fundamental subspaces and which spaces they live in.

Dimensions of the Four Subspaces
nn
mm
R=R =
rank r\text{rank } \htmlClass{lc-blue}{r}
dim⁡C(AT)=  ?dim⁡C(A)=  ?dim⁡N(A)=  ?dim⁡N(AT)=  ?\begin{alignedat}{2} \dim \rowsp{C(A^T)} &= \;? &\qquad \dim \colsp{C(A)} &= \;? \\[2pt] \dim \nullsp{N(A)} &= \;? &\qquad \dim \leftnull{N(A^T)} &= \;? \end{alignedat}

For any m×nm \times n matrix AA with rank r\rank{r}:

In the input space Rn:dim⁡C(AT)=r⏟non-zero pivot rows,dim⁡N(A)=n−r⏟free columnsIn the output space Rm:dim⁡C(A)=r⏟pivot columns,dim⁡N(AT)=m−r⏟zero rows\displaystyle \begin{aligned} & \text{In the input space } \mathbb{R}^n: \\ \dim \rowsp{C(A^T)} &= \underbrace{\rank{r}}_{\text{non-zero pivot rows}}, \\ \dim \nullsp{N(A)} &= \underbrace{\nulldim{n - r}}_{\text{free columns}} \\[0.6em] & \text{In the output space } \mathbb{R}^m: \\ \dim \colsp{C(A)} &= \underbrace{\rank{r}}_{\text{pivot columns}}, \\ \dim \leftnull{N(A^T)} &= \underbrace{\nulldim{m - r}}_{\text{zero rows}} \end{aligned}

Here are the steps for a 5×75 \times 7 matrix AA of rank r=3\rank{r} = 3:

Text solution
Match the matrix size 5×75 \times 7 to the input and output spacesWith m=5m = 5 rows and n=7n = 7 columns, inputs have 77 components (in R7\mathbb{R}^7) and outputs have 55 components (in R5\mathbb{R}^5).
Split the input space R7\mathbb{R}^7 using the r=3\rank{r} = 3 pivotsThe 33 pivots sit in 33 non-zero rows for C(AT)\rowsp{C(A^T)} and leave 7−3=47 - 3 = 4 free columns for N(A)\nullsp{N(A)}, accounting for all 77 dimensions of R7\mathbb{R}^7:
dim⁡C(AT)\displaystyle \dim \rowsp{C(A^T)} =r\displaystyle = \rank{r} =3,dim⁡N(A)\displaystyle = 3, \qquad \dim \nullsp{N(A)} =n−r\displaystyle = \nulldim{n - r} =7−3\displaystyle = 7 - 3 =4\displaystyle = 4
Split the output space R5\mathbb{R}^5 using the r=3\rank{r} = 3 pivotsThe 33 pivots sit in 33 pivot columns for C(A)\colsp{C(A)} and leave 5−3=25 - 3 = 2 zero rows for N(AT)\leftnull{N(A^T)}, accounting for all 55 dimensions of R5\mathbb{R}^5:
dim⁡C(A)\displaystyle \dim \colsp{C(A)} =r\displaystyle = \rank{r} =3,dim⁡N(AT)\displaystyle = 3, \qquad \dim \leftnull{N(A^T)} =m−r\displaystyle = \nulldim{m - r} =5−3\displaystyle = 5 - 3 =2\displaystyle = 2
input space Rn\text{input space } \mathbb{R}^n
output space Rm\text{output space } \mathbb{R}^m
Row Space\text{Row Space}
C(AT)\htmlClass{lc-blue}{C(A^T)}
dim⁡r\dim \htmlClass{lc-blue}{r}
Null Space\text{Null Space}
N(A)\htmlClass{lc-orange}{N(A)}
dim⁡n−r\dim \htmlClass{lc-orange}{n - r}
Column Space\text{Column Space}
C(A)\htmlClass{lc-blue}{C(A)}
dim⁡r\dim \htmlClass{lc-blue}{r}
Left Null Space\text{Left Null Space}
N(AT)\htmlClass{lc-orange}{N(A^T)}
dim⁡m−r\dim \htmlClass{lc-orange}{m - r}
A\htmlClass{lc-violet}{A}
1-to-1\text{1-to-1}
A\htmlClass{lc-violet}{A}
00

In the input space, every vector xx splits at a right angle into a Row Space part in C(AT)\rowsp{C(A^T)} and a Null Space part in N(A)\nullsp{N(A)}. Changing the Null Space part never changes the output Ax\act{A}x, because only the Row Space part maps into C(A)\colsp{C(A)} while the Null Space part goes to 00.

Every input splits at a right angle into a Row Space part and a Null Space part, and only the Row Space part ever reaches the output.

Key concepts

An m×nm \times n matrix AA of rank r\rank{r} splits its input space Rn\mathbb{R}^n and its output space Rm\mathbb{R}^m into two perpendicular pairs of subspaces:

How much work to do depends on what the question asks

If you only need the dimensions (say, for a 5×75 \times 7 matrix of rank r=3\rank{r} = 3), don't row-reduce at all. Just count pivots: dim⁡C(AT)=dim⁡C(A)=r=3\dim \rowsp{C(A^T)} = \dim \colsp{C(A)} = \rank{r} = 3, dim⁡N(A)=n−r=4\dim \nullsp{N(A)} = \nulldim{n - r} = 4 in R7\mathbb{R}^7, and dim⁡N(AT)=m−r=2\dim \leftnull{N(A^T)} = \nulldim{m - r} = 2 in R5\mathbb{R}^5. If you only need bases for C(AT)\rowsp{C(A^T)}, N(A)\nullsp{N(A)}, or C(A)\colsp{C(A)}, row-reducing AA to RR is enough; only attach II to form [A∣I][A \mid I] when you also need a basis for the Left Null Space N(AT)\leftnull{N(A^T)}.

Three spaces come from the reduced matrix, but C(A)\colsp{C(A)} comes from the original AA

After reducing [A∣I][A \mid I] to [12023−100011−2100000−1−11]\left[\begin{array}{cccc|ccc} \pivone{\mathbf{1}} & 2 & 0 & 2 & 3 & -1 & 0 \\ 0 & 0 & \pivtwo{\mathbf{1}} & 1 & -2 & 1 & 0 \\ 0 & 0 & 0 & 0 & \leftnull{-1} & \leftnull{-1} & \leftnull{1} \end{array}\right], read the Row Space C(AT)\rowsp{C(A^T)} straight from the non-zero left rows {[1202],[0011]}\left\{ \rowsp{\begin{bmatrix} 1 \\ 2 \\ 0 \\ 2 \end{bmatrix}}, \rowsp{\begin{bmatrix} 0 \\ 0 \\ 1 \\ 1 \end{bmatrix}} \right\}, the Null Space N(A)\nullsp{N(A)} from the free left columns {[−2100],[−20−11]}\left\{ \nullsp{\begin{bmatrix} -2 \\ 1 \\ 0 \\ 0 \end{bmatrix}}, \nullsp{\begin{bmatrix} -2 \\ 0 \\ -1 \\ 1 \end{bmatrix}} \right\}, and the Left Null Space N(AT)\leftnull{N(A^T)} from the right-hand row {[−1−11]}\left\{ \leftnull{\begin{bmatrix} -1 \\ -1 \\ 1 \end{bmatrix}} \right\} opposite the zero row. For the Column Space C(A)\colsp{C(A)}, row operations changed the columns, so use the pivot positions (Columns 1 and 3) to take the original columns {[123],[134]}\left\{ \pivone{\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}}, \pivtwo{\begin{bmatrix} 1 \\ 3 \\ 4 \end{bmatrix}} \right\} from AA.

Perpendicular pairs test membership with a dot product

In Rn\mathbb{R}^n, N(A)⊥C(AT)\nullsp{N(A)} \perp \rowsp{C(A^T)} because Ax=0\act{A}x = 0; in Rm\mathbb{R}^m, N(AT)⊥C(A)\leftnull{N(A^T)} \perp \colsp{C(A)} because yTA=0T\leftnull{y^T} \act{A} = 0^T. Any vector in one space must have a dot product of 00 with every vector in its perpendicular space, and splitting an input xx into a Row Space part plus a Null Space part means only the Row Space part produces the output Ax\act{A}x.

7

Practice

1

A matrix AA has size 6×46 \times 4 and rank r=3r = 3. For each of the four fundamental subspaces (C(AT)\rowsp{C(A^T)}, N(A)\nullsp{N(A)}, C(A)\colsp{C(A)}, and N(AT)\leftnull{N(A^T)}), state which space (R4\mathbb{R}^4 or R6\mathbb{R}^6) it lives in and find its dimension.

Match the 44 columns of AA to the number of components in an input xx, and the 66 rows of AA to the number of components in an output b\out{b}.

With m=6m = 6 rows and n=4n = 4 columns, the input space is R4\mathbb{R}^4 and the output space is R6\mathbb{R}^6 (be careful not to read 6×46 \times 4 in order as "inputs in R6\mathbb{R}^6, outputs in R4\mathbb{R}^4," which swaps the two spaces and gives the wrong null space dimensions dim⁡N(A)=6−3=3\dim N(A) = 6 - 3 = 3 and dim⁡N(AT)=4−3=1\dim N(A^T) = 4 - 3 = 1).
Row Space C(AT)\rowsp{C(A^T)}: lives in R4\mathbb{R}^4, dimension r=3\rank{r} = 3.
Null Space N(A)\nullsp{N(A)}: lives in R4\mathbb{R}^4, dimension n−r=4−3=1\nulldim{n - r} = 4 - 3 = 1.
Column Space C(A)\colsp{C(A)}: lives in R6\mathbb{R}^6, dimension r=3\rank{r} = 3.
Left Null Space N(AT)\leftnull{N(A^T)}: lives in R6\mathbb{R}^6, dimension m−r=6−3=3\nulldim{m - r} = 6 - 3 = 3.
2

For A=[131226351334]A = \begin{bmatrix} 1 & 3 & 1 & 2 \\ 2 & 6 & 3 & 5 \\ 1 & 3 & 3 & 4 \end{bmatrix}, row-reduce the augmented matrix [A∣I][A \mid I] and write a basis for each of the four fundamental subspaces: C(AT)\rowsp{C(A^T)}, N(A)\nullsp{N(A)}, C(A)\colsp{C(A)}, and N(AT)\leftnull{N(A^T)}.

Attach the 3×33 \times 3 identity matrix on the right, clear Column 1 and then Column 3, and check which basis must be taken from the original matrix AA rather than the reduced matrix.

Row-reducing [A∣I][A \mid I] gives:
[131210026350101334001]→R2←R2−2R1R3←R3−R1[13121000011−2100022−101]→R3←R3−2R2R1←R1−R2[13013−100011−21000003−21]\displaystyle \begin{aligned} & \left[\begin{array}{cccc|ccc} 1 & 3 & 1 & 2 & 1 & 0 & 0 \\ 2 & 6 & 3 & 5 & 0 & 1 & 0 \\ 1 & 3 & 3 & 4 & 0 & 0 & 1 \end{array}\right] \\ &\quad \xrightarrow{\substack{R_2 \leftarrow R_2 - 2R_1 \\ R_3 \leftarrow R_3 - R_1}} \\ &\quad \left[\begin{array}{cccc|ccc} 1 & 3 & 1 & 2 & 1 & 0 & 0 \\ 0 & 0 & 1 & 1 & -2 & 1 & 0 \\ 0 & 0 & 2 & 2 & -1 & 0 & 1 \end{array}\right] \\ &\quad \xrightarrow{\substack{R_3 \leftarrow R_3 - 2R_2 \\ R_1 \leftarrow R_1 - R_2}} \\ &\quad \left[\begin{array}{cccc|ccc} \pivone{\mathbf{1}} & 3 & 0 & 1 & 3 & -1 & 0 \\ 0 & 0 & \pivtwo{\mathbf{1}} & 1 & -2 & 1 & 0 \\ 0 & 0 & 0 & 0 & \leftnull{\mathbf{3}} & \leftnull{\mathbf{-2}} & \leftnull{\mathbf{1}} \end{array}\right] \end{aligned}
When reading N(A)\nullsp{N(A)} from free columns 2 and 4, remember to negate the free entries so they cancel the pivot columns (forgetting to flip the signs gives the wrong vectors [3100]\begin{bmatrix} 3 \\ 1 \\ 0 \\ 0 \end{bmatrix} and [1011]\begin{bmatrix} 1 \\ 0 \\ 1 \\ 1 \end{bmatrix}). When writing C(A)\colsp{C(A)}, take pivot columns 1 and 3 from the original matrix AA (taking them from RR gives the wrong basis {[100],[010]}\left\{ \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \right\}).
Basis for C(AT)\rowsp{C(A^T)}: {[1301],[0011]}\left\{ \rowsp{\begin{bmatrix} 1 \\ 3 \\ 0 \\ 1 \end{bmatrix}}, \rowsp{\begin{bmatrix} 0 \\ 0 \\ 1 \\ 1 \end{bmatrix}} \right\}
Basis for N(A)\nullsp{N(A)}: {[−3100],[−10−11]}\left\{ \nullsp{\begin{bmatrix} -3 \\ 1 \\ 0 \\ 0 \end{bmatrix}}, \nullsp{\begin{bmatrix} -1 \\ 0 \\ -1 \\ 1 \end{bmatrix}} \right\}
Basis for C(A)\colsp{C(A)}: {[121],[133]}\left\{ \pivone{\begin{bmatrix} 1 \\ 2 \\ 1 \end{bmatrix}}, \pivtwo{\begin{bmatrix} 1 \\ 3 \\ 3 \end{bmatrix}} \right\}
Basis for N(AT)\leftnull{N(A^T)}: {[3−21]}\left\{ \leftnull{\begin{bmatrix} 3 \\ -2 \\ 1 \end{bmatrix}} \right\}
3

Suppose you are only given the reduced row echelon form R=[10−2014000]R = \begin{bmatrix} 1 & 0 & -2 \\ 0 & 1 & 4 \\ 0 & 0 & 0 \end{bmatrix} of an unknown 3×33 \times 3 matrix AA, without AA itself. Which of the four fundamental subspaces can you find a basis for from RR alone? Find those bases, and explain why you cannot write down a basis for C(A)\colsp{C(A)} from RR alone.

Ask which subspaces stay unchanged when you combine rows, and which ones lose information once the bottom row turns into zeros.

You can find bases for the Row Space C(AT)\rowsp{C(A^T)} and the Null Space N(A)\nullsp{N(A)}:
Basis for C(AT)\rowsp{C(A^T)}: {[10−2],[014]}\left\{ \rowsp{\begin{bmatrix} 1 \\ 0 \\ -2 \end{bmatrix}}, \rowsp{\begin{bmatrix} 0 \\ 1 \\ 4 \end{bmatrix}} \right\} from the non-zero rows of RR, because row operations do not change the row space.
Basis for N(A)\nullsp{N(A)}: {[2−41]}\left\{ \nullsp{\begin{bmatrix} 2 \\ -4 \\ 1 \end{bmatrix}} \right\} from the free third column of RR, because row operations do not change column relationships (watch the signs when setting x3=1x_3 = 1: copying the column entries directly gives the wrong vector [−241]\begin{bmatrix} -2 \\ 4 \\ 1 \end{bmatrix}).
You cannot find a basis for C(A)\colsp{C(A)} from RR alone because row operations can change the columns: RR's pivot columns {[100],[010]}\left\{ \pivone{\begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}}, \pivtwo{\begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}} \right\} lie in the plane z=0z = 0, but the columns of AA need not; RR only tells you that Columns 1 and 2 of the original matrix AA are the pivot columns, and similarly you would need the row operations that produced the zero row to find a basis for N(AT)\leftnull{N(A^T)}.
4

For a 3×43 \times 4 matrix AA, the vector y=[2−13]\leftnull{y} = \leftnull{\begin{bmatrix} 2 \\ -1 \\ 3 \end{bmatrix}} is in the Left Null Space N(AT)\leftnull{N(A^T)}, and the vector r=[1−204]\rowsp{r} = \rowsp{\begin{bmatrix} 1 \\ -2 \\ 0 \\ 4 \end{bmatrix}} is in the Row Space C(AT)\rowsp{C(A^T)}. (a) If b=[45c]\out{b} = \out{\begin{bmatrix} 4 \\ 5 \\ c \end{bmatrix}} is in the Column Space C(A)\colsp{C(A)}, what must cc be? (b) If x=[6d7−1]\nullsp{x} = \nullsp{\begin{bmatrix} 6 \\ d \\ 7 \\ -1 \end{bmatrix}} is in the Null Space N(A)\nullsp{N(A)}, what must dd be?

Pair up the subspaces that live in the same space (R3\mathbb{R}^3 or R4\mathbb{R}^4) and set the dot product of perpendicular vectors equal to zero.

(a) Since N(AT)⊥C(A)\leftnull{N(A^T)} \perp \colsp{C(A)} in R3\mathbb{R}^3, their dot product is zero:
y⋅b\displaystyle \leftnull{y} \cdot \out{b} =(2)(4)+(−1)(5)+(3)(c)\displaystyle = (\leftnull{2})(\out{4}) + (\leftnull{-1})(\out{5}) + (\leftnull{3})(c) =3+3c\displaystyle = 3 + 3c =0\displaystyle = 0
Watch the sign when subtracting 33 from both sides (solving 3+3c=03 + 3c = 0 without negating gives the wrong answer c=1c = 1):
c\displaystyle c =−1\displaystyle = -1
(b) Since N(A)⊥C(AT)\nullsp{N(A)} \perp \rowsp{C(A^T)} in R4\mathbb{R}^4, their dot product is zero:
r⋅x\displaystyle \rowsp{r} \cdot \nullsp{x} =(1)(6)+(−2)(d)+(0)(7)+(4)(−1)\displaystyle = (\rowsp{1})(\nullsp{6}) + (\rowsp{-2})(d) + (\rowsp{0})(\nullsp{7}) + (\rowsp{4})(\nullsp{-1}) =2−2d\displaystyle = 2 - 2d =0\displaystyle = 0
Be careful with the minus sign on (4)(−1)=−4(4)(-1) = -4 (adding +4+4 instead gives 10−2d=010 - 2d = 0 and the wrong answer d=5d = 5):
d\displaystyle d =1\displaystyle = 1
5

A 4×54 \times 5 matrix AA has a Null Space N(A)\nullsp{N(A)} with basis {[1−2100],[200−11]}\left\{ \nullsp{\begin{bmatrix} 1 \\ -2 \\ 1 \\ 0 \\ 0 \end{bmatrix}}, \nullsp{\begin{bmatrix} 2 \\ 0 \\ 0 \\ -1 \\ 1 \end{bmatrix}} \right\}. (a) Find the rank r\rank{r} of AA and the dimensions of C(AT)\rowsp{C(A^T)}, C(A)\colsp{C(A)}, and N(AT)\leftnull{N(A^T)}. (b) An input vector xx in R5\mathbb{R}^5 is split into a Row Space part xr\rowsp{x_r} and a Null Space part xn=[4−42−11]\nullsp{x_n} = \nullsp{\begin{bmatrix} 4 \\ -4 \\ 2 \\ -1 \\ 1 \end{bmatrix}}, so x=xr+xnx = \rowsp{x_r} + \nullsp{x_n}. If Axr=[2−143]\act{A}\rowsp{x_r} = \out{\begin{bmatrix} 2 \\ -1 \\ 4 \\ 3 \end{bmatrix}}, what is Ax\act{A}x, and why?

(a) Use the number of basis vectors in N(A)\nullsp{N(A)} as the number of free columns n−r\nulldim{n - r}. (b) Multiply A(xr+xn)\act{A}(\rowsp{x_r} + \nullsp{x_n}) and use what AA does to any vector in the Null Space.

(a) The matrix has m=4m = 4 rows and n=5n = 5 columns, and the basis for N(A)\nullsp{N(A)} has 22 vectors, so dim⁡N(A)=n−r=5−r=2\dim \nullsp{N(A)} = \nulldim{n - r} = 5 - \rank{r} = 2 (don't mistake the 22 basis vectors of N(A)\nullsp{N(A)} for the rank r=2r = 2, or subtract from m=4m = 4 to get 4−2=24 - 2 = 2). Solving 5−r=25 - \rank{r} = 2 gives:
r=3,dim⁡C(AT)=r=3,dim⁡C(A)=r=3,dim⁡N(AT)=m−r=4−3=1\displaystyle \begin{aligned} \rank{r} &= 3, \\ \dim \rowsp{C(A^T)} &= \rank{r} = 3, \\ \dim \colsp{C(A)} &= \rank{r} = 3, \\ \dim \leftnull{N(A^T)} &= \nulldim{m - r} = 4 - 3 = 1 \end{aligned}
(b) Multiplying AA by x=xr+xnx = \rowsp{x_r} + \nullsp{x_n} sends the Null Space part xn\nullsp{x_n} to zero (Axn=0\act{A}\nullsp{x_n} = 0), so only the Row Space part xr\rowsp{x_r} produces the output:
Ax=Axr+Axn=[2−143]+[0000]=[2−143]\displaystyle \act{A}x = \act{A}\rowsp{x_r} + \act{A}\nullsp{x_n} = \out{\begin{bmatrix} 2 \\ -1 \\ 4 \\ 3 \end{bmatrix}} + \begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \end{bmatrix} = \begin{bmatrix} 2 \\ -1 \\ 4 \\ 3 \end{bmatrix}