When a matrix A multiplies an input vector x to produce an output vector b, we want to know which outputs A can actually reach and which inputs produce them.
1
The Column Space
The Column SpaceC(A) is the space of all outputs b that a matrix A can reach by combining its columns. For the 3×4 matrix
A=1232461343710
inputs have 4 components (in R4) and outputs have 3 components (in R3). To find a basis for C(A), we keep only the columns that point in new directions and drop the redundant ones.
The Column Space
Text solution
Combine the columns of A to form an outputMultiplying A by an input x uses the four entries of x as weights on the columns of A to build an output b in R3:
x1123+x2246+x3134+x43710=reachable output in C(A)b1b2b3
Spot the redundant columnsColumn 2 is double Column 1, and Column 4 is built from Columns 1 and 3, so neither one lets us reach a new direction in R3:
Keep the independent columns for the basisOnly Columns 1 and 3 point in new directions, spanning a 2D plane inside R3:
Basis for C(A)=⎩⎨⎧123,134⎭⎬⎫
In 3D output space, every multiple x43710 of the redundant fourth column reaches the exact same point as 2x4123+x4134, so adding the fourth column never moves the output off the 2D Column Space plane.
Every multiple of the fourth column stays inside the 2D plane, reaching the exact same point that a matching combination of the first and third columns already reaches on its own.
2
Finding the Column Space and Null Space from R
When a matrix is too complicated to spot redundant columns by eye, we use elimination to get the reduced row echelon form R. From R, we can find both the Column Space C(A) in the output space and the Null SpaceN(A): the space of all inputs x in R4 that the matrix sends to zero (Ax=0).
Finding the Column Space and Null Space from R
A=1232461343710
C(A)=?N(A)=?
Text solution
Subtract multiples of Row 1 to clear the first columnSubtracting 2×Row 1 from Row 2 and 3×Row 1 from Row 3 zeroes out the entries below the top-left pivot without changing how the columns relate to one another:
Use Row 2 to clear the third column and get RSubtracting Row 2 from Row 3 and from Row 1 isolates the pivots in Columns 1 and 3, showing r=2 independent columns:
100200111311R3←R3−R2R1←R1−R2R (pivots in Columns 1 and 3)100200010210
Take the pivot columns from the original matrix A for C(A)Row operations change the columns so the columns of R lie in the plane z=0. We use R only to see that Columns 1 and 3 have the pivots, then go back to A for the vectors:
Basis for C(A)=⎩⎨⎧123,134⎭⎬⎫
Read the special solutions for N(A) from the free columns of RSetting one free variable (x2 or x4) to 1 and the other to 0 gives the input weights that make the columns cancel out to Ax=0:
Basis for N(A)=⎩⎨⎧x2=1,x4=0−2100,x2=0,x4=1−20−11⎭⎬⎫
123
134
x
y
z
C(A)
100
010
x
y
z
C(R)
3
The Row Space
Looking at the columns of A gave us the Column Space in R3. Looking at the rows of A gives us the Row Space: all combinations of the rows of A, which have 4 numbers each and live in the 4D input space R4. To write rows as standard column vectors, we take the transpose AT so the rows of A become the columns of AT, and write the Row Space as C(AT).
The Row Space
input space R4
output space R3
C(A)
N(A)
?
?
A
A
0
Text solution
Turn the rows of A into columns by taking the transpose ATTaking the transpose puts the three 4-component rows of A into column form in R4:
AT=1213243736410
Take the non-zero rows of R for the basis of C(AT)Because row operations only combine rows with one another, they never leave the Row Space, so we can write the non-zero rows of R as column vectors without going back to A:
Basis for C(AT)=⎩⎨⎧1202,0011⎭⎬⎫
Check that the Row Space and Null Space are perpendicularMultiplying A by x takes the dot product of each row with x, so Ax=0 means every vector in N(A) has a dot product of zero with every vector in C(AT):
1202⋅−2100=(1)(−2)+(2)(1)+(0)(0)+(2)(0)=0
1202⋅−20−11=(1)(−2)+(2)(0)+(0)(−1)+(2)(1)=0
4
The Left Null Space
In the 4D input space, the Null Space N(A) is perpendicular to the Row Space C(AT). In the 3D output space, the matching space that is perpendicular to the 2D Column Space plane C(A) is the Left Null SpaceN(AT): all vectors y whose weights combine the rows of A into a zero row (yTA=0T, or ATy=0).
The Left Null Space
x
y
z
C(A)
N(AT)=?
Text solution
Write the row combination that produced the zero row of RIn A, Row 3 is Row 1 plus Row 2, so subtracting Row 1 and Row 2 from Row 3 cancels all entries to zero:
−1(Row 1)−1(Row 2)+1(Row 3)=[0000]
Collect the row weights into a basis vector y for N(AT)Writing those weights as a column vector y satisfies yTA=0T, or ATy=0:
Basis for N(AT)=⎩⎨⎧−1−11⎭⎬⎫
Check that N(AT) is perpendicular to C(A)Because −1(top)−1(middle)+1(bottom)=0 in every column of A, any output b built from those columns has a dot product of zero with y:
−1−11⋅123=−1−2+3=0,−1−11⋅134=−1−3+4=0
In 3D output space, the dot product y⋅b=−b1−b2+b3 equals 0 and the angle between b and y=−1−11 is 90∘ precisely when b lies in the Column Space plane C(A).
A vector in 3D output space lies on the column space plane right when it sits at a right angle to the left null space vector, giving a dot product of zero.
5
Finding All Four Subspaces with [A∣I]
When you cannot spot by eye which combination of rows made the zero row in R, attach an identity matrix I on the right of A before row-reducing. Doing the same row steps across [A∣I] turns the left side into R while the right side automatically records the row combinations, letting you read off bases for all four subspaces from one matrix.
Finding All Four Subspaces with [A | I]
1232461343710100010001
C(AT)N(A)=?=?C(A)N(AT)=?=?
Text solution
Attach the 3×3 identity matrix I on the right of AThe ones down the diagonal of I keep track of how many of Row 1, Row 2, and Row 3 we use:
[A∣I]=1232461343710100010001
Clear the first column across both sidesSubtracting 2×Row 1 from Row 2 and 3×Row 1 from Row 3 records −2 and −3 on the right side:
Clear the third column across both sides to reach R on the leftSubtracting the new Row 2 from Row 3 and from Row 1 finishes R on the left and leaves the row combination for the zero row at the bottom right:
Read all four bases from the reduced matrix and ATake the non-zero left rows for C(AT), the free left columns for N(A), the pivot columns from the original matrix A for C(A), and the right-hand row opposite the zero row for N(AT):
Basis for C(AT)=⎩⎨⎧1202,0011⎭⎬⎫,Basis for N(A)=⎩⎨⎧−2100,−20−11⎭⎬⎫
Basis for C(A)Basis for N(AT)=⎩⎨⎧123,134⎭⎬⎫,=⎩⎨⎧−1−11⎭⎬⎫
6
Dimensions of the Four Subspaces
Even if you are not given the numbers inside an m×n matrix A, knowing its size and its rank r (the number of pivots in R) tells you the dimensions of all four fundamental subspaces and which spaces they live in.
Dimensions of the Four Subspaces
n
m
R=
rank r
dimC(AT)dimN(A)=?=?dimC(A)dimN(AT)=?=?
For any m×n matrix A with rank r:
dimC(AT)dimN(A)dimC(A)dimN(AT)In the input space Rn:=non-zero pivot rowsr,=free columnsn−rIn the output space Rm:=pivot columnsr,=zero rowsm−r
Here are the steps for a 5×7 matrix A of rank r=3:
Text solution
Match the matrix size 5×7 to the input and output spacesWith m=5 rows and n=7 columns, inputs have 7 components (in R7) and outputs have 5 components (in R5).
Split the input space R7 using the r=3 pivotsThe 3 pivots sit in 3 non-zero rows for C(AT) and leave 7−3=4 free columns for N(A), accounting for all 7 dimensions of R7:
dimC(AT)=r=3,dimN(A)=n−r=7−3=4
Split the output space R5 using the r=3 pivotsThe 3 pivots sit in 3 pivot columns for C(A) and leave 5−3=2 zero rows for N(AT), accounting for all 5 dimensions of R5:
dimC(A)=r=3,dimN(AT)=m−r=5−3=2
input space Rn
output space Rm
Row Space
C(AT)
dimr
Null Space
N(A)
dimn−r
Column Space
C(A)
dimr
Left Null Space
N(AT)
dimm−r
A
1-to-1
A
0
In the input space, every vector x splits at a right angle into a Row Space part in C(AT) and a Null Space part in N(A). Changing the Null Space part never changes the output Ax, because only the Row Space part maps into C(A) while the Null Space part goes to 0.
Every input splits at a right angle into a Row Space part and a Null Space part, and only the Row Space part ever reaches the output.
Key concepts
An m×n matrix A of rank r splits its input space Rn and its output space Rm into two perpendicular pairs of subspaces:
How much work to do depends on what the question asks
If you only need the dimensions (say, for a 5×7 matrix of rank r=3), don't row-reduce at all. Just count pivots: dimC(AT)=dimC(A)=r=3,dimN(A)=n−r=4 in R7, and dimN(AT)=m−r=2 in R5. If you only need bases for C(AT),N(A), or C(A), row-reducing A to R is enough; only attach I to form [A∣I] when you also need a basis for the Left Null Space N(AT).
Three spaces come from the reduced matrix, but C(A) comes from the original A
After reducing [A∣I] to 1002000102103−2−1−11−1001, read the Row Space C(AT) straight from the non-zero left rows ⎩⎨⎧1202,0011⎭⎬⎫, the Null Space N(A) from the free left columns ⎩⎨⎧−2100,−20−11⎭⎬⎫, and the Left Null Space N(AT) from the right-hand row ⎩⎨⎧−1−11⎭⎬⎫ opposite the zero row. For the Column Space C(A), row operations changed the columns, so use the pivot positions (Columns 1 and 3) to take the original columns ⎩⎨⎧123,134⎭⎬⎫ from A.
Perpendicular pairs test membership with a dot product
In Rn,N(A)⊥C(AT) because Ax=0; in Rm,N(AT)⊥C(A) because yTA=0T. Any vector in one space must have a dot product of 0 with every vector in its perpendicular space, and splitting an input x into a Row Space part plus a Null Space part means only the Row Space part produces the output Ax.
7
Practice
1
A matrix A has size 6×4 and rank r=3. For each of the four fundamental subspaces (C(AT),N(A),C(A), and N(AT)), state which space (R4 or R6) it lives in and find its dimension.
Be careful not to read 6×4 in order as "inputs in R6, outputs in R4," which swaps the two spaces and gives the wrong null space dimensions dimN(A)=6−3=3 and dimN(AT)=4−3=1.
Match the 4 columns of A to the number of components in an input x, and the 6 rows of A to the number of components in an output b.
With m=6 rows and n=4 columns, the input space is R4 and the output space is R6 (be careful not to read 6×4 in order as "inputs in R6, outputs in R4," which swaps the two spaces and gives the wrong null space dimensions dimN(A)=6−3=3 and dimN(AT)=4−3=1).
Row Space C(AT): lives in R4, dimension r=3.
Null Space N(A): lives in R4, dimension n−r=4−3=1.
Column Space C(A): lives in R6, dimension r=3.
Left Null Space N(AT): lives in R6, dimension m−r=6−3=3.
2
For A=121363133254, row-reduce the augmented matrix [A∣I] and write a basis for each of the four fundamental subspaces: C(AT),N(A),C(A), and N(AT).
Attach the 3×3 identity matrix on the right, clear Column 1 and then Column 3, and check which basis must be taken from the original matrix A rather than the reduced matrix.
When reading N(A) from free columns 2 and 4, remember to negate the free entries so they cancel the pivot columns (forgetting to flip the signs gives the wrong vectors 3100 and 1011). When writing C(A), take pivot columns 1 and 3 from the original matrix A (taking them from R gives the wrong basis ⎩⎨⎧100,010⎭⎬⎫).
Basis for C(AT):⎩⎨⎧1301,0011⎭⎬⎫
Basis for N(A):⎩⎨⎧−3100,−10−11⎭⎬⎫
Basis for C(A):⎩⎨⎧121,133⎭⎬⎫
Basis for N(AT):⎩⎨⎧3−21⎭⎬⎫
3
Suppose you are only given the reduced row echelon form R=100010−240 of an unknown 3×3 matrix A, without A itself. Which of the four fundamental subspaces can you find a basis for from R alone? Find those bases, and explain why you cannot write down a basis for C(A) from R alone.
Ask which subspaces stay unchanged when you combine rows, and which ones lose information once the bottom row turns into zeros.
You can find bases for the Row Space C(AT) and the Null Space N(A):
Basis for C(AT):⎩⎨⎧10−2,014⎭⎬⎫ from the non-zero rows of R, because row operations do not change the row space.
Basis for N(A):⎩⎨⎧2−41⎭⎬⎫ from the free third column of R, because row operations do not change column relationships (watch the signs when setting x3=1: copying the column entries directly gives the wrong vector −241).
You cannot find a basis for C(A) from R alone because row operations can change the columns: R's pivot columns ⎩⎨⎧100,010⎭⎬⎫ lie in the plane z=0, but the columns of A need not; R only tells you that Columns 1 and 2 of the original matrix A are the pivot columns, and similarly you would need the row operations that produced the zero row to find a basis for N(AT).
4
For a 3×4 matrix A, the vector y=2−13 is in the Left Null Space N(AT), and the vector r=1−204 is in the Row Space C(AT). (a) If b=45c is in the Column Space C(A), what must c be? (b) If x=6d7−1 is in the Null Space N(A), what must d be?
Watch the sign when subtracting 3 from both sides (solving 3+3c=0 without negating gives the wrong answer c=1).
Pair up the subspaces that live in the same space (R3 or R4) and set the dot product of perpendicular vectors equal to zero.
(a) Since N(AT)⊥C(A) in R3, their dot product is zero:
y⋅b=(2)(4)+(−1)(5)+(3)(c)=3+3c=0
Watch the sign when subtracting 3 from both sides (solving 3+3c=0 without negating gives the wrong answer c=1):
c=−1
(b) Since N(A)⊥C(AT) in R4, their dot product is zero:
r⋅x=(1)(6)+(−2)(d)+(0)(7)+(4)(−1)=2−2d=0
Be careful with the minus sign on (4)(−1)=−4 (adding +4 instead gives 10−2d=0 and the wrong answer d=5):
d=1
5
A 4×5 matrix A has a Null Space N(A) with basis ⎩⎨⎧1−2100,200−11⎭⎬⎫. (a) Find the rank r of A and the dimensions of C(AT),C(A), and N(AT). (b) An input vector x in R5 is split into a Row Space part xr and a Null Space part xn=4−42−11, so x=xr+xn. If Axr=2−143, what is Ax, and why?
Don't mistake the 2 basis vectors of N(A) for the rank r=2, or subtract from m=4 to get 4−2=2.
(a) Use the number of basis vectors in N(A) as the number of free columns n−r. (b) Multiply A(xr+xn) and use what A does to any vector in the Null Space.
(a) The matrix has m=4 rows and n=5 columns, and the basis for N(A) has 2 vectors, so dimN(A)=n−r=5−r=2 (don't mistake the 2 basis vectors of N(A) for the rank r=2, or subtract from m=4 to get 4−2=2). Solving 5−r=2 gives:
rdimC(AT)dimC(A)dimN(AT)=3,=r=3,=r=3,=m−r=4−3=1
(b) Multiplying A by x=xr+xn sends the Null Space part xn to zero (Axn=0), so only the Row Space part xr produces the output: