1 00:00:01,060 --> 00:00:04,030 GILBERT STRANG: I would like you to see the big picture 2 00:00:04,030 --> 00:00:06,060 of linear algebra. 3 00:00:06,060 --> 00:00:08,440 We're not doing, in this set of videos, 4 00:00:08,440 --> 00:00:10,630 a full course on linear algebra. 5 00:00:10,630 --> 00:00:15,660 That's already on OpenCourseWare 1806. 6 00:00:15,660 --> 00:00:20,230 And now I'm concentrating on differential equations, 7 00:00:20,230 --> 00:00:25,550 but you got to see linear algebra this way. 8 00:00:25,550 --> 00:00:29,670 And this way means subspaces. 9 00:00:29,670 --> 00:00:32,850 And there are four of them in the big picture. 10 00:00:32,850 --> 00:00:37,470 And we-- previous video described the column space 11 00:00:37,470 --> 00:00:39,030 and the null space. 12 00:00:39,030 --> 00:00:41,730 Now, we've got two more, making four. 13 00:00:41,730 --> 00:00:47,830 And let me look at this matrix-- it's for subspaces-- 14 00:00:47,830 --> 00:00:50,720 and put them into the big picture. 15 00:00:50,720 --> 00:00:54,570 So the first space I'll look at is the row space. 16 00:00:54,570 --> 00:00:57,120 Now, the row space has these rows-- 17 00:00:57,120 --> 00:01:00,150 has the vector 1, 2, 3 and the vector 18 00:01:00,150 --> 00:01:05,600 4, 5, 6, two vectors there, and all their combinations. 19 00:01:05,600 --> 00:01:09,730 That's the key idea in linear algebra, linear combinations. 20 00:01:09,730 --> 00:01:13,760 So 1, 2, 3 is a vector in three dimensional space. 21 00:01:13,760 --> 00:01:16,110 4, 5, 6 is another one. 22 00:01:16,110 --> 00:01:18,260 Now, if I take all their combinations, 23 00:01:18,260 --> 00:01:24,240 do you visualize that if I have two vectors, and I add them, 24 00:01:24,240 --> 00:01:28,060 and I get another vector that's in the same plane? 25 00:01:28,060 --> 00:01:31,500 Or if I subtract them, I'm still in that plane. 26 00:01:31,500 --> 00:01:34,630 Or if I take five of one and three of another, 27 00:01:34,630 --> 00:01:36,490 I'm still in that plane. 28 00:01:36,490 --> 00:01:40,590 And I fill the plane when I take all the combinations. 29 00:01:40,590 --> 00:01:44,780 So the row space-- can I try to draw a picture here? 30 00:01:44,780 --> 00:01:46,610 It's a plane. 31 00:01:46,610 --> 00:01:48,120 This is the row space. 32 00:01:48,120 --> 00:01:50,020 I'll just put row. 33 00:01:50,020 --> 00:01:56,750 And in that plane are the vectors 1, 2, 3 34 00:01:56,750 --> 00:02:01,540 and the vectors 4, 5, 6, those two rows. 35 00:02:01,540 --> 00:02:04,140 And the plane fills our combinations. 36 00:02:04,140 --> 00:02:09,139 Well, I can't draw am infinite plane on this MIT blackboard. 37 00:02:09,139 --> 00:02:11,200 But you get the idea. 38 00:02:11,200 --> 00:02:12,330 It's a plane. 39 00:02:12,330 --> 00:02:15,450 And we're sitting in three dimensions. 40 00:02:15,450 --> 00:02:19,370 Now, the other-- so there's more. 41 00:02:19,370 --> 00:02:22,540 We've only got one-- a plane here, a flat part, 42 00:02:22,540 --> 00:02:25,880 like a tabletop, extending to infinity, 43 00:02:25,880 --> 00:02:29,710 but not filling 3D because we've got another direction. 44 00:02:29,710 --> 00:02:33,390 And in that other direction is the null space. 45 00:02:33,390 --> 00:02:34,870 That's the nice thing. 46 00:02:34,870 --> 00:02:38,680 So I would like to know the null space of that matrix. 47 00:02:38,680 --> 00:02:44,980 I'd like to solve so the null space, N of A-- I'm 48 00:02:44,980 --> 00:02:50,200 solving Av equals all 0s. 49 00:02:50,200 --> 00:02:53,130 So some combination of those three columns 50 00:02:53,130 --> 00:02:54,720 will give me the 0 column. 51 00:02:54,720 --> 00:02:58,830 Let me write it in as a 0 column. 52 00:02:58,830 --> 00:03:00,920 What could v be? 53 00:03:00,920 --> 00:03:04,380 What combination of that column, that column, and that column 54 00:03:04,380 --> 00:03:06,040 give 0, 0? 55 00:03:06,040 --> 00:03:09,320 Now, I know there are some interesting combinations 56 00:03:09,320 --> 00:03:14,750 because I-- only amounts to two equations with three unknowns, 57 00:03:14,750 --> 00:03:16,620 v1, v2, v3. 58 00:03:16,620 --> 00:03:21,860 I want to multiply that by v1, that by v2, that by v3. 59 00:03:21,860 --> 00:03:25,200 So I have three unknowns, but I've only two 0s 60 00:03:25,200 --> 00:03:27,390 to get, only two equations. 61 00:03:27,390 --> 00:03:29,470 And if I have three unknowns and two equations, 62 00:03:29,470 --> 00:03:31,700 there will be lots of solutions. 63 00:03:31,700 --> 00:03:33,330 And I can see one. 64 00:03:33,330 --> 00:03:39,710 Do you see that if I add that and that, I get 4, 10 65 00:03:39,710 --> 00:03:44,090 And that's the same-- 4, 10 is the same as 2 times 2, 5. 66 00:03:44,090 --> 00:03:47,520 In other words, I believe v equal-- 67 00:03:47,520 --> 00:03:51,280 if I took 1 of the first and 1 of the third, 68 00:03:51,280 --> 00:03:56,000 and if I subtracted 2 of the second column-- 69 00:03:56,000 --> 00:04:00,380 so Av will give me 1 of the first column, 1 70 00:04:00,380 --> 00:04:02,710 of the third column, and subtracting 71 00:04:02,710 --> 00:04:05,650 2 of the second column will give me 0, 0. 72 00:04:05,650 --> 00:04:07,840 So here's my null space. 73 00:04:07,840 --> 00:04:11,230 My null space heads off in this direction, 74 00:04:11,230 --> 00:04:17,000 in the direction of 1, minus 2, 1. 75 00:04:17,000 --> 00:04:19,940 But, of course, I get more solutions 76 00:04:19,940 --> 00:04:23,300 by multiplying v by any number. 77 00:04:23,300 --> 00:04:29,200 10 times that vector would still give me 0s and still 78 00:04:29,200 --> 00:04:30,790 be in the null space. 79 00:04:30,790 --> 00:04:33,070 So I really have-- the null space 80 00:04:33,070 --> 00:04:34,720 is a whole line of vectors. 81 00:04:34,720 --> 00:04:37,660 It's that vector and any multiple of that vector. 82 00:04:37,660 --> 00:04:42,240 So it's a whole infinite line, which is a one dimensional 83 00:04:42,240 --> 00:04:44,560 subspace, the null space. 84 00:04:44,560 --> 00:04:47,180 So the null space in my picture-- 85 00:04:47,180 --> 00:04:48,680 here is the null space. 86 00:04:48,680 --> 00:04:54,590 Well, it is not very thick is it because it's just a line. 87 00:04:54,590 --> 00:05:02,140 So I'll call this N of A, this line. 88 00:05:02,140 --> 00:05:05,400 Well, you see I'm trying to draw a three dimensional space. 89 00:05:05,400 --> 00:05:07,720 That line goes both ways. 90 00:05:07,720 --> 00:05:09,560 But it's perpendicular to the plane. 91 00:05:09,560 --> 00:05:11,570 That's the fabulous part. 92 00:05:11,570 --> 00:05:13,070 That's wonderful. 93 00:05:13,070 --> 00:05:17,640 This line, the null space, is perpendicular to this plane, 94 00:05:17,640 --> 00:05:20,140 the row space. 95 00:05:20,140 --> 00:05:21,150 You want to know why? 96 00:05:21,150 --> 00:05:22,840 You want to just see it? 97 00:05:22,840 --> 00:05:27,460 Because if I take A times v, that would be 1, 98 00:05:27,460 --> 00:05:32,570 2, 3 times v. 1, 2, 3 is perpendicular to that. 99 00:05:32,570 --> 00:05:35,560 How do I check perpendicular for two vectors? 100 00:05:35,560 --> 00:05:37,750 1, 2, 3 dot product. 101 00:05:37,750 --> 00:05:42,760 1, minus 2, 1, the dot product is 1 times 1, minus 2 times 102 00:05:42,760 --> 00:05:46,910 2-- that's 4-- plus 3 times 1, that's 3. 103 00:05:46,910 --> 00:05:49,970 1 minus 4 plus 3 is 0. 104 00:05:49,970 --> 00:05:54,120 And, similarly, 4 minus 10 plus 6 is 0. 105 00:05:54,120 --> 00:05:57,760 So this is a right angle here. 106 00:05:57,760 --> 00:06:03,320 It's a right angle, 90 degrees between those two subspaces. 107 00:06:03,320 --> 00:06:07,090 And again, in this example, one space 108 00:06:07,090 --> 00:06:09,740 is two dimensional, a plane. 109 00:06:09,740 --> 00:06:12,770 The other space is one dimensional, a perpendicular 110 00:06:12,770 --> 00:06:13,350 line. 111 00:06:13,350 --> 00:06:15,320 I can show with my hands, but I can't 112 00:06:15,320 --> 00:06:17,240 draw on this flat blackboard. 113 00:06:17,240 --> 00:06:20,800 I have the plane going infinitely far, 114 00:06:20,800 --> 00:06:24,240 and I have the line going perpendicular to it 115 00:06:24,240 --> 00:06:28,290 and meeting, of course, at 0-- at the 0 vector. 116 00:06:28,290 --> 00:06:33,630 That solves Av equals 0, and it also-- it's 117 00:06:33,630 --> 00:06:36,690 a combination, a 0 combination of the rows. 118 00:06:36,690 --> 00:06:41,210 That's half of the big picture, the row space 119 00:06:41,210 --> 00:06:43,160 and the null space. 120 00:06:43,160 --> 00:06:45,330 Now, I'm ready for the other half, which 121 00:06:45,330 --> 00:06:49,010 is a second side of the other-- the right-hand side 122 00:06:49,010 --> 00:06:53,570 of the big picture contains the column space first of all. 123 00:06:53,570 --> 00:06:56,440 So what's the column space of that matrix? 124 00:06:56,440 --> 00:06:58,460 So the column space of a matrix, we 125 00:06:58,460 --> 00:07:01,790 take all combinations of those three columns. 126 00:07:04,520 --> 00:07:07,880 And that will fill out a space. 127 00:07:07,880 --> 00:07:13,958 Now, I have-- so I take the vector 1, 4. 128 00:07:13,958 --> 00:07:20,740 And I take the vector 2, 5, maybe there. 129 00:07:20,740 --> 00:07:23,980 And then I'm going to also take the vector 3, 6. 130 00:07:23,980 --> 00:07:25,360 Well, I've got three columns. 131 00:07:25,360 --> 00:07:27,780 So I'm counting out 3, up 6. 132 00:07:27,780 --> 00:07:28,280 Good. 133 00:07:32,510 --> 00:07:36,410 Take those combinations of those vectors, and what do you get? 134 00:07:36,410 --> 00:07:39,590 This is a picture in two dimensional space 135 00:07:39,590 --> 00:07:45,620 because these columns are in two dimensions, 1,4; 2, 5; 3, 6. 136 00:07:45,620 --> 00:07:50,860 When I take the combinations of 1, 4 and 2, 5, 137 00:07:50,860 --> 00:07:53,050 those are in different directions. 138 00:07:53,050 --> 00:07:57,700 The combinations already give me all of two dimensional space, 139 00:07:57,700 --> 00:08:01,550 so the column space is the whole space, including 0, 140 00:08:01,550 --> 00:08:06,120 0 because I could take 0 of 1 plus 0 of the other vector. 141 00:08:06,120 --> 00:08:10,570 And that third column can't contribute anything new. 142 00:08:10,570 --> 00:08:12,260 It's sitting in the column space. 143 00:08:12,260 --> 00:08:14,410 It's a combination of those two. 144 00:08:14,410 --> 00:08:17,030 But the first two are independent. 145 00:08:17,030 --> 00:08:19,580 Their combinations give the whole plane. 146 00:08:19,580 --> 00:08:23,290 So the column space is the whole plane. 147 00:08:23,290 --> 00:08:24,180 Column space. 148 00:08:29,770 --> 00:08:33,070 There's not much room for our fourth subspace. 149 00:08:33,070 --> 00:08:37,110 But the fourth subspace, in this example, is quite small. 150 00:08:37,110 --> 00:08:39,929 Let me tell you about the fourth subspaces then. 151 00:08:39,929 --> 00:08:44,179 So we know the null space, N of A. 152 00:08:44,179 --> 00:08:50,020 And we know the column space, C of A. The null space was 153 00:08:50,020 --> 00:08:51,410 in this picture. 154 00:08:51,410 --> 00:08:54,840 The column space was in that picture. 155 00:08:54,840 --> 00:08:57,810 Now, what about-- what's the name for the row space? 156 00:08:57,810 --> 00:09:02,280 Well, if I transpose the matrix, the row space 157 00:09:02,280 --> 00:09:05,300 turns into the column space. 158 00:09:05,300 --> 00:09:12,680 Transpose rows into columns of the matrix A transpose. 159 00:09:12,680 --> 00:09:16,970 So by transposing a matrix, it turns these two rows 160 00:09:16,970 --> 00:09:17,845 into two columns. 161 00:09:20,750 --> 00:09:22,080 And that's what I have here. 162 00:09:22,080 --> 00:09:25,600 The row space is the-- this is the column 163 00:09:25,600 --> 00:09:28,890 space of the transpose matrix. 164 00:09:28,890 --> 00:09:29,510 I like it. 165 00:09:29,510 --> 00:09:32,840 I don't want to introduce a new letter for the row space. 166 00:09:32,840 --> 00:09:36,440 I like having just column space and null space. 167 00:09:36,440 --> 00:09:42,000 So I-- and I'm OK to go to A transpose. 168 00:09:42,000 --> 00:09:44,770 Now, what's that fourth guy? 169 00:09:44,770 --> 00:09:49,080 Oh, just by beautifulness, general principles of elegance 170 00:09:49,080 --> 00:09:49,710 here. 171 00:09:49,710 --> 00:09:52,950 If I have columns space and null space of A, 172 00:09:52,950 --> 00:09:55,440 and if I have column space of A transpose, 173 00:09:55,440 --> 00:10:01,940 the fourth guy has to be the null space of A transpose. 174 00:10:01,940 --> 00:10:05,510 Sorry, I wrote that so small, so small. 175 00:10:05,510 --> 00:10:07,550 But I did write this a little larger. 176 00:10:07,550 --> 00:10:11,400 The null space of A transpose, all the w's 177 00:10:11,400 --> 00:10:13,620 that solve that equation. 178 00:10:13,620 --> 00:10:15,490 A transpose w equals 0. 179 00:10:15,490 --> 00:10:19,130 The null space of A transpose is all the w's that 180 00:10:19,130 --> 00:10:21,370 solve that equation. 181 00:10:21,370 --> 00:10:23,610 What does that equation looks like? 182 00:10:23,610 --> 00:10:24,160 Ha! 183 00:10:24,160 --> 00:10:29,850 Well, that equation-- A transpose 184 00:10:29,850 --> 00:10:31,850 will have two columns. 185 00:10:31,850 --> 00:10:36,450 So A transpose-- this will be w1 of the first column. 186 00:10:36,450 --> 00:10:41,450 1, 2, 3, when I transpose. 187 00:10:41,450 --> 00:10:50,560 And w2 of the second column, 4, 5, 6 equaling 0, 0, 0. 188 00:10:54,820 --> 00:10:59,180 Well, now I've got, for this null space-- because my matrix 189 00:10:59,180 --> 00:11:05,140 here is 2 by 3, for this fourth subspace, 190 00:11:05,140 --> 00:11:10,700 I have three equations and only two unknowns, w1 and w2. 191 00:11:10,700 --> 00:11:13,620 And, in fact, the only solutions are 192 00:11:13,620 --> 00:11:19,080 w1 equals 0, w2 equals 0, because that's 193 00:11:19,080 --> 00:11:21,810 the only way I can get combination-- that's 194 00:11:21,810 --> 00:11:24,300 the only combination of that vector 195 00:11:24,300 --> 00:11:29,300 and that vector that gives me 0 is to take 0 of that 196 00:11:29,300 --> 00:11:30,926 and zero of that. 197 00:11:30,926 --> 00:11:36,780 Do you see that the-- in this example, the null space 198 00:11:36,780 --> 00:11:42,930 of A transpose is just-- null space of A transpose-- 199 00:11:42,930 --> 00:11:48,710 is just what I call the 0 subspace. 200 00:11:48,710 --> 00:11:51,920 The subspace that has only one puny vector 201 00:11:51,920 --> 00:11:53,680 in it, the 0 vector. 202 00:11:53,680 --> 00:11:54,730 But that's OK. 203 00:11:54,730 --> 00:11:57,370 It follows the rule for subspaces. 204 00:11:57,370 --> 00:12:01,640 And it completes the picture of four subspaces. 205 00:12:01,640 --> 00:12:10,560 In other examples, we could have all four subspaces nonzero. 206 00:12:10,560 --> 00:12:17,360 But we would have two over here that, together, complete 207 00:12:17,360 --> 00:12:19,880 the full N dimensional space. 208 00:12:19,880 --> 00:12:24,760 And over here, we have two that together complete the full M 209 00:12:24,760 --> 00:12:26,220 dimensional space. 210 00:12:26,220 --> 00:12:31,960 And here, for this matrix, M was 2, so that this is completed. 211 00:12:31,960 --> 00:12:38,910 The column space was all of R2 in this case. 212 00:12:38,910 --> 00:12:41,990 All of two dimensional space was the column space, 213 00:12:41,990 --> 00:12:45,340 and that didn't leave any room for the left null space, 214 00:12:45,340 --> 00:12:47,350 the null space of A transpose. 215 00:12:47,350 --> 00:12:49,370 So do you see that picture? 216 00:12:49,370 --> 00:12:54,380 Let we may be just sketch it once more with a clean board. 217 00:12:54,380 --> 00:12:57,360 So I have the row space. 218 00:12:57,360 --> 00:13:02,560 Let me draw it maybe going this way, the row space. 219 00:13:02,560 --> 00:13:07,820 And perpendicular to that is the null space. 220 00:13:11,650 --> 00:13:15,450 That's in-- we're here in N dimensions, 221 00:13:15,450 --> 00:13:19,000 and they are perpendicular, those spaces. 222 00:13:19,000 --> 00:13:22,220 And then, over here, I have the column space. 223 00:13:28,600 --> 00:13:36,450 And perpendicular to that is the left null space. 224 00:13:40,360 --> 00:13:44,370 And we're here in M dimensions. 225 00:13:44,370 --> 00:13:46,200 Those are our four subspaces. 226 00:13:46,200 --> 00:13:50,220 And they have-- they sit in N dimensional space, two of them, 227 00:13:50,220 --> 00:13:53,980 two of them in M dimensional space, perpendicular. 228 00:13:53,980 --> 00:13:58,250 And I could tell you something about their dimensions. 229 00:13:58,250 --> 00:14:02,260 So this row space, in that example, was two dimensional. 230 00:14:02,260 --> 00:14:03,440 It was a plane. 231 00:14:03,440 --> 00:14:08,310 In general, the dimension equals-- 232 00:14:08,310 --> 00:14:15,480 let's say R. That's an important number, the rank of A. Oh, 233 00:14:15,480 --> 00:14:17,280 that's a key number. 234 00:14:17,280 --> 00:14:21,880 Maybe, I better speak separately about the rank of a matrix. 235 00:14:21,880 --> 00:14:25,290 But I'll complete the idea here. 236 00:14:25,290 --> 00:14:29,560 So the dimension of the row space 237 00:14:29,560 --> 00:14:32,230 is the number of independent rows. 238 00:14:32,230 --> 00:14:35,440 And I call that number R. And the beauty 239 00:14:35,440 --> 00:14:38,410 is that this has the same dimension. 240 00:14:38,410 --> 00:14:44,410 Dimension Is also the rank R. Can 241 00:14:44,410 --> 00:14:47,660 I say that wonderful fact in a sentence? 242 00:14:47,660 --> 00:14:53,150 The column space and the row space have the same dimension. 243 00:14:53,150 --> 00:14:56,490 The number of independent rows equals the number 244 00:14:56,490 --> 00:14:58,200 of independent columns. 245 00:14:58,200 --> 00:15:06,740 That's like a miracle for a giant matrix, say 57 by 212, 246 00:15:06,740 --> 00:15:09,290 there might be 40 independent rows. 247 00:15:09,290 --> 00:15:11,880 Then, there would be 40 independent columns. 248 00:15:11,880 --> 00:15:15,950 And then the null space and the left null space 249 00:15:15,950 --> 00:15:17,940 have the remaining dimension. 250 00:15:17,940 --> 00:15:26,600 So the null space has dimension N minus R because, altogether-- 251 00:15:26,600 --> 00:15:28,730 together they have dimension N. 252 00:15:28,730 --> 00:15:35,540 And this has dimension M minus R because, together, they 253 00:15:35,540 --> 00:15:41,560 have dimension M. That's the picture with the dimensions put 254 00:15:41,560 --> 00:15:42,220 in. 255 00:15:42,220 --> 00:15:49,540 And let me say a little more about the idea of dimension 256 00:15:49,540 --> 00:15:51,290 in a separate video. 257 00:15:51,290 --> 00:15:53,040 Thank you.