1 00:00:11,825 --> 00:00:12,325 OK. 2 00:00:14,830 --> 00:00:15,900 cameras are rolling. 3 00:00:15,900 --> 00:00:21,840 This is lecture fourteen, starting a new chapter. 4 00:00:21,840 --> 00:00:24,660 Chapter about orthogonality. 5 00:00:24,660 --> 00:00:29,480 What it means for vectors to be orthogonal. 6 00:00:29,480 --> 00:00:33,070 What it means for subspaces to be orthogonal. 7 00:00:33,070 --> 00:00:37,370 What it means for bases to be orthogonal. 8 00:00:37,370 --> 00:00:44,460 So ninety degrees, this is a ninety-degree chapter. 9 00:00:44,460 --> 00:00:47,800 So what does it mean -- 10 00:00:47,800 --> 00:00:50,890 let me jump to subspaces. 11 00:00:50,890 --> 00:00:54,830 Because I've drawn here the big picture. 12 00:00:54,830 --> 00:00:59,910 This is the 18.06 picture here. 13 00:00:59,910 --> 00:01:04,800 And hold it down, guys. 14 00:01:04,800 --> 00:01:09,280 So this is the picture and we know a lot about that picture 15 00:01:09,280 --> 00:01:10,550 already. 16 00:01:10,550 --> 00:01:14,570 We know the dimension of every subspace. 17 00:01:14,570 --> 00:01:22,040 We know that these dimensions are r and n-r. 18 00:01:22,040 --> 00:01:26,320 We know that these dimensions are r and m-r. 19 00:01:29,680 --> 00:01:33,780 What I want to show now is what this figure is saying, 20 00:01:33,780 --> 00:01:38,140 that the angle -- 21 00:01:38,140 --> 00:01:43,520 the figure is just my attempt to draw what I'm now going to say, 22 00:01:43,520 --> 00:01:50,510 that the angle between these subspaces is ninety degrees. 23 00:01:50,510 --> 00:01:53,890 And the angle between these subspaces is ninety degrees. 24 00:01:53,890 --> 00:01:55,670 Now I have to say what does that mean? 25 00:01:55,670 --> 00:02:01,080 What does it mean for subspaces to be orthogonal? 26 00:02:01,080 --> 00:02:06,540 But I hope you appreciate the beauty of this picture, 27 00:02:06,540 --> 00:02:11,340 that that those subspaces are going 28 00:02:11,340 --> 00:02:14,170 to come out to be orthogonal. 29 00:02:14,170 --> 00:02:18,140 Those two and also those two. 30 00:02:18,140 --> 00:02:24,870 So that's like one point, one important point 31 00:02:24,870 --> 00:02:28,960 to step forward in understanding those subspaces. 32 00:02:28,960 --> 00:02:31,140 We knew what each subspace was like, 33 00:02:31,140 --> 00:02:33,500 we could compute bases for them. 34 00:02:33,500 --> 00:02:35,440 Now we know more. 35 00:02:35,440 --> 00:02:38,240 Or we will in a few minutes. 36 00:02:38,240 --> 00:02:38,750 OK. 37 00:02:38,750 --> 00:02:43,110 I have to say first of all what does it mean for two vectors 38 00:02:43,110 --> 00:02:44,920 to be orthogonal? 39 00:02:44,920 --> 00:02:46,470 So let me start with that. 40 00:02:49,560 --> 00:02:50,560 Orthogonal vectors. 41 00:02:57,010 --> 00:03:00,610 The word orthogonal is -- is just another word 42 00:03:00,610 --> 00:03:02,620 for perpendicular. 43 00:03:02,620 --> 00:03:05,510 It means that in n-dimensional space 44 00:03:05,510 --> 00:03:10,290 the angle between those vectors is ninety degrees. 45 00:03:10,290 --> 00:03:13,480 It means that they form a right triangle. 46 00:03:13,480 --> 00:03:20,940 It even means that the going way back to the Greeks that this 47 00:03:20,940 --> 00:03:29,010 angle that this triangle a vector x, a vector x, 48 00:03:29,010 --> 00:03:36,560 and a vector x+y -- of course that'll be the hypotenuse, 49 00:03:36,560 --> 00:03:42,750 so what was it the Greeks figured out and it's neat. 50 00:03:42,750 --> 00:03:45,860 It's the fact that the -- 51 00:03:45,860 --> 00:03:50,980 so these are orthogonal, this is a right angle, if -- 52 00:03:50,980 --> 00:03:57,660 so let me put the great name down, Pythagoras, 53 00:03:57,660 --> 00:04:00,540 I'm looking for -- what I looking for? 54 00:04:00,540 --> 00:04:05,980 I'm looking for the condition if you give me two vectors, how do 55 00:04:05,980 --> 00:04:08,540 I know if they're orthogonal? 56 00:04:08,540 --> 00:04:11,490 How can I tell two perpendicular vectors? 57 00:04:11,490 --> 00:04:13,830 And actually you probably know the answer. 58 00:04:13,830 --> 00:04:15,770 Let me write the answer down. 59 00:04:15,770 --> 00:04:20,550 Orthogonal vectors, what's the test for orthogonality? 60 00:04:20,550 --> 00:04:29,430 I take the dot product which I tend to write as x transpose y, 61 00:04:29,430 --> 00:04:33,060 because that's a row times a column, 62 00:04:33,060 --> 00:04:38,390 and that matrix multiplication just gives me the right thing, 63 00:04:38,390 --> 00:04:45,340 that x1y1+x2y2 and so on, so these vectors are orthogonal 64 00:04:45,340 --> 00:04:52,560 if this result x transpose y is zero. 65 00:04:52,560 --> 00:04:54,130 That's the test. 66 00:04:54,130 --> 00:04:55,340 OK. 67 00:04:55,340 --> 00:05:00,850 Can I connect that to other things? 68 00:05:00,850 --> 00:05:06,480 I mean -- it's just beautiful that here we have we're in n 69 00:05:06,480 --> 00:05:09,390 dimensions, we've got a couple of vectors, 70 00:05:09,390 --> 00:05:11,620 we want to know the angle between them, 71 00:05:11,620 --> 00:05:16,750 and the right thing to look at is the simplest thing that you 72 00:05:16,750 --> 00:05:18,930 could imagine, the dot product. 73 00:05:18,930 --> 00:05:19,600 OK. 74 00:05:19,600 --> 00:05:20,890 Now why? 75 00:05:20,890 --> 00:05:24,430 So I'm answering the question now why -- 76 00:05:24,430 --> 00:05:33,500 let's add some justification to this fact, that's the test. 77 00:05:33,500 --> 00:05:38,050 OK, so Pythagoras would say we've got a right triangle, 78 00:05:38,050 --> 00:05:41,400 if that length squared plus that length squared equals 79 00:05:41,400 --> 00:05:43,170 that length squared. 80 00:05:43,170 --> 00:05:51,800 OK, can I write it as x squared plus y squared equals 81 00:05:51,800 --> 00:05:54,890 x plus y squared? 82 00:05:54,890 --> 00:05:59,120 Now don't, please don't think that this is always true. 83 00:05:59,120 --> 00:06:03,360 This is only going to be true in this -- 84 00:06:03,360 --> 00:06:07,600 it's going to be equivalent to orthogonality. 85 00:06:07,600 --> 00:06:11,180 For other triangles of course, it's not true. 86 00:06:11,180 --> 00:06:12,540 For other triangles it's not. 87 00:06:12,540 --> 00:06:16,860 But for a right triangle somehow that fact 88 00:06:16,860 --> 00:06:18,810 should connect to that fact. 89 00:06:18,810 --> 00:06:20,240 Can we just make that connection? 90 00:06:23,580 --> 00:06:28,090 What's the connection between this test for orthogonality 91 00:06:28,090 --> 00:06:31,040 and this statement of orthogonality? 92 00:06:31,040 --> 00:06:35,520 Well, I guess I have to say what is the length squared? 93 00:06:35,520 --> 00:06:42,370 So let's continue on the board underneath with that equation. 94 00:06:42,370 --> 00:06:47,800 Give me another way to express the length squared of a vector. 95 00:06:47,800 --> 00:06:49,960 And let me just give you a vector. 96 00:06:49,960 --> 00:06:53,420 The vector one, two, three. 97 00:06:53,420 --> 00:06:55,730 That's in three dimensions. 98 00:06:55,730 --> 00:07:00,720 What is the length squared of the vector x equal one, two, 99 00:07:00,720 --> 00:07:03,410 three? 100 00:07:03,410 --> 00:07:06,820 So how do you find the length squared? 101 00:07:06,820 --> 00:07:10,710 Well, really you just, you want the length of that vector that 102 00:07:10,710 --> 00:07:15,220 goes along one -- up two, and out three -- 103 00:07:15,220 --> 00:07:19,910 and we'll come back to that right triangle stuff. 104 00:07:19,910 --> 00:07:25,928 The length squared is exactly x transpose x. 105 00:07:28,870 --> 00:07:31,230 Whenever I see x transpose x, I know 106 00:07:31,230 --> 00:07:35,040 I've got a number that's positive. 107 00:07:35,040 --> 00:07:37,750 It's a length squared unless it -- 108 00:07:37,750 --> 00:07:40,440 unless x happens to be the zero vector, 109 00:07:40,440 --> 00:07:44,500 that's the one case where the length is zero. 110 00:07:44,500 --> 00:07:49,770 So right -- this is just x1 squared plus x2 squared plus 111 00:07:49,770 --> 00:07:51,920 so on, plus xn squared. 112 00:07:51,920 --> 00:07:55,830 So one -- in the example I gave you what was the length squared 113 00:07:55,830 --> 00:07:59,230 of that vector one, two, three? 114 00:07:59,230 --> 00:08:02,070 So you square those, you get one, four and nine, 115 00:08:02,070 --> 00:08:04,310 you add, you get fourteen. 116 00:08:04,310 --> 00:08:08,590 So the vector one, two, three has length fourteen. 117 00:08:08,590 --> 00:08:13,000 So let me just put down a vector here. 118 00:08:13,000 --> 00:08:15,995 Let x be the vector one, two, three. 119 00:08:19,220 --> 00:08:22,060 Let me cook up a vector that's orthogonal to it. 120 00:08:25,550 --> 00:08:27,590 So what's the vector that's orthogonal? 121 00:08:27,590 --> 00:08:33,270 So right down here, x squared is one plus four plus nine, 122 00:08:33,270 --> 00:08:38,480 fourteen, let me cook up a vector that's orthogonal to it, 123 00:08:38,480 --> 00:08:47,170 we'll get right that that -- those two vectors are 124 00:08:47,170 --> 00:08:55,320 orthogonal, the length of y squared is five, 125 00:08:55,320 --> 00:09:02,520 and x plus y is one and two making three, 126 00:09:02,520 --> 00:09:06,290 two and minus one making one, three and zero making three, 127 00:09:06,290 --> 00:09:13,010 and the length of this squared is nine plus one plus nine, 128 00:09:13,010 --> 00:09:15,700 nineteen. 129 00:09:15,700 --> 00:09:20,810 And sure enough, I haven't proved anything. 130 00:09:20,810 --> 00:09:28,900 I've just like checked to see that my test x transpose 131 00:09:28,900 --> 00:09:31,530 y equals zero, which is true, right? 132 00:09:31,530 --> 00:09:35,370 Everybody sees that x transpose y is zero here? 133 00:09:35,370 --> 00:09:37,740 That's maybe the main point. 134 00:09:37,740 --> 00:09:41,910 That you should get really quick at doing x transpose y, 135 00:09:41,910 --> 00:09:45,980 so it's just this plus this plus this and that's zero. 136 00:09:45,980 --> 00:09:51,730 And sure enough, that clicks with fourteen plus five 137 00:09:51,730 --> 00:09:53,190 agreeing with nineteen. 138 00:09:53,190 --> 00:09:56,780 Now let me just do that in letters. 139 00:09:56,780 --> 00:10:01,110 So that's y transpose y. 140 00:10:01,110 --> 00:10:08,150 And this is x plus y transpose x plus y. 141 00:10:08,150 --> 00:10:09,290 OK. 142 00:10:09,290 --> 00:10:12,530 So I'm looking, again, this isn't always true. 143 00:10:12,530 --> 00:10:14,110 I repeat. 144 00:10:14,110 --> 00:10:18,020 This is going to be true when we have a right angle. 145 00:10:18,020 --> 00:10:20,210 And let's just -- well, of course, 146 00:10:20,210 --> 00:10:22,850 I'm just going to simplify this stuff here. 147 00:10:22,850 --> 00:10:25,960 There's an x transpose x there. 148 00:10:25,960 --> 00:10:29,850 And there's a y transpose y there. 149 00:10:29,850 --> 00:10:34,450 And there's an x transpose y. 150 00:10:34,450 --> 00:10:36,610 And there's a y transpose x. 151 00:10:41,700 --> 00:10:46,990 I knew I could do that simplification because I'm just 152 00:10:46,990 --> 00:10:50,150 doing matrix multiplication and I've just followed the rules. 153 00:10:50,150 --> 00:10:50,880 OK. 154 00:10:50,880 --> 00:10:53,650 So x transpose x-s cancel. 155 00:10:53,650 --> 00:10:55,910 Y transpose y-s cancel. 156 00:10:55,910 --> 00:10:57,950 And what about these guys? 157 00:10:57,950 --> 00:11:03,400 What can you tell me about the inner product of x with y 158 00:11:03,400 --> 00:11:06,890 and the inner product of y with x? 159 00:11:06,890 --> 00:11:09,620 Is there a difference? 160 00:11:09,620 --> 00:11:15,150 I think if we -- while we're doing real vectors, 161 00:11:15,150 --> 00:11:18,310 which is all we're doing now, there isn't a difference, 162 00:11:18,310 --> 00:11:19,660 there's no difference. 163 00:11:19,660 --> 00:11:23,690 If I take x transpose y, that'll give me zero, 164 00:11:23,690 --> 00:11:28,840 if I took y transpose x I would have the same x1y1 and x2y2 165 00:11:28,840 --> 00:11:35,280 and x3y3, it would be the same, so this is -- 166 00:11:35,280 --> 00:11:37,820 this is the same as that, this is really 167 00:11:37,820 --> 00:11:40,070 I'll knock that guy out and say two of these. 168 00:11:43,700 --> 00:11:47,360 So actually that's the -- 169 00:11:47,360 --> 00:11:51,960 this equation boiled down to this thing being zero. 170 00:11:51,960 --> 00:11:52,670 Right? 171 00:11:52,670 --> 00:11:56,010 Everything else canceled and this equation 172 00:11:56,010 --> 00:11:57,250 boiled down to that one. 173 00:11:57,250 --> 00:12:00,450 So that's really all I wanted. 174 00:12:00,450 --> 00:12:08,170 I just wanted to check that Pythagoras for a right triangle 175 00:12:08,170 --> 00:12:13,250 led me to this -- of course I cancel the two now. 176 00:12:13,250 --> 00:12:14,680 No problem. 177 00:12:14,680 --> 00:12:18,880 To x transpose y equals zero as the test. 178 00:12:18,880 --> 00:12:19,610 Fair enough. 179 00:12:19,610 --> 00:12:21,190 OK. 180 00:12:21,190 --> 00:12:23,150 You knew it was coming. 181 00:12:23,150 --> 00:12:27,010 The dot product of orthogonal vectors is zero. 182 00:12:27,010 --> 00:12:31,010 It's just -- I just want to say that's really neat. 183 00:12:31,010 --> 00:12:32,760 That it comes out so well. 184 00:12:32,760 --> 00:12:34,830 All right. 185 00:12:34,830 --> 00:12:38,760 Now what about -- so now I know if two -- 186 00:12:38,760 --> 00:12:40,700 when two vectors are orthogonal. 187 00:12:40,700 --> 00:12:45,860 By the way, what about if one of these guys is the zero vector? 188 00:12:45,860 --> 00:12:51,660 Suppose x is the zero vector, and y is whatever. 189 00:12:51,660 --> 00:12:55,010 Are they orthogonal? 190 00:12:55,010 --> 00:12:57,190 Sure. 191 00:12:57,190 --> 00:12:59,310 In math the one thing about math is you're 192 00:12:59,310 --> 00:13:01,910 supposed to follow the rules. 193 00:13:01,910 --> 00:13:05,450 So you're supposed to -- if x is the zero vector, 194 00:13:05,450 --> 00:13:09,330 you're supposed to take the zero vector dotted with y 195 00:13:09,330 --> 00:13:12,180 and of course you always get zero. 196 00:13:12,180 --> 00:13:17,340 So just so we're all sure, the zero vector 197 00:13:17,340 --> 00:13:20,730 is orthogonal to everybody. 198 00:13:20,730 --> 00:13:24,770 But what I want to -- 199 00:13:24,770 --> 00:13:32,240 what I now want to think about is subspaces. 200 00:13:32,240 --> 00:13:37,530 What does it mean for me to say that some subspace is 201 00:13:37,530 --> 00:13:39,320 orthogonal to some other subspace? 202 00:13:39,320 --> 00:13:41,110 So OK. 203 00:13:41,110 --> 00:13:42,590 Now I've got to write this down. 204 00:13:42,590 --> 00:13:47,960 So because we're defining definition of subspace 205 00:13:47,960 --> 00:14:02,390 S is orthogonal so to subspace let's say T, 206 00:14:02,390 --> 00:14:04,130 so I've got a couple of subspaces. 207 00:14:07,350 --> 00:14:11,480 And what should it mean for those guys to be orthogonal? 208 00:14:11,480 --> 00:14:14,780 It's just sort of what's the natural extension 209 00:14:14,780 --> 00:14:19,990 from orthogonal vectors to orthogonal subspaces? 210 00:14:19,990 --> 00:14:26,820 Well, and in particular, let's think 211 00:14:26,820 --> 00:14:34,410 of some orthogonal subspaces, like this wall. 212 00:14:34,410 --> 00:14:37,300 Let's say in three dimensions. 213 00:14:37,300 --> 00:14:41,320 So the blackboard extended to infinity, right, is a -- 214 00:14:41,320 --> 00:14:44,950 is a subspace, a plane, a two-dimensional subspace. 215 00:14:47,680 --> 00:14:51,310 It's a little bumpy but anyway, it's a -- 216 00:14:51,310 --> 00:14:57,001 think of it as a subspace, let me take the floor as another 217 00:14:57,001 --> 00:14:57,500 subspace. 218 00:15:01,850 --> 00:15:07,840 Again, it's not a great subspace, 219 00:15:07,840 --> 00:15:15,400 MIT only built it like so-so, but I'll 220 00:15:15,400 --> 00:15:18,840 put the origin right here. 221 00:15:18,840 --> 00:15:23,040 So the origin of the world is right there. 222 00:15:23,040 --> 00:15:23,540 OK. 223 00:15:26,730 --> 00:15:30,340 Thereby giving linear algebra its proper importance in this. 224 00:15:30,340 --> 00:15:31,190 OK. 225 00:15:31,190 --> 00:15:34,290 So there is one subspace, there's another one. 226 00:15:34,290 --> 00:15:35,940 The floor. 227 00:15:35,940 --> 00:15:42,200 And are they orthogonal? 228 00:15:42,200 --> 00:15:44,130 What does it mean for two subspaces 229 00:15:44,130 --> 00:15:46,460 to be orthogonal and in that special case 230 00:15:46,460 --> 00:15:47,650 are they orthogonal? 231 00:15:47,650 --> 00:15:48,290 All right. 232 00:15:48,290 --> 00:15:50,950 Let's finish this sentence. 233 00:15:50,950 --> 00:15:58,501 What does it mean means we have to know 234 00:15:58,501 --> 00:15:59,500 what we're talking about 235 00:15:59,500 --> 00:16:00,040 here. 236 00:16:00,040 --> 00:16:05,310 So what would be a reasonable idea of orthogonal? 237 00:16:05,310 --> 00:16:08,740 Well, let me put the right thing up. 238 00:16:08,740 --> 00:16:17,930 It means that every vector in S, every vector in S, 239 00:16:17,930 --> 00:16:23,565 is orthogonal to -- 240 00:16:27,290 --> 00:16:30,080 what I going to say? 241 00:16:30,080 --> 00:16:38,000 Every vector in T. 242 00:16:38,000 --> 00:16:42,990 That's a reasonable and it's a good 243 00:16:42,990 --> 00:16:45,560 and it's the right definition for two subspaces 244 00:16:45,560 --> 00:16:47,010 to be orthogonal. 245 00:16:47,010 --> 00:16:50,740 But I just want you to see, hey, what does that mean? 246 00:16:50,740 --> 00:16:53,620 So answer the question about the -- 247 00:16:53,620 --> 00:16:57,440 the blackboard and the floor. 248 00:16:57,440 --> 00:17:01,440 Are those two subspaces, they're two-dimensional, right, 249 00:17:01,440 --> 00:17:03,830 and we're in R^3. 250 00:17:03,830 --> 00:17:09,665 It's like a xz plane or something and a xy plane. 251 00:17:15,089 --> 00:17:17,109 Are they orthogonal? 252 00:17:17,109 --> 00:17:19,560 Is every vector in the blackboard orthogonal 253 00:17:19,560 --> 00:17:23,349 to every vector in the floor, starting from the origin 254 00:17:23,349 --> 00:17:24,140 right there? 255 00:17:27,640 --> 00:17:28,900 Yes or no? 256 00:17:28,900 --> 00:17:31,010 I could take a vote. 257 00:17:31,010 --> 00:17:35,040 Well we get some yeses and some noes. 258 00:17:35,040 --> 00:17:36,170 No is the answer. 259 00:17:36,170 --> 00:17:37,810 They're not. 260 00:17:37,810 --> 00:17:41,340 You can tell me a vector in the blackboard 261 00:17:41,340 --> 00:17:43,995 and a vector in the floor that are not orthogonal. 262 00:17:49,360 --> 00:17:54,960 Well you can tell me quite a few, I guess. 263 00:17:54,960 --> 00:17:59,140 Maybe like I could take some forty-five-degree guy 264 00:17:59,140 --> 00:18:05,730 in the blackboard, and something in the floor, 265 00:18:05,730 --> 00:18:08,580 they're not at ninety degrees, right? 266 00:18:08,580 --> 00:18:10,780 In fact, even more, you could tell me 267 00:18:10,780 --> 00:18:17,140 a vector that's in both the blackboard plane and the floor 268 00:18:17,140 --> 00:18:20,550 plane, so it's certainly not orthogonal to itself. 269 00:18:20,550 --> 00:18:25,600 So for sure, those two planes aren't orthogonal. 270 00:18:25,600 --> 00:18:26,950 What would that be? 271 00:18:26,950 --> 00:18:29,350 So what's a vector that's -- 272 00:18:29,350 --> 00:18:32,320 in both of those planes? 273 00:18:32,320 --> 00:18:35,170 It's this guy running along the crack 274 00:18:35,170 --> 00:18:39,540 there, in the intersection, the intersection. 275 00:18:39,540 --> 00:18:41,360 A vector, you know -- 276 00:18:41,360 --> 00:18:45,460 if two subspaces meet at some vector, well then for sure 277 00:18:45,460 --> 00:18:51,170 they're not orthogonal, because that vector is in one 278 00:18:51,170 --> 00:18:54,240 and it's in the other, and it's not orthogonal to itself 279 00:18:54,240 --> 00:18:55,770 unless it's zero. 280 00:18:55,770 --> 00:19:01,000 So the only I mean so orthogonal is 281 00:19:01,000 --> 00:19:04,220 for me to say these two subspaces are orthogonal first 282 00:19:04,220 --> 00:19:09,080 of all I'm certainly saying that they don't intersect 283 00:19:09,080 --> 00:19:14,550 in any nonzero vector. 284 00:19:14,550 --> 00:19:18,430 But also I mean more than that just not intersecting 285 00:19:18,430 --> 00:19:18,990 isn't good 286 00:19:18,990 --> 00:19:19,490 enough. 287 00:19:19,490 --> 00:19:26,150 So give me an example, oh, let's say in the plane, oh well, 288 00:19:26,150 --> 00:19:30,090 when do we have orthogonal subspaces in the plane? 289 00:19:30,090 --> 00:19:32,100 Yeah, tell me in the plane, so we don't -- 290 00:19:32,100 --> 00:19:34,960 there aren't that many different subspaces in the plane. 291 00:19:34,960 --> 00:19:38,385 What what have we got in the plane as possible subspaces? 292 00:19:40,900 --> 00:19:44,420 The zero vector, real small. 293 00:19:44,420 --> 00:19:47,330 A line through the origin. 294 00:19:47,330 --> 00:19:49,570 Or the whole plane. 295 00:19:49,570 --> 00:19:50,420 OK. 296 00:19:50,420 --> 00:19:55,769 Now so when is a line through the origin orthogonal 297 00:19:55,769 --> 00:19:56,560 to the whole plane? 298 00:19:59,870 --> 00:20:02,070 Never, right, never. 299 00:20:02,070 --> 00:20:05,620 When is a line through the origin orthogonal to the zero 300 00:20:05,620 --> 00:20:06,120 subspace? 301 00:20:09,310 --> 00:20:10,180 Always. 302 00:20:10,180 --> 00:20:10,910 Right. 303 00:20:10,910 --> 00:20:13,480 When is a line through the origin orthogonal 304 00:20:13,480 --> 00:20:15,920 to a different line through the origin? 305 00:20:15,920 --> 00:20:19,880 Well, that's the case that we all have a clear picture 306 00:20:19,880 --> 00:20:21,070 of, they -- 307 00:20:21,070 --> 00:20:23,610 the two lines have to meet at ninety degrees. 308 00:20:23,610 --> 00:20:28,340 They have only the -- so that's like this simple case 309 00:20:28,340 --> 00:20:29,090 I'm talking about. 310 00:20:29,090 --> 00:20:31,970 There's one subspace, there's the other subspace. 311 00:20:31,970 --> 00:20:33,770 They only meet at zero. 312 00:20:33,770 --> 00:20:35,570 And they're orthogonal. 313 00:20:35,570 --> 00:20:36,340 OK. 314 00:20:36,340 --> 00:20:37,930 Now. 315 00:20:37,930 --> 00:20:43,540 So we now know what it means for two subspaces to be orthogonal. 316 00:20:43,540 --> 00:20:47,240 And now I want to say that this is true for the row 317 00:20:47,240 --> 00:20:49,080 space and the null space. 318 00:20:49,080 --> 00:20:49,740 OK. 319 00:20:49,740 --> 00:20:53,880 So that's the neat fact. 320 00:20:53,880 --> 00:21:05,720 So row space is orthogonal to the null space. 321 00:21:05,720 --> 00:21:07,435 Now how did I come up with that? 322 00:21:12,540 --> 00:21:16,500 But you see the rank it's great, that means that these -- 323 00:21:16,500 --> 00:21:19,020 that these subspaces are just the right things, 324 00:21:19,020 --> 00:21:23,220 they're just cutting the whole space up 325 00:21:23,220 --> 00:21:27,560 into two perpendicular subspaces. 326 00:21:27,560 --> 00:21:28,060 OK. 327 00:21:28,060 --> 00:21:28,560 So why? 328 00:21:33,710 --> 00:21:38,040 Well, what have I got to work with? 329 00:21:38,040 --> 00:21:41,230 All I know is the null space. 330 00:21:41,230 --> 00:21:46,360 The null space has vectors that solve Ax equals zero. 331 00:21:46,360 --> 00:21:49,780 So this is a guy x. 332 00:21:49,780 --> 00:21:53,940 x is in the null space. 333 00:21:53,940 --> 00:21:57,350 Then Ax is zero. 334 00:21:57,350 --> 00:22:03,250 So why is it orthogonal to the rows of A? 335 00:22:03,250 --> 00:22:05,340 If I write down Ax equals zero, which 336 00:22:05,340 --> 00:22:07,720 is all I know about the null space, 337 00:22:07,720 --> 00:22:13,130 then I guess I want you to see that that's telling me, 338 00:22:13,130 --> 00:22:16,430 just that equation right there is telling me 339 00:22:16,430 --> 00:22:19,670 that the rows of A, let me write it out. 340 00:22:19,670 --> 00:22:24,000 There's row one of A. 341 00:22:24,000 --> 00:22:24,540 Row two. 342 00:22:27,290 --> 00:22:32,300 Row m of A. that's A. 343 00:22:32,300 --> 00:22:34,530 And it's multiplying X. 344 00:22:34,530 --> 00:22:36,560 And it's producing zero. 345 00:22:36,560 --> 00:22:37,060 OK. 346 00:22:41,550 --> 00:22:47,200 Written out that way you'll see it. 347 00:22:47,200 --> 00:22:50,200 So I'm saying that a vector in the row space 348 00:22:50,200 --> 00:22:54,250 is perpendicular to this guy X in the null space. 349 00:22:54,250 --> 00:22:57,310 And you see why? 350 00:22:57,310 --> 00:22:59,550 Because this equation is telling you 351 00:22:59,550 --> 00:23:06,330 that row one of A multiplying that's a dot product, right? 352 00:23:06,330 --> 00:23:11,480 Row one of A dot product with this x is producing this zero. 353 00:23:11,480 --> 00:23:15,970 So x is orthogonal to the first row. 354 00:23:15,970 --> 00:23:17,670 And to the second row. 355 00:23:17,670 --> 00:23:20,630 Row two of A, x is giving that zero. 356 00:23:20,630 --> 00:23:23,590 Row m of A times x is giving that zero. 357 00:23:23,590 --> 00:23:25,020 So x is -- 358 00:23:25,020 --> 00:23:27,380 the equation is telling me that x 359 00:23:27,380 --> 00:23:31,309 is orthogonal to all the rows. 360 00:23:31,309 --> 00:23:32,600 Right, it's just sitting there. 361 00:23:32,600 --> 00:23:36,380 That's all we -- it had to be sitting there because we 362 00:23:36,380 --> 00:23:40,000 didn't know anything more about the null space than this. 363 00:23:40,000 --> 00:23:46,120 And now I guess to be totally complete here 364 00:23:46,120 --> 00:23:49,350 I'd now check that x is orthogonal 365 00:23:49,350 --> 00:23:51,720 to each separate row. 366 00:23:51,720 --> 00:23:55,710 But what else strictly speaking do I have to do? 367 00:23:58,670 --> 00:24:02,340 To show that those subspaces are orthogonal, 368 00:24:02,340 --> 00:24:05,580 I have to take this x in the null space and show that 369 00:24:05,580 --> 00:24:10,380 it's orthogonal to every vector in the row space, 370 00:24:10,380 --> 00:24:12,740 every vector in the row space, so what -- 371 00:24:12,740 --> 00:24:15,330 what else is in the row space? 372 00:24:15,330 --> 00:24:18,970 This row is in the row space, that row is in the row space, 373 00:24:18,970 --> 00:24:22,870 they're all there, but it's not the whole row space, 374 00:24:22,870 --> 00:24:24,990 right, we just have to like remember, what does it 375 00:24:24,990 --> 00:24:29,610 mean, what does that word space telling us? 376 00:24:29,610 --> 00:24:34,470 And what else is in the row space? 377 00:24:34,470 --> 00:24:37,810 Besides the rows? 378 00:24:37,810 --> 00:24:41,990 All their combinations. 379 00:24:41,990 --> 00:24:44,680 So I really have to check that sure enough 380 00:24:44,680 --> 00:24:46,770 if x is perpendicular to row one, 381 00:24:46,770 --> 00:24:49,720 row two, all the different separate rows, 382 00:24:49,720 --> 00:24:54,110 then also x is perpendicular to a combination of the rows. 383 00:24:54,110 --> 00:24:57,000 And that's just matrix multiplication again. 384 00:24:57,000 --> 00:25:03,300 You know, I have row one transpose x is zero, 385 00:25:03,300 --> 00:25:11,000 so on, row two transpose x is zero, 386 00:25:11,000 --> 00:25:16,800 so I'm entitled to multiply that by some c1, this by some c2, 387 00:25:16,800 --> 00:25:20,700 I still have zeroes, I'm entitled to add, 388 00:25:20,700 --> 00:25:24,760 so I have c1 row one so -- so all this when I put that 389 00:25:24,760 --> 00:25:33,290 together that's big parentheses c1 row one plus c2 row two 390 00:25:33,290 --> 00:25:34,780 and so on. 391 00:25:34,780 --> 00:25:38,220 Transpose x is zero. 392 00:25:38,220 --> 00:25:38,720 Right? 393 00:25:38,720 --> 00:25:41,380 I just added the zeroes and got zero, 394 00:25:41,380 --> 00:25:43,760 and I just added these following the rule. 395 00:25:46,550 --> 00:25:48,490 No big deal. 396 00:25:48,490 --> 00:25:51,890 The whole point was right sitting in that. 397 00:25:51,890 --> 00:25:54,610 OK. 398 00:25:54,610 --> 00:26:02,480 So if I come back to this figure now I'm like a happier person. 399 00:26:02,480 --> 00:26:05,090 Because I have this -- 400 00:26:05,090 --> 00:26:10,730 I now see how those subspaces are oriented. 401 00:26:10,730 --> 00:26:14,090 And these subspaces are also oriented. 402 00:26:14,090 --> 00:26:20,200 Well, actually why is that orthogonality? 403 00:26:20,200 --> 00:26:23,660 Well, it's the same statement for A transpose 404 00:26:23,660 --> 00:26:25,330 that that one was for A. 405 00:26:25,330 --> 00:26:27,670 So I won't take time to prove it again 406 00:26:27,670 --> 00:26:32,500 because we've checked it for every matrix 407 00:26:32,500 --> 00:26:35,960 and A transpose is just as good a matrix as A. 408 00:26:35,960 --> 00:26:39,520 So we're orthogonal over there. 409 00:26:39,520 --> 00:26:46,180 So we really have carved up this -- 410 00:26:46,180 --> 00:26:50,070 this was like carving up m-dimensional space 411 00:26:50,070 --> 00:26:56,070 into two subspaces and this one was carving up 412 00:26:56,070 --> 00:27:01,810 n-dimensional space into two subspaces. 413 00:27:01,810 --> 00:27:06,500 And well, one more thing here. 414 00:27:06,500 --> 00:27:07,550 One more important thing. 415 00:27:11,110 --> 00:27:13,330 Let me move into three dimensions. 416 00:27:15,990 --> 00:27:22,780 Tell me a couple of orthogonal subspaces in three dimensions 417 00:27:22,780 --> 00:27:27,450 that somehow don't carve up the whole space, 418 00:27:27,450 --> 00:27:30,330 there's stuff left there. 419 00:27:30,330 --> 00:27:34,830 I'm thinking of a couple of orthogonal lines. 420 00:27:34,830 --> 00:27:38,550 If I -- suppose I'm in three dimensions, R^3. 421 00:27:38,550 --> 00:27:43,510 And I have one line, one one-dimensional subspace, 422 00:27:43,510 --> 00:27:46,030 and a perpendicular one. 423 00:27:46,030 --> 00:27:51,170 Could those be the row space and the null space? 424 00:27:51,170 --> 00:27:54,590 Could those be the row space and the null space? 425 00:27:54,590 --> 00:28:00,520 Could I be in three dimensions and have 426 00:28:00,520 --> 00:28:05,930 a row space that's a line and a null space that's a line? 427 00:28:05,930 --> 00:28:07,270 No. 428 00:28:07,270 --> 00:28:10,180 Why not? 429 00:28:10,180 --> 00:28:11,840 Because the dimensions aren't right. 430 00:28:11,840 --> 00:28:12,340 Right? 431 00:28:12,340 --> 00:28:14,080 The dimensions are no good. 432 00:28:14,080 --> 00:28:19,050 The dimensions here, r and n-r, they add up to three, 433 00:28:19,050 --> 00:28:21,490 they add up to n. 434 00:28:21,490 --> 00:28:23,220 If I take -- 435 00:28:23,220 --> 00:28:26,800 just follow that example -- 436 00:28:26,800 --> 00:28:30,910 if the row space is one-dimensional, 437 00:28:30,910 --> 00:28:36,030 suppose A is what's a good in R^3, 438 00:28:36,030 --> 00:28:39,560 I want a one-dimensional row space, let me take one, two, 439 00:28:39,560 --> 00:28:43,470 five, two, four, ten. 440 00:28:43,470 --> 00:28:45,330 What's the dimension of that row space? 441 00:28:48,590 --> 00:28:50,220 One. 442 00:28:50,220 --> 00:28:52,260 What's the dimension of the null space? 443 00:28:56,160 --> 00:28:59,130 Tell what's the null space look like in that case? 444 00:28:59,130 --> 00:29:01,670 The row space is a line, right? 445 00:29:01,670 --> 00:29:06,440 One-dimensional, it's just a line through one, two, five. 446 00:29:06,440 --> 00:29:08,370 Geometrically what's the row space look like? 447 00:29:13,950 --> 00:29:15,660 What's its dimension? 448 00:29:15,660 --> 00:29:20,920 So here r here n is three, the rank 449 00:29:20,920 --> 00:29:26,610 is one, so the dimension of the null space, 450 00:29:26,610 --> 00:29:32,180 so I'm looking at this x, x1, x2, x3. 451 00:29:32,180 --> 00:29:33,680 To give zero. 452 00:29:33,680 --> 00:29:43,300 So the dimension of the null space is we all know is two. 453 00:29:43,300 --> 00:29:43,800 Right. 454 00:29:43,800 --> 00:29:45,440 It's a plane. 455 00:29:45,440 --> 00:29:49,770 And now actually we know, we see better, what plane is it? 456 00:29:49,770 --> 00:29:52,660 What plane is it? 457 00:29:52,660 --> 00:29:57,500 It's the plane that's perpendicular to one, 458 00:29:57,500 --> 00:29:59,950 two, five. 459 00:29:59,950 --> 00:30:00,450 Right? 460 00:30:00,450 --> 00:30:01,350 We now see. 461 00:30:01,350 --> 00:30:04,710 In fact the two, four, ten didn't actually 462 00:30:04,710 --> 00:30:07,000 have any effect at all. 463 00:30:07,000 --> 00:30:09,750 I could have just ignored that. 464 00:30:09,750 --> 00:30:14,340 That didn't change the row space or the null space. 465 00:30:14,340 --> 00:30:17,051 I'll just make that one equation. 466 00:30:17,051 --> 00:30:17,550 Yeah. 467 00:30:17,550 --> 00:30:18,000 OK. 468 00:30:18,000 --> 00:30:18,499 Sure. 469 00:30:18,499 --> 00:30:21,210 That's the easiest to deal with. 470 00:30:21,210 --> 00:30:22,130 One equation. 471 00:30:22,130 --> 00:30:24,440 Three unknowns. 472 00:30:24,440 --> 00:30:32,670 And I want to ask -- 473 00:30:32,670 --> 00:30:37,180 what would the equation give me the null space, 474 00:30:37,180 --> 00:30:41,380 and you would have said back in September 475 00:30:41,380 --> 00:30:43,410 you would have said it gives you a plane, 476 00:30:43,410 --> 00:30:46,530 and we're completely right. 477 00:30:46,530 --> 00:30:49,530 And the plane it gives you, the normal vector, 478 00:30:49,530 --> 00:30:53,610 you remember in calculus, there was this dumb normal vector 479 00:30:53,610 --> 00:30:54,760 called N. 480 00:30:54,760 --> 00:30:55,496 Well there it is. 481 00:30:55,496 --> 00:30:56,120 One, two, five. 482 00:30:56,120 --> 00:30:56,620 OK. 483 00:30:56,620 --> 00:31:08,820 What is the what's the point I want to make here? 484 00:31:08,820 --> 00:31:10,010 I want to make -- 485 00:31:10,010 --> 00:31:14,300 I want to emphasize that not only are the -- 486 00:31:14,300 --> 00:31:15,410 let me write it in words. 487 00:31:19,930 --> 00:31:33,900 So I want to write the null space and the row space are 488 00:31:33,900 --> 00:31:40,290 orthogonal, that's this neat fact, which we've -- 489 00:31:40,290 --> 00:31:43,240 we've just checked from Ax equals zero, 490 00:31:43,240 --> 00:31:48,750 but now I want to say more because there's a little more 491 00:31:48,750 --> 00:31:51,190 that's true. 492 00:31:51,190 --> 00:31:54,940 Their dimensions add to the whole space. 493 00:31:54,940 --> 00:31:57,660 So that's like a little extra information. 494 00:31:57,660 --> 00:31:59,860 That it's not like I could have -- 495 00:31:59,860 --> 00:32:03,030 I couldn't have a line and a line in three dimensions. 496 00:32:03,030 --> 00:32:07,130 Those don't add up one and one don't add to three. 497 00:32:07,130 --> 00:32:17,820 So I used the word orthogonal complements in R^n. 498 00:32:17,820 --> 00:32:20,000 And the idea of this word complement 499 00:32:20,000 --> 00:32:28,720 is that the orthogonal complement of a row space 500 00:32:28,720 --> 00:32:32,950 contains not just some vectors that are orthogonal to it, 501 00:32:32,950 --> 00:32:34,400 but all. 502 00:32:34,400 --> 00:32:36,050 So what does that mean? 503 00:32:36,050 --> 00:32:42,260 That means that the null space contains all, not just 504 00:32:42,260 --> 00:32:51,270 some but all, vectors that are perpendicular to the row space. 505 00:32:51,270 --> 00:32:52,260 OK. 506 00:32:52,260 --> 00:33:04,190 Really what I've done in this half of the lecture is just 507 00:33:04,190 --> 00:33:09,100 notice some of the nice geometry that -- 508 00:33:09,100 --> 00:33:12,030 that we didn't pick up before because we didn't discuss 509 00:33:12,030 --> 00:33:15,010 perpendicular vectors before. 510 00:33:15,010 --> 00:33:16,800 But it was all sitting there. 511 00:33:16,800 --> 00:33:18,440 And now we picked it up. 512 00:33:18,440 --> 00:33:21,370 That these vectors are orthogonal complements. 513 00:33:21,370 --> 00:33:23,740 And I guess I even call this part 514 00:33:23,740 --> 00:33:26,730 one of the fundamental theorem of linear algebra. 515 00:33:26,730 --> 00:33:32,330 The fundamental theorem of linear algebra 516 00:33:32,330 --> 00:33:37,230 is about these four subspaces, so part one 517 00:33:37,230 --> 00:33:41,660 is about their dimension, maybe I should call it part two now. 518 00:33:41,660 --> 00:33:44,600 Their dimensions we got. 519 00:33:44,600 --> 00:33:49,010 Now we're getting their orthogonality, that's part two. 520 00:33:49,010 --> 00:33:54,640 And part three will be about bases for them. 521 00:33:54,640 --> 00:33:57,290 Orthogonal bases. 522 00:33:57,290 --> 00:34:00,520 So that's coming up. 523 00:34:00,520 --> 00:34:01,570 OK. 524 00:34:01,570 --> 00:34:10,870 So I'm happy with that geometry right now. 525 00:34:10,870 --> 00:34:11,980 OK. 526 00:34:11,980 --> 00:34:12,949 OK. 527 00:34:12,949 --> 00:34:16,239 Now what's my next goal in this chapter? 528 00:34:16,239 --> 00:34:19,087 Here's the main problem of the chapter. 529 00:34:19,087 --> 00:34:21,420 The main problem of the chapter is -- so this is coming. 530 00:34:21,420 --> 00:34:22,378 It's coming attraction. 531 00:34:22,378 --> 00:34:37,160 This is the very last chapter that's about Ax=b. 532 00:34:44,030 --> 00:34:48,690 I would like to solve that system of equations 533 00:34:48,690 --> 00:34:50,710 when there is no solution. 534 00:34:54,560 --> 00:34:56,880 You may say what a ridiculous thing to do. 535 00:34:56,880 --> 00:35:00,670 But I have to say it's done all the time. 536 00:35:00,670 --> 00:35:02,440 In fact it has to be done. 537 00:35:02,440 --> 00:35:07,510 You get -- so the problem is solve -- 538 00:35:07,510 --> 00:35:20,626 the best possible solve I'll put quote Ax=b when there is no 539 00:35:20,626 --> 00:35:21,125 solution. 540 00:35:24,470 --> 00:35:25,930 And of course what does that mean? 541 00:35:25,930 --> 00:35:29,550 b isn't in the column space. 542 00:35:29,550 --> 00:35:35,720 And it's quite typical if this matrix A is rectangular if I -- 543 00:35:35,720 --> 00:35:40,370 maybe I have m equations and that's bigger than the number 544 00:35:40,370 --> 00:35:47,840 of unknowns, then for sure the rank is not m, 545 00:35:47,840 --> 00:35:51,910 the rank couldn't be m now, so there'll be a lot of right-hand 546 00:35:51,910 --> 00:36:00,300 sides with no solution, but here's an example. 547 00:36:00,300 --> 00:36:02,760 Some satellite is buzzing along. 548 00:36:02,760 --> 00:36:06,196 You measure its position. 549 00:36:06,196 --> 00:36:07,570 You make a thousand measurements. 550 00:36:10,190 --> 00:36:13,940 So that gives you a thousand equations for the -- 551 00:36:13,940 --> 00:36:18,340 for the parameters that -- that give the position. 552 00:36:18,340 --> 00:36:20,350 But there aren't a thousand parameters, 553 00:36:20,350 --> 00:36:23,310 there's just maybe six or something. 554 00:36:23,310 --> 00:36:26,930 Or you're measuring the -- you're doing questionnaires. 555 00:36:29,690 --> 00:36:36,210 You're measuring resistances. 556 00:36:36,210 --> 00:36:37,650 You're taking pulses. 557 00:36:37,650 --> 00:36:41,120 You're measuring somebody's pulse rate. 558 00:36:41,120 --> 00:36:42,190 There's just one unknown. 559 00:36:42,190 --> 00:36:42,690 OK. 560 00:36:42,690 --> 00:36:44,890 The pulse rate. 561 00:36:44,890 --> 00:36:46,950 So you measure it once, OK, fine, 562 00:36:46,950 --> 00:36:49,510 but if you really want to know it, 563 00:36:49,510 --> 00:36:54,800 you measure it multiple times, but then the measurements have 564 00:36:54,800 --> 00:36:57,120 noise in them, so there's -- 565 00:36:57,120 --> 00:36:59,820 the problem is that in many many problems 566 00:36:59,820 --> 00:37:03,740 we've got too many equations and they've got 567 00:37:03,740 --> 00:37:05,240 noise in the right-hand side. 568 00:37:05,240 --> 00:37:11,780 So Ax=b I can't expect to solve it exactly right, 569 00:37:11,780 --> 00:37:13,230 because I don't even know what -- 570 00:37:13,230 --> 00:37:18,680 there's a measurement mistake in b. 571 00:37:18,680 --> 00:37:21,790 But there's information too. 572 00:37:21,790 --> 00:37:25,120 There's a lot of information about x in there. 573 00:37:25,120 --> 00:37:30,070 And what I want to do is like separate the noise, the junk, 574 00:37:30,070 --> 00:37:33,860 from the information. 575 00:37:33,860 --> 00:37:38,680 And so this is a straightforward linear algebra problem. 576 00:37:38,680 --> 00:37:41,600 How do I solve, what's the best solution? 577 00:37:41,600 --> 00:37:43,600 OK. 578 00:37:43,600 --> 00:37:45,070 Now. 579 00:37:45,070 --> 00:37:52,190 I want to say so that's like describes the problem 580 00:37:52,190 --> 00:37:55,010 in an algebraic way. 581 00:37:55,010 --> 00:37:58,820 I got some equations, I'm looking for the best solution. 582 00:37:58,820 --> 00:38:02,020 Well, one way to find it is -- one way to start, 583 00:38:02,020 --> 00:38:09,580 one way to find a solution is throw away equations until 584 00:38:09,580 --> 00:38:12,340 you've got a nice, square invertible system and solve 585 00:38:12,340 --> 00:38:14,760 that. 586 00:38:14,760 --> 00:38:18,620 That's not satisfactory. 587 00:38:18,620 --> 00:38:21,410 There's no reason in these measurements 588 00:38:21,410 --> 00:38:23,300 to say these measurements are perfect 589 00:38:23,300 --> 00:38:25,970 and these measurements are useless. 590 00:38:25,970 --> 00:38:27,990 We want to use all the measurements 591 00:38:27,990 --> 00:38:32,140 to get the best information, to get the maximum information. 592 00:38:32,140 --> 00:38:33,170 But how? 593 00:38:33,170 --> 00:38:34,390 OK. 594 00:38:34,390 --> 00:38:40,020 Let me anticipate a matrix that's going to show up. 595 00:38:40,020 --> 00:38:43,720 This A is typically rectangular. 596 00:38:43,720 --> 00:38:47,390 But a matrix that shows up whenever you have -- 597 00:38:47,390 --> 00:38:52,260 and we chapter three was all about rectangular matrices. 598 00:38:52,260 --> 00:38:54,840 And we know when this is solvable, 599 00:38:54,840 --> 00:38:58,020 you could do elimination on it, right? 600 00:38:58,020 --> 00:39:00,600 But I'm thinking hey, you do elimination 601 00:39:00,600 --> 00:39:03,299 and you get equation zero equal other non-zeroes. 602 00:39:03,299 --> 00:39:05,590 I'm thinking we really -- elimination is going to fail. 603 00:39:05,590 --> 00:39:05,630 So that's our question. 604 00:39:05,630 --> 00:39:07,046 Elimination will get us down to -- 605 00:39:16,320 --> 00:39:18,700 will tell us if there is a solution or not. 606 00:39:18,700 --> 00:39:22,600 But I'm now thinking not. 607 00:39:22,600 --> 00:39:24,130 So what are we going to do? 608 00:39:24,130 --> 00:39:24,720 OK. 609 00:39:24,720 --> 00:39:25,320 All right. 610 00:39:25,320 --> 00:39:30,810 I want to tell you to jump ahead to the matrix that 611 00:39:30,810 --> 00:39:32,310 will play a key role. 612 00:39:32,310 --> 00:39:35,780 So this is the matrix that you want to understand 613 00:39:35,780 --> 00:39:38,950 for this chapter four. 614 00:39:38,950 --> 00:39:47,740 And it's the matrix A transpose A. 615 00:39:47,740 --> 00:39:53,130 What's -- tell me some things about that matrix. 616 00:39:53,130 --> 00:39:58,640 So A is this m by n matrix, rectangular, but now 617 00:39:58,640 --> 00:40:03,100 I'm saying that the good matrix that shows up in the end 618 00:40:03,100 --> 00:40:05,130 is A transpose A. 619 00:40:05,130 --> 00:40:09,120 So tell me something about that. 620 00:40:09,120 --> 00:40:14,010 Is it -- what's the first thing you know about A transpose A. 621 00:40:14,010 --> 00:40:15,390 It's square. 622 00:40:15,390 --> 00:40:16,780 Right? 623 00:40:16,780 --> 00:40:21,740 Square because this is m by n and this is n by m. 624 00:40:21,740 --> 00:40:24,090 So this is the result is n by n. 625 00:40:24,090 --> 00:40:25,240 Good. 626 00:40:25,240 --> 00:40:26,230 Square. 627 00:40:26,230 --> 00:40:27,990 What else? 628 00:40:27,990 --> 00:40:28,970 It's symmetric. 629 00:40:28,970 --> 00:40:29,535 Good. 630 00:40:29,535 --> 00:40:30,160 It's symmetric. 631 00:40:35,920 --> 00:40:39,650 Because you remember how to do that. 632 00:40:39,650 --> 00:40:44,270 If we transpose that matrix let's transpose it, 633 00:40:44,270 --> 00:40:48,510 A transpose A, if I transpose it, 634 00:40:48,510 --> 00:40:55,230 then that comes first transposed, this comes second, 635 00:40:55,230 --> 00:41:01,990 transposed, and then transposing twice is leaves it -- 636 00:41:01,990 --> 00:41:05,100 brings it back to the same so it's symmetric. 637 00:41:05,100 --> 00:41:06,620 Good. 638 00:41:06,620 --> 00:41:11,340 Now we now know how to ask more about a matrix. 639 00:41:14,050 --> 00:41:20,110 I'm interested in is it invertible? 640 00:41:20,110 --> 00:41:23,810 If not, what's its null space? 641 00:41:23,810 --> 00:41:26,940 So I want to know about -- because you're going to see, 642 00:41:26,940 --> 00:41:32,260 well, let me -- let me even, well I shouldn't do this, 643 00:41:32,260 --> 00:41:33,350 but I will. 644 00:41:33,350 --> 00:41:37,460 Let me tell you what equation to solve 645 00:41:37,460 --> 00:41:42,300 when you can't solve that one. 646 00:41:42,300 --> 00:41:47,450 The good equation comes from multiplying both sides 647 00:41:47,450 --> 00:41:53,150 by A transpose, so the good equation that you get to 648 00:41:53,150 --> 00:41:54,820 is this one. 649 00:41:54,820 --> 00:42:00,790 A transpose Ax equals A transpose b. 650 00:42:04,960 --> 00:42:08,700 That will be the central equation in the chapter. 651 00:42:08,700 --> 00:42:10,950 So I think why not tell it to you. 652 00:42:10,950 --> 00:42:13,290 Why not admit it right away. 653 00:42:13,290 --> 00:42:14,440 OK. 654 00:42:14,440 --> 00:42:15,260 I have to -- 655 00:42:15,260 --> 00:42:17,960 I should really give x. 656 00:42:17,960 --> 00:42:27,750 I want to sort of indicate that this x isn't I mean this x was 657 00:42:27,750 --> 00:42:30,580 the solution to that equation if it existed, 658 00:42:30,580 --> 00:42:33,440 but probably didn't. 659 00:42:33,440 --> 00:42:39,020 Now let me give this a different name, x hat. 660 00:42:39,020 --> 00:42:45,740 Because I'm hoping this one will have a solution. 661 00:42:45,740 --> 00:42:49,680 And I'm saying that it's my best solution. 662 00:42:49,680 --> 00:42:52,400 I'll have to say what does best mean. 663 00:42:52,400 --> 00:42:55,720 But that's going to be my -- my plan. 664 00:42:55,720 --> 00:42:59,650 I'm going to say that the best solution solves this equation. 665 00:42:59,650 --> 00:43:03,770 So you see right away why I'm so interested in this matrix 666 00:43:03,770 --> 00:43:05,620 A transpose A. 667 00:43:05,620 --> 00:43:07,130 And in its invertibility. 668 00:43:07,130 --> 00:43:07,630 OK. 669 00:43:10,210 --> 00:43:11,720 Now, when is it invertible? 670 00:43:14,290 --> 00:43:14,900 OK. 671 00:43:14,900 --> 00:43:22,040 Let me take a case, let me just do an example and then -- 672 00:43:22,040 --> 00:43:25,910 I'll just pick a matrix here. 673 00:43:25,910 --> 00:43:28,510 Just so we see what A transpose A looks like. 674 00:43:28,510 --> 00:43:35,010 So let me take a matrix A one, one, one, one, two, five. 675 00:43:35,010 --> 00:43:37,190 Just to invent a matrix. 676 00:43:37,190 --> 00:43:40,600 So there's a matrix A. 677 00:43:40,600 --> 00:43:48,390 Notice that it has M equal three rows and N equal two columns. 678 00:43:48,390 --> 00:43:50,950 Its rank is -- 679 00:43:50,950 --> 00:43:55,200 the rank of that matrix is two. 680 00:43:55,200 --> 00:43:58,460 Right, yeah, the columns are independent. 681 00:43:58,460 --> 00:44:01,200 Does Ax equal b? 682 00:44:01,200 --> 00:44:09,086 If I look at Ax=b, so x is just x1 x2, and b is b1 b2 b3. 683 00:44:12,700 --> 00:44:15,300 Do I expect to solve Ax=b? 684 00:44:15,300 --> 00:44:17,760 What's -- no way, right? 685 00:44:17,760 --> 00:44:21,440 I mean linear algebra's great, but solving 686 00:44:21,440 --> 00:44:24,040 three equations with only two unknowns usually 687 00:44:24,040 --> 00:44:26,020 we can't do it. 688 00:44:26,020 --> 00:44:30,050 We can only solve it if this vector is b is what? 689 00:44:33,230 --> 00:44:37,780 I can solve that equation if that vector b1 b2 b3 690 00:44:37,780 --> 00:44:41,700 is in the column space. 691 00:44:41,700 --> 00:44:44,650 If it's a combination of those columns then fine. 692 00:44:44,650 --> 00:44:47,030 But usually it won't be. 693 00:44:47,030 --> 00:44:49,580 The combinations just fill up a plane 694 00:44:49,580 --> 00:44:52,780 and most vectors aren't on that plane. 695 00:44:52,780 --> 00:44:56,630 So what I'm saying is that I'm going to work 696 00:44:56,630 --> 00:44:59,990 with the matrix A transpose A. 697 00:44:59,990 --> 00:45:03,600 And I just want to figure out in this example what 698 00:45:03,600 --> 00:45:07,780 A transpose A is. 699 00:45:07,780 --> 00:45:09,390 So it's two by two. 700 00:45:09,390 --> 00:45:13,320 The first entry is a three, the next entry is an eight, 701 00:45:13,320 --> 00:45:15,160 this entry is -- 702 00:45:18,040 --> 00:45:20,910 what's that entry? 703 00:45:20,910 --> 00:45:23,110 Eight, for sure. 704 00:45:23,110 --> 00:45:25,270 We knew it had to be, and this entry 705 00:45:25,270 --> 00:45:32,630 is, what's that now, getting out my trusty calculator, thirty, 706 00:45:32,630 --> 00:45:35,880 is that right? 707 00:45:35,880 --> 00:45:37,580 And is that matrix invertible? 708 00:45:37,580 --> 00:45:38,080 Thirty. 709 00:45:38,080 --> 00:45:40,960 There's an A transpose A. 710 00:45:40,960 --> 00:45:42,620 And it is invertible, right? 711 00:45:42,620 --> 00:45:46,040 Three, eight is not a multiple of eight, thirty -- 712 00:45:46,040 --> 00:45:48,050 and it's invertible. 713 00:45:48,050 --> 00:45:51,420 And that's the normal, that's what I expect. 714 00:45:51,420 --> 00:45:56,730 So this is I want to show. 715 00:45:56,730 --> 00:45:59,740 So here's the final -- here's the key point. 716 00:45:59,740 --> 00:46:04,660 The null space of A transpose A -- 717 00:46:04,660 --> 00:46:06,645 it's not going to be always invertible. 718 00:46:09,650 --> 00:46:11,450 Tell me a matrix -- 719 00:46:11,450 --> 00:46:15,640 I have to say that I can't say A transpose A is always 720 00:46:15,640 --> 00:46:16,770 invertible. 721 00:46:16,770 --> 00:46:19,590 Because that's asking too much. 722 00:46:19,590 --> 00:46:22,860 I mean what could the matrix A be, for example, 723 00:46:22,860 --> 00:46:27,330 so that A transpose A was not invertible? 724 00:46:27,330 --> 00:46:29,110 Well, it even could be the zero matrix. 725 00:46:29,110 --> 00:46:32,250 I mean that's like extreme case. 726 00:46:32,250 --> 00:46:38,960 Suppose I make this rank -- 727 00:46:38,960 --> 00:46:46,090 suppose I change to that A. 728 00:46:46,090 --> 00:46:49,240 Now I figure out A transpose A again and I get -- 729 00:46:49,240 --> 00:46:49,840 what do I get? 730 00:46:53,890 --> 00:46:58,360 I get nine, I get nine of course and here I 731 00:46:58,360 --> 00:47:02,434 get what's that entry? 732 00:47:02,434 --> 00:47:02,975 Twenty-seven. 733 00:47:06,900 --> 00:47:09,100 And is that matrix invertible? 734 00:47:09,100 --> 00:47:09,600 No. 735 00:47:12,110 --> 00:47:13,470 And why do I -- 736 00:47:13,470 --> 00:47:16,590 I knew it wouldn't be invertible anyway. 737 00:47:16,590 --> 00:47:23,190 Because this matrix only has rank one. 738 00:47:23,190 --> 00:47:26,100 And if I have a product of matrices of rank one, 739 00:47:26,100 --> 00:47:30,340 the product is not going to have a rank bigger than one. 740 00:47:30,340 --> 00:47:33,810 So I'm not surprised that the answer only has rank one. 741 00:47:33,810 --> 00:47:36,730 And that's what I -- 742 00:47:36,730 --> 00:47:41,200 always happens, that the rank of A transpose A 743 00:47:41,200 --> 00:47:44,460 comes out to equal the rank of A. 744 00:47:44,460 --> 00:47:49,580 So, yes, so the null space of A transpose A 745 00:47:49,580 --> 00:47:57,580 equals the null space of A, the rank of A transpose A 746 00:47:57,580 --> 00:48:01,840 equals the rank of A. 747 00:48:01,840 --> 00:48:11,460 So let's -- as soon as I can why that's true. 748 00:48:11,460 --> 00:48:18,920 But let's draw from that what the fact that I want. 749 00:48:18,920 --> 00:48:24,810 This tells me that this square symmetric matrix is invertible 750 00:48:24,810 --> 00:48:26,650 if -- 751 00:48:26,650 --> 00:48:30,460 so here's my conclusion. 752 00:48:30,460 --> 00:48:40,400 A transpose A is invertible if exactly when -- 753 00:48:40,400 --> 00:48:46,390 exactly if this null space is only got the zero vector. 754 00:48:46,390 --> 00:48:51,700 Which means the columns of A are independent. 755 00:48:51,700 --> 00:48:54,342 So A transpose A is invertible exactly 756 00:48:54,342 --> 00:48:55,550 if A has independent columns. 757 00:48:55,550 --> 00:49:12,290 That's the fact that I need about A transpose A. 758 00:49:12,290 --> 00:49:17,160 And then you'll see next time how A transpose A 759 00:49:17,160 --> 00:49:18,780 enters everything. 760 00:49:18,780 --> 00:49:22,130 Next lecture is actually a crucial one. 761 00:49:22,130 --> 00:49:26,280 Here I'm preparing for it by getting us 762 00:49:26,280 --> 00:49:29,000 thinking about A transpose A. 763 00:49:29,000 --> 00:49:31,970 And its rank is the same as the rank of A, 764 00:49:31,970 --> 00:49:34,310 and we can decide when it's invertible. 765 00:49:34,310 --> 00:49:34,980 OK. 766 00:49:34,980 --> 00:49:35,938 So I'll see you Friday. 767 00:49:35,938 --> 00:49:37,140 Thanks.