1 00:00:14,160 --> 00:00:14,860 Okay. 2 00:00:14,860 --> 00:00:21,760 This is the lecture on the singular value decomposition. 3 00:00:21,760 --> 00:00:27,030 But everybody calls it the SVD. 4 00:00:27,030 --> 00:00:34,350 So this is the final and best factorization of a matrix. 5 00:00:34,350 --> 00:00:37,270 Let me tell you what's coming. 6 00:00:37,270 --> 00:00:43,330 The factors will be, orthogonal matrix, diagonal matrix, 7 00:00:43,330 --> 00:00:45,610 orthogonal matrix. 8 00:00:45,610 --> 00:00:48,610 So it's things that we've seen before, 9 00:00:48,610 --> 00:00:53,180 these special good matrices, orthogonal diagonal. 10 00:00:53,180 --> 00:00:58,810 The new point is that we need two orthogonal matrices. 11 00:00:58,810 --> 00:01:02,720 A can be any matrix whatsoever. 12 00:01:02,720 --> 00:01:07,140 Any matrix whatsoever has this singular value decomposition, 13 00:01:07,140 --> 00:01:11,870 so a diagonal one in the middle, but I need two different -- 14 00:01:11,870 --> 00:01:17,030 probably different orthogonal matrices to be able to do this. 15 00:01:17,030 --> 00:01:17,560 Okay. 16 00:01:17,560 --> 00:01:22,030 And this factorization has jumped into importance 17 00:01:22,030 --> 00:01:27,240 and is properly, I think, maybe the bringing together 18 00:01:27,240 --> 00:01:29,620 of everything in this course. 19 00:01:29,620 --> 00:01:36,970 One thing we'll bring together is the very good family 20 00:01:36,970 --> 00:01:39,070 of matrices that we just studied, 21 00:01:39,070 --> 00:01:41,120 symmetric, positive, definite. 22 00:01:41,120 --> 00:01:43,730 Do you remember the stories with those guys? 23 00:01:43,730 --> 00:01:49,160 Because they were symmetric, their eigenvectors were 24 00:01:49,160 --> 00:01:53,880 orthogonal, so I could produce an orthogonal matrix -- 25 00:01:53,880 --> 00:01:56,070 this is my usual one. 26 00:01:56,070 --> 00:01:59,630 My usual one is the eigenvectors and eigenvalues In 27 00:01:59,630 --> 00:02:05,540 the symmetric case, the eigenvectors are orthogonal, 28 00:02:05,540 --> 00:02:09,430 so I've got the good -- my ordinary s has become 29 00:02:09,430 --> 00:02:12,180 an especially good Q. 30 00:02:12,180 --> 00:02:15,610 And positive definite, my ordinary lambda 31 00:02:15,610 --> 00:02:18,260 has become a positive lambda. 32 00:02:18,260 --> 00:02:24,420 So that's the singular value decomposition in case 33 00:02:24,420 --> 00:02:28,240 our matrix is symmetric positive definite -- 34 00:02:28,240 --> 00:02:30,650 in that case, I don't need two -- 35 00:02:30,650 --> 00:02:35,795 U and a V -- one orthogonal matrix will do for both sides. 36 00:02:38,970 --> 00:02:41,260 So this would be no good in general, 37 00:02:41,260 --> 00:02:45,190 because usually the eigenvector matrix isn't orthogonal. 38 00:02:45,190 --> 00:02:49,900 So that's not what I'm after. 39 00:02:49,900 --> 00:02:56,790 I'm looking for orthogonal times diagonal times orthogonal. 40 00:02:56,790 --> 00:03:00,250 And let me show you what that means and where it comes from. 41 00:03:00,250 --> 00:03:01,880 Okay. 42 00:03:01,880 --> 00:03:02,680 What does it mean? 43 00:03:05,440 --> 00:03:10,310 You remember the picture of any linear transformation. 44 00:03:10,310 --> 00:03:16,670 This was, like, the most important figure in 45 00:03:16,670 --> 00:03:19,810 And what I looking for now? 46 00:03:19,810 --> 00:03:26,020 A typical vector in the row space -- 47 00:03:26,020 --> 00:03:28,940 typical vector, let me call it v1, 48 00:03:28,940 --> 00:03:35,470 gets taken over to some vector in the column space, say u1. 49 00:03:35,470 --> 00:03:37,850 So u1 is Av1. 50 00:03:41,350 --> 00:03:43,320 Okay. 51 00:03:43,320 --> 00:03:48,800 Now, another vector gets taken over here somewhere. 52 00:03:48,800 --> 00:03:50,290 What I looking for? 53 00:03:50,290 --> 00:03:54,590 In this SVD, this singular value decomposition, 54 00:03:54,590 --> 00:03:59,950 what I'm looking for is an orthogonal basis here 55 00:03:59,950 --> 00:04:06,130 that gets knocked over into an orthogonal basis over there. 56 00:04:06,130 --> 00:04:11,810 See that's pretty special, to have an orthogonal basis 57 00:04:11,810 --> 00:04:18,180 in the row space that goes over into an orthogonal basis -- 58 00:04:18,180 --> 00:04:21,490 so this is like a right angle and this is a right angle -- 59 00:04:21,490 --> 00:04:25,700 into an orthogonal basis in the column space. 60 00:04:25,700 --> 00:04:30,940 So that's our goal, is to find -- 61 00:04:30,940 --> 00:04:34,230 do you see how things are coming together? 62 00:04:34,230 --> 00:04:37,590 First of all, can I find an orthogonal basis 63 00:04:37,590 --> 00:04:39,960 for this row space? 64 00:04:39,960 --> 00:04:41,100 Of course. 65 00:04:41,100 --> 00:04:44,210 No big deal to find an orthogonal basis. 66 00:04:44,210 --> 00:04:46,870 Graham Schmidt tells me how to do it. 67 00:04:46,870 --> 00:04:50,710 Start with any old basis and grind through Graham Schmidt, 68 00:04:50,710 --> 00:04:54,100 out comes an orthogonal basis. 69 00:04:54,100 --> 00:04:58,280 But then, if I just take any old orthogonal basis, then 70 00:04:58,280 --> 00:05:00,990 when I multiply by A, there's no reason 71 00:05:00,990 --> 00:05:04,770 why it should be orthogonal over here. 72 00:05:04,770 --> 00:05:06,800 So I'm looking for this special set 73 00:05:06,800 --> 00:05:13,850 up where A takes these basis vectors into orthogonal vectors 74 00:05:13,850 --> 00:05:14,780 over there. 75 00:05:14,780 --> 00:05:18,570 Now, you might have noticed that the null space 76 00:05:18,570 --> 00:05:19,400 I didn't include. 77 00:05:19,400 --> 00:05:22,400 Why don't I stick that in? 78 00:05:22,400 --> 00:05:25,770 You remember our usual figure had a little null space 79 00:05:25,770 --> 00:05:28,730 and a little null space. 80 00:05:28,730 --> 00:05:31,650 And those are no problems. 81 00:05:31,650 --> 00:05:34,000 Those null spaces are going to show up 82 00:05:34,000 --> 00:05:37,510 as zeroes on the diagonal of sigma, 83 00:05:37,510 --> 00:05:43,730 so that doesn't present any difficulty. 84 00:05:43,730 --> 00:05:46,250 Our difficulty is to find these. 85 00:05:46,250 --> 00:05:48,220 So do you see what this will mean? 86 00:05:48,220 --> 00:05:57,950 This will mean that A times these v-s, v1, v2, up to -- 87 00:05:57,950 --> 00:06:01,460 what's the dimension of this row space? 88 00:06:01,460 --> 00:06:03,180 Vr. 89 00:06:03,180 --> 00:06:07,390 Sorry, make that V a little smaller -- up to vr. 90 00:06:10,650 --> 00:06:12,990 So that's -- 91 00:06:12,990 --> 00:06:16,680 Av1 is going to be the first column, 92 00:06:16,680 --> 00:06:20,250 so here's what I'm achieving. 93 00:06:20,250 --> 00:06:24,390 Oh, I'm not only going to make these orthogonal, 94 00:06:24,390 --> 00:06:27,030 but why not make them orthonormal? 95 00:06:27,030 --> 00:06:28,880 Make them unit vectors. 96 00:06:28,880 --> 00:06:34,380 So maybe the unit vector is here, is the u1, 97 00:06:34,380 --> 00:06:38,510 and this might be a multiple of it. 98 00:06:38,510 --> 00:06:45,560 So really, what's happening is Av1 is some multiple of u1, 99 00:06:45,560 --> 00:06:46,660 right? 100 00:06:46,660 --> 00:06:50,040 These guys will be unit vectors and they'll 101 00:06:50,040 --> 00:06:52,940 go over into multiples of unit vectors 102 00:06:52,940 --> 00:06:56,820 and the multiple I'm not going to call lambda anymore. 103 00:06:56,820 --> 00:06:58,090 I'm calling it sigma. 104 00:06:58,090 --> 00:07:01,480 So that's the number -- the stretching number. 105 00:07:01,480 --> 00:07:06,000 And similarly, Av2 is sigma two u2. 106 00:07:06,000 --> 00:07:09,000 This is my goal. 107 00:07:09,000 --> 00:07:13,730 And now I want to express that goal in matrix language. 108 00:07:13,730 --> 00:07:15,340 That's the usual step. 109 00:07:15,340 --> 00:07:18,400 Think of what you want and then express it 110 00:07:18,400 --> 00:07:20,420 as a matrix multiplication. 111 00:07:20,420 --> 00:07:25,800 So Av1 is sigma one u1 -- 112 00:07:25,800 --> 00:07:27,960 actually, here we go. 113 00:07:27,960 --> 00:07:30,710 Let me pull out these -- 114 00:07:30,710 --> 00:07:36,765 u1, u2 to ur and then a matrix with the sigmas. 115 00:07:40,460 --> 00:07:45,970 Everything now is going to be in that little part 116 00:07:45,970 --> 00:07:46,720 of the blackboard. 117 00:07:46,720 --> 00:07:52,880 Do you see that this equation says what I'm 118 00:07:52,880 --> 00:07:55,740 trying to do with my figure. 119 00:07:55,740 --> 00:08:00,770 A times the first basis vector should be sigma one times 120 00:08:00,770 --> 00:08:04,580 the other basis -- the other first basis vector. 121 00:08:04,580 --> 00:08:08,150 These are the basis vectors in the row space, 122 00:08:08,150 --> 00:08:11,470 these are the basis vectors in the column space 123 00:08:11,470 --> 00:08:14,680 and these are the multiplying factors. 124 00:08:14,680 --> 00:08:23,660 So Av2 is sigma two times u2, Avr is sigma r times ur. 125 00:08:23,660 --> 00:08:27,170 And then we've got a whole lot of zeroes and maybe some zeroes 126 00:08:27,170 --> 00:08:31,480 at the end, but that's the heart of it. 127 00:08:31,480 --> 00:08:34,064 And now if I express that in -- 128 00:08:40,000 --> 00:08:43,350 as matrices, because you knew that was coming -- 129 00:08:43,350 --> 00:08:45,140 that's what I have. 130 00:08:45,140 --> 00:08:51,520 So, this is my goal, to find an orthogonal basis 131 00:08:51,520 --> 00:08:55,500 in the orthonormal, even -- basis in the row space 132 00:08:55,500 --> 00:09:00,420 and an orthonormal basis in the column space so that I've sort 133 00:09:00,420 --> 00:09:02,590 of diagonalized the matrix. 134 00:09:02,590 --> 00:09:05,530 The matrix A is, like, getting converted 135 00:09:05,530 --> 00:09:08,660 to this diagonal matrix sigma. 136 00:09:08,660 --> 00:09:13,140 And you notice that usually I have to allow myself 137 00:09:13,140 --> 00:09:15,980 two different bases. 138 00:09:15,980 --> 00:09:19,230 My little comment about symmetric positive definite 139 00:09:19,230 --> 00:09:24,860 was the one case where it's A Q equal Q sigma, 140 00:09:24,860 --> 00:09:27,430 where V and U are the same Q. 141 00:09:27,430 --> 00:09:32,310 But mostly, you know, I'm going to take a matrix like -- oh, 142 00:09:32,310 --> 00:09:38,240 let me take a matrix like four four minus three three. 143 00:09:38,240 --> 00:09:39,540 Okay. 144 00:09:39,540 --> 00:09:42,420 There's a two by two matrix. 145 00:09:42,420 --> 00:09:48,430 It's invertible, so it has rank two. 146 00:09:48,430 --> 00:09:51,210 So I'm going to look for two vectors, 147 00:09:51,210 --> 00:09:55,230 v1 and v2 in the row space, and U -- 148 00:09:55,230 --> 00:10:00,770 so I'm going to look for v1, v2 in the row space, 149 00:10:00,770 --> 00:10:05,350 which of course is R^2. 150 00:10:05,350 --> 00:10:10,280 And I'm going to look for u1, u2 in the column space, 151 00:10:10,280 --> 00:10:15,620 which of course is also R^2, and I'm going to look for numbers 152 00:10:15,620 --> 00:10:21,160 sigma one and sigma two so that it all comes out right. 153 00:10:21,160 --> 00:10:26,060 So these guys are orthonormal, these guys are orthonormal 154 00:10:26,060 --> 00:10:28,820 and these are the scaling factors. 155 00:10:28,820 --> 00:10:33,510 So I'll do that example as soon as I get the matrix 156 00:10:33,510 --> 00:10:35,030 picture straight. 157 00:10:35,030 --> 00:10:36,280 Okay. 158 00:10:36,280 --> 00:10:39,380 Do you see that this expresses what I want? 159 00:10:39,380 --> 00:10:45,360 Can I just say two words about null spaces? 160 00:10:45,360 --> 00:10:52,010 If there's some null space, then we 161 00:10:52,010 --> 00:10:56,080 want to stick in a basis for those, for that. 162 00:10:56,080 --> 00:11:02,700 So here comes a basis for the null space, v(r+1) down to vm. 163 00:11:02,700 --> 00:11:06,590 So if we only had an r dimensional row space 164 00:11:06,590 --> 00:11:11,160 and the other n-r dimensions were in the null space -- okay, 165 00:11:11,160 --> 00:11:14,360 we'll take an orthogonal -- orthonormal basis there. 166 00:11:14,360 --> 00:11:15,430 No problem. 167 00:11:15,430 --> 00:11:18,500 And then we'll just get zeroes. 168 00:11:18,500 --> 00:11:21,700 So, actually, w- those zeroes will come out 169 00:11:21,700 --> 00:11:25,040 on the diagonal matrix. 170 00:11:25,040 --> 00:11:33,250 So I'll complete that to an orthonormal basis for the whole 171 00:11:33,250 --> 00:11:35,420 space, R^m. 172 00:11:35,420 --> 00:11:38,080 I complete this to an orthonormal basis for the whole 173 00:11:38,080 --> 00:11:42,420 space R^n and I complete that with zeroes. 174 00:11:42,420 --> 00:11:46,870 Null spaces are no problem here. 175 00:11:46,870 --> 00:11:52,570 So really the true problem is in a matrix like that, 176 00:11:52,570 --> 00:11:56,170 which isn't symmetric, so I can't use its eigenvectors, 177 00:11:56,170 --> 00:11:58,110 they're not orthogonal -- 178 00:11:58,110 --> 00:12:02,740 but somehow I have to get these orthogonal -- in fact, 179 00:12:02,740 --> 00:12:09,470 orthonormal guys that make it work. 180 00:12:09,470 --> 00:12:13,990 I have to find these orthonormal guys, these orthonormal guys 181 00:12:13,990 --> 00:12:22,960 and I want Av1 to be sigma one u1 and Av2 to be sigma two u2. 182 00:12:22,960 --> 00:12:23,460 Okay. 183 00:12:26,790 --> 00:12:28,260 That's my goal. 184 00:12:28,260 --> 00:12:34,310 Here's the matrices that are going to get me there. 185 00:12:34,310 --> 00:12:36,500 Now these are orthogonal matrices. 186 00:12:36,500 --> 00:12:41,610 I can put that -- if I multiply on both sides by V inverse, 187 00:12:41,610 --> 00:12:47,740 I have A equals U sigma V inverse, 188 00:12:47,740 --> 00:12:53,230 and of course you know the other way I can write V inverse. 189 00:12:53,230 --> 00:12:57,200 This is one of those square orthogonal matrices, 190 00:12:57,200 --> 00:13:02,731 so it's the same as U sigma V transpose. 191 00:13:02,731 --> 00:13:03,230 Okay. 192 00:13:06,020 --> 00:13:08,900 Here's my problem. 193 00:13:08,900 --> 00:13:14,580 I've got two orthogonal matrices here. 194 00:13:14,580 --> 00:13:17,330 And I don't want to find them both at once. 195 00:13:17,330 --> 00:13:21,120 So I want to cook up some expression that 196 00:13:21,120 --> 00:13:26,630 will make the Us disappear. 197 00:13:26,630 --> 00:13:29,010 I would like to make the Us disappear and leave me 198 00:13:29,010 --> 00:13:31,070 only with the Vs. 199 00:13:31,070 --> 00:13:34,200 And here's how to do it. 200 00:13:34,200 --> 00:13:37,520 It's the same combination that keeps showing up 201 00:13:37,520 --> 00:13:41,040 whenever we have a general rectangular matrix, 202 00:13:41,040 --> 00:13:46,640 then it's A transpose A, that's the great matrix. 203 00:13:46,640 --> 00:13:48,760 That's the great matrix. 204 00:13:48,760 --> 00:13:50,530 That's the matrix that's symmetric, 205 00:13:50,530 --> 00:13:53,910 and in fact positive definite or at least 206 00:13:53,910 --> 00:13:55,240 positive semi-definite. 207 00:13:55,240 --> 00:13:58,300 This is the matrix with nice properties, so let's see what 208 00:13:58,300 --> 00:13:59,230 will it be? 209 00:13:59,230 --> 00:14:03,100 So if I took the transpose, then, I would have -- 210 00:14:03,100 --> 00:14:06,140 A transpose A will be what? 211 00:14:06,140 --> 00:14:06,920 What do I have? 212 00:14:06,920 --> 00:14:12,580 If I transpose that I have V sigma transpose U transpose, 213 00:14:12,580 --> 00:14:14,900 that's the A transpose. 214 00:14:14,900 --> 00:14:18,350 Now the A -- 215 00:14:18,350 --> 00:14:19,290 and what have I got? 216 00:14:21,930 --> 00:14:25,960 Looks like worse, because it's got six things now together, 217 00:14:25,960 --> 00:14:30,660 but it's going to collapse into something good. 218 00:14:30,660 --> 00:14:34,300 What does U transpose U collapse into? 219 00:14:34,300 --> 00:14:35,770 I, the identity. 220 00:14:35,770 --> 00:14:37,330 So that's the key point. 221 00:14:37,330 --> 00:14:42,050 This is the identity and we don't have U anymore. 222 00:14:42,050 --> 00:14:44,380 And sigma transpose times sigma, those 223 00:14:44,380 --> 00:14:48,300 are diagonal matrixes, so their product is just 224 00:14:48,300 --> 00:14:50,590 going to have sigma squareds on the diagonal. 225 00:14:50,590 --> 00:14:52,650 So do you see what we've got here? 226 00:14:52,650 --> 00:14:57,440 This is V times this easy matrix sigma 227 00:14:57,440 --> 00:15:03,080 one squared sigma two squared times V transpose. 228 00:15:03,080 --> 00:15:05,840 This is the A transpose A. 229 00:15:05,840 --> 00:15:07,590 This is -- let me copy down -- 230 00:15:07,590 --> 00:15:09,185 A transpose A is that. 231 00:15:12,760 --> 00:15:14,170 Us are out of the picture, now. 232 00:15:14,170 --> 00:15:17,900 I'm only having to choose the Vs, and what are these Vs? 233 00:15:17,900 --> 00:15:19,370 And what are these sigmas? 234 00:15:19,370 --> 00:15:26,750 Do you know what the Vs are? 235 00:15:26,750 --> 00:15:28,650 They're the eigenvectors that -- see, 236 00:15:28,650 --> 00:15:34,120 this is a perfect eigenvector, eigenvalue, 237 00:15:34,120 --> 00:15:41,920 Q lambda Q transpose for the matrix A transpose A. 238 00:15:41,920 --> 00:15:45,020 A itself is nothing special. 239 00:15:45,020 --> 00:15:48,620 But A transpose A will be special. 240 00:15:48,620 --> 00:15:50,700 It'll be symmetric positive definite, 241 00:15:50,700 --> 00:15:55,340 so this will be its eigenvectors and this'll be its eigenvalues. 242 00:15:55,340 --> 00:15:59,100 And the eigenvalues'll be positive because this thing's 243 00:15:59,100 --> 00:16:02,400 positive definite. 244 00:16:02,400 --> 00:16:04,650 So this is my method. 245 00:16:04,650 --> 00:16:07,310 This tells me what the Vs are. 246 00:16:07,310 --> 00:16:09,210 And how I going to find the Us? 247 00:16:11,730 --> 00:16:18,570 Well, one way would be to look at A A transpose. 248 00:16:18,570 --> 00:16:21,650 Multiply A by A transpose in the opposite order. 249 00:16:21,650 --> 00:16:24,340 That will stick the Vs in the middle, 250 00:16:24,340 --> 00:16:27,130 knock them out, and leave me with the Us. 251 00:16:27,130 --> 00:16:29,620 So here's the overall picture, then. 252 00:16:29,620 --> 00:16:36,270 The Vs are the eigenvectors of A transpose A. 253 00:16:36,270 --> 00:16:38,180 The Us are the eigenvectors of A A 254 00:16:38,180 --> 00:16:39,860 transpose, which are different. 255 00:16:39,860 --> 00:16:43,940 And the sigmas are the square roots 256 00:16:43,940 --> 00:16:48,730 of these and the positive square roots, 257 00:16:48,730 --> 00:16:50,580 so we have positive sigmas. 258 00:16:50,580 --> 00:16:52,780 Let me do it for that example. 259 00:16:52,780 --> 00:16:56,570 This is really what you should know 260 00:16:56,570 --> 00:17:01,460 and be able to do for the SVD. 261 00:17:01,460 --> 00:17:02,520 Okay. 262 00:17:02,520 --> 00:17:03,740 Let me take that matrix. 263 00:17:03,740 --> 00:17:06,550 So what's my first step? 264 00:17:06,550 --> 00:17:12,191 Compute A transpose A, because I want its eigenvectors. 265 00:17:12,191 --> 00:17:12,690 Okay. 266 00:17:12,690 --> 00:17:16,690 So I have to compute A transpose A. 267 00:17:16,690 --> 00:17:22,240 So A transpose is four four minus three three, 268 00:17:22,240 --> 00:17:26,970 and A is four four minus three three, 269 00:17:26,970 --> 00:17:30,810 and I do that multiplication and I get sixteen -- 270 00:17:30,810 --> 00:17:32,930 I get twenty five -- 271 00:17:32,930 --> 00:17:36,300 I get sixteen minus nine -- 272 00:17:36,300 --> 00:17:37,870 is that seven? 273 00:17:37,870 --> 00:17:40,260 And it better come out symmetric. 274 00:17:40,260 --> 00:17:43,190 And -- oh, okay, and then it comes out 25. 275 00:17:43,190 --> 00:17:43,690 Okay. 276 00:17:47,360 --> 00:17:52,030 So, I want its eigenvectors and its eigenvalues. 277 00:17:52,030 --> 00:17:55,680 Its eigenvectors will be the Vs, its eigenvalues 278 00:17:55,680 --> 00:17:58,990 will be the squares of the sigmas. 279 00:17:58,990 --> 00:17:59,570 Okay. 280 00:17:59,570 --> 00:18:04,900 What are the eigenvalues and eigenvectors of this guy? 281 00:18:04,900 --> 00:18:10,690 Have you seen that two by two example enough to recognize 282 00:18:10,690 --> 00:18:16,970 that the eigenvectors are -- that one one is an eigenvector? 283 00:18:16,970 --> 00:18:19,210 So this here is A transpose A. 284 00:18:19,210 --> 00:18:22,300 I'm looking for its eigenvectors. 285 00:18:22,300 --> 00:18:27,620 So its eigenvectors, I think, are one one and one minus one, 286 00:18:27,620 --> 00:18:29,800 because if I multiply that matrix 287 00:18:29,800 --> 00:18:32,390 by one one, what do I get? 288 00:18:32,390 --> 00:18:37,720 If I multiply that matrix by one one, I get 32 32, 289 00:18:37,720 --> 00:18:41,530 which is 32 of one one. 290 00:18:41,530 --> 00:18:45,240 So there's the first eigenvector, 291 00:18:45,240 --> 00:18:49,360 and there's the eigenvalue for A transpose A. 292 00:18:49,360 --> 00:18:59,120 So I'm going to take its square root for sigma. 293 00:18:59,120 --> 00:18:59,620 Okay. 294 00:18:59,620 --> 00:19:01,310 What's the eigenvector that goes -- 295 00:19:01,310 --> 00:19:03,410 eigenvalue that goes with this one? 296 00:19:03,410 --> 00:19:05,730 If I do that multiplication, what do I get? 297 00:19:05,730 --> 00:19:12,420 I get some multiple of one minus one, and what is that multiple? 298 00:19:12,420 --> 00:19:13,250 Looks like 18. 299 00:19:16,470 --> 00:19:17,340 Okay. 300 00:19:17,340 --> 00:19:20,880 So those are the two eigenvectors, but -- oh, 301 00:19:20,880 --> 00:19:24,300 just a moment, I didn't normalize them. 302 00:19:24,300 --> 00:19:27,310 To make everything absolutely right, 303 00:19:27,310 --> 00:19:30,180 I ought to normalize these eigenvectors, 304 00:19:30,180 --> 00:19:33,370 divide by their length, square root of two. 305 00:19:33,370 --> 00:19:42,890 So all these guys should be true unit vectors and, of course, 306 00:19:42,890 --> 00:19:46,900 that normalization didn't change the 32 and the 18. 307 00:19:46,900 --> 00:19:48,500 Okay. 308 00:19:48,500 --> 00:19:51,980 So I'm happy with the Vs. 309 00:19:51,980 --> 00:19:53,380 Here are the Vs. 310 00:19:53,380 --> 00:19:57,490 So now let me put together the pieces here. 311 00:19:57,490 --> 00:19:59,340 Here's my A. 312 00:19:59,340 --> 00:20:01,060 Here's my A. 313 00:20:01,060 --> 00:20:03,550 Let me write down A again. 314 00:20:07,860 --> 00:20:16,150 If life is right, we should get U, which I don't yet know -- 315 00:20:16,150 --> 00:20:19,870 U I don't yet know, sigma I do now know. 316 00:20:19,870 --> 00:20:21,040 What's sigma? 317 00:20:21,040 --> 00:20:24,080 So I'm looking for a U sigma V transpose. 318 00:20:24,080 --> 00:20:28,115 U, the diagonal guy and V transpose. 319 00:20:31,500 --> 00:20:32,030 Okay. 320 00:20:32,030 --> 00:20:33,740 Let's just see that come out right. 321 00:20:33,740 --> 00:20:36,780 So what are the sigmas? 322 00:20:36,780 --> 00:20:39,130 They're the square roots of these things. 323 00:20:39,130 --> 00:20:43,820 So square root of 32 and square root of 18. 324 00:20:47,900 --> 00:20:49,120 Zero zero. 325 00:20:49,120 --> 00:20:50,210 Okay. 326 00:20:50,210 --> 00:20:52,040 What are the Vs? 327 00:20:52,040 --> 00:20:53,810 They're these two. 328 00:20:53,810 --> 00:20:56,900 And I have to transpose -- 329 00:20:56,900 --> 00:20:59,450 maybe that just leaves me with ones -- 330 00:20:59,450 --> 00:21:03,790 with one over square root of two in that row and the other one 331 00:21:03,790 --> 00:21:06,870 is one over square root of two minus one 332 00:21:06,870 --> 00:21:08,010 over square root of two. 333 00:21:11,590 --> 00:21:15,030 Now finally, I've got to know the Us. 334 00:21:15,030 --> 00:21:18,570 Well, actually, one way to do -- since I now know all the other 335 00:21:18,570 --> 00:21:21,100 pieces, I could put those together and figure out what 336 00:21:21,100 --> 00:21:22,110 the Us are. 337 00:21:22,110 --> 00:21:25,780 But let me do it the A A transpose way. 338 00:21:25,780 --> 00:21:26,280 Okay. 339 00:21:26,280 --> 00:21:27,380 Find the Us now. 340 00:21:31,390 --> 00:21:33,070 u1 and u2. 341 00:21:33,070 --> 00:21:34,650 And what are they? 342 00:21:37,220 --> 00:21:41,240 I look at A A transpose -- 343 00:21:41,240 --> 00:21:47,230 so A is supposed to be U sigma V transpose, and then 344 00:21:47,230 --> 00:21:52,652 when I transpose that I get V sigma transpose U transpose. 345 00:21:57,210 --> 00:21:59,160 So I'm just doing it in the opposite order, 346 00:21:59,160 --> 00:22:03,470 A times A transpose, and what's the good part here? 347 00:22:03,470 --> 00:22:10,480 That in the middle, V transpose V is going to be the identity. 348 00:22:10,480 --> 00:22:14,990 So this is just U sigma sigma transpose, 349 00:22:14,990 --> 00:22:22,340 that's some diagonal matrix with sigma squareds and U transpose. 350 00:22:22,340 --> 00:22:24,480 So what I seeing here? 351 00:22:24,480 --> 00:22:29,280 I'm seeing here, again, a symmetric positive definite 352 00:22:29,280 --> 00:22:32,000 or at least semi-definite matrix and I'm 353 00:22:32,000 --> 00:22:36,600 seeing its eigenvectors and its eigenvalues. 354 00:22:36,600 --> 00:22:41,590 So if I compute A A transpose, its eigenvectors 355 00:22:41,590 --> 00:22:44,060 will be the things that go into U. 356 00:22:44,060 --> 00:22:47,470 Okay, so I need to compute A A transpose. 357 00:22:47,470 --> 00:22:50,970 I guess I'm going to have to go -- 358 00:22:50,970 --> 00:22:53,450 can I move that up just a little? 359 00:22:53,450 --> 00:22:56,580 Maybe a little more and do A A transpose. 360 00:22:59,880 --> 00:23:01,820 So what's A? 361 00:23:01,820 --> 00:23:05,930 Four four minus three and three. 362 00:23:05,930 --> 00:23:07,550 And what's A transpose? 363 00:23:07,550 --> 00:23:10,310 Four four minus three and three. 364 00:23:10,310 --> 00:23:15,530 And when I do that multiplication, what do I get? 365 00:23:15,530 --> 00:23:18,750 Sixteen and sixteen, thirty two. 366 00:23:18,750 --> 00:23:21,630 Uh, that one comes out zero. 367 00:23:21,630 --> 00:23:26,940 Oh, so this is a lucky case and that one comes out 18. 368 00:23:26,940 --> 00:23:31,560 So this is an accident that A A transpose 369 00:23:31,560 --> 00:23:38,030 happens to come out diagonal, so we know easily its eigenvectors 370 00:23:38,030 --> 00:23:38,990 and eigenvalues. 371 00:23:38,990 --> 00:23:43,130 So its eigenvectors -- what's the first eigenvector for this 372 00:23:43,130 --> 00:23:45,150 A A transpose matrix? 373 00:23:45,150 --> 00:23:49,940 It's just one zero, and when I do that multiplication, 374 00:23:49,940 --> 00:23:54,020 I get 32 times one zero. 375 00:23:54,020 --> 00:23:57,380 And the other eigenvector is just zero one 376 00:23:57,380 --> 00:24:00,350 and when I multiply by that I get 18. 377 00:24:00,350 --> 00:24:04,720 So this is A A transpose. 378 00:24:04,720 --> 00:24:08,910 Multiplying that gives me the 32 A A transpose. 379 00:24:08,910 --> 00:24:14,860 Multiplying this guy gives me First of all, 380 00:24:14,860 --> 00:24:18,590 I got 32 and 18 again. 381 00:24:18,590 --> 00:24:19,740 Am I surprised? 382 00:24:19,740 --> 00:24:24,420 You know, it's clearly not an accident. 383 00:24:24,420 --> 00:24:29,190 The eigenvalues of A A transpose were exactly the same 384 00:24:29,190 --> 00:24:37,820 as the eigenvalues of -- this one was A transpose A. 385 00:24:37,820 --> 00:24:40,140 That's no surprise at all. 386 00:24:40,140 --> 00:24:47,140 The eigenvalues of A B are the same as the eigenvalues of B A. 387 00:24:47,140 --> 00:24:50,530 That's a very nice fact, that eigenvalues 388 00:24:50,530 --> 00:24:55,310 stay the same if I switch the order of multiplication. 389 00:24:55,310 --> 00:25:01,650 So no surprise to see 32 and What I learned -- 390 00:25:01,650 --> 00:25:05,550 first the check that things were numerically correct, 391 00:25:05,550 --> 00:25:07,950 but now I've learned these eigenvectors, 392 00:25:07,950 --> 00:25:11,980 and actually they're about as nice as can be. 393 00:25:11,980 --> 00:25:16,045 They're the best orthogonal matrix, just the identity. 394 00:25:18,511 --> 00:25:19,010 Okay. 395 00:25:21,940 --> 00:25:26,400 So my claim is that it ought to all fit together, 396 00:25:26,400 --> 00:25:31,560 that these numbers should come out right. 397 00:25:31,560 --> 00:25:33,440 The numbers should come out right 398 00:25:33,440 --> 00:25:41,070 because the matrix multiplications use 399 00:25:41,070 --> 00:25:42,450 the properties that we want. 400 00:25:42,450 --> 00:25:42,970 Okay. 401 00:25:42,970 --> 00:25:44,370 Shall we just check that? 402 00:25:44,370 --> 00:25:47,020 Here's the identity, so not doing anything -- 403 00:25:47,020 --> 00:25:50,530 square root of 32 is multiplying that row, 404 00:25:50,530 --> 00:25:53,670 so that square root of 32 divided by square root of two 405 00:25:53,670 --> 00:25:58,150 means square root of 16, four, correct? 406 00:25:58,150 --> 00:26:01,680 And square root of 18 is divided by square root of two, 407 00:26:01,680 --> 00:26:07,570 so that leaves me square root of 9, which is three, but -- 408 00:26:07,570 --> 00:26:11,100 well, Professor Strang, you see the problem? 409 00:26:11,100 --> 00:26:12,740 Why is that -- 410 00:26:12,740 --> 00:26:13,240 okay. 411 00:26:13,240 --> 00:26:16,790 Why I getting minus three three here 412 00:26:16,790 --> 00:26:19,965 and here I'm getting three minus three? 413 00:26:24,640 --> 00:26:26,240 Phooey. 414 00:26:26,240 --> 00:26:27,105 I don't know why. 415 00:26:30,980 --> 00:26:34,650 It shouldn't have happened, but it did. 416 00:26:34,650 --> 00:26:38,402 Now, okay, you could say, well, just -- 417 00:26:41,300 --> 00:26:43,560 the eigenvector there could have -- 418 00:26:43,560 --> 00:26:47,410 I could have had the minus sign here for that eigenvector, 419 00:26:47,410 --> 00:26:49,710 but I'm not happy about that. 420 00:26:49,710 --> 00:26:50,210 Hmm. 421 00:26:50,210 --> 00:26:50,710 Okay. 422 00:26:55,740 --> 00:26:58,480 So I realize there's a little catch here somewhere 423 00:26:58,480 --> 00:27:02,630 and I may not see it until Wednesday. 424 00:27:02,630 --> 00:27:04,830 Which then gives you a very important reason 425 00:27:04,830 --> 00:27:09,930 to come back on Wednesday, to catch that sine difference. 426 00:27:09,930 --> 00:27:14,310 So what did I do illegally? 427 00:27:14,310 --> 00:27:22,550 I think I put the eigenvectors in that matrix V transpose -- 428 00:27:22,550 --> 00:27:24,160 okay, I'm going to have to think. 429 00:27:24,160 --> 00:27:29,460 Why did that come out with with the opposite sines? 430 00:27:29,460 --> 00:27:30,510 So you see -- 431 00:27:30,510 --> 00:27:35,000 I mean, if I had a minus there, I would be all right, 432 00:27:35,000 --> 00:27:36,490 but I don't want that. 433 00:27:36,490 --> 00:27:45,331 I want positive entries down the diagonal of sigma squared. 434 00:27:45,331 --> 00:27:45,830 Okay. 435 00:27:45,830 --> 00:27:51,590 It'll come to me, but, I'm going to leave this example 436 00:27:51,590 --> 00:27:57,600 to finish. 437 00:27:57,600 --> 00:27:58,920 Okay. 438 00:27:58,920 --> 00:28:02,560 And the beauty of, these sliding boards 439 00:28:02,560 --> 00:28:05,780 is I can make that go away. 440 00:28:05,780 --> 00:28:10,910 Can I,-- let me not do it, though, yet. 441 00:28:10,910 --> 00:28:15,090 Let me take a second example. 442 00:28:15,090 --> 00:28:18,720 Let me take a second example where the matrix is singular. 443 00:28:21,940 --> 00:28:24,090 So rank one. 444 00:28:24,090 --> 00:28:32,390 Okay, so let me take as an example two, 445 00:28:32,390 --> 00:28:38,770 where my matrix A is going to be rectangular again -- 446 00:28:38,770 --> 00:28:43,220 let me just make it four three eight six. 447 00:28:47,590 --> 00:28:48,090 Okay. 448 00:28:48,090 --> 00:28:50,830 That's a rank one matrix. 449 00:28:50,830 --> 00:28:57,430 So that has a null space and only a one dimensional row 450 00:28:57,430 --> 00:28:59,370 space and column space. 451 00:28:59,370 --> 00:29:05,510 So actually, my picture becomes easy for this matrix, 452 00:29:05,510 --> 00:29:09,510 because what's my row space for this one? 453 00:29:09,510 --> 00:29:12,450 So this is two by two. 454 00:29:12,450 --> 00:29:17,000 So my pictures are both two dimensional. 455 00:29:17,000 --> 00:29:22,390 My row space is all multiples of that vector four three. 456 00:29:22,390 --> 00:29:24,760 So the whole -- the row space is just a line, right? 457 00:29:27,770 --> 00:29:29,380 That's the row space. 458 00:29:29,380 --> 00:29:33,050 And the null space, of course, is the perpendicular line. 459 00:29:33,050 --> 00:29:46,480 So the row space for this matrix is multiples of four three. 460 00:29:46,480 --> 00:29:47,680 Typical row. 461 00:29:47,680 --> 00:29:48,520 Okay. 462 00:29:48,520 --> 00:29:50,060 What's the column space? 463 00:29:50,060 --> 00:29:55,980 The columns are all multiples of four eight, three six, one two. 464 00:29:55,980 --> 00:30:00,315 The column space, then, goes in, like, this direction. 465 00:30:03,040 --> 00:30:07,490 So the column space -- 466 00:30:07,490 --> 00:30:09,650 when I look at those columns, the column space -- 467 00:30:09,650 --> 00:30:12,660 so it's only one dimensional, because the rank is one. 468 00:30:12,660 --> 00:30:21,480 It's multiples of four eight. 469 00:30:21,480 --> 00:30:22,370 Okay. 470 00:30:22,370 --> 00:30:26,360 And what's the null space of A transpose? 471 00:30:26,360 --> 00:30:30,270 It's the perpendicular guy. 472 00:30:30,270 --> 00:30:35,150 So this was the null space of A and this is 473 00:30:35,150 --> 00:30:38,921 the null space of A transpose. 474 00:30:38,921 --> 00:30:39,420 Okay. 475 00:30:42,260 --> 00:30:48,650 What I want to say here is that choosing these orthogonal bases 476 00:30:48,650 --> 00:30:53,750 for the row space and the column space is, like, no problem. 477 00:30:53,750 --> 00:30:55,800 They're only one dimensional. 478 00:30:55,800 --> 00:30:58,020 So what should V be? 479 00:30:58,020 --> 00:31:02,620 V should be -- v1, but -- yes, v1, rather -- 480 00:31:02,620 --> 00:31:05,540 v1 is supposed to be a unit vector. 481 00:31:05,540 --> 00:31:08,640 There's only one v1 to choose here, 482 00:31:08,640 --> 00:31:11,210 only one dimension in the row space. 483 00:31:11,210 --> 00:31:13,800 I just want to make it a unit vector. 484 00:31:13,800 --> 00:31:17,180 So v1 will be -- 485 00:31:17,180 --> 00:31:24,570 it'll be this vector, but made into a unit vector, so four -- 486 00:31:24,570 --> 00:31:25,970 point eight point six. 487 00:31:28,660 --> 00:31:30,670 Four fifths, three fifths. 488 00:31:30,670 --> 00:31:33,040 And what will be u1? 489 00:31:33,040 --> 00:31:35,740 u1 will be the unit vector there. 490 00:31:35,740 --> 00:31:40,820 So I want to turn four eight or one two into a unit vector, 491 00:31:40,820 --> 00:31:44,150 so u1 will be -- 492 00:31:44,150 --> 00:31:47,760 let's see, if it's one two, then what multiple of one two 493 00:31:47,760 --> 00:31:49,230 do I want? 494 00:31:49,230 --> 00:31:51,240 That has length square root of five, 495 00:31:51,240 --> 00:31:53,460 so I have to divide by square root of five. 496 00:31:56,140 --> 00:31:58,470 Let me complete the singular value 497 00:31:58,470 --> 00:32:01,660 decomposition for this matrix. 498 00:32:01,660 --> 00:32:09,540 So this matrix, four three eight six, is -- 499 00:32:09,540 --> 00:32:11,520 so I know what u1 -- 500 00:32:11,520 --> 00:32:19,570 here's A and I want to get U the basis in the column space. 501 00:32:19,570 --> 00:32:24,020 And it has to start with this guy, one 502 00:32:24,020 --> 00:32:27,290 over square root of five two over square root of five. 503 00:32:30,520 --> 00:32:36,350 Then I want the sigma. 504 00:32:36,350 --> 00:32:37,540 Okay. 505 00:32:37,540 --> 00:32:40,370 What are we expecting now for sigma? 506 00:32:44,400 --> 00:32:47,010 This is only a rank one matrix. 507 00:32:47,010 --> 00:32:51,720 We're only expecting a sigma one, which I have to find, 508 00:32:51,720 --> 00:32:55,040 but zeroes here. 509 00:32:55,040 --> 00:32:55,540 Okay. 510 00:32:55,540 --> 00:32:57,420 So what's sigma one? 511 00:32:57,420 --> 00:33:02,950 It should be the -- 512 00:33:02,950 --> 00:33:05,480 where did these sigmas come from? 513 00:33:05,480 --> 00:33:08,370 They came from A transpose A, so I -- 514 00:33:08,370 --> 00:33:10,750 can I do that little calculation over here? 515 00:33:10,750 --> 00:33:19,840 A transpose A is four three -- four three eight six times four 516 00:33:19,840 --> 00:33:23,340 three eight six. 517 00:33:23,340 --> 00:33:26,240 This had better -- this is a rank one matrix, 518 00:33:26,240 --> 00:33:29,170 this is going to be -- the whole thing will have rank one, 519 00:33:29,170 --> 00:33:40,240 that's 16 and 64 is 80, 12 and 48 is 60, 12 and 48 is 60, 520 00:33:40,240 --> 00:33:43,880 9 and 36 is 45. 521 00:33:43,880 --> 00:33:45,520 Okay. 522 00:33:45,520 --> 00:33:47,340 It's a rank one matrix. 523 00:33:47,340 --> 00:33:48,230 Of course. 524 00:33:48,230 --> 00:33:52,450 Every row is a multiple of four three. 525 00:33:52,450 --> 00:33:56,820 And what's the eigen -- what are the eigenvalues of that matrix? 526 00:33:56,820 --> 00:33:59,730 So this is the calculation -- this is like practicing, 527 00:33:59,730 --> 00:34:00,230 now. 528 00:34:00,230 --> 00:34:04,510 What are the eigenvalues of this rank one matrix? 529 00:34:04,510 --> 00:34:08,389 Well, tell me one eigenvalue, since the rank is only one, 530 00:34:08,389 --> 00:34:11,920 one eigenvalue is going to be zero. 531 00:34:11,920 --> 00:34:14,320 And then you know that the other eigenvalue 532 00:34:14,320 --> 00:34:19,330 is going to be a hundred and twenty five. 533 00:34:19,330 --> 00:34:22,760 So that's sigma squared, right, in A transpose A. 534 00:34:22,760 --> 00:34:28,780 So this will be the square root of a hundred and twenty five. 535 00:34:28,780 --> 00:34:34,900 And then finally, the V transpose -- 536 00:34:34,900 --> 00:34:37,750 the Vs will be -- 537 00:34:37,750 --> 00:34:41,830 there's v1, and what's v2? 538 00:34:41,830 --> 00:34:45,170 What's v2 in the -- 539 00:34:45,170 --> 00:34:50,739 how do I make this into an orthonormal basis? 540 00:34:50,739 --> 00:34:55,480 Well, v2 is, in the null space direction. 541 00:34:55,480 --> 00:34:59,810 It's perpendicular to that, so point six and minus point 542 00:34:59,810 --> 00:35:00,790 eight. 543 00:35:00,790 --> 00:35:04,400 So those are the Vs that go in here. 544 00:35:04,400 --> 00:35:11,920 Point eight, point six and point six minus point eight. 545 00:35:11,920 --> 00:35:12,420 Okay. 546 00:35:14,960 --> 00:35:17,410 And I guess I better finish this guy. 547 00:35:17,410 --> 00:35:21,580 So this guy, all I want is to complete the orthonormal basis 548 00:35:21,580 --> 00:35:23,670 -- it'll be coming from there. 549 00:35:23,670 --> 00:35:28,770 It'll be a two over square root of five and a minus one 550 00:35:28,770 --> 00:35:31,330 over square root of five. 551 00:35:31,330 --> 00:35:35,100 Let me take square root of five out of that matrix 552 00:35:35,100 --> 00:35:38,080 to make it look better. 553 00:35:38,080 --> 00:35:44,850 So one over square root of five times one two two minus one. 554 00:35:47,560 --> 00:35:49,920 Okay. 555 00:35:49,920 --> 00:35:53,160 So there I have -- including the square root of five -- 556 00:35:53,160 --> 00:35:56,870 that's an orthogonal matrix, that's an orthogonal matrix, 557 00:35:56,870 --> 00:36:01,190 that's a diagonal matrix and its rank is only one. 558 00:36:01,190 --> 00:36:03,800 And now if I do that multiplication, 559 00:36:03,800 --> 00:36:07,550 I pray that it comes out right. 560 00:36:10,200 --> 00:36:12,022 The square root of five will cancel 561 00:36:12,022 --> 00:36:13,480 into that square root of one twenty 562 00:36:13,480 --> 00:36:16,390 five and leave me with the square root of 25, which 563 00:36:16,390 --> 00:36:20,190 is five, and five will multiply these numbers 564 00:36:20,190 --> 00:36:24,410 and I'll get whole numbers and out will come A. 565 00:36:24,410 --> 00:36:25,230 Okay. 566 00:36:25,230 --> 00:36:31,110 That's like a second example showing how the null space guy 567 00:36:31,110 --> 00:36:39,680 -- so this -- this vector and this one were multiplied 568 00:36:39,680 --> 00:36:40,620 by this zero. 569 00:36:40,620 --> 00:36:46,370 So they were easy to deal with. 570 00:36:46,370 --> 00:36:50,960 Tthe key ones are the ones in the column space and the row 571 00:36:50,960 --> 00:36:51,480 space. 572 00:36:51,480 --> 00:36:57,920 Do you see how I'm getting columns here, diagonal here, 573 00:36:57,920 --> 00:37:02,330 rows here, coming together to produce A. 574 00:37:02,330 --> 00:37:06,790 Okay, that's the singular value decomposition. 575 00:37:06,790 --> 00:37:13,370 So, let me think what I want to add to complete this topic. 576 00:37:18,130 --> 00:37:22,970 So that's two examples. 577 00:37:22,970 --> 00:37:25,740 And now let's think what we're really doing. 578 00:37:25,740 --> 00:37:33,770 We're choosing the right basis for the four subspaces 579 00:37:33,770 --> 00:37:35,010 of linear algebra. 580 00:37:35,010 --> 00:37:39,640 Let me write this down. 581 00:37:39,640 --> 00:37:52,785 So v1 up to vr is an orthonormal basis for the row space. 582 00:37:58,220 --> 00:38:05,570 u1 up to ur is an orthonormal basis for the column space. 583 00:38:09,420 --> 00:38:14,930 And then I just finish those out by v(r+1), 584 00:38:14,930 --> 00:38:20,350 the rest up to vn is an orthonormal basis for the null 585 00:38:20,350 --> 00:38:20,850 space. 586 00:38:24,510 --> 00:38:35,220 And finally, u(r+1) up to is an orthonormal basis for the null 587 00:38:35,220 --> 00:38:36,320 space of A transpose. 588 00:38:39,670 --> 00:38:45,360 Do you see that we finally got the bases right? 589 00:38:45,360 --> 00:38:51,170 They're right because they're orthonormal, and also -- 590 00:38:51,170 --> 00:38:55,100 again, Graham Schmidt would have done this in chapter four. 591 00:38:55,100 --> 00:39:00,920 Here we needed eigenvalues, because these bases 592 00:39:00,920 --> 00:39:03,070 make the matrix diagonal. 593 00:39:03,070 --> 00:39:09,400 A times V I is a multiple of U I. 594 00:39:09,400 --> 00:39:11,250 So I'll put "and" -- 595 00:39:14,540 --> 00:39:16,640 the matrix has been made diagonal. 596 00:39:16,640 --> 00:39:24,730 When we choose these bases, there's no coupling between Vs 597 00:39:24,730 --> 00:39:26,740 and no coupling between Us. 598 00:39:26,740 --> 00:39:30,550 Each A -- A times each V is in the direction 599 00:39:30,550 --> 00:39:31,980 of the corresponding U. 600 00:39:31,980 --> 00:39:36,700 So it's exactly the right basis for the four 601 00:39:36,700 --> 00:39:38,130 fundamental subspaces. 602 00:39:38,130 --> 00:39:41,570 And of course, their dimensions are what we know. 603 00:39:41,570 --> 00:39:44,850 The dimension of the row space is 604 00:39:44,850 --> 00:39:49,450 the rank r, and so is the dimension of the column space. 605 00:39:49,450 --> 00:39:51,190 The dimension of the null space is 606 00:39:51,190 --> 00:39:54,620 n-r, that's how many vectors we need, 607 00:39:54,620 --> 00:39:59,790 and m-r basis vectors for the left null space, the null space 608 00:39:59,790 --> 00:40:01,960 of A transpose. 609 00:40:01,960 --> 00:40:04,620 Okay. 610 00:40:04,620 --> 00:40:05,850 I'm going to stop there. 611 00:40:05,850 --> 00:40:10,720 I could develop further from the SVD, 612 00:40:10,720 --> 00:40:14,140 but we'll see it again in the very last lectures 613 00:40:14,140 --> 00:40:14,949 of the course. 614 00:40:14,949 --> 00:40:15,740 So there's the SVD. 615 00:40:15,740 --> 00:40:17,290 Thanks.