1 00:00:00,000 --> 00:00:00,030 2 00:00:00,030 --> 00:00:02,330 The following content is provided under a Creative 3 00:00:02,330 --> 00:00:03,610 Commons license. 4 00:00:03,610 --> 00:00:05,770 Your support will help MIT OpenCourseWare 5 00:00:05,770 --> 00:00:09,930 continue to offer high quality educational resources for free. 6 00:00:09,930 --> 00:00:12,530 To make a donation or to view additional materials 7 00:00:12,530 --> 00:00:15,620 from hundreds of MIT courses visit MIT OpenCourseWare 8 00:00:15,620 --> 00:00:20,920 at ocw.mit.edu. 9 00:00:20,920 --> 00:00:25,670 PROFESSOR STRANG: Finally we get to positive definite matrices. 10 00:00:25,670 --> 00:00:29,180 I've used the word and now it's time to pin it down. 11 00:00:29,180 --> 00:00:33,620 And so this would be my thank you for staying with it 12 00:00:33,620 --> 00:00:37,490 while we do this important preliminary stuff 13 00:00:37,490 --> 00:00:39,780 about linear algebra. 14 00:00:39,780 --> 00:00:41,850 So starting the next lecture we'll 15 00:00:41,850 --> 00:00:46,660 really make a big start on engineering applications. 16 00:00:46,660 --> 00:00:51,250 But these matrices are going to be the key to everything. 17 00:00:51,250 --> 00:00:58,290 And I'll call these matrices K. And positive definite, 18 00:00:58,290 --> 00:01:03,200 I will only use that word about a symmetric matrix. 19 00:01:03,200 --> 00:01:07,130 So the matrix is already symmetric 20 00:01:07,130 --> 00:01:09,730 and that means it has real eigenvalues 21 00:01:09,730 --> 00:01:12,820 and many other important properties, 22 00:01:12,820 --> 00:01:14,930 orthogonal eigenvectors. 23 00:01:14,930 --> 00:01:17,960 And now we're asking for more. 24 00:01:17,960 --> 00:01:25,630 And it's that extra bit that is terrific 25 00:01:25,630 --> 00:01:28,260 in all kinds of applications. 26 00:01:28,260 --> 00:01:31,090 So if I can do this bit of linear algebra. 27 00:01:31,090 --> 00:01:38,880 So what's coming then, my review session this afternoon at four, 28 00:01:38,880 --> 00:01:41,460 I'm very happy that we've got, I think, 29 00:01:41,460 --> 00:01:47,150 the best MATLAB problem ever invented, in 18.085 anyway. 30 00:01:47,150 --> 00:01:51,410 So that should get onto the website probably by tomorrow. 31 00:01:51,410 --> 00:01:55,670 And Peter Buchak is like the MATLAB person. 32 00:01:55,670 --> 00:01:59,590 So his review sessions are Friday at noon. 33 00:01:59,590 --> 00:02:02,890 And I just saw him and suggested Friday at noon 34 00:02:02,890 --> 00:02:06,750 he might as well just stay in here. 35 00:02:06,750 --> 00:02:09,740 And knowing that that isn't maybe 36 00:02:09,740 --> 00:02:11,050 a good hour for everybody. 37 00:02:11,050 --> 00:02:16,390 So you could see him also outside of that hour. 38 00:02:16,390 --> 00:02:19,670 But that's the hour he will be ready 39 00:02:19,670 --> 00:02:23,520 for all kinds of questions about MATLAB or about the homeworks. 40 00:02:23,520 --> 00:02:28,720 Actually you'll be probably thinking more about, also 41 00:02:28,720 --> 00:02:35,680 about the homework questions on this topic. 42 00:02:35,680 --> 00:02:40,910 Ready for positive definite? 43 00:02:40,910 --> 00:02:43,750 You said yes, right? 44 00:02:43,750 --> 00:02:50,180 And you have a hint about these things. 45 00:02:50,180 --> 00:02:54,880 So we have a symmetric matrix and the beauty 46 00:02:54,880 --> 00:02:58,190 is that it brings together all of linear algebra. 47 00:02:58,190 --> 00:03:01,670 Including elimination, that's when we see pivots. 48 00:03:01,670 --> 00:03:05,930 Including determinants which are closely related to the pivots. 49 00:03:05,930 --> 00:03:08,180 And what do I mean by upper left? 50 00:03:08,180 --> 00:03:13,310 I mean that if I have a three by three symmetric matrix 51 00:03:13,310 --> 00:03:16,350 and I want to test it for positive definite, 52 00:03:16,350 --> 00:03:18,690 and I guess actually this would be the easiest 53 00:03:18,690 --> 00:03:23,140 test if I had a tiny matrix, three by three, 54 00:03:23,140 --> 00:03:27,120 and I had numbers, then this would be a good test. 55 00:03:27,120 --> 00:03:31,413 The determinants-- By upper left determinants I mean that one 56 00:03:31,413 --> 00:03:33,170 by one determinant. 57 00:03:33,170 --> 00:03:36,220 So just that first number has to be positive. 58 00:03:36,220 --> 00:03:39,900 Then the two by two determinant, that times that minus that 59 00:03:39,900 --> 00:03:42,230 times that has to be positive. 60 00:03:42,230 --> 00:03:44,420 Oh I've already been saying that. 61 00:03:44,420 --> 00:03:46,290 Let me just put in some letters. 62 00:03:46,290 --> 00:03:48,100 So a has to be positive. 63 00:03:48,100 --> 00:03:51,620 This is symmetric, so a times c has 64 00:03:51,620 --> 00:03:54,310 to be bigger than b squared. 65 00:03:54,310 --> 00:03:57,340 So that will tell us. 66 00:03:57,340 --> 00:03:59,230 And then for two by two we finish. 67 00:03:59,230 --> 00:04:03,256 For three by three we would also require the three 68 00:04:03,256 --> 00:04:05,880 by three determinant to be positive. 69 00:04:05,880 --> 00:04:08,440 But already here you're seeing one point 70 00:04:08,440 --> 00:04:11,470 about a positive definite matrix. 71 00:04:11,470 --> 00:04:14,430 Its diagonal will have to be positive. 72 00:04:14,430 --> 00:04:20,020 And somehow its diagonal has to be not just above zero, 73 00:04:20,020 --> 00:04:25,140 but somehow it has to defeat b squared. 74 00:04:25,140 --> 00:04:30,690 So the diagonal has to be somehow more positive 75 00:04:30,690 --> 00:04:33,890 than whatever negative stuff might 76 00:04:33,890 --> 00:04:35,780 come from off the diagonal. 77 00:04:35,780 --> 00:04:41,250 That's why I would need a*c > b squared. 78 00:04:41,250 --> 00:04:43,120 So both of those will be positive 79 00:04:43,120 --> 00:04:48,600 and their product has to be bigger than the other guy. 80 00:04:48,600 --> 00:04:52,650 And then finally, a third test is that all the eigenvalues 81 00:04:52,650 --> 00:04:53,970 are positive. 82 00:04:53,970 --> 00:04:56,500 And of course if I give you a three by three matrix, 83 00:04:56,500 --> 00:04:58,340 that's probably not the easiest test 84 00:04:58,340 --> 00:05:00,670 since you'd have to find the eigenvalues. 85 00:05:00,670 --> 00:05:05,490 Much easier to find the determinants or the pivots. 86 00:05:05,490 --> 00:05:10,400 Actually, just while I'm at it, so the first pivot of course 87 00:05:10,400 --> 00:05:12,170 is a itself. 88 00:05:12,170 --> 00:05:15,140 No difficulty there. 89 00:05:15,140 --> 00:05:19,580 The second pivot turns out to be the ratio of a*c - 90 00:05:19,580 --> 00:05:22,140 b squared to a. 91 00:05:22,140 --> 00:05:25,840 So the connection between pivots and determinants 92 00:05:25,840 --> 00:05:28,210 is just really close. 93 00:05:28,210 --> 00:05:30,320 Pivots are ratios of determinants, 94 00:05:30,320 --> 00:05:31,250 if you work it out. 95 00:05:31,250 --> 00:05:34,930 The second pivot, maybe I would call that d_2, 96 00:05:34,930 --> 00:05:40,110 is the ratio of a*c - b squared over a. 97 00:05:40,110 --> 00:05:43,040 In other words it's c minus b^2 / a. 98 00:05:43,040 --> 00:05:48,250 99 00:05:48,250 --> 00:05:51,790 Determinants are positive and vice versa. 100 00:05:51,790 --> 00:05:54,130 Then it's fantastic that the eigenvalues 101 00:05:54,130 --> 00:05:56,740 come into the picture. 102 00:05:56,740 --> 00:06:00,510 So those are three ways, three important properties 103 00:06:00,510 --> 00:06:02,870 of a positive definite matrix. 104 00:06:02,870 --> 00:06:08,770 But I'd like to make the definition something different. 105 00:06:08,770 --> 00:06:11,200 Now I'm coming to the meaning. 106 00:06:11,200 --> 00:06:14,420 If I think of those as the tests, that's done. 107 00:06:14,420 --> 00:06:24,370 Now the meaning of positive definite. 108 00:06:24,370 --> 00:06:27,230 The meaning of positive definite and the applications 109 00:06:27,230 --> 00:06:32,540 are closely related to looking for a minimum. 110 00:06:32,540 --> 00:06:39,900 And so what I'm going to bring in here-- So it's symmetric. 111 00:06:39,900 --> 00:06:47,610 Now for a symmetric matrix I want to introduce the energy. 112 00:06:47,610 --> 00:06:51,580 So this is the reason why it has so many applications 113 00:06:51,580 --> 00:06:54,750 and such important physical meaning is 114 00:06:54,750 --> 00:06:56,910 that what I'm about to introduce, 115 00:06:56,910 --> 00:07:01,600 which is a function of x, and here 116 00:07:01,600 --> 00:07:07,940 it is, it's x transpose times A, not A, 117 00:07:07,940 --> 00:07:16,770 I'm sticking with K for my matrix, times x. 118 00:07:16,770 --> 00:07:20,870 I think of that as some f(x). 119 00:07:20,870 --> 00:07:24,430 And let's just see what it would be if the matrix was 120 00:07:24,430 --> 00:07:29,510 two by two, [a, b; b, c]. 121 00:07:29,510 --> 00:07:33,330 Suppose that's my matrix. 122 00:07:33,330 --> 00:07:36,770 We want to get a handle on what, this is the first time 123 00:07:36,770 --> 00:07:42,040 I've ever written something that has x's times x's. 124 00:07:42,040 --> 00:07:45,190 So it's going to be quadratic. 125 00:07:45,190 --> 00:07:48,330 They're going to be x's times x's. 126 00:07:48,330 --> 00:07:52,190 And x is a general vector of the right size 127 00:07:52,190 --> 00:07:55,100 so it's got components x_1, x_2. 128 00:07:55,100 --> 00:07:57,850 And there it's transposed, so it's a row. 129 00:07:57,850 --> 00:08:01,920 And now I put in the [a, b; b, c]. 130 00:08:01,920 --> 00:08:05,680 And then I put in x again. 131 00:08:05,680 --> 00:08:09,090 This is going to give me a very nice, simple, important 132 00:08:09,090 --> 00:08:11,830 expression. 133 00:08:11,830 --> 00:08:14,470 Depending on x_1 and x_2. 134 00:08:14,470 --> 00:08:18,090 Now what is, can we do that multiplication? 135 00:08:18,090 --> 00:08:25,210 Maybe above I'll do the multiplication of this pair 136 00:08:25,210 --> 00:08:28,370 and then I have the other guy to bring in. 137 00:08:28,370 --> 00:08:31,230 So here, that would be ax_1+bx_2. 138 00:08:31,230 --> 00:08:35,040 139 00:08:35,040 --> 00:08:36,850 And this would be bx_1+cx_2. 140 00:08:36,850 --> 00:08:39,550 141 00:08:39,550 --> 00:08:44,722 So that's the first, that's this times this. 142 00:08:44,722 --> 00:08:45,680 What am I going to get? 143 00:08:45,680 --> 00:08:48,910 What shape, what size is this result going to be? 144 00:08:48,910 --> 00:08:56,140 This K is n by n. x is a column vector. n by one. x transpose, 145 00:08:56,140 --> 00:08:58,800 what's the shape of x transpose? 146 00:08:58,800 --> 00:09:00,070 One by n? 147 00:09:00,070 --> 00:09:02,400 So what's the total result? 148 00:09:02,400 --> 00:09:03,100 One by one. 149 00:09:03,100 --> 00:09:04,270 Just a number. 150 00:09:04,270 --> 00:09:05,070 Just a function. 151 00:09:05,070 --> 00:09:06,560 It's a number. 152 00:09:06,560 --> 00:09:11,450 But it depends on the x's and the matrix inside. 153 00:09:11,450 --> 00:09:12,830 Can we do it now? 154 00:09:12,830 --> 00:09:16,280 So I've got this to multiply by this. 155 00:09:16,280 --> 00:09:19,900 Do you see an x_1 squared showing up? 156 00:09:19,900 --> 00:09:21,590 Yes, from there times there. 157 00:09:21,590 --> 00:09:24,770 And what's it multiplied by? 158 00:09:24,770 --> 00:09:26,260 The a. 159 00:09:26,260 --> 00:09:30,830 The first term is this times the ax_1 is a x_1 squared. 160 00:09:30,830 --> 00:09:33,480 So that's our first quadratic. 161 00:09:33,480 --> 00:09:36,360 Now there'd be an x_1, x_2. 162 00:09:36,360 --> 00:09:39,820 Let me leave that for a minute and find the x_2 squared 163 00:09:39,820 --> 00:09:41,350 because it's easy. 164 00:09:41,350 --> 00:09:43,440 So where am I going to get x_2 squared? 165 00:09:43,440 --> 00:09:46,650 I'm going to get that from x_2, second guy 166 00:09:46,650 --> 00:09:49,150 here times second guy here. 167 00:09:49,150 --> 00:09:54,830 There's a c x_2 squared. 168 00:09:54,830 --> 00:09:58,630 So you're seeing already where the diagonal shows up. 169 00:09:58,630 --> 00:10:02,330 The diagonal a, c, whatever, is multiplying 170 00:10:02,330 --> 00:10:04,600 the perfect squares. 171 00:10:04,600 --> 00:10:08,820 And it'll be the off-diagonal that multiplies the x_1 x_2. 172 00:10:08,820 --> 00:10:11,210 We might call those the cross terms. 173 00:10:11,210 --> 00:10:13,160 And what do we get for that then? 174 00:10:13,160 --> 00:10:16,180 We have x_1 times this guy. 175 00:10:16,180 --> 00:10:20,200 So that's a cross term. bx_1*x_2, right? 176 00:10:20,200 --> 00:10:24,270 And here's another one coming from x_2 times this guy. 177 00:10:24,270 --> 00:10:27,780 And what's that one? 178 00:10:27,780 --> 00:10:30,080 It's also bx_1*x_2. 179 00:10:30,080 --> 00:10:33,750 So x_1, multiply that, x_2 multiply that, 180 00:10:33,750 --> 00:10:37,210 and so what do we have for this cross term here? 181 00:10:37,210 --> 00:10:39,600 Two of them. 182 00:10:39,600 --> 00:10:40,670 2bx_1*x_2. 183 00:10:40,670 --> 00:10:43,350 184 00:10:43,350 --> 00:10:48,160 In other words, that b and that b came together 185 00:10:48,160 --> 00:10:50,490 in the 2bx_1*x_2. 186 00:10:50,490 --> 00:10:56,280 So here's my energy. 187 00:10:56,280 --> 00:10:58,400 Can I just loosely call it energy? 188 00:10:58,400 --> 00:11:02,960 And then as we get to applications we'll see why. 189 00:11:02,960 --> 00:11:06,400 So I'm interested in that because it 190 00:11:06,400 --> 00:11:12,390 has important meaning. 191 00:11:12,390 --> 00:11:16,770 Well, so now I'm ready to define positive definite matrices. 192 00:11:16,770 --> 00:11:19,690 So I'll call that number four. 193 00:11:19,690 --> 00:11:23,860 But I'm going to give it a big star. 194 00:11:23,860 --> 00:11:29,670 Even more because it's the sort of key. 195 00:11:29,670 --> 00:11:34,010 So the test will be, you can probably guess it, 196 00:11:34,010 --> 00:11:41,630 I look at this expression, that x transpose Ax. 197 00:11:41,630 --> 00:11:45,250 And if it's a positive definite matrix 198 00:11:45,250 --> 00:11:47,340 and this represents energy, the key 199 00:11:47,340 --> 00:11:50,750 will be that this should be positive. 200 00:11:50,750 --> 00:11:57,120 This one should be positive for all x's. 201 00:11:57,120 --> 00:11:59,700 Well, with one exception, of course. 202 00:11:59,700 --> 00:12:07,530 All x's except, which vector is it? x=0 would just give me-- 203 00:12:07,530 --> 00:12:17,390 See, I put K. My default for a matrix, but should be, it's K. 204 00:12:17,390 --> 00:12:22,340 Except x=0, except the zero vector. 205 00:12:22,340 --> 00:12:23,730 Of course. 206 00:12:23,730 --> 00:12:29,940 If x_1 and x_2 are both zero. 207 00:12:29,940 --> 00:12:34,720 Now that looks a little maybe less straightforward, 208 00:12:34,720 --> 00:12:37,760 I would say, because it's a statement about this 209 00:12:37,760 --> 00:12:41,940 is true for all x_1 and x_2. 210 00:12:41,940 --> 00:12:44,760 And we better do some examples and draw a picture. 211 00:12:44,760 --> 00:12:51,670 Let me draw a picture right away. 212 00:12:51,670 --> 00:12:54,750 So here's x_1 direction. 213 00:12:54,750 --> 00:12:56,420 Here's x_2 direction. 214 00:12:56,420 --> 00:13:05,960 And here is the x transpose Ax, my function. 215 00:13:05,960 --> 00:13:09,160 So this depends on two variables. 216 00:13:09,160 --> 00:13:13,370 So it's going to be a sort of a surface if I draw it. 217 00:13:13,370 --> 00:13:16,650 Now, what point do we absolutely know? 218 00:13:16,650 --> 00:13:21,050 And I put A again. 219 00:13:21,050 --> 00:13:29,560 I am so sorry. 220 00:13:29,560 --> 00:13:30,840 Well, we know one point. 221 00:13:30,840 --> 00:13:34,660 It's there whatever that matrix might be. 222 00:13:34,660 --> 00:13:35,680 It's there. 223 00:13:35,680 --> 00:13:37,240 Zero, right? 224 00:13:37,240 --> 00:13:40,300 You just told me that if both x's are zero then 225 00:13:40,300 --> 00:13:42,760 we automatically get zero. 226 00:13:42,760 --> 00:13:47,100 Now what do you think the shape of this curve, 227 00:13:47,100 --> 00:13:51,280 the shape of this graph is going to look like? 228 00:13:51,280 --> 00:13:54,920 The point is, if we're positive definite now. 229 00:13:54,920 --> 00:13:58,980 So I'm drawing the picture for positive definite. 230 00:13:58,980 --> 00:14:04,610 So my definition is that the energy goes up. 231 00:14:04,610 --> 00:14:06,650 It's positive, right? 232 00:14:06,650 --> 00:14:10,320 When I leave, when I move away from that point I go upwards. 233 00:14:10,320 --> 00:14:13,540 That point will be a minimum. 234 00:14:13,540 --> 00:14:17,740 Let me just draw it roughly. 235 00:14:17,740 --> 00:14:23,500 So it sort of goes up like this. 236 00:14:23,500 --> 00:14:30,670 These cheap 2-D boards and I've got a three-dimensional picture 237 00:14:30,670 --> 00:14:34,370 here. 238 00:14:34,370 --> 00:14:35,780 But you see it somehow? 239 00:14:35,780 --> 00:14:40,300 What word or what's your visualization? 240 00:14:40,300 --> 00:14:43,100 It has a minimum there. 241 00:14:43,100 --> 00:14:45,840 That's why minimization, which was like, 242 00:14:45,840 --> 00:14:49,140 the core problem in calculus, is here now. 243 00:14:49,140 --> 00:14:54,870 But for functions of two x's or n x's. 244 00:14:54,870 --> 00:14:58,930 We're up the dimension over the basic minimum problem 245 00:14:58,930 --> 00:15:03,730 of calculus. 246 00:15:03,730 --> 00:15:05,260 It's sort of like a parabola. 247 00:15:05,260 --> 00:15:07,960 Its cross-sections cutting down through the thing 248 00:15:07,960 --> 00:15:10,550 would be just parabolas because of the x squared. 249 00:15:10,550 --> 00:15:12,890 I'm going to call this a bowl. 250 00:15:12,890 --> 00:15:16,050 It's a short word. 251 00:15:16,050 --> 00:15:16,840 Do you see it? 252 00:15:16,840 --> 00:15:18,280 It opens up. 253 00:15:18,280 --> 00:15:20,870 That's the key point, that it opens upward. 254 00:15:20,870 --> 00:15:22,920 And let's do some examples. 255 00:15:22,920 --> 00:15:26,000 Tell me some positive definite. 256 00:15:26,000 --> 00:15:29,170 So positive definite and then let 257 00:15:29,170 --> 00:15:35,910 me here put some not positive definite cases. 258 00:15:35,910 --> 00:15:38,360 Tell me a matrix. 259 00:15:38,360 --> 00:15:41,050 Well, what's the easiest, first matrix 260 00:15:41,050 --> 00:15:44,710 that occurs to you as a positive definite matrix? 261 00:15:44,710 --> 00:15:49,230 The identity. 262 00:15:49,230 --> 00:15:52,090 That passes all our tests, its eigenvalues are one, 263 00:15:52,090 --> 00:15:54,890 its pivots are one, the determinants are one. 264 00:15:54,890 --> 00:15:58,830 And the function is x_1 squared plus x_2 265 00:15:58,830 --> 00:16:05,520 squared with no b in it. 266 00:16:05,520 --> 00:16:08,590 It's just a perfect bowl, perfectly symmetric, 267 00:16:08,590 --> 00:16:12,190 the way it would come off a potter's wheel. 268 00:16:12,190 --> 00:16:16,190 Now let me take one that's maybe not so, 269 00:16:16,190 --> 00:16:18,410 let me put a nine there. 270 00:16:18,410 --> 00:16:20,310 So I'm off to a reasonable start. 271 00:16:20,310 --> 00:16:24,090 I have an x_1 squared and a nine x_2 squared. 272 00:16:24,090 --> 00:16:25,890 And now I want to ask you, what could I 273 00:16:25,890 --> 00:16:30,410 put in there that would leave it positive definite? 274 00:16:30,410 --> 00:16:33,430 Well, give me a couple of possibilities. 275 00:16:33,430 --> 00:16:37,800 What's a nice, not too big now, that's the thing. 276 00:16:37,800 --> 00:16:38,550 Two. 277 00:16:38,550 --> 00:16:39,850 Two would be fine. 278 00:16:39,850 --> 00:16:42,540 So if I had a two there and a two there I would have 279 00:16:42,540 --> 00:16:47,000 a 4x_1*x_2 and it would, like, this, 280 00:16:47,000 --> 00:16:54,720 instead of being a circle, which it was for the identity, 281 00:16:54,720 --> 00:16:59,230 the plane there would cut out a ellipse instead. 282 00:16:59,230 --> 00:17:02,750 But it would be a good ellipse. 283 00:17:02,750 --> 00:17:05,730 Because we're doing squares, we're really, 284 00:17:05,730 --> 00:17:08,960 the Greeks understood these second degree things 285 00:17:08,960 --> 00:17:18,540 and they would have known this would have been an ellipse. 286 00:17:18,540 --> 00:17:23,070 How high can I go with that two or where do I have to stop? 287 00:17:23,070 --> 00:17:27,660 Where would I have to, if I wanted to change the two, 288 00:17:27,660 --> 00:17:33,080 let me just focus on that one, suppose I wanted to change it. 289 00:17:33,080 --> 00:17:36,510 First of all, give me one that's, 290 00:17:36,510 --> 00:17:38,900 how about the borderline. 291 00:17:38,900 --> 00:17:40,610 Three would be the borderline. 292 00:17:40,610 --> 00:17:41,410 Why's that? 293 00:17:41,410 --> 00:17:48,100 Because at three we have nine minus nine for the determinant. 294 00:17:48,100 --> 00:17:51,500 So the determinant is zero. 295 00:17:51,500 --> 00:17:53,350 Of course it passed the first test. 296 00:17:53,350 --> 00:17:54,920 One by one was okay. 297 00:17:54,920 --> 00:18:03,840 But two by two was not, was at the borderline. 298 00:18:03,840 --> 00:18:07,850 What else should I think? 299 00:18:07,850 --> 00:18:11,230 Oh, that's a very interesting case. 300 00:18:11,230 --> 00:18:13,320 The borderline. 301 00:18:13,320 --> 00:18:15,660 You know, it almost makes it. 302 00:18:15,660 --> 00:18:21,860 But can you tell me the eigenvalues of that matrix? 303 00:18:21,860 --> 00:18:25,100 Don't do any quadratic equations. 304 00:18:25,100 --> 00:18:28,220 How do I know, what's one eigenvalue of a matrix? 305 00:18:28,220 --> 00:18:30,330 You made it singular, right? 306 00:18:30,330 --> 00:18:31,770 You made that matrix singular. 307 00:18:31,770 --> 00:18:32,730 Determinant zero. 308 00:18:32,730 --> 00:18:36,030 So one of its eigenvalues is zero. 309 00:18:36,030 --> 00:18:40,180 And the other one is visible by looking at the trace. 310 00:18:40,180 --> 00:18:44,090 I just quickly mentioned that if I add the diagonal, 311 00:18:44,090 --> 00:18:47,850 I get the same answer as if I add the two eigenvalues. 312 00:18:47,850 --> 00:18:51,180 So that other eigenvalue must be ten. 313 00:18:51,180 --> 00:18:54,940 And this is entirely typical, that ten and zero, 314 00:18:54,940 --> 00:18:58,640 the extreme eigenvalues, lambda_max and lambda_min, 315 00:18:58,640 --> 00:19:04,300 are bigger than-- these diagonal guys are inside. 316 00:19:04,300 --> 00:19:08,180 They're inside, between zero and ten 317 00:19:08,180 --> 00:19:12,580 and it's these terms that enter somehow and gave us 318 00:19:12,580 --> 00:19:15,880 an eigenvalue of ten and an eigenvalue of zero. 319 00:19:15,880 --> 00:19:20,320 I guess I'm tempted to try to draw that figure. 320 00:19:20,320 --> 00:19:25,560 Just to get a feeling of what's with that one. 321 00:19:25,560 --> 00:19:30,140 It always helps to get the borderline case. 322 00:19:30,140 --> 00:19:32,320 So what's with this one? 323 00:19:32,320 --> 00:19:35,630 Let me see what my quadratic would be. 324 00:19:35,630 --> 00:19:37,760 Can I just change it up here? 325 00:19:37,760 --> 00:19:38,970 Rather than rewriting it. 326 00:19:38,970 --> 00:19:42,000 So I'm going to, I'll put it up here. 327 00:19:42,000 --> 00:19:46,240 So I have to change that four to what? 328 00:19:46,240 --> 00:19:48,810 Now that I'm looking at this matrix. 329 00:19:48,810 --> 00:19:51,960 That four is now a six. 330 00:19:51,960 --> 00:19:53,900 Six. 331 00:19:53,900 --> 00:19:56,880 This is my guy for this one. 332 00:19:56,880 --> 00:19:58,862 Which is not positive definite. 333 00:19:58,862 --> 00:20:00,320 Let me tell you right away the word 334 00:20:00,320 --> 00:20:02,260 that I would use for this one. 335 00:20:02,260 --> 00:20:06,820 I would call it positive semi-definite because it's 336 00:20:06,820 --> 00:20:09,500 almost there, but not quite. 337 00:20:09,500 --> 00:20:15,300 So semi-definite allows the matrix to be singular. 338 00:20:15,300 --> 00:20:18,180 So semi-definite, maybe I'll do it 339 00:20:18,180 --> 00:20:22,350 in green what semi-definite would be. 340 00:20:22,350 --> 00:20:31,700 Semi-def would be eigenvalues greater than or equal zero. 341 00:20:31,700 --> 00:20:35,160 Determinants greater than or equal zero. 342 00:20:35,160 --> 00:20:39,440 Pivots greater than zero if they're there or then 343 00:20:39,440 --> 00:20:41,410 we run out of pivots. 344 00:20:41,410 --> 00:20:44,490 You could say greater than or equal to zero then. 345 00:20:44,490 --> 00:20:48,740 And energy, greater than or equal to zero 346 00:20:48,740 --> 00:20:53,110 for semi-definite. 347 00:20:53,110 --> 00:20:58,380 And when would the energy, what x's, what would be the like, 348 00:20:58,380 --> 00:21:01,090 you could say the ground states or something, 349 00:21:01,090 --> 00:21:04,420 what x's-- So greater than or equal to zero, 350 00:21:04,420 --> 00:21:08,080 emphasize that possibility of equal in the semi-definite 351 00:21:08,080 --> 00:21:10,100 case. 352 00:21:10,100 --> 00:21:17,300 Suppose I have a semi-definite matrix, yeah, I've got one. 353 00:21:17,300 --> 00:21:19,690 But it's singular. 354 00:21:19,690 --> 00:21:26,160 So that means a singular matrix takes some vector x to zero. 355 00:21:26,160 --> 00:21:27,700 Right? 356 00:21:27,700 --> 00:21:30,080 If my matrix is actually singular, 357 00:21:30,080 --> 00:21:32,990 then there'll be an x where Kx is zero. 358 00:21:32,990 --> 00:21:35,000 And then, of course, multiplying by x transpose, 359 00:21:35,000 --> 00:21:36,350 I'm still at zero. 360 00:21:36,350 --> 00:21:41,900 So the x's, the zero energy guys, this is straightforward, 361 00:21:41,900 --> 00:21:49,180 the zero energy guys, the ones where x transpose Kx is zero, 362 00:21:49,180 --> 00:21:56,730 will happen when Kx is zero. 363 00:21:56,730 --> 00:22:03,210 If Kx is zero, and we'll see it in that example. 364 00:22:03,210 --> 00:22:05,140 Let's see it in that example. 365 00:22:05,140 --> 00:22:12,300 What's the x for which, I could say 366 00:22:12,300 --> 00:22:15,170 in the null space, what's the vector x 367 00:22:15,170 --> 00:22:18,830 that that matrix kills? 368 00:22:18,830 --> 00:22:21,490 369 00:22:21,490 --> 00:22:23,146 [3, -1], right? 370 00:22:23,146 --> 00:22:24,770 The vector [3, -1]. 371 00:22:24,770 --> 00:22:30,600 [3, -1] gives me [0, 0]. 372 00:22:30,600 --> 00:22:35,360 That's the vector that-- So I get 3-3, the zero, 9-9, 373 00:22:35,360 --> 00:22:36,710 the zero. 374 00:22:36,710 --> 00:22:40,700 So I believe that this thing will 375 00:22:40,700 --> 00:22:44,750 be-- Is it zero at [3, -1]? 376 00:22:44,750 --> 00:22:47,530 I think that it has to be, right? 377 00:22:47,530 --> 00:22:52,040 If I take x_1 to be three and x_2 to be minus one, 378 00:22:52,040 --> 00:22:54,400 I think I've got zero energy here. 379 00:22:54,400 --> 00:22:59,630 Do I? x_1 squared will be nine and nine x_2 squared 380 00:22:59,630 --> 00:23:01,210 will be nine more. 381 00:23:01,210 --> 00:23:04,160 And what will be this 6x_1*x_2? 382 00:23:04,160 --> 00:23:09,250 What will that come out for this x_1 and x_2? 383 00:23:09,250 --> 00:23:10,530 Minus 18. 384 00:23:10,530 --> 00:23:11,950 Had to, right? 385 00:23:11,950 --> 00:23:15,620 So I'd get nine from there, nine from there, minus 18, zero. 386 00:23:15,620 --> 00:23:18,660 So the graph for this positive semi-definite 387 00:23:18,660 --> 00:23:21,050 will look a bit like this. 388 00:23:21,050 --> 00:23:26,060 There'll be a direction in which it doesn't climb. 389 00:23:26,060 --> 00:23:29,560 It doesn't go below the base, right? 390 00:23:29,560 --> 00:23:31,340 It's never negative. 391 00:23:31,340 --> 00:23:33,190 This is now the semi-definite picture. 392 00:23:33,190 --> 00:23:36,260 But it can run along the base. 393 00:23:36,260 --> 00:23:40,390 And it will for the vector x_1=3, x_2=-1, 394 00:23:40,390 --> 00:23:42,520 I don't know where that is, one, two, three, 395 00:23:42,520 --> 00:23:45,910 and then maybe minus one. 396 00:23:45,910 --> 00:23:51,520 Along some line here the graph doesn't go up. 397 00:23:51,520 --> 00:23:56,940 It's sitting, can you imagine that sitting in the base? 398 00:23:56,940 --> 00:24:05,870 I'm not Rembrandt here, but in the other direction it goes up. 399 00:24:05,870 --> 00:24:08,240 Oh, the hell with that one. 400 00:24:08,240 --> 00:24:10,650 Do you see, sort of? 401 00:24:10,650 --> 00:24:14,270 It's like a trough, would you say? 402 00:24:14,270 --> 00:24:16,740 I mean, it's like a, you know, a bit 403 00:24:16,740 --> 00:24:19,720 of a drainpipe or something. 404 00:24:19,720 --> 00:24:27,300 It's running along the ground, along this [3, -1] direction 405 00:24:27,300 --> 00:24:30,180 and in the other directions it does go up. 406 00:24:30,180 --> 00:24:35,480 So it's shaped like this with the base not climbing. 407 00:24:35,480 --> 00:24:39,020 Whereas here, there's no bad direction. 408 00:24:39,020 --> 00:24:40,900 Climbs every way you go. 409 00:24:40,900 --> 00:24:45,440 So that's positive definite and that's positive semi-definite. 410 00:24:45,440 --> 00:24:50,770 Well suppose I push it a little further. 411 00:24:50,770 --> 00:24:56,510 Let me make a place here for a matrix that isn't 412 00:24:56,510 --> 00:25:01,350 even positive semi-definite. 413 00:25:01,350 --> 00:25:05,180 Now it's just going to go down somewhere. 414 00:25:05,180 --> 00:25:07,010 I'll start again with one and nine 415 00:25:07,010 --> 00:25:09,850 and tell me what to put in now. 416 00:25:09,850 --> 00:25:13,480 So this is going to be a case where the off-diagonal is 417 00:25:13,480 --> 00:25:15,740 too big, it wins. 418 00:25:15,740 --> 00:25:18,160 And prevents positive definite. 419 00:25:18,160 --> 00:25:21,410 So what number would you like here? 420 00:25:21,410 --> 00:25:22,770 Five? 421 00:25:22,770 --> 00:25:27,960 Five is certainly plenty. 422 00:25:27,960 --> 00:25:30,600 So now I have [1, 5; 5, 9]. 423 00:25:30,600 --> 00:25:37,440 Let me take a little space on a board just to show you. 424 00:25:37,440 --> 00:25:42,190 Sorry about that. 425 00:25:42,190 --> 00:25:44,450 So I'm going to do the [1, 5; 5, 9] 426 00:25:44,450 --> 00:25:46,800 just because they're all important, 427 00:25:46,800 --> 00:25:49,480 but then we're coming back to positive definite. 428 00:25:49,480 --> 00:25:57,510 So if it's [1, 5; 5, 9] and I do that usual x transpose 429 00:25:57,510 --> 00:26:03,890 Kx and I do the multiplication out, I see the one x_1 squared 430 00:26:03,890 --> 00:26:06,750 and I see the nine x_2 squareds. 431 00:26:06,750 --> 00:26:11,690 And how many x_1*x_2's do I see? 432 00:26:11,690 --> 00:26:15,480 Five from there, five from there, ten. 433 00:26:15,480 --> 00:26:20,800 And I believe that can be negative. 434 00:26:20,800 --> 00:26:23,220 The fact of having all nice plus signs 435 00:26:23,220 --> 00:26:25,970 is not going to help it because we can choose, 436 00:26:25,970 --> 00:26:30,030 as we already did, x_1 to be like a negative number and x_2 437 00:26:30,030 --> 00:26:31,350 to be a positive. 438 00:26:31,350 --> 00:26:35,180 And we can get this guy to be negative and make it, 439 00:26:35,180 --> 00:26:41,040 in this case we can make it defeat these positive parts. 440 00:26:41,040 --> 00:26:43,590 What choice would do it? 441 00:26:43,590 --> 00:26:46,560 Let me take x_1 to be minus one and tell me 442 00:26:46,560 --> 00:26:53,760 an x_2 that's good enough to show that this thing is not 443 00:26:53,760 --> 00:26:56,520 positive definite or even semi-definite, 444 00:26:56,520 --> 00:26:58,360 it goes downhill. 445 00:26:58,360 --> 00:26:59,570 Take x_2 equal? 446 00:26:59,570 --> 00:27:04,030 What do you say? 447 00:27:04,030 --> 00:27:05,460 1/2? 448 00:27:05,460 --> 00:27:07,250 Yeah, I don't want too big an x_2 449 00:27:07,250 --> 00:27:10,910 because if I have too big an x_2, then this'll be important. 450 00:27:10,910 --> 00:27:14,550 Does 1/2 do it? 451 00:27:14,550 --> 00:27:18,040 So I've got 1/4, that's positive, but not very. 452 00:27:18,040 --> 00:27:23,610 9/4, so I'm up to 10/4, but this guy is what? 453 00:27:23,610 --> 00:27:26,531 Ten and the minus is minus five. 454 00:27:26,531 --> 00:27:27,030 Yeah. 455 00:27:27,030 --> 00:27:35,370 So that absolutely goes, at this one I come out less than zero. 456 00:27:35,370 --> 00:27:37,170 And I might as well complete. 457 00:27:37,170 --> 00:27:43,890 So this is the case where I would call it indefinite. 458 00:27:43,890 --> 00:27:45,190 Indefinite. 459 00:27:45,190 --> 00:27:49,240 It goes up, like if x_2 is zero, then 460 00:27:49,240 --> 00:27:52,170 it's just got x_1 squared, that's up. 461 00:27:52,170 --> 00:27:55,750 If x_1 is zero, it's only got x_2 squared, that's up. 462 00:27:55,750 --> 00:27:58,890 But there are other directions where it goes downhill. 463 00:27:58,890 --> 00:28:02,100 So it goes either up-- it goes both up in some ways 464 00:28:02,100 --> 00:28:03,160 and down in others. 465 00:28:03,160 --> 00:28:07,760 And what kind of a graph, what kind of a surface 466 00:28:07,760 --> 00:28:12,160 would I now have for x transpose for this x transpose, 467 00:28:12,160 --> 00:28:15,690 this indefinite guy? 468 00:28:15,690 --> 00:28:26,400 So up in some ways and down in others. 469 00:28:26,400 --> 00:28:35,150 This gets really hard to draw. 470 00:28:35,150 --> 00:28:39,890 I believe that if you ride horses 471 00:28:39,890 --> 00:28:43,090 you have an edge on visualizing this. 472 00:28:43,090 --> 00:28:45,760 So it's called, what kind of a point's it called? 473 00:28:45,760 --> 00:28:50,260 Saddle point, it's called a saddle point. 474 00:28:50,260 --> 00:28:53,890 So what's a saddle point? 475 00:28:53,890 --> 00:28:56,460 That's not bad, right? 476 00:28:56,460 --> 00:28:58,650 So this is a direction where it went up. 477 00:28:58,650 --> 00:29:02,210 This is a direction where it went down. 478 00:29:02,210 --> 00:29:09,440 And so it sort of fills in somehow. 479 00:29:09,440 --> 00:29:18,330 Or maybe, if you don't, I mean, who rides horses now? 480 00:29:18,330 --> 00:29:25,190 Actually maybe something we do do is drive over mountains. 481 00:29:25,190 --> 00:29:35,380 So the path, if the road is sort of well-chosen, 482 00:29:35,380 --> 00:29:39,250 the road will go, it'll look for the, 483 00:29:39,250 --> 00:29:43,540 this would be-- Yeah, here's our road. 484 00:29:43,540 --> 00:29:46,310 We would do as little climbing as possible. 485 00:29:46,310 --> 00:29:48,860 The mountain would go like this, sort of. 486 00:29:48,860 --> 00:29:52,320 So this would be like, the bottom part 487 00:29:52,320 --> 00:29:55,630 looking along the peaks of the mountains. 488 00:29:55,630 --> 00:29:59,410 But it's the top part looking along the driving direction. 489 00:29:59,410 --> 00:30:05,590 So driving, it's a maximum, but in the mountain range direction 490 00:30:05,590 --> 00:30:06,320 it's a minimum. 491 00:30:06,320 --> 00:30:10,500 So it's a saddle point. 492 00:30:10,500 --> 00:30:14,810 So that's what you get from a typical symmetric matrix. 493 00:30:14,810 --> 00:30:19,010 And if it was minus five it would still be the same saddle 494 00:30:19,010 --> 00:30:23,480 point, would still be 9-25, it would still 495 00:30:23,480 --> 00:30:27,130 be negative and a saddle. 496 00:30:27,130 --> 00:30:30,200 Positive guys are our thing. 497 00:30:30,200 --> 00:30:32,000 Alright. 498 00:30:32,000 --> 00:30:36,230 So now back to positive definite. 499 00:30:36,230 --> 00:30:40,650 With these four tests and then the discussion 500 00:30:40,650 --> 00:30:45,680 of semi-definite. 501 00:30:45,680 --> 00:30:49,150 Very key, that energy. 502 00:30:49,150 --> 00:30:51,970 Let me just look ahead a moment. 503 00:30:51,970 --> 00:30:58,650 Most physical problems, many, many physical problems, 504 00:30:58,650 --> 00:31:00,010 you have an option. 505 00:31:00,010 --> 00:31:04,810 Either you solve some equations, either you find the solution 506 00:31:04,810 --> 00:31:10,040 from our equations, Ku=f, typically. 507 00:31:10,040 --> 00:31:12,670 Matrix equation or differential equation. 508 00:31:12,670 --> 00:31:20,940 Or there's another option of minimizing some function. 509 00:31:20,940 --> 00:31:23,640 Some energy. 510 00:31:23,640 --> 00:31:25,850 And it gives the same equations. 511 00:31:25,850 --> 00:31:30,680 So this minimizing energy will be a second way 512 00:31:30,680 --> 00:31:36,720 to describe the applications. 513 00:31:36,720 --> 00:31:40,760 Now can I get a number five? 514 00:31:40,760 --> 00:31:42,660 There's an important number five and then 515 00:31:42,660 --> 00:31:48,130 you know really all you need to know about symmetric matrices. 516 00:31:48,130 --> 00:31:51,630 This gives me, about positive definite matrices, 517 00:31:51,630 --> 00:32:01,750 this gives me a chance to recap. 518 00:32:01,750 --> 00:32:06,400 So I'm going to put down a number five. 519 00:32:06,400 --> 00:32:16,540 Because this is where the matrices come from. 520 00:32:16,540 --> 00:32:17,480 Really important. 521 00:32:17,480 --> 00:32:20,390 And it's where they'll come from in all these applications 522 00:32:20,390 --> 00:32:23,230 that chapter two is going to be all about, that we're 523 00:32:23,230 --> 00:32:25,600 going to start. 524 00:32:25,600 --> 00:32:28,500 So they come, these positive definite matrices, 525 00:32:28,500 --> 00:32:34,290 so this is another way to, it's a test 526 00:32:34,290 --> 00:32:39,180 for positive definite matrices and it's, actually, it's 527 00:32:39,180 --> 00:32:40,930 where they come from. 528 00:32:40,930 --> 00:32:44,340 So here's a positive definite matrix. 529 00:32:44,340 --> 00:32:54,270 They come from A transpose A. A fundamental message is 530 00:32:54,270 --> 00:32:57,090 that if I have just an average matrix, 531 00:32:57,090 --> 00:33:01,840 possibly rectangular, could be a square but not symmetric, 532 00:33:01,840 --> 00:33:07,220 then sooner or later, in fact usually sooner, 533 00:33:07,220 --> 00:33:10,430 you end up looking at A transpose A. 534 00:33:10,430 --> 00:33:11,940 We've seen that already. 535 00:33:11,940 --> 00:33:15,390 And we already know that A transpose A is square, 536 00:33:15,390 --> 00:33:17,950 we already know it's symmetric and now 537 00:33:17,950 --> 00:33:20,980 we're going to know that it's positive definite. 538 00:33:20,980 --> 00:33:25,140 So matrices like A transpose A are positive definite 539 00:33:25,140 --> 00:33:28,140 or possibly semi-definite. 540 00:33:28,140 --> 00:33:29,660 There's that possibility. 541 00:33:29,660 --> 00:33:31,920 If A was the zero matrix, of course, 542 00:33:31,920 --> 00:33:33,820 we would just get the zero matrix which 543 00:33:33,820 --> 00:33:37,460 would be only semi-definite, or other ways 544 00:33:37,460 --> 00:33:42,120 to get a semi-definite. 545 00:33:42,120 --> 00:33:46,720 So I'm saying that if K, if I have a matrix, any matrix, 546 00:33:46,720 --> 00:33:50,770 and I form A transpose A, I get a positive definite matrix 547 00:33:50,770 --> 00:33:56,780 or maybe just semi-definite, but not indefinite. 548 00:33:56,780 --> 00:34:01,850 Can we see why? 549 00:34:01,850 --> 00:34:11,900 Why is this positive definite or semi-? 550 00:34:11,900 --> 00:34:13,740 So that's my question. 551 00:34:13,740 --> 00:34:16,490 And the answer is really worth-- it's 552 00:34:16,490 --> 00:34:19,070 just neat and worth seeing. 553 00:34:19,070 --> 00:34:23,970 So do I want to look at the pivots of A transpose A? 554 00:34:23,970 --> 00:34:25,970 No. 555 00:34:25,970 --> 00:34:27,910 They're something, but whatever they are, 556 00:34:27,910 --> 00:34:30,460 I can't really follow those well. 557 00:34:30,460 --> 00:34:34,870 Or the eigenvalues very well, or the determinants. 558 00:34:34,870 --> 00:34:36,790 None of those come out nicely. 559 00:34:36,790 --> 00:34:41,690 But the real guy works perfectly. 560 00:34:41,690 --> 00:34:46,060 So look at x transpose Kx. 561 00:34:46,060 --> 00:34:48,580 562 00:34:48,580 --> 00:34:57,190 So I'm just doing-- following my instinct here. 563 00:34:57,190 --> 00:35:00,590 So if K is A transpose A, my claim 564 00:35:00,590 --> 00:35:07,050 is, what am I saying then about this energy? 565 00:35:07,050 --> 00:35:13,090 What is it that I want to discover and understand? 566 00:35:13,090 --> 00:35:15,630 Why it's positive. 567 00:35:15,630 --> 00:35:18,820 Why does taking any matrix, multiplying 568 00:35:18,820 --> 00:35:27,670 by its transpose produce something that's positive? 569 00:35:27,670 --> 00:35:30,520 Can you see any reason why that quantity, 570 00:35:30,520 --> 00:35:33,930 which looks kind of messy, I just 571 00:35:33,930 --> 00:35:37,530 want to look at it the right way to see 572 00:35:37,530 --> 00:35:41,510 why that should be positive, that should come out positive. 573 00:35:41,510 --> 00:35:44,820 So I'm not going to get into numbers, 574 00:35:44,820 --> 00:35:47,640 I'm not going to get into diagonals and off-diagonals. 575 00:35:47,640 --> 00:35:52,260 I'm just going to do one thing to understand 576 00:35:52,260 --> 00:35:57,990 that particular combination, x transpose A transpose Ax. 577 00:35:57,990 --> 00:35:59,870 What shall I do? 578 00:35:59,870 --> 00:36:06,130 Anybody see what I might do? 579 00:36:06,130 --> 00:36:10,190 Yeah, you're seeing here if you look at it again, 580 00:36:10,190 --> 00:36:12,210 what are you seeing here? 581 00:36:12,210 --> 00:36:14,220 Tell me again. 582 00:36:14,220 --> 00:36:20,500 If I take Ax together, then what's the other half? 583 00:36:20,500 --> 00:36:23,090 It's the transpose of Ax. 584 00:36:23,090 --> 00:36:25,900 So I just want to write that as, I just want 585 00:36:25,900 --> 00:36:29,000 to think of it that way, as Ax. 586 00:36:29,000 --> 00:36:32,270 And here's the transpose of Ax. 587 00:36:32,270 --> 00:36:33,040 Right? 588 00:36:33,040 --> 00:36:36,050 Because transposes of Ax, so transpose 589 00:36:36,050 --> 00:36:39,890 guys in the opposite order, and the multiplication-- 590 00:36:39,890 --> 00:36:41,230 This is the great. 591 00:36:41,230 --> 00:36:44,380 I call these proof by parenthesis 592 00:36:44,380 --> 00:36:47,150 because I'm just putting parentheses in the right place, 593 00:36:47,150 --> 00:36:53,940 but the key law of matrix multiplication 594 00:36:53,940 --> 00:36:58,920 is that, that I can put (AB)C is the same as A(BC). 595 00:36:58,920 --> 00:37:01,810 596 00:37:01,810 --> 00:37:04,170 That rule, which is just multiply it out 597 00:37:04,170 --> 00:37:06,150 and you see that parentheses are not 598 00:37:06,150 --> 00:37:08,430 needed because if you keep them in the right order 599 00:37:08,430 --> 00:37:12,300 you can do this first, or you can do this first. 600 00:37:12,300 --> 00:37:13,720 Same answer. 601 00:37:13,720 --> 00:37:15,480 What do I learn from that? 602 00:37:15,480 --> 00:37:17,020 What was the point? 603 00:37:17,020 --> 00:37:20,510 This is some vector, I don't know especially what it is, 604 00:37:20,510 --> 00:37:21,900 times its transpose. 605 00:37:21,900 --> 00:37:24,910 So that's the length squared. 606 00:37:24,910 --> 00:37:27,210 What's the key fact about that? 607 00:37:27,210 --> 00:37:30,000 That it is never negative. 608 00:37:30,000 --> 00:37:41,070 It's always greater than zero or possibly equal. 609 00:37:41,070 --> 00:37:44,280 When does that quantity equal zero? 610 00:37:44,280 --> 00:37:45,580 When Ax is zero. 611 00:37:45,580 --> 00:37:47,060 When Ax is zero. 612 00:37:47,060 --> 00:37:49,050 Because this is a vector. 613 00:37:49,050 --> 00:37:50,830 That's the same vector transposed. 614 00:37:50,830 --> 00:37:52,500 And everybody's got that picture. 615 00:37:52,500 --> 00:37:56,970 When I take any y transpose y, I get 616 00:37:56,970 --> 00:38:00,440 y_1 squared plus y_2 squared through y_n squared. 617 00:38:00,440 --> 00:38:05,450 And I get a positive answer except if the vector is zero. 618 00:38:05,450 --> 00:38:11,720 So it's zero when Ax is zero. 619 00:38:11,720 --> 00:38:13,990 So that's going to be the key. 620 00:38:13,990 --> 00:38:16,680 If I pick any matrix A, and I can just 621 00:38:16,680 --> 00:38:21,670 take an example, but chapter-- the applications 622 00:38:21,670 --> 00:38:23,730 are just going to be full of examples. 623 00:38:23,730 --> 00:38:29,110 Where the problem begins with a matrix A and then 624 00:38:29,110 --> 00:38:34,960 A transpose shows up and it's the combination A transpose A 625 00:38:34,960 --> 00:38:36,100 that we work with. 626 00:38:36,100 --> 00:38:40,150 And we're just learning that it's positive definite. 627 00:38:40,150 --> 00:38:46,440 Unless, shall I just hang on since I've got here, 628 00:38:46,440 --> 00:38:53,470 I have to say when is it, have to get these two possibilities. 629 00:38:53,470 --> 00:38:56,950 Positive definite or only semi-definite. 630 00:38:56,950 --> 00:39:05,820 So what's the key to that borderline question? 631 00:39:05,820 --> 00:39:09,680 This thing will be only semi-definite 632 00:39:09,680 --> 00:39:12,270 if there's a solution to Ax=0. 633 00:39:12,270 --> 00:39:16,000 634 00:39:16,000 --> 00:39:23,560 If there is an x, well, there's always the zero vector. 635 00:39:23,560 --> 00:39:26,730 Zero vector I can't expect to be positive. 636 00:39:26,730 --> 00:39:31,300 So I'm looking for if there's an x so 637 00:39:31,300 --> 00:39:42,130 that Ax is zero but x is not zero, 638 00:39:42,130 --> 00:39:48,420 then I'll only be semi-definite. 639 00:39:48,420 --> 00:39:50,320 That's the test. 640 00:39:50,320 --> 00:39:52,730 If there is a solution to Ax=0. 641 00:39:52,730 --> 00:39:55,420 642 00:39:55,420 --> 00:39:58,320 When we see applications that'll mean 643 00:39:58,320 --> 00:40:03,110 there's a displacement with no stretching. 644 00:40:03,110 --> 00:40:09,580 We might have a line of springs and when 645 00:40:09,580 --> 00:40:16,570 could the line of springs displace with no stretching? 646 00:40:16,570 --> 00:40:18,390 When it's free-free, right? 647 00:40:18,390 --> 00:40:24,010 If I have a line of springs and no supports at the ends, 648 00:40:24,010 --> 00:40:26,860 then that would be the case where it could shift over 649 00:40:26,860 --> 00:40:29,350 by the [1, 1, 1] vector. 650 00:40:29,350 --> 00:40:33,230 So that would be the case where the matrix is only singular. 651 00:40:33,230 --> 00:40:34,410 We know that. 652 00:40:34,410 --> 00:40:37,440 The matrix is now positive semi-definite. 653 00:40:37,440 --> 00:40:38,880 We just learned that. 654 00:40:38,880 --> 00:40:43,330 So the free-free matrix, like B, both 655 00:40:43,330 --> 00:40:48,690 ends free, or C. So our answer is 656 00:40:48,690 --> 00:40:56,170 going to be that K and T are positive definite. 657 00:40:56,170 --> 00:40:59,470 And our other two guys, the singular ones, of course, 658 00:40:59,470 --> 00:41:00,590 just don't make it. 659 00:41:00,590 --> 00:41:04,000 B at both ends, the free-free line of springs, 660 00:41:04,000 --> 00:41:07,040 it can shift without stretching. 661 00:41:07,040 --> 00:41:10,810 Since Ax will measure the stretching when it just 662 00:41:10,810 --> 00:41:13,630 shifts rigid motion, the Ax is zero 663 00:41:13,630 --> 00:41:16,110 and we see only positive definite. 664 00:41:16,110 --> 00:41:19,030 And also C, the circular one. 665 00:41:19,030 --> 00:41:21,460 There it can displace with no stretching 666 00:41:21,460 --> 00:41:24,360 because it can just turn in the circle. 667 00:41:24,360 --> 00:41:45,220 So these guys will be only positive semi-definite. 668 00:41:45,220 --> 00:41:49,340 Maybe I better say this another way. 669 00:41:49,340 --> 00:41:51,790 When is this positive definite? 670 00:41:51,790 --> 00:41:54,930 Can I use just a different sentence 671 00:41:54,930 --> 00:41:57,270 to describe this possibility? 672 00:41:57,270 --> 00:42:03,830 This is positive definite provided, 673 00:42:03,830 --> 00:42:08,270 so what I'm going to write now is to remove this possibility 674 00:42:08,270 --> 00:42:10,140 and get positive definite. 675 00:42:10,140 --> 00:42:16,010 This is positive definite provided, now, 676 00:42:16,010 --> 00:42:17,490 I could say it this way. 677 00:42:17,490 --> 00:42:25,130 The A has independent columns. 678 00:42:25,130 --> 00:42:29,080 So I just needed to give you another way of looking at this 679 00:42:29,080 --> 00:42:33,400 Ax=0 question. 680 00:42:33,400 --> 00:42:37,220 If A has independent columns, what does that mean? 681 00:42:37,220 --> 00:42:40,920 That means that the only solution to Ax=0 is the zero 682 00:42:40,920 --> 00:42:42,800 solution. 683 00:42:42,800 --> 00:42:47,640 In other words, it means that this thing works perfectly 684 00:42:47,640 --> 00:42:50,520 and gives me positive. 685 00:42:50,520 --> 00:42:53,080 When A has independent columns. 686 00:42:53,080 --> 00:43:05,470 Let's just remember our K, T, B, C. So here's a matrix, 687 00:43:05,470 --> 00:43:11,150 so let me take the T matrix, that's this one, this guy. 688 00:43:11,150 --> 00:43:15,600 And then the third column is [0, -1, 2]. 689 00:43:15,600 --> 00:43:19,960 Those three columns are independent. 690 00:43:19,960 --> 00:43:21,250 They point off. 691 00:43:21,250 --> 00:43:23,110 They don't lie in a plane. 692 00:43:23,110 --> 00:43:27,140 They point off in three different directions. 693 00:43:27,140 --> 00:43:34,820 And then there are no solutions to, no x's that go Kx=0. 694 00:43:34,820 --> 00:43:39,140 695 00:43:39,140 --> 00:43:41,840 So that would be a case of independent columns. 696 00:43:41,840 --> 00:43:45,000 Let me make a case of dependent columns. 697 00:43:45,000 --> 00:43:47,730 So, and I'm going to make it B now. 698 00:43:47,730 --> 00:43:51,750 Now the columns of that matrix are dependent. 699 00:43:51,750 --> 00:43:54,280 There's a combination of them that give zero. 700 00:43:54,280 --> 00:43:56,530 They all lie in the same plane. 701 00:43:56,530 --> 00:44:00,210 There's a solution to that matrix times x equal zero. 702 00:44:00,210 --> 00:44:02,440 What combination of those columns 703 00:44:02,440 --> 00:44:05,850 shows me that they are dependent? 704 00:44:05,850 --> 00:44:09,000 That some combination of those three columns, 705 00:44:09,000 --> 00:44:10,880 some amount of this plus some amount 706 00:44:10,880 --> 00:44:12,630 of this plus some amount of that column 707 00:44:12,630 --> 00:44:15,350 gives me the zero vector. 708 00:44:15,350 --> 00:44:17,600 You see the combination. 709 00:44:17,600 --> 00:44:21,550 What should I take? [1, 1, 1] again. 710 00:44:21,550 --> 00:44:22,300 No surprise. 711 00:44:22,300 --> 00:44:26,050 That's the vector [1, 1, 1] that we 712 00:44:26,050 --> 00:44:29,840 know is in the-- everything shifting the same amount, 713 00:44:29,840 --> 00:44:36,680 nothing stretching. 714 00:44:36,680 --> 00:44:40,430 Talking fast here about positive definite matrices. 715 00:44:40,430 --> 00:44:42,360 This is the key. 716 00:44:42,360 --> 00:44:44,980 Let's just ask a few questions about positive definite 717 00:44:44,980 --> 00:44:49,510 matrices as a way to practice. 718 00:44:49,510 --> 00:44:50,940 Suppose I had one. 719 00:44:50,940 --> 00:44:52,560 Positive definite. 720 00:44:52,560 --> 00:44:57,390 What about its inverse? 721 00:44:57,390 --> 00:45:02,340 Is that positive definite or not? 722 00:45:02,340 --> 00:45:06,160 So I've got a positive definite one, it's not singular, 723 00:45:06,160 --> 00:45:09,990 it's got positive eigenvalues, everything else. 724 00:45:09,990 --> 00:45:14,240 It's inverse will be symmetric, so I'm 725 00:45:14,240 --> 00:45:16,330 allowed to think about it. 726 00:45:16,330 --> 00:45:20,530 Will it be positive definite? 727 00:45:20,530 --> 00:45:23,670 What do you think? 728 00:45:23,670 --> 00:45:27,000 Well, you've got a whole bunch of tests 729 00:45:27,000 --> 00:45:30,090 to sort of mentally run through. 730 00:45:30,090 --> 00:45:35,580 Pivots of the inverse, you don't want to touch that stuff. 731 00:45:35,580 --> 00:45:36,690 Determinants, no. 732 00:45:36,690 --> 00:45:39,230 What about eigenvalues? 733 00:45:39,230 --> 00:45:41,090 What would be the eigenvalues if I 734 00:45:41,090 --> 00:45:43,640 have this positive definite symmetric matrix, 735 00:45:43,640 --> 00:45:46,800 its eigenvalues are one, four, five. 736 00:45:46,800 --> 00:45:49,700 What can you tell me about the eigenvalues 737 00:45:49,700 --> 00:45:53,030 of the inverse matrix? 738 00:45:53,030 --> 00:45:54,080 They're the inverses. 739 00:45:54,080 --> 00:45:56,690 So those three eigenvalues are? 740 00:45:56,690 --> 00:46:00,890 1, 1/4, 1/5, what's the conclusion here? 741 00:46:00,890 --> 00:46:02,080 It is positive definite. 742 00:46:02,080 --> 00:46:04,430 Those are all positive, it is positive definite. 743 00:46:04,430 --> 00:46:07,940 So if I invert a positive definite matrix, 744 00:46:07,940 --> 00:46:11,350 I'm still positive definite. 745 00:46:11,350 --> 00:46:13,660 All the tests would have to pass. 746 00:46:13,660 --> 00:46:17,720 It's just I'm looking each time for the easiest test. 747 00:46:17,720 --> 00:46:22,380 Let me look now, for the easiest test on K_1+K_2. 748 00:46:22,380 --> 00:46:25,470 749 00:46:25,470 --> 00:46:29,980 Suppose that's positive definite and that's positive definite. 750 00:46:29,980 --> 00:46:33,670 What if I add them? 751 00:46:33,670 --> 00:46:35,940 What do you think? 752 00:46:35,940 --> 00:46:38,680 Well, we hope so. 753 00:46:38,680 --> 00:46:42,840 But we have to say which of my one, two, three, four, five 754 00:46:42,840 --> 00:46:45,660 would be a good way to see it. 755 00:46:45,660 --> 00:46:48,300 Would be a good way to see it. 756 00:46:48,300 --> 00:46:50,830 Good question. 757 00:46:50,830 --> 00:46:53,560 Four? 758 00:46:53,560 --> 00:46:55,280 We certainly don't want to touch pivots 759 00:46:55,280 --> 00:46:58,860 and now we don't want to touch eigenvalues either. 760 00:46:58,860 --> 00:47:03,400 Of course, if number four works, others will also work. 761 00:47:03,400 --> 00:47:05,650 The eigenvalues will come out positive. 762 00:47:05,650 --> 00:47:08,170 But not too easy to say what they are. 763 00:47:08,170 --> 00:47:14,450 Let's try test number four. 764 00:47:14,450 --> 00:47:15,060 So K_1. 765 00:47:15,060 --> 00:47:18,760 766 00:47:18,760 --> 00:47:20,020 What's the test? 767 00:47:20,020 --> 00:47:23,430 So test number four tells us that this part, 768 00:47:23,430 --> 00:47:28,670 x transpose K_1*x, that that part is positive, right? 769 00:47:28,670 --> 00:47:30,900 That that part is positive. 770 00:47:30,900 --> 00:47:33,040 If we know that's positive definite. 771 00:47:33,040 --> 00:47:37,030 Now, about K_2 we also know that for every x, you see it's 772 00:47:37,030 --> 00:47:38,720 for every x, that helps. 773 00:47:38,720 --> 00:47:40,900 Don't let me put x_2 there. 774 00:47:40,900 --> 00:47:47,670 For every x, this will be positive. 775 00:47:47,670 --> 00:47:52,670 And now what's the step I want to take? 776 00:47:52,670 --> 00:47:57,030 To get some information on the matrix K_1+K_2. 777 00:47:57,030 --> 00:47:59,590 778 00:47:59,590 --> 00:48:01,280 I should add. 779 00:48:01,280 --> 00:48:06,330 If I add these guys, you see that it just, then I 780 00:48:06,330 --> 00:48:14,160 can write that as, I can write that this way. 781 00:48:14,160 --> 00:48:17,650 And what have I learned? 782 00:48:17,650 --> 00:48:19,890 I've learned that that's positive, even greater than, 783 00:48:19,890 --> 00:48:21,680 except for the zero vector. 784 00:48:21,680 --> 00:48:23,930 Because this was greater than, this is greater than. 785 00:48:23,930 --> 00:48:27,170 If I add two positive numbers, the energies are positive 786 00:48:27,170 --> 00:48:29,380 and the energies just add. 787 00:48:29,380 --> 00:48:34,660 The energies just add. 788 00:48:34,660 --> 00:48:40,170 So that definition four was the good way, just nice, easy way 789 00:48:40,170 --> 00:48:44,960 to see that if I have a couple of positive definite matrices, 790 00:48:44,960 --> 00:48:47,020 a couple of positive energies, I'm really 791 00:48:47,020 --> 00:48:49,570 coupling the two systems. 792 00:48:49,570 --> 00:48:53,030 This is associated somehow. 793 00:48:53,030 --> 00:48:55,190 I've got two systems, I'm putting them together 794 00:48:55,190 --> 00:49:00,260 and the energy is just even more positive. 795 00:49:00,260 --> 00:49:05,860 It's more positive either of these guys because I'm adding. 796 00:49:05,860 --> 00:49:09,070 As I'm speaking here, will you allow 797 00:49:09,070 --> 00:49:14,870 me to try test number five, this A transpose A business? 798 00:49:14,870 --> 00:49:21,271 Suppose K_1 was A transpose A. If it's positive definite, 799 00:49:21,271 --> 00:49:21,770 it will. 800 00:49:21,770 --> 00:49:31,190 Be And suppose K_2 is B transpose B. 801 00:49:31,190 --> 00:49:33,280 If it's positive definite, it will be. 802 00:49:33,280 --> 00:49:41,390 Now I would like to write the sum somehow as, 803 00:49:41,390 --> 00:49:43,990 in this something transpose something. 804 00:49:43,990 --> 00:49:46,220 And I just do it now because I think 805 00:49:46,220 --> 00:49:50,020 it's like, you won't perhaps have thought of this way 806 00:49:50,020 --> 00:49:54,000 to do it. 807 00:49:54,000 --> 00:49:56,070 Watch. 808 00:49:56,070 --> 00:50:01,830 Suppose I create the matrix [A; B]. 809 00:50:01,830 --> 00:50:03,210 That'll be my new matrix. 810 00:50:03,210 --> 00:50:11,100 Say, call it C. Am I allowed to do that? 811 00:50:11,100 --> 00:50:13,210 I mean, that creates a matrix? 812 00:50:13,210 --> 00:50:18,110 These A and B, they had the same number of columns, n. 813 00:50:18,110 --> 00:50:20,300 So I can put one over the other and I still 814 00:50:20,300 --> 00:50:22,370 have something with n columns. 815 00:50:22,370 --> 00:50:26,800 So that's my new matrix C. And now I want C transpose. 816 00:50:26,800 --> 00:50:31,490 By the way, I'd call that a block matrix. 817 00:50:31,490 --> 00:50:35,990 You know, instead of numbers, it's got two blocks in there. 818 00:50:35,990 --> 00:50:37,980 Block matrices are really handy. 819 00:50:37,980 --> 00:50:43,310 Now what's the transpose of that block matrix? 820 00:50:43,310 --> 00:50:47,490 You just have faith, just have faith with blocks. 821 00:50:47,490 --> 00:50:48,880 It's just like numbers. 822 00:50:48,880 --> 00:50:55,970 If I had a matrix [1; 5] then I'd get a row one, five. 823 00:50:55,970 --> 00:50:57,810 But what do you think? 824 00:50:57,810 --> 00:51:01,270 This is worth thinking about even after class. 825 00:51:01,270 --> 00:51:05,470 What would be, if this C matrix is this block A above B, 826 00:51:05,470 --> 00:51:07,870 what do you think for C transpose? 827 00:51:07,870 --> 00:51:11,510 A transpose, B transpose side by side. 828 00:51:11,510 --> 00:51:15,720 Just put in numbers and you'd see it. 829 00:51:15,720 --> 00:51:17,990 And now I'm going to take C transpose 830 00:51:17,990 --> 00:51:23,880 times C. I'm calling it C now instead of A 831 00:51:23,880 --> 00:51:26,360 because I've used the A in the first guy 832 00:51:26,360 --> 00:51:31,780 and I've used B in the second one and now I'm ready for C. 833 00:51:31,780 --> 00:51:35,490 How do you multiply block matrices? 834 00:51:35,490 --> 00:51:37,980 Again, you just have faith. 835 00:51:37,980 --> 00:51:39,750 What do you think? 836 00:51:39,750 --> 00:51:41,820 Tell me the answer. 837 00:51:41,820 --> 00:51:43,880 A transpose, I multiply that by that 838 00:51:43,880 --> 00:51:47,180 just as if they were numbers. 839 00:51:47,180 --> 00:51:52,020 And I add that times that just as if they were numbers. 840 00:51:52,020 --> 00:51:55,072 And what do I have? 841 00:51:55,072 --> 00:51:55,780 I've got K_1+K_2. 842 00:51:55,780 --> 00:51:58,600 843 00:51:58,600 --> 00:52:05,000 So I've written K_1, this is K_1+K_2 and this is in my form 844 00:52:05,000 --> 00:52:09,080 C transpose C that I was looking for, that number five was 845 00:52:09,080 --> 00:52:10,860 looking for. 846 00:52:10,860 --> 00:52:12,920 So it's done it. 847 00:52:12,920 --> 00:52:13,660 It's done it. 848 00:52:13,660 --> 00:52:19,310 The fact of getting A-- K_1 in this form, K_2 in this form. 849 00:52:19,310 --> 00:52:21,720 And I just made a block matrix and I got K_1+K_2. 850 00:52:21,720 --> 00:52:25,730 851 00:52:25,730 --> 00:52:29,730 That's not a big deal in itself, but block matrices 852 00:52:29,730 --> 00:52:32,060 are really handy. 853 00:52:32,060 --> 00:52:36,260 It's good to take that step with matrices. 854 00:52:36,260 --> 00:52:39,930 Think of, possibly, the entries as coming in blocks 855 00:52:39,930 --> 00:52:42,600 and not just one at a time. 856 00:52:42,600 --> 00:52:44,330 Well, thank you, okay. 857 00:52:44,330 --> 00:52:50,700 I swear Friday we'll start applications 858 00:52:50,700 --> 00:52:53,000 in all kinds of engineering problems 859 00:52:53,000 --> 00:52:55,277 and you'll have new applications. 860 00:52:55,277 --> 00:52:55,777