1 00:00:05,560 --> 00:00:14,460 OK, this is the lecture on positive definite matrices. 2 00:00:14,460 --> 00:00:20,770 I made a start on those briefly in a previous lecture. 3 00:00:20,770 --> 00:00:26,180 One point I wanted to make was the way that this topic brings 4 00:00:26,180 --> 00:00:30,300 the whole course together, pivots, determinants, 5 00:00:30,300 --> 00:00:36,190 eigenvalues, and something new- four plot instability 6 00:00:36,190 --> 00:00:42,050 and then something new in this expression, x transpose Ax, 7 00:00:42,050 --> 00:00:46,040 actually that's the guy to watch in this lecture. 8 00:00:46,040 --> 00:00:50,410 So, so the topic is positive definite matrix, 9 00:00:50,410 --> 00:00:53,580 and what's my goal? 10 00:00:53,580 --> 00:00:58,440 First, first goal is, how can I tell if a matrix is 11 00:00:58,440 --> 00:00:59,970 positive definite? 12 00:00:59,970 --> 00:01:02,490 So I would like to have tests to see 13 00:01:02,490 --> 00:01:06,100 if you give me a, a five by five matrix, 14 00:01:06,100 --> 00:01:09,000 how do I tell if it's positive definite? 15 00:01:09,000 --> 00:01:12,020 More important is, what does it mean? 16 00:01:12,020 --> 00:01:14,700 Why are we so interested in this property 17 00:01:14,700 --> 00:01:16,440 of positive definiteness? 18 00:01:16,440 --> 00:01:21,410 And then, at the end comes some geometry. 19 00:01:21,410 --> 00:01:25,080 Ellipses are connected with positive definite things. 20 00:01:25,080 --> 00:01:28,520 Hyperbolas are not connected with positive definite things, 21 00:01:28,520 --> 00:01:33,350 so we- it's this, we, there's a geometry too, 22 00:01:33,350 --> 00:01:38,410 but mostly it's linear algebra and -- 23 00:01:38,410 --> 00:01:43,020 this application of how do you recognize 'em when you have 24 00:01:43,020 --> 00:01:45,181 a minim is pretty neat. 25 00:01:45,181 --> 00:01:45,680 OK. 26 00:01:45,680 --> 00:01:50,090 I'm gonna begin with two by two. 27 00:01:50,090 --> 00:01:52,280 All matrices are symmetric, right? 28 00:01:52,280 --> 00:01:55,180 That's understood; the matrix is symmetric, 29 00:01:55,180 --> 00:02:00,650 now my question is, is it positive definite? 30 00:02:00,650 --> 00:02:04,030 Now, here are some -- 31 00:02:04,030 --> 00:02:09,570 each one of these is a complete test for positive definiteness. 32 00:02:09,570 --> 00:02:15,010 If I know the eigenvalues, my test is are they positive? 33 00:02:15,010 --> 00:02:18,140 Are they all positive? 34 00:02:18,140 --> 00:02:21,650 If I know these -- 35 00:02:21,650 --> 00:02:24,170 so, A is really -- 36 00:02:24,170 --> 00:02:27,570 I look at that number A, here, as the, as the one 37 00:02:27,570 --> 00:02:33,420 by one determinant, and here's the two by two determinant. 38 00:02:33,420 --> 00:02:35,510 So this is the determinant test. 39 00:02:35,510 --> 00:02:39,280 This is the eigenvalue test, this is the determinant test. 40 00:02:39,280 --> 00:02:46,040 Are the determinants growing in s- of all, of all end, sort of, 41 00:02:46,040 --> 00:02:48,640 can I call them leading submatrices, 42 00:02:48,640 --> 00:02:52,140 they're the first ones the northwest, 43 00:02:52,140 --> 00:02:56,510 Seattle submatrices coming down from from there, 44 00:02:56,510 --> 00:02:59,570 they all, all those determinants have to be positive, 45 00:02:59,570 --> 00:03:03,020 and then another test is the pivots. 46 00:03:03,020 --> 00:03:06,370 The pivots of a two by two matrix 47 00:03:06,370 --> 00:03:13,070 are the number A for sure, and, since the product 48 00:03:13,070 --> 00:03:15,410 is the determinant, the second pivot 49 00:03:15,410 --> 00:03:18,980 must be the determinant divided by A. 50 00:03:18,980 --> 00:03:24,970 And then in here is gonna come my favorite and my new idea, 51 00:03:24,970 --> 00:03:30,450 the, the, the the one to catch, about x transpose Ax 52 00:03:30,450 --> 00:03:32,640 being positive. 53 00:03:32,640 --> 00:03:35,280 But we'll have to look at this guy. 54 00:03:35,280 --> 00:03:42,780 This gets, like a star, because for most, presentations, 55 00:03:42,780 --> 00:03:45,610 the definition of positive definiteness 56 00:03:45,610 --> 00:03:49,370 would be this number four and these numbers one two three 57 00:03:49,370 --> 00:03:51,260 would be test four. 58 00:03:51,260 --> 00:03:51,760 OK. 59 00:03:51,760 --> 00:03:56,160 Maybe I'll tuck this, where, you know, 60 00:03:56,160 --> 00:03:56,660 OK. 61 00:03:56,660 --> 00:04:00,000 So I'll have to look at this x transpose Ax. 62 00:04:00,000 --> 00:04:07,760 Can you, can we just be sure, how 63 00:04:07,760 --> 00:04:13,130 do we know that the eigenvalue test and the determinant test, 64 00:04:13,130 --> 00:04:17,170 pick out the same matrices, and let me, 65 00:04:17,170 --> 00:04:19,797 let's just do a few examples. 66 00:04:19,797 --> 00:04:20,380 Some examples. 67 00:04:24,610 --> 00:04:31,220 Let me pick the matrix two, six, six, tell me, 68 00:04:31,220 --> 00:04:36,360 what number do I have to put there for the matrix 69 00:04:36,360 --> 00:04:37,440 to be positive definite? 70 00:04:40,150 --> 00:04:42,090 Tell me a sufficiently large number 71 00:04:42,090 --> 00:04:45,020 that would make it positive definite? 72 00:04:45,020 --> 00:04:47,830 Let's just practice with these conditions in the two 73 00:04:47,830 --> 00:04:49,370 by two case. 74 00:04:49,370 --> 00:04:51,490 Now, when I ask you that, you don't 75 00:04:51,490 --> 00:04:56,270 wanna find the eigenvalues, you would use the determinant test 76 00:04:56,270 --> 00:05:00,340 for that, so, the first or the pivot test, 77 00:05:00,340 --> 00:05:03,750 that, that guy is certainly positive, that had to happen, 78 00:05:03,750 --> 00:05:05,070 and it's OK. 79 00:05:05,070 --> 00:05:09,030 How large a number here -- the number had better be more than 80 00:05:09,030 --> 00:05:10,200 what? 81 00:05:10,200 --> 00:05:14,350 More than eighteen, right, because if it's eight -- 82 00:05:14,350 --> 00:05:15,400 no. 83 00:05:15,400 --> 00:05:18,550 More than what? 84 00:05:18,550 --> 00:05:21,390 Nineteen, is it? 85 00:05:21,390 --> 00:05:28,965 If I have a nineteen here, is that positive definite? 86 00:05:32,590 --> 00:05:36,160 I get thirty eight minus thirty six, that's OK. 87 00:05:36,160 --> 00:05:39,900 If I had an eighteen, let me play it really close. 88 00:05:39,900 --> 00:05:44,850 If I have an eighteen there, then I positive definite? 89 00:05:44,850 --> 00:05:46,640 Not quite. 90 00:05:46,640 --> 00:05:49,830 I would call this guy positive, so it's 91 00:05:49,830 --> 00:05:53,550 useful just to see that this the borderline. 92 00:05:53,550 --> 00:05:55,640 That matrix is on the borderline, 93 00:05:55,640 --> 00:05:58,880 I would call that matrix positive semi-definite. 94 00:06:04,670 --> 00:06:07,900 And what are the eigenvalues of that matrix, 95 00:06:07,900 --> 00:06:11,690 just since we're given eigenvalues of two by twos, 96 00:06:11,690 --> 00:06:19,460 when it's semi-definite, but not definite, then the -- 97 00:06:19,460 --> 00:06:22,540 I'm squeezing this eigenvalue test down, -- 98 00:06:22,540 --> 00:06:28,370 what's the eigenvalue that I know this matrix has? 99 00:06:28,370 --> 00:06:31,320 What kind of a matrix have I got here? 100 00:06:31,320 --> 00:06:37,680 It's a singular matrix, one of its eigenvalues is zero. 101 00:06:37,680 --> 00:06:43,570 That has an eigenvalue zero, and the other eigenvalue is -- 102 00:06:43,570 --> 00:06:47,020 from the trace, twenty. 103 00:06:47,020 --> 00:06:47,620 OK. 104 00:06:47,620 --> 00:06:51,560 So that, that matrix has eigenvalues greater than 105 00:06:51,560 --> 00:06:55,710 or equal to zero, and it's that "equal to" that brought this 106 00:06:55,710 --> 00:06:58,000 word "semi-definite" in. 107 00:06:58,000 --> 00:07:01,240 And, the what are the pivots of that matrix? 108 00:07:01,240 --> 00:07:03,810 So the pivots, so the eigenvalues 109 00:07:03,810 --> 00:07:10,580 are zero and twenty, the pivots are, well, the pivot is two, 110 00:07:10,580 --> 00:07:12,090 and what's the next pivot? 111 00:07:15,840 --> 00:07:17,430 There isn't one. 112 00:07:17,430 --> 00:07:20,900 We got a singular matrix here, it'll only have one pivot. 113 00:07:20,900 --> 00:07:24,920 You see that that's a rank one matrix, two six is a -- 114 00:07:24,920 --> 00:07:27,710 six eighteen is a multiple of two six, 115 00:07:27,710 --> 00:07:32,380 the matrix is singular it only has one pivot, 116 00:07:32,380 --> 00:07:38,050 so the pivot test doesn't quite 117 00:07:38,050 --> 00:07:40,710 The -- let me do the x transpose Ax. 118 00:07:40,710 --> 00:07:41,210 pass. 119 00:07:41,210 --> 00:07:45,280 Now this is -- 120 00:07:45,280 --> 00:07:49,130 the novelty now. 121 00:07:49,130 --> 00:07:49,960 OK. 122 00:07:49,960 --> 00:07:54,470 What I looking at when I look at this new combination, 123 00:07:54,470 --> 00:07:56,360 x transpose Ax. 124 00:07:56,360 --> 00:08:02,140 x is any vector now, so lemme compute, so any vector, 125 00:08:02,140 --> 00:08:08,960 lemme call its components x1 and x2, so that's x. 126 00:08:08,960 --> 00:08:11,300 And I put in here A. 127 00:08:11,300 --> 00:08:15,500 Let's, let's use this example two six, six eighteen, 128 00:08:15,500 --> 00:08:20,500 and here's x transposed, so x1 x2. 129 00:08:20,500 --> 00:08:25,580 We're back to real matrices, after that last lecture 130 00:08:25,580 --> 00:08:29,480 that- that said what to do in the complex case, let's 131 00:08:29,480 --> 00:08:32,010 come back to real matrices. 132 00:08:32,010 --> 00:08:36,780 So here's x transpose Ax, and what I'm interested 133 00:08:36,780 --> 00:08:42,720 in is, what do I get if I multiply those together? 134 00:08:42,720 --> 00:08:48,410 I get some function of x1 and x2, and what is it? 135 00:08:48,410 --> 00:08:51,700 Let's see, if I do this multiplication, so I do it, 136 00:08:51,700 --> 00:08:55,220 lemme, just, I'll just do it slowly, x1, x2, 137 00:08:55,220 --> 00:09:01,980 if I multiply that matrix, this is 2x1, and 6x2s, 138 00:09:01,980 --> 00:09:07,400 and the next row is 6x1s and 18x2s, 139 00:09:07,400 --> 00:09:11,530 and now I do this final step and what do I have? 140 00:09:11,530 --> 00:09:16,810 I've got 2x1 squareds, got 2X1 squareds 141 00:09:16,810 --> 00:09:23,080 is coming from this two, I've got this one gives me eighteen, 142 00:09:23,080 --> 00:09:25,630 well, shall I do the ones in the middle? 143 00:09:25,630 --> 00:09:29,530 How many x1 x2s do I have? 144 00:09:29,530 --> 00:09:33,330 Let's see, x1 times that 6x2 would be six of 'em, 145 00:09:33,330 --> 00:09:38,300 and then x2 times this one will be six more, I get twelve. 146 00:09:38,300 --> 00:09:44,290 So, in here is going, this is the number a, this is 147 00:09:44,290 --> 00:09:49,190 the number 2b, and in here is -- 148 00:09:49,190 --> 00:09:54,960 x2 times eighteen x2 will be eighteen x2 squareds and this 149 00:09:54,960 --> 00:09:57,630 is the number c. 150 00:09:57,630 --> 00:10:02,010 So it's ax1 -- it's like ax squared. 151 00:10:02,010 --> 00:10:05,140 2bxy and cy squared. 152 00:10:05,140 --> 00:10:11,280 For my first point that I wanted to make by just doing out 153 00:10:11,280 --> 00:10:15,720 a multiplication is, that is as soon as you give me the matrix, 154 00:10:15,720 --> 00:10:19,240 as soon as you give me the matrix, I can -- 155 00:10:19,240 --> 00:10:22,860 those are the numbers that appear in -- 156 00:10:22,860 --> 00:10:25,580 I'll call this thing a quadratic, 157 00:10:25,580 --> 00:10:29,140 you see, it's not linear anymore. 158 00:10:29,140 --> 00:10:32,360 Ax is linear, but now I've got an x transpose coming 159 00:10:32,360 --> 00:10:35,680 in, I'm up to degree two, up to degree two, 160 00:10:35,680 --> 00:10:38,880 maybe quadratic is the -- 161 00:10:38,880 --> 00:10:39,480 use the word. 162 00:10:39,480 --> 00:10:41,980 A quadratic form. 163 00:10:41,980 --> 00:10:46,610 It's purely degree two, there's no linear part, 164 00:10:46,610 --> 00:10:48,440 there's no constant part, there certainly 165 00:10:48,440 --> 00:10:53,540 no cubes or fourth powers, it's all second degree. 166 00:10:53,540 --> 00:10:55,780 And my question is -- 167 00:10:55,780 --> 00:11:00,120 is that quantity positive or not? 168 00:11:00,120 --> 00:11:08,250 That's -- for every x1 and x2, that is my new definition -- 169 00:11:08,250 --> 00:11:11,920 that's my definition of a positive definite matrix. 170 00:11:11,920 --> 00:11:17,970 If this quantity is positive, if, if, if, it's positive 171 00:11:17,970 --> 00:11:25,140 for all x's and y's, all x1 x2s, then I call them -- 172 00:11:25,140 --> 00:11:29,170 then that's the matrix is positive definite. 173 00:11:29,170 --> 00:11:34,340 Now, is this guy passing our test? 174 00:11:34,340 --> 00:11:38,120 Well we have, we anticipated the answer here by, 175 00:11:38,120 --> 00:11:42,420 by asking first about eigenvalues and pivots, 176 00:11:42,420 --> 00:11:44,820 and what happened? 177 00:11:44,820 --> 00:11:46,350 It failed. 178 00:11:46,350 --> 00:11:49,490 It barely failed. 179 00:11:49,490 --> 00:11:53,670 If I had made this eighteen down to a seven, 180 00:11:53,670 --> 00:11:58,120 it would've totally failed. 181 00:11:58,120 --> 00:12:01,540 I do that with the eraser, and then I'll put back eighteen, 182 00:12:01,540 --> 00:12:07,170 because, seven is such a total disaster, but if -- 183 00:12:07,170 --> 00:12:10,240 I'll keep seven for a second. 184 00:12:10,240 --> 00:12:15,170 Is that thing in any way positive definite? 185 00:12:15,170 --> 00:12:18,480 No, absolutely not. 186 00:12:18,480 --> 00:12:21,990 I don't know its eigenvalues, but I know for sure one of them 187 00:12:21,990 --> 00:12:24,230 is negative. 188 00:12:24,230 --> 00:12:28,940 Its pivots are two and then the next pivot would be 189 00:12:28,940 --> 00:12:32,250 the determinant over two, and the determinant is -- what, 190 00:12:32,250 --> 00:12:35,090 what's the determinant of this thing? 191 00:12:35,090 --> 00:12:37,620 Fourteen minus thirty six, I've got 192 00:12:37,620 --> 00:12:39,820 a determinant minus twenty two. 193 00:12:39,820 --> 00:12:43,330 The next pivot will be -- the pivots now, 194 00:12:43,330 --> 00:12:48,480 of this thing are two and minus eleven or something. 195 00:12:48,480 --> 00:12:51,320 Their product being minus twenty two the determinant. 196 00:12:51,320 --> 00:12:53,620 This thing is not positive definite. 197 00:12:53,620 --> 00:12:57,200 What would be -- let me look at the x transpose Ax for this 198 00:12:57,200 --> 00:12:57,700 guy. 199 00:12:57,700 --> 00:13:00,460 What's -- if I do out this multiplication, 200 00:13:00,460 --> 00:13:04,560 this eighteen is temporarily changing to a seven. 201 00:13:04,560 --> 00:13:08,130 This eighteen is temporarily changing to a seven, 202 00:13:08,130 --> 00:13:16,610 and I know that there's some numbers x1 and x2 203 00:13:16,610 --> 00:13:24,880 for which that thing, that function, is negative. 204 00:13:24,880 --> 00:13:28,640 And I'm desperately trying to think what they are. 205 00:13:28,640 --> 00:13:30,040 Maybe you can see. 206 00:13:30,040 --> 00:13:33,940 Can you tell me a value of x1 and x2 207 00:13:33,940 --> 00:13:36,675 that makes this quantity negative? 208 00:13:40,400 --> 00:13:43,180 Oh, maybe one and minus one? 209 00:13:43,180 --> 00:13:49,270 Yes, that's -- in this case, that, will work, right, 210 00:13:49,270 --> 00:13:54,150 if I took x1 to be one, and x2 to be minus one, 211 00:13:54,150 --> 00:13:57,170 then I always get something positive, the two, 212 00:13:57,170 --> 00:14:01,367 and the seven minus one squared, but this would be minus twelve 213 00:14:01,367 --> 00:14:02,950 and the whole thing would be negative; 214 00:14:02,950 --> 00:14:07,620 I would have two minus twelve plus seven, a negative. 215 00:14:07,620 --> 00:14:11,550 If I drew the graph, can I get the little picture in 216 00:14:11,550 --> 00:14:12,050 here? 217 00:14:12,050 --> 00:14:16,430 If I draw the graph of this thing? 218 00:14:16,430 --> 00:14:22,730 So, graphs, of the function f(x,y), or f(x), 219 00:14:22,730 --> 00:14:29,620 so I say here f(x,y) equal this -- x transpose Ax, this, 220 00:14:29,620 --> 00:14:32,040 this this ax squared, 2bxy, and cy squared. 221 00:14:32,040 --> 00:14:46,600 And, let's take the example, with these numbers. 222 00:14:49,210 --> 00:14:54,650 OK, so here's the x axis, here's the y axis, and here's -- 223 00:14:54,650 --> 00:14:56,380 up is the function. 224 00:14:56,380 --> 00:14:59,390 z, if you like, or f. 225 00:14:59,390 --> 00:15:03,940 I apologize, and let me, just once in my life here, 226 00:15:03,940 --> 00:15:08,780 put an arrow over these, these, axes so you see them. 227 00:15:08,780 --> 00:15:13,300 That's the vector and I just, see, instead of x1 and x2, 228 00:15:13,300 --> 00:15:16,220 I made them x- the components x and y. 229 00:15:16,220 --> 00:15:16,930 OK. 230 00:15:16,930 --> 00:15:23,660 So, so, what's a graph of 2x squared, twelve xy, and seven y 231 00:15:23,660 --> 00:15:25,710 squared? 232 00:15:25,710 --> 00:15:27,660 I'd like to see -- 233 00:15:27,660 --> 00:15:31,260 I not the greatest artist, but let's -- 234 00:15:31,260 --> 00:15:38,400 tell me something about this graph of this function. 235 00:15:42,560 --> 00:15:44,740 Whoa, tell me one point that it goes through. 236 00:15:47,970 --> 00:15:49,470 The origin. 237 00:15:49,470 --> 00:15:50,990 Right? 238 00:15:50,990 --> 00:15:56,450 Even this artist can get this thing to go through the origin, 239 00:15:56,450 --> 00:16:00,421 when these are zero, I, I certainly get zero. 240 00:16:00,421 --> 00:16:00,920 OK. 241 00:16:00,920 --> 00:16:02,540 Some more points. 242 00:16:02,540 --> 00:16:06,570 If x is one and y is zero, then I'm going upwards, 243 00:16:06,570 --> 00:16:09,060 so I'm going up this way, and I'm, I'm 244 00:16:09,060 --> 00:16:12,180 going up, like, two x squared in that direction. 245 00:16:12,180 --> 00:16:14,766 So -- that's meant to be a parabola. 246 00:16:18,200 --> 00:16:23,150 And, suppose x stays zero and y increases. 247 00:16:23,150 --> 00:16:26,760 Well, y could be positive or negative; it's seven y squared. 248 00:16:26,760 --> 00:16:30,530 Is this function going upward? 249 00:16:30,530 --> 00:16:35,640 In the x direction it's going upward, and in the y direction 250 00:16:35,640 --> 00:16:39,900 it's going upwards, and if x equals y 251 00:16:39,900 --> 00:16:42,140 then the forty-five degree direction is certainly 252 00:16:42,140 --> 00:16:44,810 going upwards; because then we'd have what, 253 00:16:44,810 --> 00:16:49,600 about, everything would be positive, but what? 254 00:16:49,600 --> 00:16:54,810 This function -- what's the graph of this function? 255 00:16:54,810 --> 00:16:55,310 Look like? 256 00:16:55,310 --> 00:17:00,450 Tell me the word that describes the graph of this function. 257 00:17:00,450 --> 00:17:02,980 This is the non-positive definite here, 258 00:17:02,980 --> 00:17:07,660 everybody's with me here, for some reason 259 00:17:07,660 --> 00:17:10,740 got started in a negative direction, your case that 260 00:17:10,740 --> 00:17:12,750 isn't positive definite. 261 00:17:12,750 --> 00:17:17,190 And what's the graph look like that goes up, but does it -- 262 00:17:17,190 --> 00:17:22,710 do we have a minimum here, does it go from, from the origin? 263 00:17:22,710 --> 00:17:24,500 Completely? 264 00:17:24,500 --> 00:17:28,710 No, because we just checked that this thing failed. 265 00:17:28,710 --> 00:17:33,490 It failed along the direction when x was minus y -- 266 00:17:33,490 --> 00:17:37,080 we have a saddle point, let me put myself, let me, 267 00:17:37,080 --> 00:17:39,350 to the least, tell you the word. 268 00:17:39,350 --> 00:17:46,470 This thing, goes up in some directions, 269 00:17:46,470 --> 00:17:53,330 but down in other directions, and if we actually knew what 270 00:17:53,330 --> 00:17:59,300 a saddle looked like or thinks saddles do that -- 271 00:17:59,300 --> 00:18:06,560 the way your legs go is, like, down, up, the way, you, 272 00:18:06,560 --> 00:18:16,200 looking like, forward, and, the, and drawing the thing is even 273 00:18:16,200 --> 00:18:17,330 worse than describing -- 274 00:18:17,330 --> 00:18:20,760 I'm just going to say in some directions we go up 275 00:18:20,760 --> 00:18:28,290 and in other directions, there is, a saddle -- 276 00:18:28,290 --> 00:18:30,330 Now I'm sorry I put that on the front board, 277 00:18:30,330 --> 00:18:34,450 you have no way to cover it, but it's a saddle. 278 00:18:34,450 --> 00:18:35,360 OK. 279 00:18:35,360 --> 00:18:38,950 And, and this is a saddle point, it's 280 00:18:38,950 --> 00:18:43,900 the, it's the point that's at the maximum in some directions 281 00:18:43,900 --> 00:18:46,770 and at the minimum in other directions. 282 00:18:46,770 --> 00:18:50,840 And actually, the perfect directions to look 283 00:18:50,840 --> 00:18:53,670 are the eigenvector directions. 284 00:18:53,670 --> 00:18:55,260 We'll see that. 285 00:18:55,260 --> 00:19:03,730 So this is, not a positive definite matrix. 286 00:19:03,730 --> 00:19:04,280 OK. 287 00:19:04,280 --> 00:19:08,190 Now I'm coming back to this example, 288 00:19:08,190 --> 00:19:14,400 getting rid of this seven, let's move it up to twenty. 289 00:19:14,400 --> 00:19:18,420 Let's, let's let's make the thing really positive definite. 290 00:19:18,420 --> 00:19:19,470 OK. 291 00:19:19,470 --> 00:19:22,990 So this is, this number's now twenty. 292 00:19:22,990 --> 00:19:24,360 c is now twenty. 293 00:19:24,360 --> 00:19:24,860 OK. 294 00:19:24,860 --> 00:19:30,800 Now that passes the test, which I haven't proved, of course, 295 00:19:30,800 --> 00:19:34,960 it passes the test for positive pivots. 296 00:19:34,960 --> 00:19:40,270 It passes the test for positive eigenvalues. 297 00:19:40,270 --> 00:19:43,160 How can you tell that the eigenvalues of that matrix 298 00:19:43,160 --> 00:19:45,850 are positive without actually finding them? 299 00:19:45,850 --> 00:19:49,400 Of course, two by two I could find them, but can you see -- 300 00:19:49,400 --> 00:19:51,530 how do I know they're positive? 301 00:19:51,530 --> 00:19:53,270 I know that their product is -- 302 00:19:53,270 --> 00:19:58,020 I know that lambda one times lambda two is positive, why? 303 00:19:58,020 --> 00:20:01,750 Because that's the determinant, right, 304 00:20:01,750 --> 00:20:06,000 lambda one times lambda two is the determinant, which is forty 305 00:20:06,000 --> 00:20:07,730 minus thirty-six is four. 306 00:20:07,730 --> 00:20:11,120 So the determinant is four. 307 00:20:11,120 --> 00:20:16,160 And the trace, the sum down the diagonal, is twenty-two. 308 00:20:16,160 --> 00:20:20,630 So, they multiply to give four. 309 00:20:20,630 --> 00:20:22,850 So that leaves the possibility they're either 310 00:20:22,850 --> 00:20:25,730 both positive or both negative. 311 00:20:25,730 --> 00:20:29,979 But if they're both negative, the trace couldn't be 312 00:20:29,979 --> 00:20:31,520 So they're both positive. twenty-two. 313 00:20:31,520 --> 00:20:34,170 So both of the eigenvalues that are positive, 314 00:20:34,170 --> 00:20:36,220 both of the pivots are positive -- 315 00:20:36,220 --> 00:20:39,820 the determinants are positive, and I believe that this 316 00:20:39,820 --> 00:20:47,260 function is positive everywhere except at zero, zero, 317 00:20:47,260 --> 00:20:48,240 of course. 318 00:20:48,240 --> 00:20:52,330 When I write down this condition, 319 00:20:52,330 --> 00:20:54,670 So I believe that x transposed, let 320 00:20:54,670 --> 00:21:00,280 me copy, x transpose Ax is positive, except, of course, 321 00:21:00,280 --> 00:21:07,450 at the minimum point, at zero, of course, 322 00:21:07,450 --> 00:21:08,830 I don't expect miracles. 323 00:21:11,760 --> 00:21:15,720 So what does its graph look like, and how do I check, 324 00:21:15,720 --> 00:21:18,730 and how do I check that this really is positive? 325 00:21:22,300 --> 00:21:24,960 So we take it's graph for a minute. 326 00:21:24,960 --> 00:21:27,200 What would be the graph of that function -- 327 00:21:27,200 --> 00:21:29,400 it does not have a saddle point. 328 00:21:29,400 --> 00:21:31,280 Let me -- I'll raise the board, here, 329 00:21:31,280 --> 00:21:33,320 and stay with this example for a while. 330 00:21:36,280 --> 00:21:40,660 So I want to do the graph of -- here's my function, 331 00:21:40,660 --> 00:21:46,940 two x squared, twelve xy-s, that could be positive or negative, 332 00:21:46,940 --> 00:21:48,510 and twenty y squared. 333 00:21:48,510 --> 00:21:56,290 But my point is, so you're seeing the underlying point is, 334 00:21:56,290 --> 00:21:59,910 that, the things are positive definite 335 00:21:59,910 --> 00:22:05,730 when in some way, these, these pure squares, squares 336 00:22:05,730 --> 00:22:10,180 we know to be positive, and when those kind of overwhelm 337 00:22:10,180 --> 00:22:13,527 this guy, who could be m- positive or negative, 338 00:22:13,527 --> 00:22:15,110 because some like or have same or have 339 00:22:15,110 --> 00:22:19,760 same or different signs, when these are big enough 340 00:22:19,760 --> 00:22:22,850 they overwhelm this guy and make the total thing positive, 341 00:22:22,850 --> 00:22:25,750 and what would the graph now look like? 342 00:22:25,750 --> 00:22:32,940 Let me draw the x - well, let me draw the x direction, the y 343 00:22:32,940 --> 00:22:40,970 direction, and the origin, at zero, zero, I'm there, 344 00:22:40,970 --> 00:22:45,650 where do I go as I move away from the origin? 345 00:22:45,650 --> 00:22:50,780 Where do I go as I move away from the origin? 346 00:22:50,780 --> 00:22:53,240 I'm sure that I go up. 347 00:22:53,240 --> 00:22:56,660 The origin, the center point here, 348 00:22:56,660 --> 00:23:01,730 is a minim because this thing I believe, and we better see why, 349 00:23:01,730 --> 00:23:07,620 it's, the graph is like a bowl, the graph is like a bowl shape, 350 00:23:07,620 --> 00:23:09,805 it's -- here's the minimum. 351 00:23:15,660 --> 00:23:19,030 And because we've got a pure quadratic, 352 00:23:19,030 --> 00:23:23,410 we know it sits at the origin, we know it's tangent plane, 353 00:23:23,410 --> 00:23:30,970 the first derivatives are zero, so, we know, first derivatives, 354 00:23:30,970 --> 00:23:37,410 First derivatives are all zero, but that's 355 00:23:37,410 --> 00:23:38,560 not enough for a minimum. 356 00:23:38,560 --> 00:23:43,020 It's first derivatives were zero here. 357 00:23:45,780 --> 00:23:51,200 So, the partial derivatives, the first derivatives, are zero. 358 00:23:51,200 --> 00:23:56,720 Again, because first derivatives are gonna have an x or an a y, 359 00:23:56,720 --> 00:23:59,730 or a y in them, they'll be zero at the origin. 360 00:23:59,730 --> 00:24:03,340 It's the second derivatives that control everything. 361 00:24:03,340 --> 00:24:07,810 It's the second derivatives that this matrix is telling us, 362 00:24:07,810 --> 00:24:10,630 and somehow -- 363 00:24:10,630 --> 00:24:11,780 here's my point. 364 00:24:11,780 --> 00:24:16,710 You remember in Calculus, how did you decide on a minimum? 365 00:24:16,710 --> 00:24:20,010 First requirement was, that the derivative had to be 366 00:24:20,010 --> 00:24:20,940 zero. 367 00:24:20,940 --> 00:24:25,890 But then you didn't know if you had a minimum or a maximum. 368 00:24:25,890 --> 00:24:27,500 To know that you had a minimum, you 369 00:24:27,500 --> 00:24:30,300 had to look at the second derivative. 370 00:24:30,300 --> 00:24:33,400 The second derivative had to be positive, 371 00:24:33,400 --> 00:24:36,900 the slope had to be increasing as you 372 00:24:36,900 --> 00:24:40,010 went through the minimum point. 373 00:24:40,010 --> 00:24:43,470 The curvature had to go upwards, and that's 374 00:24:43,470 --> 00:24:46,910 what we're doing now in two dimensions, 375 00:24:46,910 --> 00:24:49,280 and in n dimensions. 376 00:24:49,280 --> 00:24:52,030 So we're doing what we did in Calculus. 377 00:24:52,030 --> 00:24:55,150 Second derivative positive, m- will now 378 00:24:55,150 --> 00:24:58,590 become that the matrix of second derivatives 379 00:24:58,590 --> 00:25:00,810 is positive definite. 380 00:25:00,810 --> 00:25:02,650 Can I just -- 381 00:25:02,650 --> 00:25:05,670 like a translation of -- 382 00:25:05,670 --> 00:25:11,890 this is how minimum are coming in, ithe beginning of Calculus 383 00:25:11,890 --> 00:25:14,640 -- 384 00:25:14,640 --> 00:25:22,730 we had a minimum was associated with second derivative, 385 00:25:22,730 --> 00:25:24,170 being positive. 386 00:25:24,170 --> 00:25:27,340 And first derivative zero, of course. 387 00:25:27,340 --> 00:25:36,120 Derivative, first derivative, but it 388 00:25:36,120 --> 00:25:39,870 was the second derivative that told us we had a minimum. 389 00:25:39,870 --> 00:25:43,510 And now, in 18.06, in linear algebra, 390 00:25:43,510 --> 00:25:47,260 we'll have a minim for our function now, 391 00:25:47,260 --> 00:25:53,040 our function will have, for your function be a function not 392 00:25:53,040 --> 00:26:00,290 of just x but several variables, the way functions really 393 00:26:00,290 --> 00:26:03,370 are in real life, the minimum will 394 00:26:03,370 --> 00:26:15,140 be when the matrix of second derivatives, the matrix 395 00:26:15,140 --> 00:26:17,590 here was one by one, there was just one second derivative, 396 00:26:17,590 --> 00:26:20,560 now we've got lots. 397 00:26:20,560 --> 00:26:25,335 Is positive definite. 398 00:26:29,070 --> 00:26:31,450 So positive for a number translates 399 00:26:31,450 --> 00:26:34,450 into positive definite for a matrix. 400 00:26:34,450 --> 00:26:38,860 And it this brings everything you check pivots, 401 00:26:38,860 --> 00:26:41,680 you check determinants, you check all your values, 402 00:26:41,680 --> 00:26:44,910 or you check this minimum stuff. 403 00:26:44,910 --> 00:26:45,410 OK. 404 00:26:45,410 --> 00:26:49,190 Let me come back to this graph. 405 00:26:49,190 --> 00:26:50,595 That graph goes upwards. 406 00:26:53,220 --> 00:26:54,740 And I'll have to see why. 407 00:26:54,740 --> 00:26:58,360 How do I know that this, that this function is always 408 00:26:58,360 --> 00:26:59,980 positive? 409 00:26:59,980 --> 00:27:04,770 Can you look at that and tell that it's always positive? 410 00:27:04,770 --> 00:27:08,710 Maybe two by two, you could feel pretty sure, 411 00:27:08,710 --> 00:27:14,530 but what's the good way to show that this thing is always 412 00:27:14,530 --> 00:27:20,630 If we can express it, as, in terms of squares, positive? 413 00:27:20,630 --> 00:27:24,010 because that's what we know for any x and y, 414 00:27:24,010 --> 00:27:26,760 whatever, if we're squaring something 415 00:27:26,760 --> 00:27:29,470 we certainly are not negative. 416 00:27:29,470 --> 00:27:32,690 So I believe that this expression, this function, 417 00:27:32,690 --> 00:27:37,190 could be written as a sum of squares. 418 00:27:37,190 --> 00:27:41,170 Can you tell me -- 419 00:27:41,170 --> 00:27:43,460 see, because all the problems, the headaches 420 00:27:43,460 --> 00:27:47,230 are coming from this xy term. 421 00:27:47,230 --> 00:27:50,500 If we can get expressions -- if we can get that inside 422 00:27:50,500 --> 00:27:53,660 a square, so actually, what we're doing is something 423 00:27:53,660 --> 00:27:58,160 called, that you've seen called completing the square. 424 00:27:58,160 --> 00:28:01,210 Let me start the square and you complete it. 425 00:28:01,210 --> 00:28:05,820 OK, I think we have two of x plus, 426 00:28:05,820 --> 00:28:09,400 now I don't remember how many y-s we need, but you'll 427 00:28:09,400 --> 00:28:10,725 figure it out, squared. 428 00:28:13,750 --> 00:28:20,470 How many y-s should I put in here, to make -- 429 00:28:20,470 --> 00:28:23,950 what do I want to do, the two x squared-s will be correct, 430 00:28:23,950 --> 00:28:25,770 right? 431 00:28:25,770 --> 00:28:28,990 What I want to do is put in the right number of y-s 432 00:28:28,990 --> 00:28:33,270 to get the twelve xy correct. 433 00:28:33,270 --> 00:28:34,885 And what is that number of y-s? 434 00:28:37,430 --> 00:28:39,170 Let's see, I've got two times, and so 435 00:28:39,170 --> 00:28:41,770 I really want six xy-s to come out of here, 436 00:28:41,770 --> 00:28:44,650 I think maybe if I put three there, 437 00:28:44,650 --> 00:28:48,590 does that look right to you? 438 00:28:48,590 --> 00:28:53,940 I have two- this is, we can mentally, multiply out, 439 00:28:53,940 --> 00:28:56,420 that's X squared, that's right, that's 440 00:28:56,420 --> 00:28:59,580 six X Y, times the two gives from, right, 441 00:28:59,580 --> 00:29:02,660 and how many Y squareds have I now got? 442 00:29:02,660 --> 00:29:05,950 How many Y squareds have I now got from this term? 443 00:29:05,950 --> 00:29:07,750 Eighteen. 444 00:29:07,750 --> 00:29:12,000 Eighteen was the key number, remember? 445 00:29:12,000 --> 00:29:16,630 Now if I want to make it twenty, then I've got two left. 446 00:29:16,630 --> 00:29:18,100 Two y squared-s. 447 00:29:18,100 --> 00:29:25,660 That's completing the square, and it's, now 448 00:29:25,660 --> 00:29:28,120 I can see that that function is positive, 449 00:29:28,120 --> 00:29:30,090 because it's all squares. 450 00:29:33,340 --> 00:29:35,840 I've got two squares, added together, 451 00:29:35,840 --> 00:29:38,110 I couldn't go negative. 452 00:29:38,110 --> 00:29:42,070 What if I went back to that seven? 453 00:29:42,070 --> 00:29:45,080 If instead of twenty that number was a seven, then 454 00:29:45,080 --> 00:29:47,610 what would happen? 455 00:29:47,610 --> 00:29:51,010 This would still be correct, I'd still have this square, 456 00:29:51,010 --> 00:29:53,670 to get the two x squared and the twelve xy, 457 00:29:53,670 --> 00:29:58,800 and I'd have eighteen y squared and then what would I do here? 458 00:29:58,800 --> 00:30:03,760 I'd have to remove eleven y squared-s, right, 459 00:30:03,760 --> 00:30:08,200 if I only had a seven here, then instead of -- 460 00:30:08,200 --> 00:30:13,040 when I had a twenty I had two more to put in, when I had 461 00:30:13,040 --> 00:30:16,740 an eighteen, which was the marginal case, 462 00:30:16,740 --> 00:30:19,780 I had no more to put in. 463 00:30:19,780 --> 00:30:23,370 When I had a seven, which was the case below zero, 464 00:30:23,370 --> 00:30:29,870 the indefinite case, I had minus eleven. 465 00:30:29,870 --> 00:30:36,360 Now, so, you can see now, that this thing is a bowl. 466 00:30:36,360 --> 00:30:36,860 OK. 467 00:30:36,860 --> 00:30:42,200 It's going upwards, if I cut it at a plane, z equal to one, 468 00:30:42,200 --> 00:30:47,830 say, I would get, I would get a curve, what would 469 00:30:47,830 --> 00:30:50,280 be the equation for that curve? 470 00:30:50,280 --> 00:30:53,190 If I cut it at height one, the equation 471 00:30:53,190 --> 00:30:56,980 would be this thing equal to one. 472 00:30:56,980 --> 00:30:59,910 And that curve would be an ellipse. 473 00:30:59,910 --> 00:31:02,560 So actually, already, I've blocked 474 00:31:02,560 --> 00:31:09,380 into the lecture, the different pieces that we're aiming for. 475 00:31:09,380 --> 00:31:12,620 We're aiming for the tests, which this passed; 476 00:31:12,620 --> 00:31:17,360 we're aiming for the connection to a minimum, which this -- 477 00:31:17,360 --> 00:31:22,420 which we see in the graph, and if we chop it up, 478 00:31:22,420 --> 00:31:25,180 if we set this thing equal to one, 479 00:31:25,180 --> 00:31:27,560 if I set that thing equal to one, that -- 480 00:31:27,560 --> 00:31:30,490 what that gives me is, the cross-section. 481 00:31:30,490 --> 00:31:37,750 It gives me this, this curve, and its equation 482 00:31:37,750 --> 00:31:41,200 is this thing equals one, and that's an ellipse. 483 00:31:41,200 --> 00:31:44,660 Whereas if I cut through a saddle point, 484 00:31:44,660 --> 00:31:47,900 I get a hyperbola. 485 00:31:47,900 --> 00:31:53,820 But this minimum stuff is really what I'm most interested 486 00:31:53,820 --> 00:31:54,610 OK. 487 00:31:54,610 --> 00:31:55,110 in. 488 00:31:55,110 --> 00:31:55,610 OK. 489 00:31:55,610 --> 00:32:00,300 By -- I just have to ask, do you recognize, I mean, 490 00:32:00,300 --> 00:32:04,460 these numbers here, the two that appeared outside, 491 00:32:04,460 --> 00:32:08,090 the three that appeared inside, the two that appeared there -- 492 00:32:08,090 --> 00:32:13,690 actually, those numbers come from elimination. 493 00:32:13,690 --> 00:32:18,890 Completing the square is our good old method 494 00:32:18,890 --> 00:32:24,910 of Gaussian elimination, in expressed 495 00:32:24,910 --> 00:32:27,280 in terms of these squares. 496 00:32:27,280 --> 00:32:30,600 The -- let me show you what I mean. 497 00:32:30,600 --> 00:32:34,050 I just think those numbers are no accident, 498 00:32:34,050 --> 00:32:40,050 If I take my matrix two, six, six, and twenty, 499 00:32:40,050 --> 00:32:45,050 and I do elimination, then the pivot is two 500 00:32:45,050 --> 00:32:50,150 and I take three, what's the multiplier? 501 00:32:50,150 --> 00:32:54,180 How much of row one do I take away from row two? 502 00:32:54,180 --> 00:32:55,360 Three. 503 00:32:55,360 --> 00:32:58,700 So what you're seeing in this, completing the square, 504 00:32:58,700 --> 00:33:05,180 is the pivots outside and the multiplier inside. 505 00:33:05,180 --> 00:33:06,680 Just do that again? 506 00:33:06,680 --> 00:33:12,840 The pivot is two, three -- three of those away from that gives 507 00:33:12,840 --> 00:33:18,100 me two, six, zero, and what's the second pivot? 508 00:33:18,100 --> 00:33:21,550 Three of this away from this, three sixes'll be eighteen, 509 00:33:21,550 --> 00:33:23,380 and the second pivot will also be a 510 00:33:23,380 --> 00:33:24,040 two. 511 00:33:24,040 --> 00:33:32,490 So that's the U, this is the A, and of course the L 512 00:33:32,490 --> 00:33:37,900 was one, zero, one, and the multiplier was three. 513 00:33:40,540 --> 00:33:48,210 So, completing the square is elimination. 514 00:33:48,210 --> 00:33:54,230 Why I happy to see, happy to see that coming together? 515 00:33:54,230 --> 00:34:01,190 Because I know about elimination for m by m matrices. 516 00:34:01,190 --> 00:34:07,740 I just started talking about completing the square, here, 517 00:34:07,740 --> 00:34:10,320 for two by twos. 518 00:34:10,320 --> 00:34:12,989 But now I see what's going on. 519 00:34:12,989 --> 00:34:17,909 Completing the square really amounts to splitting this thing 520 00:34:17,909 --> 00:34:21,610 into a sum of squares, so what's the critical thing -- 521 00:34:21,610 --> 00:34:25,380 I have a lot of squares, and inside those squares 522 00:34:25,380 --> 00:34:28,020 are multipliers but they're squares, 523 00:34:28,020 --> 00:34:31,790 and the question is, what's outside these squares? 524 00:34:31,790 --> 00:34:35,210 When I complete the square, what are the numbers that go 525 00:34:35,210 --> 00:34:36,179 outside? 526 00:34:36,179 --> 00:34:37,342 They're the pivots. 527 00:34:39,889 --> 00:34:45,510 They're the pivots, and that's why positive pivots give me 528 00:34:45,510 --> 00:34:47,409 sum of squares. 529 00:34:47,409 --> 00:34:50,270 Positive pivots, those pivots are the numbers 530 00:34:50,270 --> 00:34:53,600 that go outside the squares, so positive pivots, 531 00:34:53,600 --> 00:34:57,850 sum of squares, everything positive, graph goes up, 532 00:34:57,850 --> 00:35:03,050 a minimum at the origin, it's all connected together; 533 00:35:03,050 --> 00:35:04,710 all connected together. 534 00:35:04,710 --> 00:35:08,110 And in the two by two case, you can see those connections, 535 00:35:08,110 --> 00:35:16,100 but linear algebra now can go up to three by three, m by m. 536 00:35:16,100 --> 00:35:17,950 Let's do that next. 537 00:35:17,950 --> 00:35:20,680 Can I just, before I leave two by two, 538 00:35:20,680 --> 00:35:24,940 I've written this expression "matrix of second derivatives." 539 00:35:24,940 --> 00:35:27,165 What's the matrix of second derivatives? 540 00:35:29,690 --> 00:35:31,970 That's one second derivative now, 541 00:35:31,970 --> 00:35:40,020 but if I'm in two dimensions, I have a two by two matrix, 542 00:35:40,020 --> 00:35:46,480 it's the second x derivative, the second x derivative goes 543 00:35:46,480 --> 00:35:49,070 there -- 544 00:35:49,070 --> 00:35:56,360 shall I write it -- fxx, if you like, fxx, 545 00:35:56,360 --> 00:36:00,170 that means the second derivative of f in the x direction. 546 00:36:00,170 --> 00:36:04,500 fyy, second derivative in the y direction. 547 00:36:04,500 --> 00:36:07,970 Those are the pure derivatives, second derivatives. 548 00:36:07,970 --> 00:36:10,950 They have to be positive. 549 00:36:10,950 --> 00:36:13,430 For a minimum. 550 00:36:13,430 --> 00:36:15,590 This number has to be positive for a minimum. 551 00:36:15,590 --> 00:36:17,730 That number has to be positive for a minimum. 552 00:36:17,730 --> 00:36:20,410 But, that's not enough. 553 00:36:20,410 --> 00:36:23,680 Those numbers have to somehow be big enough 554 00:36:23,680 --> 00:36:30,090 to overcome this cross-derivative, 555 00:36:30,090 --> 00:36:31,780 Why is the matrix symmetric? 556 00:36:31,780 --> 00:36:35,730 Because the second derivative f with respect to x and y is 557 00:36:35,730 --> 00:36:37,950 equal to -- 558 00:36:37,950 --> 00:36:42,170 I can, that's the beautiful fact about second derivatives, is 559 00:36:42,170 --> 00:36:46,480 I can do those in either order and I get the same thing. 560 00:36:46,480 --> 00:36:54,410 So this is the same as that, and so, that's 561 00:36:54,410 --> 00:36:57,300 the matrix of second derivatives. 562 00:36:57,300 --> 00:37:01,280 And the test is, it has to be positive definite. 563 00:37:01,280 --> 00:37:05,960 You might remember, from, tucked in somewhere 564 00:37:05,960 --> 00:37:08,990 near the end of eighteen o' two or at least in the book, 565 00:37:08,990 --> 00:37:13,590 was the condition for a minimum, For a function 566 00:37:13,590 --> 00:37:15,470 of two variables. 567 00:37:15,470 --> 00:37:17,180 Let's -- when do you have a minimum? 568 00:37:17,180 --> 00:37:19,740 For a function of two variables, believe me, 569 00:37:19,740 --> 00:37:24,200 that's what Calculus is for. 570 00:37:24,200 --> 00:37:29,270 The condition is first derivatives have to be zero. 571 00:37:29,270 --> 00:37:32,630 And the matrix of second derivatives 572 00:37:32,630 --> 00:37:35,280 has to be positive definite. 573 00:37:35,280 --> 00:37:39,650 So you maybe remember there was an fxx times an fyy that 574 00:37:39,650 --> 00:37:42,610 had to be bigger than an an fxy squared, 575 00:37:42,610 --> 00:37:46,630 that's just our determinant, two by two. 576 00:37:46,630 --> 00:37:51,480 But now, we now know the answer for three by three, 577 00:37:51,480 --> 00:37:57,770 m by m, because we can do elimination by m by m matrices, 578 00:37:57,770 --> 00:38:01,660 we can connect eigenvalues of m by m matrices, 579 00:38:01,660 --> 00:38:06,070 we can do sum of squares, sum of m squares instead of only two 580 00:38:06,070 --> 00:38:11,470 squares; and so let's take a, let 581 00:38:11,470 --> 00:38:14,000 me go over here to do a three by three example. 582 00:38:17,110 --> 00:38:18,510 So, three by three example. 583 00:38:23,445 --> 00:38:23,945 OK. 584 00:38:27,060 --> 00:38:29,113 Oh, let me -- 585 00:38:32,980 --> 00:38:34,670 shall I use my favorite matrix? 586 00:38:37,210 --> 00:38:39,080 You've seen this matrix before. 587 00:38:45,340 --> 00:38:49,680 Yes, let's use the good matrix, four by one, oops, open. 588 00:38:56,130 --> 00:38:57,950 Is that matrix positive definite? 589 00:39:01,300 --> 00:39:04,440 What's -- so I'm going to ask questions about this matrix, 590 00:39:04,440 --> 00:39:08,260 is it positive definite, first of all? 591 00:39:08,260 --> 00:39:10,970 What's the function associated with that matrix, 592 00:39:10,970 --> 00:39:12,780 what's the x transpose Ax? 593 00:39:16,260 --> 00:39:18,860 Is -- do we have a minimum for that function, 594 00:39:18,860 --> 00:39:19,865 at zero? 595 00:39:22,460 --> 00:39:25,380 And then even what's the geometry? 596 00:39:25,380 --> 00:39:26,060 OK. 597 00:39:26,060 --> 00:39:29,200 First of all, is the matrix positive definite, 598 00:39:29,200 --> 00:39:33,460 now I've given you the numbers there so you can take 599 00:39:33,460 --> 00:39:35,370 the determinants, maybe that's the quickest, 600 00:39:35,370 --> 00:39:36,910 is that what you would do mentally, 601 00:39:36,910 --> 00:39:39,660 if I give you all a matrix on a quiz and say 602 00:39:39,660 --> 00:39:43,010 is it positive definite or not? 603 00:39:43,010 --> 00:39:47,240 I would take that determinant and I'd give the answer two. 604 00:39:47,240 --> 00:39:49,130 I would take that determinant and I 605 00:39:49,130 --> 00:39:55,380 would give the answer for that two by two determinant, 606 00:39:55,380 --> 00:39:57,400 I'd give the answer three, and anybody 607 00:39:57,400 --> 00:40:04,220 remember the answer for the three by three determinant? 608 00:40:04,220 --> 00:40:08,220 It was four, you remember for these special matrices, 609 00:40:08,220 --> 00:40:13,190 when we do determinants, they went up two, three, four, five, 610 00:40:13,190 --> 00:40:15,950 six, they just went up linearly. 611 00:40:15,950 --> 00:40:22,940 So that matrix has -- the determinants are two, three, 612 00:40:22,940 --> 00:40:24,615 and four. 613 00:40:24,615 --> 00:40:25,115 Pivots. 614 00:40:27,630 --> 00:40:31,160 What are the pivots for that matrix? 615 00:40:31,160 --> 00:40:34,470 I'll tell you, they're -- the first pivot is two, 616 00:40:34,470 --> 00:40:37,520 the next pivot is three over two, 617 00:40:37,520 --> 00:40:39,730 the next pivot is four over three. 618 00:40:42,590 --> 00:40:45,170 Because, the product of the pivots 619 00:40:45,170 --> 00:40:47,550 has to give me those determinants. 620 00:40:47,550 --> 00:40:50,370 The product of these two pivots gives me that determinant; 621 00:40:50,370 --> 00:40:53,574 the product of all the pivots gives me that determinant. 622 00:40:53,574 --> 00:40:54,615 What are the eigenvalues? 623 00:41:03,020 --> 00:41:05,717 Oh, I don't know. 624 00:41:08,600 --> 00:41:12,860 The eigenvalues I've got, what do I have a cubic equation -- 625 00:41:12,860 --> 00:41:14,320 a degree three equation? 626 00:41:17,230 --> 00:41:19,350 There are three eigenvalues to find. 627 00:41:23,350 --> 00:41:25,200 If I believe what I've said today, 628 00:41:25,200 --> 00:41:27,800 what do I know about these eigenvalues, 629 00:41:27,800 --> 00:41:30,710 even though I don't know the exact numbers. 630 00:41:30,710 --> 00:41:34,535 I -- I remember the numbers. 631 00:41:37,100 --> 00:41:39,740 Because these matrices are so important 632 00:41:39,740 --> 00:41:43,750 that people figure them. 633 00:41:43,750 --> 00:41:47,440 But -- what do you believe to be true about these three 634 00:41:47,440 --> 00:41:52,950 eigenvalues -- you believe that they are all positive. 635 00:41:52,950 --> 00:41:54,270 They're all positive. 636 00:41:54,270 --> 00:42:00,910 I think that they are two minus square root of two, two, 637 00:42:00,910 --> 00:42:02,630 and two plus the square root of two. 638 00:42:02,630 --> 00:42:03,720 I think. 639 00:42:03,720 --> 00:42:05,490 Let me just -- 640 00:42:05,490 --> 00:42:08,280 I can't write those numbers down without checking 641 00:42:08,280 --> 00:42:10,870 the simple checks, what the first simple check is 642 00:42:10,870 --> 00:42:15,110 the trace, so if I add those numbers I get six 643 00:42:15,110 --> 00:42:18,970 and if I add those numbers I get six. 644 00:42:18,970 --> 00:42:24,150 The other simple test is the determinant, if I -- 645 00:42:24,150 --> 00:42:26,930 can you do this, can you multiply those numbers 646 00:42:26,930 --> 00:42:29,240 together? 647 00:42:29,240 --> 00:42:32,610 I guess we can multiply by two out there. 648 00:42:32,610 --> 00:42:35,010 What's two minus square root of two times two 649 00:42:35,010 --> 00:42:39,530 plus square root of two, that'll be four minus two, 650 00:42:39,530 --> 00:42:41,610 that'll be two, yeah, two times two, 651 00:42:41,610 --> 00:42:46,280 that's got the determinant, right, so it's got, 652 00:42:46,280 --> 00:42:49,810 it's got a chance of being correct and I think it is. 653 00:42:49,810 --> 00:42:51,990 Now, what's the x transpose Ax? 654 00:42:51,990 --> 00:42:54,610 I better give myself enough room for that. 655 00:42:54,610 --> 00:42:59,180 x transpose Ax for this guy. 656 00:42:59,180 --> 00:43:07,290 It's two x1 squareds, and two x2 squareds, and two x3 squareds. 657 00:43:07,290 --> 00:43:10,800 Those come from the diagonal, those were easy. 658 00:43:10,800 --> 00:43:13,490 Now off the diagonal there's a minus and a minus, 659 00:43:13,490 --> 00:43:20,010 they come together there'll be a minus two minus two whats? 660 00:43:20,010 --> 00:43:26,390 Are coming from this one two and two one position, is the x1 x2. 661 00:43:26,390 --> 00:43:30,900 I'm doing mentally a multiplication 662 00:43:30,900 --> 00:43:34,300 of this matrix times a row vector 663 00:43:34,300 --> 00:43:37,680 on the left times a column vector on the right, 664 00:43:37,680 --> 00:43:41,730 and I know that these numbers show up in the answer. 665 00:43:41,730 --> 00:43:45,060 The diagonal is the perfect square, 666 00:43:45,060 --> 00:43:49,110 this off diagonal is a minus two x1 x2, 667 00:43:49,110 --> 00:43:55,280 and there are no x1 x3-s, and there're minus two x2 x3-s. 668 00:43:55,280 --> 00:43:58,860 And I believe that that expression is always positive. 669 00:44:01,670 --> 00:44:06,010 I believe that that curve, that graph, really, 670 00:44:06,010 --> 00:44:09,720 of that function, this is my function f, 671 00:44:09,720 --> 00:44:14,770 and I'm in more dimensions now than I can draw, it -- 672 00:44:14,770 --> 00:44:20,080 but the graph of that function goes upwards. 673 00:44:20,080 --> 00:44:22,320 It's a bowl. 674 00:44:22,320 --> 00:44:26,320 Or maybe the right word is -- 675 00:44:26,320 --> 00:44:34,210 just forgot, what's a long word for bowl? 676 00:44:34,210 --> 00:44:42,570 Hm, maybe paraboloid, I think, paraboloid comes in. 677 00:44:42,570 --> 00:44:46,150 I'll edit the tape and get that word in. 678 00:44:46,150 --> 00:44:54,410 Bowl, let's say, is, that, so that, and if I can -- 679 00:44:54,410 --> 00:44:56,700 I could complete the squares, I could write that 680 00:44:56,700 --> 00:44:59,740 as the sum of three squares, and those three squares 681 00:44:59,740 --> 00:45:02,780 would get multiplied by the three pivots. 682 00:45:02,780 --> 00:45:05,440 And the pivots are all positive. 683 00:45:05,440 --> 00:45:08,760 So I would have positive pivots times squares, 684 00:45:08,760 --> 00:45:11,630 the net result would be a positive function 685 00:45:11,630 --> 00:45:14,520 and a bowl which goes upwards. 686 00:45:14,520 --> 00:45:19,230 And then, finally, if I cut -- if I slice through this bowl, 687 00:45:19,230 --> 00:45:23,730 if I -- now I'm asking you to stretch your visualization 688 00:45:23,730 --> 00:45:26,680 here, because I'm in four dimensions, 689 00:45:26,680 --> 00:45:33,610 I've got x1 x2 x3 in the base, and this function is z, or f, 690 00:45:33,610 --> 00:45:35,000 or something. 691 00:45:35,000 --> 00:45:38,320 And its graph is going up. 692 00:45:38,320 --> 00:45:40,530 But I'm in four dimensions, because I've got three 693 00:45:40,530 --> 00:45:43,410 in the base and then the upward direction, 694 00:45:43,410 --> 00:45:49,640 but now if I cut through this four-dimensional picture, 695 00:45:49,640 --> 00:45:55,720 at level one, so, suppose I cut through this thing 696 00:45:55,720 --> 00:45:58,100 at height one. 697 00:45:58,100 --> 00:46:01,470 So I take all the points that are at height one. 698 00:46:04,980 --> 00:46:07,770 That gives me -- 699 00:46:07,770 --> 00:46:14,180 it gave me an ellipse over there, in that two by two case, 700 00:46:14,180 --> 00:46:19,350 in this case, this will be the equation of an ellipsoid, 701 00:46:19,350 --> 00:46:20,830 a football in other words. 702 00:46:23,770 --> 00:46:25,170 Well, not quite a football. 703 00:46:25,170 --> 00:46:26,450 A lopsided football. 704 00:46:26,450 --> 00:46:30,960 What would be, can I try to describe to you what 705 00:46:30,960 --> 00:46:35,590 the ellipsoid will look like, this ellipsoid, 706 00:46:35,590 --> 00:46:40,440 I'm sorry that the, that I've ended the matrix right -- 707 00:46:40,440 --> 00:46:46,230 at the point, let's -- let me be sure you've seen the equation. 708 00:46:46,230 --> 00:46:51,130 Two x1 squared, two x2 squared, two x3 squared, minus two 709 00:46:51,130 --> 00:46:57,000 of the cross parts, equal what? 710 00:46:57,000 --> 00:47:00,720 That is the equation of a football, so what 711 00:47:00,720 --> 00:47:03,780 do I mean by a football or an ellipsoid? 712 00:47:03,780 --> 00:47:12,520 I mean that, well, I'll draw a few. 713 00:47:12,520 --> 00:47:21,750 It's like that, it's got a center, 714 00:47:21,750 --> 00:47:31,990 and it's got it's got three principal directions. 715 00:47:31,990 --> 00:47:32,830 This ellipsoid. 716 00:47:32,830 --> 00:47:35,660 So -- you see what I'm saying, if we have a sphere then all 717 00:47:35,660 --> 00:47:36,890 directions would be the same. 718 00:47:39,530 --> 00:47:44,750 If we had a true football, or it's closer to a rugby ball, 719 00:47:44,750 --> 00:47:48,110 really, because it's more curved than a football, 720 00:47:48,110 --> 00:47:52,270 it would have one long direction and the other two 721 00:47:52,270 --> 00:47:54,020 would be equal. 722 00:47:54,020 --> 00:47:56,240 That would be like having a matrix 723 00:47:56,240 --> 00:47:59,700 that had one eigenvalue repeated. 724 00:47:59,700 --> 00:48:02,080 And then one other different. 725 00:48:02,080 --> 00:48:04,940 So this sphere comes from, like, the identity matrix, 726 00:48:04,940 --> 00:48:07,600 all eigenvalues the same. 727 00:48:07,600 --> 00:48:12,370 Our rugby ball comes from a case where -- 728 00:48:12,370 --> 00:48:17,070 three, the three, two of the three eigenvalues are the same. 729 00:48:17,070 --> 00:48:20,000 But we know how the case where -- the typical case, 730 00:48:20,000 --> 00:48:23,340 where the three eigenvalues were all different. 731 00:48:23,340 --> 00:48:25,551 So this will have -- 732 00:48:28,440 --> 00:48:31,830 How do I say it, if I look at this ellipsoid correctly, 733 00:48:31,830 --> 00:48:37,200 it'll have a major axis, it'll have a middle axis, 734 00:48:37,200 --> 00:48:41,520 and it'll have a minor axis. 735 00:48:41,520 --> 00:48:44,940 And those three axes will be in the direction 736 00:48:44,940 --> 00:48:47,450 of the eigenvectors. 737 00:48:47,450 --> 00:48:50,760 And the lengths of those axes will be 738 00:48:50,760 --> 00:48:52,496 determined by the eigenvalues. 739 00:48:55,260 --> 00:48:56,770 I can get -- 740 00:48:56,770 --> 00:49:02,520 turn this all into linear algebra, because we have -- 741 00:49:02,520 --> 00:49:06,410 the right thing we know about eigenvectors and eigenvalues, 742 00:49:06,410 --> 00:49:08,120 for that matrix is what? 743 00:49:08,120 --> 00:49:11,970 Of -- let me just tell you that, repeat the main linear algebra 744 00:49:11,970 --> 00:49:13,070 point. 745 00:49:13,070 --> 00:49:18,970 How could we turn what I said into algebra; 746 00:49:18,970 --> 00:49:25,050 we would write this A as Q, the eigenvector matrix, 747 00:49:25,050 --> 00:49:30,370 times lambda, the eigenvalue matrix times Q transposed. 748 00:49:30,370 --> 00:49:33,010 The principal axis theorem, we'll call it, 749 00:49:33,010 --> 00:49:33,900 now. 750 00:49:33,900 --> 00:49:37,980 The eigenvectors tell us the directions of the principal 751 00:49:37,980 --> 00:49:39,110 axes. 752 00:49:39,110 --> 00:49:43,240 The eigenvalues tell us the lengths of those axes, actually 753 00:49:43,240 --> 00:49:45,160 the lengths, or the half-lengths, 754 00:49:45,160 --> 00:49:48,970 or one over the eigenvalues, it turns out. 755 00:49:48,970 --> 00:49:53,260 And that is the matrix factorization 756 00:49:53,260 --> 00:49:55,490 which is the most important matrix 757 00:49:55,490 --> 00:50:00,540 factorization in our eigenvalue material so far. 758 00:50:00,540 --> 00:50:04,320 That's diagonalization for a symmetric matrix, 759 00:50:04,320 --> 00:50:09,350 so instead of the inverse I can write the transposed. 760 00:50:09,350 --> 00:50:09,930 OK. 761 00:50:09,930 --> 00:50:14,140 I've -- so what I've tried today is to tell you the -- 762 00:50:14,140 --> 00:50:20,510 what's going on with positive definite matrices. 763 00:50:20,510 --> 00:50:23,020 Ah, you see all how all these pieces are there 764 00:50:23,020 --> 00:50:25,721 and linear algebra connects them. 765 00:50:25,721 --> 00:50:26,220 OK. 766 00:50:28,760 --> 00:50:30,680 See you on Friday.