1 00:00:10,510 --> 00:00:11,010 OK. 2 00:00:11,010 --> 00:00:11,635 Shall we start? 3 00:00:11,635 --> 00:00:15,360 This is the second lecture on eigenvalues. 4 00:00:15,360 --> 00:00:18,350 So the first lecture was -- 5 00:00:18,350 --> 00:00:22,350 reached the key equation, A x equal lambda x. 6 00:00:22,350 --> 00:00:27,090 x is the eigenvector and lambda's the eigenvalue. 7 00:00:27,090 --> 00:00:30,220 Now to use that. 8 00:00:30,220 --> 00:00:36,330 And the, the good way to, after we've found -- 9 00:00:36,330 --> 00:00:39,400 so, so job one is to find the eigenvalues 10 00:00:39,400 --> 00:00:41,990 and find the eigenvectors. 11 00:00:41,990 --> 00:00:44,750 Now after we've found them, what do we do with them? 12 00:00:44,750 --> 00:00:50,360 Well, the good way to see that is diagonalize the matrix. 13 00:00:50,360 --> 00:00:52,850 So the matrix is A. 14 00:00:52,850 --> 00:00:55,130 And I want to show -- first of all, 15 00:00:55,130 --> 00:00:57,190 this is like the basic fact. 16 00:00:57,190 --> 00:00:58,720 This, this formula. 17 00:00:58,720 --> 00:01:03,210 That's, that's the key to today's lecture. 18 00:01:03,210 --> 00:01:06,810 This matrix A, I put its eigenvectors 19 00:01:06,810 --> 00:01:09,660 in the columns of a matrix S. 20 00:01:09,660 --> 00:01:12,740 So S will be the eigenvector matrix. 21 00:01:12,740 --> 00:01:19,560 And I want to look at this magic combination S inverse A S. 22 00:01:19,560 --> 00:01:22,830 So can I show you how that -- 23 00:01:22,830 --> 00:01:24,200 what happens there? 24 00:01:24,200 --> 00:01:27,900 And notice, there's an S inverse. 25 00:01:27,900 --> 00:01:33,690 We have to be able to invert this eigenvector matrix S. 26 00:01:33,690 --> 00:01:38,330 So for that, we need n independent eigenvectors. 27 00:01:38,330 --> 00:01:41,440 So that's the, that's the case. 28 00:01:41,440 --> 00:01:42,130 OK. 29 00:01:42,130 --> 00:01:53,740 So suppose we have n linearly independent eigenvectors 30 00:01:53,740 --> 00:02:11,830 of A. Put them in the columns of this matrix S. 31 00:02:11,830 --> 00:02:15,430 So I'm naturally going to call that the eigenvector matrix, 32 00:02:15,430 --> 00:02:18,730 because it's got the eigenvectors in its columns. 33 00:02:18,730 --> 00:02:21,730 And all I want to do is show you what happens 34 00:02:21,730 --> 00:02:24,960 when you multiply A times S. 35 00:02:24,960 --> 00:02:29,200 So A times S. 36 00:02:29,200 --> 00:02:34,490 So this is A times the matrix with the first eigenvector 37 00:02:34,490 --> 00:02:37,300 in its first column, the second eigenvector 38 00:02:37,300 --> 00:02:42,510 in its second column, the n-th eigenvector in its n-th column. 39 00:02:42,510 --> 00:02:46,300 And how I going to do this matrix multiplication? 40 00:02:46,300 --> 00:02:49,890 Well, certainly I'll do it a column at a time. 41 00:02:49,890 --> 00:02:52,510 And what do I get. 42 00:02:52,510 --> 00:02:55,480 A times the first column gives me the first column 43 00:02:55,480 --> 00:02:57,820 of the answer, but what is it? 44 00:02:57,820 --> 00:03:00,010 That's an eigenvector. 45 00:03:00,010 --> 00:03:04,280 A times x1 is equal to the lambda times the x1. 46 00:03:04,280 --> 00:03:06,870 And that lambda's we're -- we'll call lambda one, 47 00:03:06,870 --> 00:03:09,590 of course. 48 00:03:09,590 --> 00:03:11,420 So that's the first column. 49 00:03:11,420 --> 00:03:14,000 Ax1 is the same as lambda one x1. 50 00:03:14,000 --> 00:03:16,730 A x2 is lambda two x2. 51 00:03:16,730 --> 00:03:22,120 So on, along to in the n-th column we now how lambda n xn. 52 00:03:24,690 --> 00:03:27,110 Looking good, but the next step is even 53 00:03:27,110 --> 00:03:28,120 better. 54 00:03:28,120 --> 00:03:31,280 So for the next step, I want to separate out 55 00:03:31,280 --> 00:03:36,100 those eigenvalues, those, those multiplying numbers, 56 00:03:36,100 --> 00:03:38,880 from the x-s. 57 00:03:38,880 --> 00:03:41,920 So then I'll have just what I want. 58 00:03:41,920 --> 00:03:42,530 OK. 59 00:03:42,530 --> 00:03:44,990 So how, how I going to separate out? 60 00:03:44,990 --> 00:03:47,150 So that, that number lambda one is 61 00:03:47,150 --> 00:03:49,760 multiplying the first column. 62 00:03:49,760 --> 00:03:52,840 So if I want to factor it out of the first column, 63 00:03:52,840 --> 00:03:55,400 I better put -- 64 00:03:55,400 --> 00:03:58,830 here is going to be x1, and that's 65 00:03:58,830 --> 00:04:02,180 going to multiply this matrix lambda 66 00:04:02,180 --> 00:04:05,050 one in the first entry and all zeros. 67 00:04:05,050 --> 00:04:08,230 Do you see that that, that's going to come out 68 00:04:08,230 --> 00:04:10,220 right for the first column? 69 00:04:10,220 --> 00:04:13,370 Because w- we remember how -- how we're going back to that 70 00:04:13,370 --> 00:04:15,310 original punchline. 71 00:04:15,310 --> 00:04:21,260 That if I want a number to multiply x1 then 72 00:04:21,260 --> 00:04:24,310 I can do it by putting x1 in that column, 73 00:04:24,310 --> 00:04:27,280 in the first column, and putting that number there. 74 00:04:27,280 --> 00:04:28,850 Th- u- what I going to have here? 75 00:04:28,850 --> 00:04:30,040 I'm going to have lambda -- 76 00:04:30,040 --> 00:04:33,800 I'm going to have x1, x2, ... 77 00:04:33,800 --> 00:04:34,890 ,xn. 78 00:04:34,890 --> 00:04:37,150 These are going to be my columns again. 79 00:04:37,150 --> 00:04:39,740 I'm getting S back again. 80 00:04:39,740 --> 00:04:41,380 I'm getting S back again. 81 00:04:41,380 --> 00:04:44,560 But now what's it multiplied by, on the right it's 82 00:04:44,560 --> 00:04:48,080 multiplied by? 83 00:04:48,080 --> 00:04:54,730 If I want lambda n xn in the last column, how do I do it? 84 00:04:54,730 --> 00:04:57,670 Well, the last column here will be -- 85 00:04:57,670 --> 00:05:02,630 I'll take the last column, use these coefficients, 86 00:05:02,630 --> 00:05:05,050 put the lambda n down there, and it 87 00:05:05,050 --> 00:05:09,970 will multiply that n-th column and give me lambda n xn. 88 00:05:09,970 --> 00:05:13,130 There, there you see matrix multiplication just working 89 00:05:13,130 --> 00:05:14,850 for us. 90 00:05:14,850 --> 00:05:17,410 So I started with A S. 91 00:05:17,410 --> 00:05:21,010 I wrote down what it meant, A times each eigenvector. 92 00:05:21,010 --> 00:05:23,690 That gave me lambda time the eigenvector. 93 00:05:23,690 --> 00:05:25,700 And then when I peeled off the lambdas, 94 00:05:25,700 --> 00:05:30,690 they were on the right-hand side, so I've got S, my matrix, 95 00:05:30,690 --> 00:05:31,750 back again. 96 00:05:31,750 --> 00:05:45,500 And this matrix, this diagonal matrix, the eigenvalue matrix, 97 00:05:45,500 --> 00:05:49,210 and I call it capital lambda. 98 00:05:49,210 --> 00:05:52,830 Using capital letters for matrices and lambda 99 00:05:52,830 --> 00:05:54,980 to prompt me that it's, that it's 100 00:05:54,980 --> 00:05:56,850 eigenvalues that are in there. 101 00:05:56,850 --> 00:05:58,840 So you see that the eigenvalues are just 102 00:05:58,840 --> 00:06:01,480 sitting down that diagonal? 103 00:06:01,480 --> 00:06:07,340 If I had a column x2 here, I would want the lambda two 104 00:06:07,340 --> 00:06:10,970 in the two two position, in the diagonal position, 105 00:06:10,970 --> 00:06:15,140 to multiply that x2 and give me the lambda two x2. 106 00:06:15,140 --> 00:06:16,730 That's my formula. 107 00:06:16,730 --> 00:06:19,770 A S is S lambda. 108 00:06:23,000 --> 00:06:23,720 OK. 109 00:06:23,720 --> 00:06:27,530 That's the -- you see, it's just a calculation. 110 00:06:27,530 --> 00:06:31,390 Now -- I mentioned, and I have to mention again, 111 00:06:31,390 --> 00:06:37,210 this business about n independent eigenvectors. 112 00:06:37,210 --> 00:06:40,050 As it stands, this is all fine, whether -- 113 00:06:40,050 --> 00:06:43,240 I mean, I could be repeating the same eigenvector, but -- 114 00:06:43,240 --> 00:06:45,550 I'm not interested in that. 115 00:06:45,550 --> 00:06:50,600 I want to be able to invert S, and that's where this comes in. 116 00:06:50,600 --> 00:06:53,690 This n independent eigenvectors business 117 00:06:53,690 --> 00:06:56,990 comes in to tell me that that matrix is invertible. 118 00:06:56,990 --> 00:07:01,380 So let me, on the next board, write down what I've got. 119 00:07:01,380 --> 00:07:06,780 A S equals S lambda. 120 00:07:06,780 --> 00:07:11,890 And now I'm, I can multiply on the left by S inverse. 121 00:07:11,890 --> 00:07:13,839 So this is really -- 122 00:07:17,970 --> 00:07:20,660 I can do that, provided S is invertible. 123 00:07:20,660 --> 00:07:26,320 Provided my assumption of n independent eigenvectors is 124 00:07:26,320 --> 00:07:27,260 satisfied. 125 00:07:27,260 --> 00:07:31,470 And I mentioned at the end of last time, and I'll say again, 126 00:07:31,470 --> 00:07:35,930 that there's a small number of matrices for -- 127 00:07:35,930 --> 00:07:38,680 that don't have n independent eigenvectors. 128 00:07:38,680 --> 00:07:42,580 So I've got to discuss that, that technical point. 129 00:07:42,580 --> 00:07:48,030 But the great -- the most matrices that we see have n di- 130 00:07:48,030 --> 00:07:51,050 n independent eigenvectors, and we can diagonalize. 131 00:07:51,050 --> 00:07:54,090 This is diagonalization. 132 00:07:54,090 --> 00:07:57,890 I could also write it, and I often will, 133 00:07:57,890 --> 00:07:59,680 the other way round. 134 00:07:59,680 --> 00:08:03,140 If I multiply on the right by S inverse, 135 00:08:03,140 --> 00:08:06,130 if I took this equation at the top and multiplied on the right 136 00:08:06,130 --> 00:08:08,090 by S inverse, I could -- 137 00:08:08,090 --> 00:08:10,900 I would have A left here. 138 00:08:10,900 --> 00:08:15,790 Now S inverse is coming from the right. 139 00:08:15,790 --> 00:08:18,530 So can you keep those two straight? 140 00:08:18,530 --> 00:08:21,491 A multiplies its eigenvectors, that's how I keep them 141 00:08:21,491 --> 00:08:21,990 straight. 142 00:08:21,990 --> 00:08:24,530 So A multiplies S. 143 00:08:24,530 --> 00:08:26,020 A multiplies S. 144 00:08:26,020 --> 00:08:30,700 And then this S inverse makes the whole thing diagonal. 145 00:08:30,700 --> 00:08:33,750 And this is another way of saying the same thing, 146 00:08:33,750 --> 00:08:36,700 putting the Ss on the other side of the equation. 147 00:08:36,700 --> 00:08:40,179 A is S lambda S inverse. 148 00:08:40,179 --> 00:08:44,520 So that's the, that's the new factorization. 149 00:08:44,520 --> 00:08:50,550 That's the replacement for L U from elimination or Q R for -- 150 00:08:50,550 --> 00:08:52,210 from Gram-Schmidt. 151 00:08:52,210 --> 00:08:56,850 And notice that the matrix -- so it's, it's a matrix times 152 00:08:56,850 --> 00:09:00,420 a diagonal matrix times the inverse of the first one. 153 00:09:00,420 --> 00:09:02,410 It's, that's the combination that we'll 154 00:09:02,410 --> 00:09:05,530 see throughout this chapter. 155 00:09:05,530 --> 00:09:09,650 This combination with an S and an S inverse. 156 00:09:09,650 --> 00:09:10,250 OK. 157 00:09:10,250 --> 00:09:14,080 Can I just begin to use that? 158 00:09:14,080 --> 00:09:17,730 For example, what about A squared? 159 00:09:17,730 --> 00:09:20,880 What are the eigenvalues and eigenvectors of A squared? 160 00:09:20,880 --> 00:09:23,040 That's a straightforward question with a, 161 00:09:23,040 --> 00:09:25,380 with an absolutely clean answer. 162 00:09:25,380 --> 00:09:28,930 So let me, let me consider A squared. 163 00:09:28,930 --> 00:09:33,560 So I start with A x equal lambda x. 164 00:09:33,560 --> 00:09:36,070 And I'm headed for A squared. 165 00:09:36,070 --> 00:09:38,610 So let me multiply both sides by A. 166 00:09:38,610 --> 00:09:41,290 That's one way to get A squared on the left. 167 00:09:41,290 --> 00:09:46,190 So -- I should write these if-s in here. 168 00:09:46,190 --> 00:09:51,010 If A x equals lambda x, then I multiply by A, 169 00:09:51,010 --> 00:09:56,200 so I get A squared x equals -- well, I'm multiplying by A, 170 00:09:56,200 --> 00:09:58,670 so that's lambda A x. 171 00:09:58,670 --> 00:10:02,520 That lambda was a number, so I just put it on the left. 172 00:10:02,520 --> 00:10:06,740 And what do I -- tell me how to make that look better. 173 00:10:06,740 --> 00:10:10,220 What have I got here for if, if A 174 00:10:10,220 --> 00:10:13,220 has the eigenvalue lambda and eigenvector 175 00:10:13,220 --> 00:10:16,010 x, what's up with A squared? 176 00:10:16,010 --> 00:10:18,430 A squared x, I just multiplied by A, 177 00:10:18,430 --> 00:10:23,070 but now for Ax I'm going to substitute lambda x. 178 00:10:23,070 --> 00:10:25,010 So I've got lambda squared x. 179 00:10:28,120 --> 00:10:31,370 So from that simple calculation, I -- 180 00:10:31,370 --> 00:10:37,330 my conclusion is that the eigenvalues of A squared are 181 00:10:37,330 --> 00:10:39,310 lambda squared. 182 00:10:39,310 --> 00:10:40,384 And the eigenvectors -- 183 00:10:40,384 --> 00:10:41,550 I always think about both of 184 00:10:41,550 --> 00:10:42,150 those. 185 00:10:42,150 --> 00:10:44,390 What can I say about the eigenvalues? 186 00:10:44,390 --> 00:10:45,560 They're squared. 187 00:10:45,560 --> 00:10:48,220 What can I say about the eigenvectors? 188 00:10:48,220 --> 00:10:49,730 They're the same. 189 00:10:49,730 --> 00:10:54,090 The same x as in -- as for A. 190 00:10:54,090 --> 00:11:00,320 Now let me see that also from this formula. 191 00:11:00,320 --> 00:11:05,160 How can I see what A squared is looking like from this formula? 192 00:11:05,160 --> 00:11:08,480 So let me -- that was one way to do it. 193 00:11:08,480 --> 00:11:12,510 Let me do it by just taking A squared from that. 194 00:11:12,510 --> 00:11:18,990 A squared is S lambda S inverse -- that's A -- 195 00:11:18,990 --> 00:11:22,620 times S lambda S inverse -- that's A, which is? 196 00:11:25,530 --> 00:11:30,170 This is the beauty of eigenvalues, eigenvectors. 197 00:11:30,170 --> 00:11:34,000 Having that S inverse and S is the identity, 198 00:11:34,000 --> 00:11:40,220 so I've got S lambda squared S inverse. 199 00:11:40,220 --> 00:11:43,380 Do you see what that's telling me? 200 00:11:43,380 --> 00:11:46,570 It's, it's telling me the same thing that I just learned here, 201 00:11:46,570 --> 00:11:49,630 but in the -- in a matrix form. 202 00:11:49,630 --> 00:11:52,720 It's telling me that the S is the same, 203 00:11:52,720 --> 00:11:57,130 the eigenvectors are the same, but the eigenvalues 204 00:11:57,130 --> 00:11:58,530 are squared. 205 00:11:58,530 --> 00:12:01,020 Because this is -- what's lambda squared? 206 00:12:01,020 --> 00:12:02,800 That's still diagonal. 207 00:12:02,800 --> 00:12:05,180 It's got little lambda one squared, 208 00:12:05,180 --> 00:12:07,190 lambda two squared, down to lambda n 209 00:12:07,190 --> 00:12:09,280 squared o- on that diagonal. 210 00:12:09,280 --> 00:12:13,760 Those are the eigenvalues, as we just learned, of A squared. 211 00:12:13,760 --> 00:12:14,500 OK. 212 00:12:14,500 --> 00:12:19,590 So -- somehow those eigenvalues and eigenvectors are really 213 00:12:19,590 --> 00:12:25,060 giving you a way to -- 214 00:12:25,060 --> 00:12:29,460 see what's going on inside a matrix. 215 00:12:29,460 --> 00:12:32,100 Of course I can continue that for -- 216 00:12:32,100 --> 00:12:36,270 to the K-th power, A to the K-th power. 217 00:12:36,270 --> 00:12:39,370 If I multiply, if I have K of these together, 218 00:12:39,370 --> 00:12:42,920 do you see how S inverse S will keep canceling 219 00:12:42,920 --> 00:12:44,900 in the, in the inside? 220 00:12:44,900 --> 00:12:48,740 I'll have the S outside at the far left, 221 00:12:48,740 --> 00:12:54,390 and lambda will be in there K times, and S inverse. 222 00:12:54,390 --> 00:12:56,670 So what's that telling me? 223 00:12:56,670 --> 00:12:59,420 That's telling me that the eigenvalues 224 00:12:59,420 --> 00:13:04,340 of A, of A to the K-th power are the K-th powers. 225 00:13:04,340 --> 00:13:08,390 The eigenvalues of A cubed are the cubes of the eigenvalues of 226 00:13:08,390 --> 00:13:15,000 A. And the eigenvectors are the same, the same. 227 00:13:15,000 --> 00:13:15,600 OK. 228 00:13:15,600 --> 00:13:20,750 In other words, eigenvalues and eigenvectors 229 00:13:20,750 --> 00:13:25,910 give a great way to understand the powers of a matrix. 230 00:13:25,910 --> 00:13:28,380 If I take the square of a matrix, 231 00:13:28,380 --> 00:13:30,540 or the hundredth power of a matrix, 232 00:13:30,540 --> 00:13:34,780 the pivots are all over the place. 233 00:13:34,780 --> 00:13:39,690 L U, if I multiply L U times L U times L U times L U 234 00:13:39,690 --> 00:13:44,610 a hundred times, I've got a hundred L Us. 235 00:13:44,610 --> 00:13:46,370 I can't do anything with them. 236 00:13:46,370 --> 00:13:50,180 But when I multiply S lambda S inverse by itself, 237 00:13:50,180 --> 00:13:54,720 when I look at the eigenvector picture a hundred times, 238 00:13:54,720 --> 00:13:59,810 I get a hundred or ninety-nine of these guys canceling out 239 00:13:59,810 --> 00:14:03,390 inside, and I get A to the hundredth 240 00:14:03,390 --> 00:14:05,900 is S lambda to the hundredth S inverse. 241 00:14:05,900 --> 00:14:09,870 I mean, eigenvalues tell you about powers 242 00:14:09,870 --> 00:14:16,290 of a matrix in a way that we had no way to approach previously. 243 00:14:16,290 --> 00:14:21,220 For example, when does -- 244 00:14:21,220 --> 00:14:24,790 when do the powers of a matrix go to zero? 245 00:14:24,790 --> 00:14:29,330 I would call that matrix stable, maybe. 246 00:14:29,330 --> 00:14:31,940 So I could write down a theorem. 247 00:14:31,940 --> 00:14:36,720 I'll write it as a theorem just to use that word 248 00:14:36,720 --> 00:14:40,350 to emphasize that here I'm getting this great fact 249 00:14:40,350 --> 00:14:42,610 from this eigenvalue picture. 250 00:14:42,610 --> 00:14:43,230 OK. 251 00:14:43,230 --> 00:14:53,720 A to the K approaches zero as K goes, as K gets bigger if what? 252 00:14:53,720 --> 00:14:57,840 What's the w- how can I tell, for a matrix A, 253 00:14:57,840 --> 00:14:59,850 if its powers go to zero? 254 00:15:02,650 --> 00:15:07,520 What's -- somewhere inside that matrix is that information. 255 00:15:07,520 --> 00:15:11,510 That information is not present in the pivots. 256 00:15:11,510 --> 00:15:13,270 It's present in the eigenvalues. 257 00:15:13,270 --> 00:15:17,010 What do I need for the -- to know that if I take higher 258 00:15:17,010 --> 00:15:20,170 and higher powers of A, that this matrix gets smaller 259 00:15:20,170 --> 00:15:21,030 and smaller? 260 00:15:21,030 --> 00:15:24,330 Well, S and S inverse are not moving. 261 00:15:24,330 --> 00:15:26,810 So it's this guy that has to get small. 262 00:15:26,810 --> 00:15:29,720 And that's easy to -- to understand. 263 00:15:29,720 --> 00:15:32,595 The requirement is all eigenvalues -- 264 00:15:35,330 --> 00:15:38,040 so what is the requirement? 265 00:15:38,040 --> 00:15:41,170 The eigenvalues have to be less than one. 266 00:15:41,170 --> 00:15:45,310 Now I have to wrote that absolute value, 267 00:15:45,310 --> 00:15:48,360 because those eigenvalues could be negative, 268 00:15:48,360 --> 00:15:50,550 they could be complex numbers. 269 00:15:50,550 --> 00:15:53,170 So I'm taking the absolute value. 270 00:15:53,170 --> 00:15:56,750 If all of those are below one. 271 00:15:56,750 --> 00:16:05,000 That's, in fact, we practically see why. 272 00:16:05,000 --> 00:16:13,100 And let me just say that I'm operating on one assumption 273 00:16:13,100 --> 00:16:15,740 here, and I got to keep remembering 274 00:16:15,740 --> 00:16:18,690 that that assumption is still present. 275 00:16:18,690 --> 00:16:21,420 That assumption was that I had a full set of, 276 00:16:21,420 --> 00:16:24,370 of n independent eigenvectors. 277 00:16:24,370 --> 00:16:30,830 If I don't have that, then this approach is not working. 278 00:16:30,830 --> 00:16:37,090 So again, a pure eigenvalue approach, eigenvector approach, 279 00:16:37,090 --> 00:16:40,470 needs n independent eigenvectors. 280 00:16:40,470 --> 00:16:42,900 If we don't have n independent eigenvectors, 281 00:16:42,900 --> 00:16:46,490 we can't diagonalize the matrix. 282 00:16:46,490 --> 00:16:50,820 We can't get to a diagonal matrix. 283 00:16:50,820 --> 00:16:55,960 This diagonalization is only possible 284 00:16:55,960 --> 00:16:58,581 if S inverse makes sense. 285 00:16:58,581 --> 00:16:59,080 OK. 286 00:16:59,080 --> 00:17:02,800 Can I, can I follow up on that point now? 287 00:17:02,800 --> 00:17:07,099 So you see why -- what we get and, and why we want it, 288 00:17:07,099 --> 00:17:11,490 because we get information about the powers of a matrix just 289 00:17:11,490 --> 00:17:14,940 immediately from the eigenvalues. 290 00:17:14,940 --> 00:17:15,500 OK. 291 00:17:15,500 --> 00:17:22,329 Now let me follow up on this, business of which matrices 292 00:17:22,329 --> 00:17:25,030 are diagonalizable. 293 00:17:25,030 --> 00:17:28,220 Sorry about that long word. 294 00:17:28,220 --> 00:17:32,940 So a matrix is, is sure -- so here's, here's the main point. 295 00:17:32,940 --> 00:17:37,600 A is sure to be -- 296 00:17:37,600 --> 00:17:50,110 to have N independent eigenvectors and, and be -- 297 00:17:50,110 --> 00:18:00,210 now here comes that word -- diagonalizable if, if -- 298 00:18:00,210 --> 00:18:05,810 so we might as well get the nice case out in the open. 299 00:18:05,810 --> 00:18:13,330 The nice case is when -- if all the lambdas are different. 300 00:18:19,520 --> 00:18:28,326 That means, that means no repeated eigenvalues. 301 00:18:30,920 --> 00:18:32,000 OK. 302 00:18:32,000 --> 00:18:34,410 That's the nice case. 303 00:18:34,410 --> 00:18:39,970 If my matrix, and most -- if I do a random matrix in Matlab 304 00:18:39,970 --> 00:18:43,140 and compute its eigenvalues -- 305 00:18:43,140 --> 00:18:55,540 so if I computed if I took eig of rand of ten ten, gave, 306 00:18:55,540 --> 00:18:58,640 gave that Matlab command, the -- 307 00:18:58,640 --> 00:19:01,190 we'd get a random ten by ten matrix, 308 00:19:01,190 --> 00:19:03,990 we would get a list of its ten eigenvalues, 309 00:19:03,990 --> 00:19:08,040 and they would be different. 310 00:19:08,040 --> 00:19:10,470 They would be distinct is the best word. 311 00:19:10,470 --> 00:19:13,880 I would have -- a random matrix will have ten distinct -- 312 00:19:13,880 --> 00:19:18,090 a ten by ten matrix will have ten distinct eigenvalues. 313 00:19:18,090 --> 00:19:25,500 And if it does, the eigenvectors are automatically independent. 314 00:19:25,500 --> 00:19:26,930 So that's a nice fact. 315 00:19:26,930 --> 00:19:29,730 I'll refer you to the text for the proof. 316 00:19:29,730 --> 00:19:36,360 That, that A is sure to have n independent eigenvectors 317 00:19:36,360 --> 00:19:41,610 if the eigenvalues are different, if. 318 00:19:41,610 --> 00:19:43,890 If all the, if all eigenvalues are different. 319 00:19:43,890 --> 00:19:47,810 It's just if some lambdas are repeated, 320 00:19:47,810 --> 00:19:50,560 then I have to look more closely. 321 00:19:50,560 --> 00:19:55,050 If an eigenvalue is repeated, I have to look, I have to count, 322 00:19:55,050 --> 00:19:56,260 I have to check. 323 00:19:56,260 --> 00:19:59,560 Has it got -- say it's repeated three times. 324 00:19:59,560 --> 00:20:02,030 So what's a possibility for the -- 325 00:20:02,030 --> 00:20:05,893 so here is the, here is the repeated possibility. 326 00:20:11,000 --> 00:20:16,490 And, and let me emphasize the conclusion. 327 00:20:16,490 --> 00:20:21,410 That if I have repeated eigenvalues, I may or may not, 328 00:20:21,410 --> 00:20:35,271 I may or may not have, have n independent eigenvectors. 329 00:20:35,271 --> 00:20:35,770 I might. 330 00:20:35,770 --> 00:20:41,240 I, I, you know, this isn't a completely negative case. 331 00:20:41,240 --> 00:20:43,260 The identity matrix -- 332 00:20:43,260 --> 00:20:46,380 suppose I take the ten by ten identity matrix. 333 00:20:46,380 --> 00:20:50,750 What are the eigenvalues of that matrix? 334 00:20:50,750 --> 00:20:55,370 So just, just take the easiest matrix, the identity. 335 00:20:55,370 --> 00:21:00,470 If I look for its eigenvalues, they're all ones. 336 00:21:00,470 --> 00:21:04,410 So that eigenvalue one is repeated ten times. 337 00:21:04,410 --> 00:21:07,340 But there's no shortage of eigenvectors for the identity 338 00:21:07,340 --> 00:21:08,250 matrix. 339 00:21:08,250 --> 00:21:10,760 In fact, every vector is an eigenvector. 340 00:21:10,760 --> 00:21:13,610 So I can take ten independent vectors. 341 00:21:13,610 --> 00:21:16,530 Oh, well, what happens to everything -- 342 00:21:16,530 --> 00:21:18,590 if A is the identity matrix, let's 343 00:21:18,590 --> 00:21:21,650 just think that one through in our head. 344 00:21:21,650 --> 00:21:27,440 If A is the identity matrix, then it's 345 00:21:27,440 --> 00:21:28,830 got plenty of eigenvectors. 346 00:21:28,830 --> 00:21:30,910 I choose ten independent vectors. 347 00:21:30,910 --> 00:21:32,560 They're the columns of S. 348 00:21:32,560 --> 00:21:37,380 And, and what do I get from S inverse A S? 349 00:21:37,380 --> 00:21:39,400 I get I again, right? 350 00:21:39,400 --> 00:21:42,210 If A is the identity -- and of course that's the correct 351 00:21:42,210 --> 00:21:43,790 lambda. 352 00:21:43,790 --> 00:21:46,790 The matrix was already diagonal. 353 00:21:46,790 --> 00:21:48,970 So if the matrix is already diagonal, 354 00:21:48,970 --> 00:21:53,800 then the, the lambda is the same as the matrix. 355 00:21:53,800 --> 00:21:56,380 A diagonal matrix has got its eigenvalues 356 00:21:56,380 --> 00:21:59,170 sitting right there in front of you. 357 00:21:59,170 --> 00:22:01,790 Now if it's triangular, the eigenvalues 358 00:22:01,790 --> 00:22:04,820 are still sitting there, but so let's 359 00:22:04,820 --> 00:22:08,460 take a case where it's triangular. 360 00:22:08,460 --> 00:22:14,870 Suppose A is like, two one two zero. 361 00:22:17,920 --> 00:22:23,290 So there's a case that's going to be trouble. 362 00:22:23,290 --> 00:22:25,040 There's a case that's going to be trouble. 363 00:22:25,040 --> 00:22:26,360 First of all, what are the -- 364 00:22:26,360 --> 00:22:29,410 I mean, we just -- 365 00:22:29,410 --> 00:22:31,190 if we start with a matrix, the first thing 366 00:22:31,190 --> 00:22:32,890 we do, practically without thinking 367 00:22:32,890 --> 00:22:36,130 is compute the eigenvalues and eigenvectors. 368 00:22:36,130 --> 00:22:36,630 OK. 369 00:22:36,630 --> 00:22:38,360 So what are the eigenvalues? 370 00:22:38,360 --> 00:22:41,130 You can tell me right away what they are. 371 00:22:41,130 --> 00:22:43,880 They're two and two, right. 372 00:22:43,880 --> 00:22:47,770 It's a triangular matrix, so when I do this determinant, 373 00:22:47,770 --> 00:22:51,680 shall I do this determinant of A minus lambda I? 374 00:22:51,680 --> 00:22:59,990 I'll get this two minus lambda one zero two minus lambda, 375 00:22:59,990 --> 00:23:01,700 right? 376 00:23:01,700 --> 00:23:06,890 I take that determinant, so I make those into vertical bars 377 00:23:06,890 --> 00:23:09,130 to mean determinant. 378 00:23:09,130 --> 00:23:10,810 And what's the determinant? 379 00:23:10,810 --> 00:23:13,140 It's two minus lambda squared. 380 00:23:13,140 --> 00:23:14,570 What are the roots? 381 00:23:14,570 --> 00:23:17,410 Lambda equal two twice. 382 00:23:17,410 --> 00:23:22,640 So the eigenvalues are lambda equals two and two. 383 00:23:22,640 --> 00:23:23,580 OK, fine. 384 00:23:23,580 --> 00:23:26,810 Now the next step, find the eigenvectors. 385 00:23:26,810 --> 00:23:31,940 So I look for eigenvectors, and what do I find for this guy? 386 00:23:31,940 --> 00:23:33,530 Eigenvectors for this guy, when I 387 00:23:33,530 --> 00:23:38,340 subtract two minus the identity, so A minus two 388 00:23:38,340 --> 00:23:42,280 I has zeros here. 389 00:23:45,420 --> 00:23:48,740 And I'm looking for the null space. 390 00:23:48,740 --> 00:23:50,390 What's, what are the eigenvectors? 391 00:23:50,390 --> 00:23:56,540 They're the -- the null space of A minus lambda I. 392 00:23:56,540 --> 00:23:59,200 The null space is only one dimensional. 393 00:23:59,200 --> 00:24:03,700 This is a case where I don't have enough eigenvectors. 394 00:24:03,700 --> 00:24:07,940 My algebraic multiplicity is two. 395 00:24:07,940 --> 00:24:10,520 I would say, when I see, when I count 396 00:24:10,520 --> 00:24:16,210 how often the eigenvalue is repeated, 397 00:24:16,210 --> 00:24:18,410 that's the algebraic multiplicity. 398 00:24:18,410 --> 00:24:20,650 That's the multiplicity, how many times 399 00:24:20,650 --> 00:24:22,770 is it the root of the polynomial? 400 00:24:22,770 --> 00:24:28,340 My polynomial is two minus lambda squared. 401 00:24:28,340 --> 00:24:30,040 It's a double root. 402 00:24:30,040 --> 00:24:33,240 So my algebraic multiplicity is two. 403 00:24:33,240 --> 00:24:37,110 But the geometric multiplicity, which looks for vectors, 404 00:24:37,110 --> 00:24:42,130 looks for eigenvectors, and -- which means the null space 405 00:24:42,130 --> 00:24:46,500 of this thing, and the only eigenvector is one 406 00:24:46,500 --> 00:24:47,360 zero. 407 00:24:47,360 --> 00:24:50,140 That's in the null space. 408 00:24:50,140 --> 00:24:52,600 Zero one is not in the null space. 409 00:24:52,600 --> 00:24:54,540 The null space is only one dimensional. 410 00:24:54,540 --> 00:24:58,960 So there's a matrix, my -- this A or the original A, 411 00:24:58,960 --> 00:25:02,310 that are not diagonalizable. 412 00:25:02,310 --> 00:25:06,360 I can't find two independent eigenvectors. 413 00:25:06,360 --> 00:25:08,090 There's only one. 414 00:25:08,090 --> 00:25:08,700 OK. 415 00:25:08,700 --> 00:25:11,710 So that's the case that I'm -- 416 00:25:11,710 --> 00:25:15,520 that's a case that I'm not really handling. 417 00:25:15,520 --> 00:25:19,590 For example, when I wrote down up here 418 00:25:19,590 --> 00:25:24,490 that the powers went to zero if the eigenvalues were below one, 419 00:25:24,490 --> 00:25:29,210 I didn't really handle that case of repeated eigenvalues, 420 00:25:29,210 --> 00:25:33,980 because my reasoning was based on this formula. 421 00:25:33,980 --> 00:25:36,420 And this formula is based on n independent eigenvectors. 422 00:25:36,420 --> 00:25:36,920 OK. 423 00:25:36,920 --> 00:25:45,610 Just to say then, there are some matrices that we're, that, 424 00:25:45,610 --> 00:25:48,730 that we don't cover through diagonalization, 425 00:25:48,730 --> 00:25:51,070 but the great majority we do. 426 00:25:51,070 --> 00:25:51,720 OK. 427 00:25:51,720 --> 00:25:54,040 And we, we're always OK if we have 428 00:25:54,040 --> 00:25:56,550 different distinct eigenvalues. 429 00:25:56,550 --> 00:26:02,390 OK, that's the, like, the typical case. 430 00:26:02,390 --> 00:26:04,660 Because for each eigenvalue there's 431 00:26:04,660 --> 00:26:07,140 at least one eigenvector. 432 00:26:07,140 --> 00:26:11,530 The algebraic multiplicity here is one for every eigenvalue 433 00:26:11,530 --> 00:26:14,080 and the geometric multiplicity is one. 434 00:26:14,080 --> 00:26:15,580 There's one eigenvector. 435 00:26:15,580 --> 00:26:17,650 And they are independent. 436 00:26:17,650 --> 00:26:18,150 OK. 437 00:26:18,150 --> 00:26:18,650 OK. 438 00:26:21,690 --> 00:26:26,390 Now let me come back to the important case, when, 439 00:26:26,390 --> 00:26:27,770 when we're OK. 440 00:26:27,770 --> 00:26:31,820 The important case, when we are diagonalizable. 441 00:26:31,820 --> 00:26:38,060 Let me, look at -- 442 00:26:38,060 --> 00:26:42,455 so -- let me solve this equation. 443 00:26:46,680 --> 00:26:49,460 The equation will be each -- 444 00:26:49,460 --> 00:26:57,146 I start with some -- start with a given vector u0. 445 00:27:02,390 --> 00:27:06,080 And then my equation is at every step, 446 00:27:06,080 --> 00:27:11,600 I multiply what I have by A. 447 00:27:11,600 --> 00:27:16,550 That, that equation ought to be simple to handle. 448 00:27:19,940 --> 00:27:21,940 And I'd like to be able to solve it. 449 00:27:21,940 --> 00:27:26,840 How would I find -- if I start with a vector u0 and I multiply 450 00:27:26,840 --> 00:27:31,470 by A a hundred times, what have I got? 451 00:27:31,470 --> 00:27:35,310 Well, I could certainly write down a formula for the answer, 452 00:27:35,310 --> 00:27:39,295 so what, what -- so u1 is A u0. 453 00:27:42,170 --> 00:27:45,800 And u2 is -- what's u2 then? 454 00:27:45,800 --> 00:27:52,350 u2, I multiply -- u2 I get from u1 by another multiplying by A, 455 00:27:52,350 --> 00:27:55,830 so I've got A twice. 456 00:27:55,830 --> 00:28:02,120 And my formula is uk, after k steps, 457 00:28:02,120 --> 00:28:07,580 I've multiplied by A k times the original u0. 458 00:28:07,580 --> 00:28:11,220 You see what I'm doing? 459 00:28:11,220 --> 00:28:14,050 The next section is going to solve systems 460 00:28:14,050 --> 00:28:17,550 of differential equations. 461 00:28:17,550 --> 00:28:19,690 I'm going to have derivatives. 462 00:28:19,690 --> 00:28:23,370 This section is the nice one. 463 00:28:23,370 --> 00:28:26,190 It solves difference equations. 464 00:28:26,190 --> 00:28:28,550 I would call that a difference equation. 465 00:28:28,550 --> 00:28:33,550 It's -- at first order, I would call that a first-order system, 466 00:28:33,550 --> 00:28:40,150 because it connects only -- it only goes up one level. 467 00:28:40,150 --> 00:28:43,180 And I -- it's a system because these are vectors 468 00:28:43,180 --> 00:28:45,960 and that's a matrix. 469 00:28:45,960 --> 00:28:48,470 And the solution is just that. 470 00:28:48,470 --> 00:28:49,090 OK. 471 00:28:49,090 --> 00:28:55,160 But, that's a nice formula. 472 00:28:55,160 --> 00:28:57,500 That's the, like, the most compact formula 473 00:28:57,500 --> 00:29:01,630 I could ever get. u100 would be A to the one hundred u0. 474 00:29:01,630 --> 00:29:06,480 But how would I actually find u100? 475 00:29:06,480 --> 00:29:11,520 How would I find -- how would I discover what u100 is? 476 00:29:11,520 --> 00:29:13,760 Let me, let me show you how. 477 00:29:16,620 --> 00:29:18,630 Here's the idea. 478 00:29:18,630 --> 00:29:23,090 If -- so to solve, to really solve -- shall I say, 479 00:29:23,090 --> 00:29:26,920 to really solve -- 480 00:29:26,920 --> 00:29:34,500 to really solve it, I would take this initial vector u0 481 00:29:34,500 --> 00:29:39,420 and I would write it as a combination of eigenvectors. 482 00:29:39,420 --> 00:29:47,320 To really solve, write u nought as a combination, 483 00:29:47,320 --> 00:29:50,660 say certain amount of the first eigenvector 484 00:29:50,660 --> 00:29:53,480 plus a certain amount of the second eigenvector 485 00:29:53,480 --> 00:29:55,820 plus a certain amount of the last eigenvector. 486 00:30:01,790 --> 00:30:04,740 Now multiply by A. 487 00:30:04,740 --> 00:30:07,350 You want to -- you got to see the magic of eigenvectors 488 00:30:07,350 --> 00:30:08,520 working here. 489 00:30:08,520 --> 00:30:10,360 Multiply by A. 490 00:30:10,360 --> 00:30:13,910 So Au0 is what? 491 00:30:13,910 --> 00:30:16,930 So A times that. 492 00:30:16,930 --> 00:30:18,800 A times -- so what's A -- 493 00:30:18,800 --> 00:30:21,390 I can separate it out into n separate pieces, 494 00:30:21,390 --> 00:30:23,430 and that's the whole point. 495 00:30:23,430 --> 00:30:28,800 That each of those pieces is going in its own merry way. 496 00:30:28,800 --> 00:30:31,320 Each of those pieces is an eigenvector, 497 00:30:31,320 --> 00:30:35,810 and when I multiply by A, what does this piece become? 498 00:30:35,810 --> 00:30:38,450 So that's some amount of the first -- 499 00:30:38,450 --> 00:30:41,030 let's suppose the eigenvectors are normalized to be unit 500 00:30:41,030 --> 00:30:41,530 vectors. 501 00:30:44,750 --> 00:30:48,530 So that says what the eigenvector is. 502 00:30:48,530 --> 00:30:51,340 It's a -- 503 00:30:51,340 --> 00:30:55,220 And I need some multiple of it to produce u0. 504 00:30:55,220 --> 00:30:56,120 OK. 505 00:30:56,120 --> 00:30:59,470 Now when I multiply by A, what do I get? 506 00:30:59,470 --> 00:31:04,350 I get c1, which is just a factor, times Ax1, 507 00:31:04,350 --> 00:31:07,865 but Ax1 is lambda one x1. 508 00:31:10,780 --> 00:31:17,060 When I multiply this by A, I get c2 lambda two x2. 509 00:31:17,060 --> 00:31:20,740 And here I get cn lambda n xn. 510 00:31:20,740 --> 00:31:27,980 And suppose I multiply by A to the hundredth power now. 511 00:31:27,980 --> 00:31:30,840 Can we, having done it, multiplied by A, let's 512 00:31:30,840 --> 00:31:32,890 multiply by A to the hundredth. 513 00:31:32,890 --> 00:31:36,380 What happens to this first term when I multiply by A to the one 514 00:31:36,380 --> 00:31:38,130 hundredth? 515 00:31:38,130 --> 00:31:41,620 It's got that factor lambda to the hundredth. 516 00:31:41,620 --> 00:31:42,890 That's the key. 517 00:31:42,890 --> 00:31:48,440 That -- that's what I mean by going its own merry way. 518 00:31:48,440 --> 00:31:52,320 It, it is pure eigenvector. 519 00:31:52,320 --> 00:31:55,850 It's exactly in a direction where multiplication by A 520 00:31:55,850 --> 00:31:59,200 just brings in a scalar factor, lambda one. 521 00:31:59,200 --> 00:32:02,240 So a hundred times brings in this a hundred times. 522 00:32:02,240 --> 00:32:06,080 Hundred times lambda two, hundred times lambda n. 523 00:32:06,080 --> 00:32:08,830 Actually, we're -- what are we seeing here? 524 00:32:08,830 --> 00:32:15,040 We're seeing, this same, lambda capital 525 00:32:15,040 --> 00:32:19,570 lambda to the hundredth as in the, as in the diagonalization. 526 00:32:19,570 --> 00:32:22,350 And we're seeing the S matrix, the, 527 00:32:22,350 --> 00:32:24,730 the matrix S of eigenvectors. 528 00:32:24,730 --> 00:32:29,440 That's what this has got to -- this has got to amount to. 529 00:32:29,440 --> 00:32:40,030 A lambda to the hundredth power times an S times this vector c 530 00:32:40,030 --> 00:32:43,490 that's telling us how much of each one 531 00:32:43,490 --> 00:32:45,010 is in the original thing. 532 00:32:45,010 --> 00:32:49,010 So if, if I had to really find the hundredth power, 533 00:32:49,010 --> 00:32:54,200 I would take u0, I would expand it as a combination 534 00:32:54,200 --> 00:32:57,210 of eigenvectors -- this is really S, 535 00:32:57,210 --> 00:33:01,680 the eigenvector matrix, times c, the, the coefficient vector. 536 00:33:04,240 --> 00:33:07,310 And then I would immediately then, 537 00:33:07,310 --> 00:33:10,950 by inserting these hundredth powers of eigenvalues, 538 00:33:10,950 --> 00:33:15,490 I'd have the answer. 539 00:33:15,490 --> 00:33:17,880 So -- huh, there must be -- 540 00:33:17,880 --> 00:33:20,570 oh, let's see, OK. 541 00:33:20,570 --> 00:33:22,970 It's -- so, yeah. 542 00:33:22,970 --> 00:33:30,790 So if u100 is A to the hundredth times u0, and u0 is S c -- 543 00:33:30,790 --> 00:33:36,160 then you see this formula is just this formula, 544 00:33:36,160 --> 00:33:40,840 which is the way I would actually get hold of this, 545 00:33:40,840 --> 00:33:44,690 of this u100, which is -- 546 00:33:44,690 --> 00:33:47,180 let me put it here. 547 00:33:47,180 --> 00:33:48,030 u100. 548 00:33:48,030 --> 00:33:51,070 The way I would actually get hold of that, see what, 549 00:33:51,070 --> 00:33:57,400 what the solution is after a hundred steps, would be -- 550 00:33:57,400 --> 00:34:05,960 expand the initial vector into eigenvectors and let each 551 00:34:05,960 --> 00:34:10,020 eigenvector go its own way, multiplying by a hundred at -- 552 00:34:10,020 --> 00:34:13,400 by lambda at every step, and therefore by lambda 553 00:34:13,400 --> 00:34:16,030 to the hundredth power after a hundred steps. 554 00:34:16,030 --> 00:34:18,050 Can I do an example? 555 00:34:18,050 --> 00:34:20,260 So that's the formulas. 556 00:34:20,260 --> 00:34:22,540 Now let me take an example. 557 00:34:22,540 --> 00:34:29,090 I'll use the Fibonacci sequence as an example. 558 00:34:29,090 --> 00:34:31,590 So, so Fibonacci example. 559 00:34:39,830 --> 00:34:43,050 You remember the Fibonacci numbers? 560 00:34:43,050 --> 00:34:48,150 If we start with one and one as F0 -- oh, 561 00:34:48,150 --> 00:34:50,280 I think I start with zero, maybe. 562 00:34:50,280 --> 00:34:54,550 Let zero and one be the first ones. 563 00:34:54,550 --> 00:34:58,550 So there's F0 and F1, the first two Fibonacci numbers. 564 00:34:58,550 --> 00:35:02,840 Then what's the rule for Fibonacci numbers? 565 00:35:02,840 --> 00:35:04,130 Ah, they're the sum. 566 00:35:04,130 --> 00:35:08,030 The next one is the sum of those, so it's one. 567 00:35:08,030 --> 00:35:11,110 The next one is the sum of those, so it's two. 568 00:35:11,110 --> 00:35:14,010 The next one is the sum of those, so it's three. 569 00:35:14,010 --> 00:35:16,190 Well, it looks like one two three four five, 570 00:35:16,190 --> 00:35:19,350 but somehow it's not going to do that way. 571 00:35:19,350 --> 00:35:21,380 The next one is five, right. 572 00:35:21,380 --> 00:35:22,640 Two and three makes five. 573 00:35:22,640 --> 00:35:26,090 The next one is eight. 574 00:35:26,090 --> 00:35:28,370 The next one is thirteen. 575 00:35:28,370 --> 00:35:33,245 And the one hundredth Fibonacci number is what? 576 00:35:35,920 --> 00:35:37,600 That's my question. 577 00:35:37,600 --> 00:35:40,680 How could I get a formula for the hundredth number? 578 00:35:40,680 --> 00:35:44,470 And, for example, how could I answer the question, 579 00:35:44,470 --> 00:35:47,740 how fast are they growing? 580 00:35:47,740 --> 00:35:52,650 How fast are those Fibonacci numbers growing? 581 00:35:52,650 --> 00:35:54,070 They're certainly growing. 582 00:35:54,070 --> 00:35:56,270 It's not a stable case. 583 00:35:56,270 --> 00:35:59,030 Whatever the eigenvalues of whatever matrix it is, 584 00:35:59,030 --> 00:36:00,720 they're not smaller than one. 585 00:36:00,720 --> 00:36:02,540 These numbers are growing. 586 00:36:02,540 --> 00:36:04,450 But how fast are they growing? 587 00:36:04,450 --> 00:36:10,070 The answer lies in the eigenvalue. 588 00:36:10,070 --> 00:36:12,450 So I've got to find the matrix, so let me write down 589 00:36:12,450 --> 00:36:14,495 the Fibonacci rule. 590 00:36:17,610 --> 00:36:22,245 F(k+2) = F(k+1)+F k, right? 591 00:36:25,210 --> 00:36:28,280 Now that's not in my -- 592 00:36:28,280 --> 00:36:32,420 I want to write that as uk plus one and Auk. 593 00:36:32,420 --> 00:36:38,920 But right now what I've got is a single equation, not a system, 594 00:36:38,920 --> 00:36:41,140 and it's second-order. 595 00:36:41,140 --> 00:36:44,290 It's like having a second-order differential equation 596 00:36:44,290 --> 00:36:45,810 with second derivatives. 597 00:36:45,810 --> 00:36:47,580 I want to get first derivatives. 598 00:36:47,580 --> 00:36:49,200 Here I want to get first differences. 599 00:36:49,200 --> 00:36:55,910 So the way, the way to do it is to introduce uk will be 600 00:36:55,910 --> 00:36:57,960 a vector -- 601 00:36:57,960 --> 00:36:59,125 see, a small trick. 602 00:37:01,920 --> 00:37:05,330 Let uk be a vector, F(k+1) and Fk. 603 00:37:08,230 --> 00:37:12,680 So I'm going to get a two by two system, first order, 604 00:37:12,680 --> 00:37:16,890 instead of a one -- instead of a scalar system, second order, 605 00:37:16,890 --> 00:37:18,300 by a simple trick. 606 00:37:18,300 --> 00:37:22,820 I'm just going to add in an equation F(k+1) equals F(k+1). 607 00:37:22,820 --> 00:37:28,980 That will be my second equation. 608 00:37:28,980 --> 00:37:33,940 Then this is my system, this is my unknown, 609 00:37:33,940 --> 00:37:38,690 and what's my one step equation? 610 00:37:38,690 --> 00:37:45,120 So, so now u(k+1), that's -- so u(k+1) is the left side, 611 00:37:45,120 --> 00:37:47,620 and what have I got here on the right side? 612 00:37:47,620 --> 00:37:52,530 I've got some matrix multiplying uk. 613 00:37:52,530 --> 00:37:56,510 Can you, do -- can you see that all right? 614 00:37:56,510 --> 00:37:59,450 if you can see it, then you can tell me what the matrix is. 615 00:37:59,450 --> 00:38:02,860 Do you see that I'm taking my system here. 616 00:38:02,860 --> 00:38:06,550 I artificially made it into a system. 617 00:38:06,550 --> 00:38:10,540 I artificially made the unknown into a vector. 618 00:38:10,540 --> 00:38:14,260 And now I'm ready to look at and see what the matrix 619 00:38:14,260 --> 00:38:15,020 is. 620 00:38:15,020 --> 00:38:20,240 So do you see the left side, u(k+1) is F(k+2) F(k+1), 621 00:38:20,240 --> 00:38:21,940 that's just what I want. 622 00:38:21,940 --> 00:38:25,590 On the right side, this remember, this uk here -- 623 00:38:25,590 --> 00:38:29,960 let me for the moment put it as F(k+1) Fk. 624 00:38:29,960 --> 00:38:33,080 So what's the matrix? 625 00:38:33,080 --> 00:38:41,380 Well, that has a one and a one, and that has a one and a zero. 626 00:38:41,380 --> 00:38:43,080 There's the matrix. 627 00:38:43,080 --> 00:38:47,880 Do you see that that gives me the right-hand side? 628 00:38:47,880 --> 00:38:52,360 So there's the matrix A. 629 00:38:52,360 --> 00:38:56,810 And this is our friend uk. 630 00:38:56,810 --> 00:39:00,650 So we've got -- so that simple trick -- 631 00:39:00,650 --> 00:39:03,900 changed the second-order scalar problem 632 00:39:03,900 --> 00:39:05,730 to a first-order system. 633 00:39:05,730 --> 00:39:08,750 Two b- u- with two unknowns. 634 00:39:08,750 --> 00:39:10,040 With a matrix. 635 00:39:10,040 --> 00:39:13,100 And now what do I do? 636 00:39:13,100 --> 00:39:16,240 Well, before I even think, I find its eigenvalues 637 00:39:16,240 --> 00:39:18,170 and eigenvectors. 638 00:39:18,170 --> 00:39:21,170 So what are the eigenvalues and eigenvectors of that matrix? 639 00:39:23,820 --> 00:39:24,320 Let's see. 640 00:39:24,320 --> 00:39:27,083 I always -- first let me just, like, think for a minute. 641 00:39:29,720 --> 00:39:35,440 It's two by two, so this shouldn't be impossible to do. 642 00:39:35,440 --> 00:39:37,020 Let's do it. 643 00:39:37,020 --> 00:39:37,670 OK. 644 00:39:37,670 --> 00:39:43,170 So my matrix, again, is one one one zero. 645 00:39:46,170 --> 00:39:49,070 It's symmetric, by the way. 646 00:39:49,070 --> 00:39:56,070 So what I will eventually know about symmetric matrices 647 00:39:56,070 --> 00:39:59,140 is that the eigenvalues will come out real. 648 00:39:59,140 --> 00:40:02,290 I won't get any complex numbers here. 649 00:40:02,290 --> 00:40:06,210 And the eigenvectors, once I get those, 650 00:40:06,210 --> 00:40:08,520 actually will be orthogonal. 651 00:40:08,520 --> 00:40:11,190 But two by two, I'm more interested in what 652 00:40:11,190 --> 00:40:13,740 the actual numbers are. 653 00:40:13,740 --> 00:40:16,230 What do I know about the two numbers? 654 00:40:16,230 --> 00:40:18,190 Well, should do you want me to find 655 00:40:18,190 --> 00:40:19,820 this determinant of A minus 656 00:40:19,820 --> 00:40:20,629 lambda I? 657 00:40:20,629 --> 00:40:21,128 Sure. 658 00:40:23,880 --> 00:40:27,910 So it's the determinant of one minus lambda one one zero, 659 00:40:27,910 --> 00:40:28,410 right? 660 00:40:31,900 --> 00:40:33,400 Minus lambda, yes. 661 00:40:33,400 --> 00:40:33,994 God. 662 00:40:33,994 --> 00:40:34,493 OK. 663 00:40:38,030 --> 00:40:40,110 OK. 664 00:40:40,110 --> 00:40:42,240 There'll be two eigenvalues. 665 00:40:42,240 --> 00:40:45,160 What will -- tell me again what I know about the two 666 00:40:45,160 --> 00:40:47,550 eigenvalues before I go any further. 667 00:40:47,550 --> 00:40:49,590 Tell me something about these two eigenvalues. 668 00:40:49,590 --> 00:40:51,770 What do they add up to? 669 00:40:51,770 --> 00:40:55,860 Lambda one plus lambda two is? 670 00:40:55,860 --> 00:41:02,390 Is the same as the trace down the diagonal of the matrix. 671 00:41:02,390 --> 00:41:04,660 One and zero is one. 672 00:41:04,660 --> 00:41:08,320 So lambda one plus lambda two should come out to be one. 673 00:41:08,320 --> 00:41:10,710 And lambda one times lambda one times lambda two 674 00:41:10,710 --> 00:41:13,300 should come out to be the determinant, which 675 00:41:13,300 --> 00:41:15,360 is minus one. 676 00:41:15,360 --> 00:41:18,440 So I'm expecting the eigenvalues to add to one 677 00:41:18,440 --> 00:41:20,570 and to multiply to minus one. 678 00:41:20,570 --> 00:41:22,720 But let's just see it happen here. 679 00:41:22,720 --> 00:41:26,680 If I multiply this out, I get -- that times that'll be a lambda 680 00:41:26,680 --> 00:41:30,290 squared minus lambda minus one. 681 00:41:30,290 --> 00:41:30,790 Good. 682 00:41:33,830 --> 00:41:36,570 Lambda squared minus lambda minus one. 683 00:41:36,570 --> 00:41:43,250 Actually, I -- you see the b- compare that with the original 684 00:41:43,250 --> 00:41:48,655 equation that I started with. 685 00:41:48,655 --> 00:41:49,780 F(k+2) - F(k+1)-Fk is zero. 686 00:41:49,780 --> 00:42:00,330 The recursion that -- that the Fibonacci numbers satisfy is 687 00:42:00,330 --> 00:42:05,140 somehow showing up directly here for the eigenvalues when we set 688 00:42:05,140 --> 00:42:06,230 that to zero. 689 00:42:06,230 --> 00:42:06,730 WK. 690 00:42:06,730 --> 00:42:09,530 Let's solve. 691 00:42:09,530 --> 00:42:14,200 Well, I would like to be able to factor that, that quadratic, 692 00:42:14,200 --> 00:42:17,450 but I'm better off to use the quadratic formula. 693 00:42:17,450 --> 00:42:19,880 Lambda is -- let's see. 694 00:42:19,880 --> 00:42:25,860 Minus b is one plus or minus the square root of b squared, 695 00:42:25,860 --> 00:42:30,650 which is one, minus four times that times that, 696 00:42:30,650 --> 00:42:33,740 which is plus four, over two. 697 00:42:37,614 --> 00:42:39,030 So that's the square root of five. 698 00:42:42,600 --> 00:42:50,230 So the eigenvalues are lambda one is one half of one 699 00:42:50,230 --> 00:42:57,000 plus square root of five, and lambda two is one half of one 700 00:42:57,000 --> 00:42:59,220 minus square root of five. 701 00:42:59,220 --> 00:43:04,630 And sure enough, they -- those add up to one and they multiply 702 00:43:04,630 --> 00:43:06,890 to give minus one. 703 00:43:06,890 --> 00:43:07,390 OK. 704 00:43:07,390 --> 00:43:09,400 Those are the two eigenvalues. 705 00:43:09,400 --> 00:43:12,250 How -- what are those numbers approximately? 706 00:43:12,250 --> 00:43:18,060 Square root of five, well, it's more than two 707 00:43:18,060 --> 00:43:19,030 but less than three. 708 00:43:19,030 --> 00:43:19,860 Hmm. 709 00:43:19,860 --> 00:43:25,330 It'd be nice to know these numbers. 710 00:43:25,330 --> 00:43:30,480 I think, I think that -- so that number comes out bigger than 711 00:43:30,480 --> 00:43:30,980 one, right? 712 00:43:30,980 --> 00:43:31,640 That's right. 713 00:43:31,640 --> 00:43:35,070 This number comes out bigger than one. 714 00:43:35,070 --> 00:43:38,210 It's about one point six one eight or something. 715 00:43:42,610 --> 00:43:44,700 Not exactly, but. 716 00:43:44,700 --> 00:43:48,300 And suppose it's one point six. 717 00:43:48,300 --> 00:43:52,390 Just, like, I think so. 718 00:43:52,390 --> 00:43:54,800 Then what's lambda two? 719 00:43:54,800 --> 00:43:57,870 Is, is lambda two positive or negative? 720 00:43:57,870 --> 00:44:01,430 Negative, right, because I'm -- it's, obviously negative, 721 00:44:01,430 --> 00:44:07,420 and I knew that the -- so it's minus -- 722 00:44:07,420 --> 00:44:16,720 and they add up to one, so minus point six one eight, I guess. 723 00:44:16,720 --> 00:44:17,220 OK. 724 00:44:17,220 --> 00:44:17,800 A- and some more. 725 00:44:17,800 --> 00:44:18,520 Those are the two eigenvalues. 726 00:44:18,520 --> 00:44:19,830 One eigenvalue bigger than one, one eigenvalue smaller than 727 00:44:19,830 --> 00:44:20,330 one. 728 00:44:20,330 --> 00:44:22,480 Actually, that's a great situation to be in. 729 00:44:22,480 --> 00:44:25,430 Of course, the eigenvalues are different, 730 00:44:25,430 --> 00:44:29,340 so there's no doubt whatever -- is this matrix diagonalizable? 731 00:44:32,270 --> 00:44:35,280 Is this matrix diagonalizable, that original matrix A? 732 00:44:35,280 --> 00:44:35,990 Sure. 733 00:44:35,990 --> 00:44:38,030 We've got two distinct eigenvalues 734 00:44:38,030 --> 00:44:44,294 and we can find the eigenvectors in a moment. 735 00:44:44,294 --> 00:44:46,460 But they'll be independent, we'll be diagonalizable. 736 00:44:46,460 --> 00:44:54,790 And now, you, you can already answer my very first question. 737 00:44:54,790 --> 00:44:59,530 How fast are those Fibonacci numbers increasing? 738 00:44:59,530 --> 00:45:01,080 How -- those -- they're increasing, 739 00:45:01,080 --> 00:45:01,810 right? 740 00:45:01,810 --> 00:45:03,970 They're not doubling at every step. 741 00:45:03,970 --> 00:45:07,330 Let me -- let's look again at these numbers. 742 00:45:07,330 --> 00:45:09,580 Five, eight, thirteen, it's not obvious. 743 00:45:09,580 --> 00:45:14,060 The next one would be twenty-one, thirty-four. 744 00:45:14,060 --> 00:45:20,924 So to get some idea of what F one hundred is, 745 00:45:20,924 --> 00:45:21,840 can you give me any -- 746 00:45:21,840 --> 00:45:24,820 I mean the crucial number -- 747 00:45:24,820 --> 00:45:32,280 so it -- these -- it's approximately -- 748 00:45:32,280 --> 00:45:37,970 what's controlling the growth of these Fibonacci numbers? 749 00:45:37,970 --> 00:45:39,630 It's the eigenvalues. 750 00:45:39,630 --> 00:45:43,031 And which eigenvalue is controlling that growth? 751 00:45:43,031 --> 00:45:43,530 The big one. 752 00:45:43,530 --> 00:45:50,380 So F100 will be approximately some constant, c1 I guess, 753 00:45:50,380 --> 00:45:56,110 times this lambda one, this one plus square root of five 754 00:45:56,110 --> 00:46:01,300 over two, to the hundredth power. 755 00:46:01,300 --> 00:46:04,560 And the two hundredth F -- in other words, the eigenvalue -- 756 00:46:04,560 --> 00:46:08,950 the Fibonacci numbers are growing by about that factor. 757 00:46:08,950 --> 00:46:13,780 Do you see that we, we've got precise information about the, 758 00:46:13,780 --> 00:46:18,230 about the Fibonacci numbers out of the eigenvalues? 759 00:46:18,230 --> 00:46:18,940 OK. 760 00:46:18,940 --> 00:46:21,880 And again, why is that true? 761 00:46:21,880 --> 00:46:26,750 Let me go over to this board and s- show what I'm doing here. 762 00:46:26,750 --> 00:46:30,720 The -- the original initial value is some combination 763 00:46:30,720 --> 00:46:31,730 of eigenvectors. 764 00:46:35,520 --> 00:46:39,470 And then when we start -- when we start going out the theories 765 00:46:39,470 --> 00:46:42,580 of Fibonacci numbers, when we start multiplying by A 766 00:46:42,580 --> 00:46:45,980 a hundred times, it's this lambda one to the hundredth. 767 00:46:45,980 --> 00:46:51,070 This term is, is the one that's taking over. 768 00:46:51,070 --> 00:46:54,920 It's -- I mean, that's big, like one point six to the hundredth 769 00:46:54,920 --> 00:46:55,880 power. 770 00:46:55,880 --> 00:47:00,610 The second term is practically nothing, right? 771 00:47:00,610 --> 00:47:04,300 The point six, or minus point six, to the hundredth power 772 00:47:04,300 --> 00:47:08,180 is an extremely small, extremely small number. 773 00:47:08,180 --> 00:47:11,410 So this is -- there're only two terms, 774 00:47:11,410 --> 00:47:13,020 because we're two by two. 775 00:47:13,020 --> 00:47:16,430 This number is -- this piece of it is there, 776 00:47:16,430 --> 00:47:21,270 but it's, it's disappearing, where this piece is there 777 00:47:21,270 --> 00:47:23,890 and it's growing and controlling everything. 778 00:47:23,890 --> 00:47:27,230 So, so really the -- we're doing, like, 779 00:47:27,230 --> 00:47:29,100 problems that are evolving. 780 00:47:29,100 --> 00:47:33,390 We're doing dynamic u- instead of Ax=b, 781 00:47:33,390 --> 00:47:35,440 that's a static problem. 782 00:47:35,440 --> 00:47:36,930 We're now we're doing dynamics. 783 00:47:36,930 --> 00:47:39,740 A, A squared, A cubed, things are evolving in 784 00:47:39,740 --> 00:47:40,440 time. 785 00:47:40,440 --> 00:47:44,660 And the eigenvalues are the crucial, numbers. 786 00:47:44,660 --> 00:47:45,640 OK. 787 00:47:45,640 --> 00:47:52,490 I guess to complete this, I better 788 00:47:52,490 --> 00:47:56,420 write down the eigenvectors. 789 00:47:56,420 --> 00:47:59,160 So we should complete the, the whole process 790 00:47:59,160 --> 00:48:01,200 by finding the eigenvectors. 791 00:48:01,200 --> 00:48:03,820 OK, well, I have to -- up in the corner, then, 792 00:48:03,820 --> 00:48:07,670 I have to look at A minus lambda I. 793 00:48:07,670 --> 00:48:15,800 So A minus lambda I is this one minus lambda one one and minus 794 00:48:15,800 --> 00:48:16,930 lambda. 795 00:48:16,930 --> 00:48:19,990 And now can we spot an eigenvector out of that? 796 00:48:19,990 --> 00:48:23,070 That's, that's, for these two lambdas, 797 00:48:23,070 --> 00:48:24,415 this matrix is singular. 798 00:48:27,380 --> 00:48:30,350 I guess the eigenvector -- two by two ought to be, I mean, 799 00:48:30,350 --> 00:48:31,260 easy. 800 00:48:31,260 --> 00:48:33,960 So if I know that this matrix is singular, 801 00:48:33,960 --> 00:48:37,100 then u- seems to me the eigenvector 802 00:48:37,100 --> 00:48:41,340 has to be lambda and one, because that multiplication 803 00:48:41,340 --> 00:48:43,500 will give me the zero. 804 00:48:43,500 --> 00:48:47,240 And this multiplication gives me -- better give me also zero. 805 00:48:47,240 --> 00:48:48,650 Do you see why it does? 806 00:48:48,650 --> 00:48:52,670 This is the minus lambda squared plus lambda plus one. 807 00:48:52,670 --> 00:48:56,360 It's the thing that's zero because these lambdas are 808 00:48:56,360 --> 00:48:56,930 special. 809 00:48:56,930 --> 00:48:58,490 There's the eigenvector. 810 00:48:58,490 --> 00:49:07,900 x1 is lambda one one, and x2 is lambda two one. 811 00:49:07,900 --> 00:49:12,520 I did that as a little trick that was available in the two 812 00:49:12,520 --> 00:49:14,130 by two case. 813 00:49:14,130 --> 00:49:17,680 So now I finally have to -- 814 00:49:17,680 --> 00:49:20,390 oh, I have to take the initial u0 now. 815 00:49:20,390 --> 00:49:22,710 So to complete this example entirely, 816 00:49:22,710 --> 00:49:26,680 I have to say, OK, what was u0? 817 00:49:26,680 --> 00:49:28,740 u0 was F1 F0. 818 00:49:28,740 --> 00:49:40,630 So u0, the starting vector is F1 F0, and those were one and 819 00:49:40,630 --> 00:49:41,130 zero. 820 00:49:43,910 --> 00:49:47,150 So I have to use that vector. 821 00:49:47,150 --> 00:49:50,390 So I have to look for, for a multiple 822 00:49:50,390 --> 00:49:56,100 of the first eigenvector and the second to produce u0, 823 00:49:56,100 --> 00:49:58,070 the one zero 824 00:49:58,070 --> 00:49:58,570 vector. 825 00:49:58,570 --> 00:50:05,430 This is what will find c1 and c2, and then I'm done. 826 00:50:05,430 --> 00:50:10,470 Do you -- so let me instead of, in the last five seconds, 827 00:50:10,470 --> 00:50:14,920 grinding out a formula, let me repeat the idea. 828 00:50:14,920 --> 00:50:19,100 Because I'd really -- it's the idea that's central. 829 00:50:19,100 --> 00:50:21,610 When things are evolving in time -- 830 00:50:21,610 --> 00:50:25,730 let me come back to this board, because the ideas are here. 831 00:50:25,730 --> 00:50:30,400 When things are evolving in time by a first-order system, 832 00:50:30,400 --> 00:50:34,480 starting from an original u0, the key 833 00:50:34,480 --> 00:50:39,956 is find the eigenvalues and eigenvectors of A. 834 00:50:39,956 --> 00:50:41,580 That will tell -- those eigenvectors -- 835 00:50:41,580 --> 00:50:46,710 the eigenvalues will already tell you what's happening. 836 00:50:46,710 --> 00:50:48,540 Is the solution blowing up, is it 837 00:50:48,540 --> 00:50:51,390 going to zero, what's it doing. 838 00:50:51,390 --> 00:50:56,240 And then to, to find out exactly a formula, 839 00:50:56,240 --> 00:50:59,290 you have to take your u0 and write it 840 00:50:59,290 --> 00:51:03,270 as a combination of eigenvectors and then 841 00:51:03,270 --> 00:51:05,820 follow each eigenvector separately. 842 00:51:05,820 --> 00:51:10,930 And that's really what this formula, the formula for, -- 843 00:51:10,930 --> 00:51:15,190 that's what the formula for A to the K is doing. 844 00:51:15,190 --> 00:51:17,180 So remember that formula for A to the K 845 00:51:17,180 --> 00:51:21,770 is S lambda to the K S inverse. 846 00:51:21,770 --> 00:51:22,280 OK. 847 00:51:22,280 --> 00:51:24,590 That's, that's difference equations. 848 00:51:24,590 --> 00:51:33,460 And you just have to -- so the, the homework will give some 849 00:51:33,460 --> 00:51:41,180 examples, different from Fibonacci, to follow through. 850 00:51:41,180 --> 00:51:48,900 And next time will be differential equations. 851 00:51:48,900 --> 00:51:50,450 Thanks.