1 00:00:00,500 --> 00:00:02,800 The following content is provided under a Creative 2 00:00:02,800 --> 00:00:04,340 Commons license. 3 00:00:04,340 --> 00:00:06,660 Your support will help MIT OpenCourseWare 4 00:00:06,660 --> 00:00:11,020 continue to offer high quality educational resources for free. 5 00:00:11,020 --> 00:00:13,640 To make a donation or view additional materials 6 00:00:13,640 --> 00:00:17,365 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:17,365 --> 00:00:17,990 at ocw.mit.edu. 8 00:00:22,802 --> 00:00:24,260 TOM LEIGHTON: This week we're going 9 00:00:24,260 --> 00:00:26,940 to talk about recurrences, how to set them up, 10 00:00:26,940 --> 00:00:29,570 how to model a problem as a recurrent problem, 11 00:00:29,570 --> 00:00:31,950 and then how to solve them. 12 00:00:31,950 --> 00:00:35,160 This stuff is really useful when you get into algorithm design 13 00:00:35,160 --> 00:00:36,770 and algorithm analysis. 14 00:00:36,770 --> 00:00:41,060 6006 and 6046, we you use this stuff a lot. 15 00:00:41,060 --> 00:00:44,290 We're going to start with a very famous recurrent problem known 16 00:00:44,290 --> 00:00:46,430 as the Towers of Hanoi problem. 17 00:00:46,430 --> 00:00:47,920 And I'm guessing that many of you 18 00:00:47,920 --> 00:00:51,300 have seen this problem in one form or another. 19 00:00:51,300 --> 00:00:54,030 In the Tower of Hanoi problem, you've 20 00:00:54,030 --> 00:01:00,260 got three pegs and n disks, these little guys here. 21 00:01:00,260 --> 00:01:02,040 All different sizes. 22 00:01:02,040 --> 00:01:07,340 And the object is to start with a stack like this 23 00:01:07,340 --> 00:01:09,990 and move the disks around so they wind up 24 00:01:09,990 --> 00:01:13,640 with the stack on another disk. 25 00:01:13,640 --> 00:01:18,750 Now, the rules are you can only move one disk at a time. 26 00:01:18,750 --> 00:01:22,270 And you can never put a big disk over a little disk. 27 00:01:22,270 --> 00:01:24,720 So that you can't do. 28 00:01:24,720 --> 00:01:26,900 So that would have to go there. 29 00:01:26,900 --> 00:01:29,200 Then you could do this. 30 00:01:29,200 --> 00:01:34,220 And now I've gotten the top two moved over to here. 31 00:01:34,220 --> 00:01:37,650 And the goal is to take the minimum number of moves 32 00:01:37,650 --> 00:01:40,980 to get the whole stack moved over to here, 33 00:01:40,980 --> 00:01:45,010 and to even figure out if that's doable. 34 00:01:45,010 --> 00:01:51,910 Now, this was invented by a famous French mathematician 35 00:01:51,910 --> 00:01:55,470 named Edward Lucas in 1883. 36 00:01:55,470 --> 00:01:58,750 And in the original legend around this puzzle, 37 00:01:58,750 --> 00:02:00,870 there were 64 disks. 38 00:02:00,870 --> 00:02:03,770 This one has just seven on it. 39 00:02:03,770 --> 00:02:08,030 And the story was at the beginning of time, 40 00:02:08,030 --> 00:02:14,390 God placed the 64 disks on one of the three golden pegs. 41 00:02:14,390 --> 00:02:18,900 And he got an order of monks whose lifelong mission 42 00:02:18,900 --> 00:02:22,440 was to move the disks one at a time so at the end 43 00:02:22,440 --> 00:02:26,550 all 64 disks were lined up on an adjacent peg. 44 00:02:26,550 --> 00:02:29,990 And according to legend, when that task was done 45 00:02:29,990 --> 00:02:34,540 the tower would crumble and the world would end. 46 00:02:34,540 --> 00:02:37,520 Sort of explains why monks are never 47 00:02:37,520 --> 00:02:39,750 smiling, because they work forever 48 00:02:39,750 --> 00:02:42,530 and then just something bad happens. 49 00:02:42,530 --> 00:02:44,830 Now, the question we want to be able to answer 50 00:02:44,830 --> 00:02:48,220 is, how long till the world ends? 51 00:02:48,220 --> 00:02:53,300 So we can start with a simpler version. 52 00:02:53,300 --> 00:02:58,940 Say let's take three disks and start with that. 53 00:02:58,940 --> 00:03:00,890 And let's see, is it doable? 54 00:03:00,890 --> 00:03:02,660 Well, we could. 55 00:03:02,660 --> 00:03:16,740 That's one, that's two, three, four, five, six, seven. 56 00:03:16,740 --> 00:03:18,820 So it's doable with three disks. 57 00:03:18,820 --> 00:03:20,180 And we took seven moves. 58 00:03:20,180 --> 00:03:22,220 Maybe there's a better way, and that's 59 00:03:22,220 --> 00:03:25,420 part of what we'll try to study today. 60 00:03:25,420 --> 00:03:31,512 In fact, what we'd like to know is how many moves for n disks? 61 00:03:34,820 --> 00:03:36,470 And is it even finite? 62 00:03:36,470 --> 00:03:42,330 So we're going to define t sub n to be the minimum number 63 00:03:42,330 --> 00:03:49,330 of moves for n disks. 64 00:03:49,330 --> 00:03:51,880 So to take a stack of n disks and move them 65 00:03:51,880 --> 00:03:54,510 to an adjacent peg. 66 00:03:54,510 --> 00:03:58,010 So, for example, what's t1? 67 00:03:58,010 --> 00:03:58,510 One. 68 00:03:58,510 --> 00:03:59,860 I got one peg, I just move it. 69 00:03:59,860 --> 00:04:01,726 That's easy. 70 00:04:01,726 --> 00:04:05,040 t2, 3. 71 00:04:05,040 --> 00:04:07,870 Move the little guy, move the base, 72 00:04:07,870 --> 00:04:10,000 move the little guy back on top. 73 00:04:10,000 --> 00:04:12,090 That's the best you can do. 74 00:04:12,090 --> 00:04:16,510 And we've seen that t3 is at most 7. 75 00:04:16,510 --> 00:04:19,060 Maybe there's a better way. 76 00:04:19,060 --> 00:04:22,930 But let's try to first get an upper bound on this. 77 00:04:22,930 --> 00:04:26,220 Any thoughts about how we might go about a general upper 78 00:04:26,220 --> 00:04:31,710 bound for n for this problem? 79 00:04:31,710 --> 00:04:35,270 Like say I had four disks. 80 00:04:35,270 --> 00:04:37,540 Any thoughts about a strategy where 81 00:04:37,540 --> 00:04:39,510 we might be able to analyze or get an upper 82 00:04:39,510 --> 00:04:43,570 bound the number of steps for four disks here? 83 00:04:43,570 --> 00:04:44,400 Or more disks. 84 00:04:44,400 --> 00:04:45,556 Any thoughts? 85 00:04:45,556 --> 00:04:46,056 Yeah. 86 00:04:47,999 --> 00:04:51,412 AUDIENCE: You can look at the previous problem 87 00:04:51,412 --> 00:04:55,890 and then add a certain number of [INAUDIBLE]. 88 00:04:55,890 --> 00:04:57,780 TOM LEIGHTON: Look at the three disk problem. 89 00:04:57,780 --> 00:04:59,370 That's a good idea. 90 00:04:59,370 --> 00:05:03,240 In fact, you know, I could look at the three disk problem 91 00:05:03,240 --> 00:05:04,210 recursively. 92 00:05:04,210 --> 00:05:05,450 I got these three disks. 93 00:05:05,450 --> 00:05:10,160 Say I move them over a series of steps, seven steps at most, 94 00:05:10,160 --> 00:05:12,035 to put them here. 95 00:05:12,035 --> 00:05:13,282 What would I do next? 96 00:05:16,240 --> 00:05:20,000 Move the lower one and then do this recurrent problem 97 00:05:20,000 --> 00:05:22,860 again back. 98 00:05:22,860 --> 00:05:26,310 All right, so we can take a problem with n disks 99 00:05:26,310 --> 00:05:29,900 and solve it recursively. 100 00:05:29,900 --> 00:05:31,698 Let's see what happens when we do that. 101 00:05:35,087 --> 00:05:36,670 So let's look at a recursive solution. 102 00:05:43,920 --> 00:05:46,000 And let's draw it for n. 103 00:05:46,000 --> 00:05:50,220 So the first phase that you described 104 00:05:50,220 --> 00:05:52,430 is I start, here are my three pegs, 105 00:05:52,430 --> 00:05:55,900 and I start with this, 1, 2, 3 down to end here, 106 00:05:55,900 --> 00:05:58,170 and nothing on these pegs. 107 00:05:58,170 --> 00:06:02,520 And I'm going to use recursion to take the top and minus 1 108 00:06:02,520 --> 00:06:04,510 and move them out here. 109 00:06:04,510 --> 00:06:06,130 So I leave the nth guy there. 110 00:06:06,130 --> 00:06:07,680 Nothing here. 111 00:06:07,680 --> 00:06:12,870 And I go 1, 2, to n minus 1. 112 00:06:12,870 --> 00:06:17,870 How many steps does that move take, that series of moves 113 00:06:17,870 --> 00:06:20,650 take, that phase? 114 00:06:20,650 --> 00:06:23,740 In terms of t. 115 00:06:23,740 --> 00:06:28,020 t of n minus 1, because I'm taking an n minus 1 disk stack 116 00:06:28,020 --> 00:06:28,980 and moving it. 117 00:06:28,980 --> 00:06:32,030 So that is tn minus 1 steps. 118 00:06:36,230 --> 00:06:40,600 All right, now my next move is to take this guy 119 00:06:40,600 --> 00:06:41,523 and stick him here. 120 00:06:44,120 --> 00:06:46,460 So we'll draw that over here. 121 00:06:46,460 --> 00:06:49,502 So the nth guy moves there and I leave all these guys here. 122 00:06:52,030 --> 00:06:53,460 That's going to be phase two. 123 00:06:57,840 --> 00:06:59,370 I move the largest disk. 124 00:07:04,120 --> 00:07:07,590 And how many steps did that take? 125 00:07:07,590 --> 00:07:08,500 One step. 126 00:07:08,500 --> 00:07:09,590 Just move one disk. 127 00:07:13,430 --> 00:07:14,370 One step for that. 128 00:07:14,370 --> 00:07:18,620 And then my last phase is to take all these 129 00:07:18,620 --> 00:07:19,708 and move them onto here. 130 00:07:30,240 --> 00:07:34,800 And so then it looks like the final result, 1, 2, 131 00:07:34,800 --> 00:07:36,930 over n minus 1 over n. 132 00:07:36,930 --> 00:07:39,111 And how many steps this phase three take? 133 00:07:42,200 --> 00:07:43,888 tn minus 1. 134 00:07:43,888 --> 00:07:44,388 Good. 135 00:07:47,710 --> 00:07:49,660 All right, so we've got an algorithm now, 136 00:07:49,660 --> 00:07:52,080 a recursive algorithm. 137 00:07:52,080 --> 00:08:03,610 And the total time, the number of moves tn 138 00:08:03,610 --> 00:08:10,820 is at most tn minus 1 plus tn minus 1 is 2tn minus 1 139 00:08:10,820 --> 00:08:14,970 plus the one step up there. 140 00:08:14,970 --> 00:08:16,930 For example, which we already know, 141 00:08:16,930 --> 00:08:26,070 t3 is at most 2t2 plus 1, which is 2 times 3 plus 1 is 7. 142 00:08:26,070 --> 00:08:28,470 So we already knew we could do it in seven. 143 00:08:28,470 --> 00:08:33,210 What's an upper bound on the time to move four disks? 144 00:08:33,210 --> 00:08:34,559 15. 145 00:08:34,559 --> 00:08:36,330 OK? 146 00:08:36,330 --> 00:08:41,390 All right, so t4 is at most 15. 147 00:08:41,390 --> 00:08:43,070 But maybe there's a better way. 148 00:08:43,070 --> 00:08:45,740 This is just one way to do it. 149 00:08:45,740 --> 00:08:51,590 How many people think there's a better way than this approach? 150 00:08:51,590 --> 00:08:53,770 How many people think there is no better way? 151 00:08:53,770 --> 00:08:55,250 All right, very good. 152 00:08:55,250 --> 00:08:57,570 Yeah, this is the best you can do. 153 00:08:57,570 --> 00:08:58,710 And let's prove that. 154 00:08:58,710 --> 00:08:59,820 Let's get a lower bound. 155 00:09:07,500 --> 00:09:13,610 All right, to see why this is optimal, think about the step 156 00:09:13,610 --> 00:09:17,010 where the nth disk moves. 157 00:09:17,010 --> 00:09:23,600 At some point in our procedure, the biggest disk has to move. 158 00:09:23,600 --> 00:09:25,670 Now, it could move to this peg or that peg, 159 00:09:25,670 --> 00:09:26,700 doesn't really matter. 160 00:09:26,700 --> 00:09:29,835 But when it moves, what do I know 161 00:09:29,835 --> 00:09:32,250 had to have happened before that guy moved? 162 00:09:37,150 --> 00:09:42,360 Everything on top of it had to move to the other peg. 163 00:09:42,360 --> 00:09:44,840 I started with 1 through n minus 1 here. 164 00:09:44,840 --> 00:09:46,410 None of them can be here. 165 00:09:46,410 --> 00:09:49,680 They all have to be here. 166 00:09:49,680 --> 00:09:50,970 OK? 167 00:09:50,970 --> 00:09:57,330 So that means that I have to have had tn minus 1 steps. 168 00:09:57,330 --> 00:10:07,792 So at least tn minus 1 steps before the big disk moves. 169 00:10:12,520 --> 00:10:13,560 All right? 170 00:10:13,560 --> 00:10:18,850 Then there is the step when the big guy moves. 171 00:10:18,850 --> 00:10:20,570 One step for the big guy to move. 172 00:10:27,954 --> 00:10:29,578 Because it's got to move at least once. 173 00:10:33,490 --> 00:10:39,140 And then after it moves, I got to get everything else, one way 174 00:10:39,140 --> 00:10:42,500 or another, on top of it. 175 00:10:42,500 --> 00:10:46,110 At some point after the last time the big guy moves, 176 00:10:46,110 --> 00:10:48,890 everything else has to go on top. 177 00:10:48,890 --> 00:10:51,219 And that's another tn minus 1 steps. 178 00:10:55,350 --> 00:11:04,160 So after the last time the big guy moves, 179 00:11:04,160 --> 00:11:05,988 you've got to take tn minus 1 steps. 180 00:11:10,560 --> 00:11:12,640 OK, so that means there's a lower bound 181 00:11:12,640 --> 00:11:21,420 that no matter what you do tn is at least 2tn minus 1 plus 1. 182 00:11:21,420 --> 00:11:24,880 And so in fact, it's equal. 183 00:11:24,880 --> 00:11:30,730 tn equals 2tn minus 1 plus 1. 184 00:11:30,730 --> 00:11:34,750 So t3 is 7, t4 is 15. 185 00:11:34,750 --> 00:11:38,920 Any questions about that? 186 00:11:42,080 --> 00:11:44,590 All right, well we'd like to know how long till the world 187 00:11:44,590 --> 00:11:47,580 ends, which is t64. 188 00:11:47,580 --> 00:11:50,440 So we'd like to get a formula for t of n. 189 00:11:50,440 --> 00:11:54,080 We could just compute it 64 times and see. 190 00:11:54,080 --> 00:11:56,237 But much nicer just to get a formula. 191 00:11:56,237 --> 00:11:58,570 And that's what we're going to learn how to do this week 192 00:11:58,570 --> 00:12:01,570 is to get formulas, closed form expressions, 193 00:12:01,570 --> 00:12:03,304 to solve recurrences. 194 00:12:03,304 --> 00:12:05,470 And there's a bunch of different methods you can use 195 00:12:05,470 --> 00:12:07,380 and we're going to talk about them. 196 00:12:07,380 --> 00:12:12,500 The simplest is to guess the answer and then check it. 197 00:12:12,500 --> 00:12:16,750 It's called guess and verify, also known 198 00:12:16,750 --> 00:12:18,270 as the substitution method. 199 00:12:33,140 --> 00:12:36,850 Now, what's a good guess for t of n? 200 00:12:40,070 --> 00:12:41,250 2 to the n minus 1. 201 00:12:41,250 --> 00:12:49,170 You got t of 1 is 1, t of 2 is 3, t of 3 is 7, t of 4 is 15. 202 00:12:49,170 --> 00:12:57,730 You won't compute too many before you guess that t of n 203 00:12:57,730 --> 00:12:59,291 is 2 to the n minus 1. 204 00:13:02,570 --> 00:13:04,900 Now, that doesn't mean it's true. 205 00:13:04,900 --> 00:13:08,760 So how are we going to verify that guess? 206 00:13:08,760 --> 00:13:11,360 What proof technique? 207 00:13:11,360 --> 00:13:12,600 Induction. 208 00:13:12,600 --> 00:13:14,060 It just keeps coming back. 209 00:13:14,060 --> 00:13:16,880 You just never finish with induction. 210 00:13:16,880 --> 00:13:21,172 So we're going to verify this by induction. 211 00:13:28,922 --> 00:13:30,255 What's our induction hypothesis? 212 00:13:33,020 --> 00:13:33,638 Any thoughts? 213 00:13:38,280 --> 00:13:39,190 Right there. 214 00:13:39,190 --> 00:13:43,670 p of n is going to be that tn equals 2 to the n minus 1. 215 00:13:43,670 --> 00:13:44,340 Very typical. 216 00:13:47,390 --> 00:13:52,670 So the predicate is that t of n is 2 to the n minus 1. 217 00:13:52,670 --> 00:13:53,260 Base case. 218 00:13:57,970 --> 00:14:00,150 We'll pick n equals 1. 219 00:14:00,150 --> 00:14:05,260 T of 1 is 1 and that is 2 to the 1 minus 1. 220 00:14:05,260 --> 00:14:06,310 So the base case works. 221 00:14:09,120 --> 00:14:10,836 Now we'll do the inductive step. 222 00:14:16,950 --> 00:14:20,330 We're going to assume p of n. 223 00:14:20,330 --> 00:14:24,320 We're going to assume that tn equals 2 to the n minus 1 224 00:14:24,320 --> 00:14:30,940 to prove pn minus 1, which is tn plus 1 equals 2 225 00:14:30,940 --> 00:14:33,970 to the n plus 1 minus 1. 226 00:14:33,970 --> 00:14:35,960 So a very standard induction. 227 00:14:35,960 --> 00:14:39,600 So let's look at tn plus 1 now. 228 00:14:39,600 --> 00:14:45,910 tn plus 1 equals 2n plus 1. 229 00:14:45,910 --> 00:14:49,220 Yeah, 2tn plus 1. 230 00:14:49,220 --> 00:14:51,320 Now we plug in here. 231 00:14:51,320 --> 00:14:52,500 That's just the recurrence. 232 00:14:52,500 --> 00:14:56,490 Now we plug in using our assumption from the induction. 233 00:14:56,490 --> 00:15:04,580 This is 2 to the n minus 1 plus 1. 234 00:15:04,580 --> 00:15:05,330 Multiply the 2. 235 00:15:05,330 --> 00:15:07,430 That's 2 to the n plus 1. 236 00:15:07,430 --> 00:15:11,730 Minus 2 plus 1 is minus 1. 237 00:15:11,730 --> 00:15:14,651 And so we're done. 238 00:15:14,651 --> 00:15:16,150 All right, so we guessed the answer. 239 00:15:16,150 --> 00:15:17,600 It was a pretty easy guess. 240 00:15:17,600 --> 00:15:21,087 Very simple to prove it by induction. 241 00:15:21,087 --> 00:15:21,670 Any questions? 242 00:15:24,250 --> 00:15:25,850 All right, how long would it take 243 00:15:25,850 --> 00:15:31,010 me to do all seven disks on this puzzle here? 244 00:15:31,010 --> 00:15:31,795 How many steps? 245 00:15:35,118 --> 00:15:37,060 127 steps. 246 00:15:37,060 --> 00:15:40,350 How long till the end of the world? 247 00:15:40,350 --> 00:15:41,500 A long time. 248 00:15:41,500 --> 00:15:46,070 2 to the 64th minus 1 is 18 billion, billion moves. 249 00:15:46,070 --> 00:15:50,020 So going to keep those monks busy for a little while 250 00:15:50,020 --> 00:15:52,180 before the world ends. 251 00:15:52,180 --> 00:15:56,900 Now in general, this method is sort of the best method 252 00:15:56,900 --> 00:15:58,620 for solving a recurrence. 253 00:15:58,620 --> 00:16:01,950 We'll see lots of examples where it works great. 254 00:16:01,950 --> 00:16:04,530 The only problem is it requires that divine insight 255 00:16:04,530 --> 00:16:06,970 thing, that guess to guess the right answer. 256 00:16:06,970 --> 00:16:10,480 Obviously if we guess the wrong answer, it's not going to work. 257 00:16:10,480 --> 00:16:15,520 Guessing is many times easy, but not always. 258 00:16:15,520 --> 00:16:18,220 So you need other methods when you can't guess it. 259 00:16:18,220 --> 00:16:21,320 The next most common method has a lot of names. 260 00:16:21,320 --> 00:16:26,780 It's called plug and chug, expansion, iteration, 261 00:16:26,780 --> 00:16:30,572 brute force, exhaustion. 262 00:16:30,572 --> 00:16:32,530 Let me show you how that works in this example. 263 00:16:44,640 --> 00:16:46,190 OK, so plug and chug. 264 00:16:51,040 --> 00:16:54,410 So we'll solve the same recurrence. 265 00:16:54,410 --> 00:16:56,880 So we've got tn equals, and I'm going 266 00:16:56,880 --> 00:17:01,030 to write it just reversed, 1 plus 2tn. 267 00:17:01,030 --> 00:17:04,280 Because I'm going to expand out the recurrent term. 268 00:17:04,280 --> 00:17:08,109 That's why it's called expansion also. 269 00:17:08,109 --> 00:17:11,150 Well, that's equal to 1 plus 2. 270 00:17:11,150 --> 00:17:14,230 Now I expand this out using the recurrence. 271 00:17:14,230 --> 00:17:17,066 1 plus 2 tn minus 1. 272 00:17:20,048 --> 00:17:22,250 All right, now I do a little chugging here. 273 00:17:22,250 --> 00:17:24,700 So I plugged here. 274 00:17:24,700 --> 00:17:27,089 Now I'll do chug. 275 00:17:27,089 --> 00:17:33,420 That's 1 plus 2 plus 4 tn minus 1. 276 00:17:33,420 --> 00:17:34,906 Now I'm going to plug in again. 277 00:17:37,580 --> 00:17:38,738 What's that? 278 00:17:38,738 --> 00:17:39,654 AUDIENCE: [INAUDIBLE]. 279 00:17:42,700 --> 00:17:45,290 TOM LEIGHTON: You're right. 280 00:17:45,290 --> 00:17:49,140 You're right, I've got to get minus 1 here and minus 2 here. 281 00:17:49,140 --> 00:17:50,310 That is correct. 282 00:17:50,310 --> 00:17:50,810 Very good. 283 00:17:50,810 --> 00:17:52,400 I would have run into trouble. 284 00:17:52,400 --> 00:17:53,118 That's good. 285 00:17:55,730 --> 00:17:57,350 Now I plug in here. 286 00:17:57,350 --> 00:18:04,680 1 plus 2 plus 4, 1 plus 2 tn minus 3. 287 00:18:04,680 --> 00:18:05,525 Then we do the chug. 288 00:18:19,230 --> 00:18:26,770 1 plus 2 plus 4 plus 8 tn minus 3. 289 00:18:26,770 --> 00:18:29,270 All right, and now we're going to try to observe 290 00:18:29,270 --> 00:18:30,412 the pattern we're getting. 291 00:18:30,412 --> 00:18:32,870 So there's a little bit of a guess going on in this context 292 00:18:32,870 --> 00:18:33,850 still. 293 00:18:33,850 --> 00:18:34,850 But you see the pattern. 294 00:18:34,850 --> 00:18:36,760 We've got powers of 2. 295 00:18:36,760 --> 00:18:37,690 Pretty easy pattern. 296 00:18:37,690 --> 00:18:39,890 So you might say if I keep on doing this, 297 00:18:39,890 --> 00:18:43,170 I'm going to get 1 plus 2 plus 4 plus 298 00:18:43,170 --> 00:18:48,690 dot dot dot plus 2 to the i minus 1 plus 2 299 00:18:48,690 --> 00:18:52,964 to the i tn minus i. 300 00:18:52,964 --> 00:18:56,930 Got 2 to the 3 tn minus 3 there. 301 00:18:56,930 --> 00:19:01,770 And then I take this all the way down to i equals n minus 1, 1 302 00:19:01,770 --> 00:19:10,230 plus 2 plus 4 plus 2 to the n minus 2 plus 2 303 00:19:10,230 --> 00:19:12,910 to the n minus 1 t1. 304 00:19:15,570 --> 00:19:17,300 And t1 is just 1. 305 00:19:21,950 --> 00:19:24,660 So I'm just getting the sum of the powers of 2 306 00:19:24,660 --> 00:19:26,810 up to the n minus 1. 307 00:19:26,810 --> 00:19:30,180 And we know what that is from last time, for sure. 308 00:19:30,180 --> 00:19:34,000 That's just 2 to the n minus 1. 309 00:19:34,000 --> 00:19:36,570 So we've derived the answer with only a little bit 310 00:19:36,570 --> 00:19:38,930 of a-- we had to observe the pattern here, 311 00:19:38,930 --> 00:19:40,310 which is often easy to do. 312 00:19:40,310 --> 00:19:42,980 But now we've derived the answer. 313 00:19:42,980 --> 00:19:45,170 Now to be really safe, we've got to go back 314 00:19:45,170 --> 00:19:47,390 and still do verify, just to make sure 315 00:19:47,390 --> 00:19:53,580 that we didn't make a mistake in how we did the derivation here. 316 00:19:53,580 --> 00:19:55,610 OK? 317 00:19:55,610 --> 00:19:59,210 Any questions about that approach? 318 00:19:59,210 --> 00:20:01,580 You just keep plugging it back into the recurrence, 319 00:20:01,580 --> 00:20:05,330 look at the pattern you get, and try to guess it from that point 320 00:20:05,330 --> 00:20:08,040 and then solve. 321 00:20:08,040 --> 00:20:10,916 All right, let's do a little more interesting recurrence. 322 00:20:13,690 --> 00:20:15,680 This is a fairly famous one that comes up 323 00:20:15,680 --> 00:20:19,780 in a well known algorithm for sorting called merge sort. 324 00:20:19,780 --> 00:20:22,890 This is an algorithm you'll look at in 6046, 325 00:20:22,890 --> 00:20:24,380 one of the many sorting algorithms 326 00:20:24,380 --> 00:20:27,290 you'll study when you take an algorithms class. 327 00:20:39,310 --> 00:20:43,740 So you're given n numbers and your task is to sort them, 328 00:20:43,740 --> 00:20:45,140 put them in order. 329 00:20:45,140 --> 00:20:46,890 And you're allowed to do comparisons 330 00:20:46,890 --> 00:20:48,120 between the numbers. 331 00:20:48,120 --> 00:20:51,240 You can think of them as you've got a list of names, 332 00:20:51,240 --> 00:20:53,380 putting them in alphabetical order. 333 00:20:53,380 --> 00:20:57,260 How many comparisons do you have to do to make that happen? 334 00:20:57,260 --> 00:21:01,360 And this is one way to do it that's very efficient. 335 00:21:01,360 --> 00:21:05,250 So to sort n items. 336 00:21:05,250 --> 00:21:07,910 In fact, we're going to look at n being a power of 2. 337 00:21:07,910 --> 00:21:13,270 Call the items x1, x2, xn. 338 00:21:13,270 --> 00:21:17,348 n's going to be a power of 2 to make it simple for now. 339 00:21:20,186 --> 00:21:21,560 The first thing we're going to do 340 00:21:21,560 --> 00:21:24,680 is recursively sort the first half 341 00:21:24,680 --> 00:21:29,860 of the items and the second half. 342 00:21:29,860 --> 00:21:33,960 So we use recursion to short the first n over 2 items 343 00:21:33,960 --> 00:21:35,640 and the last n over 2 items. 344 00:21:46,290 --> 00:21:48,668 And then we're going to merge those two sorted lists 345 00:21:48,668 --> 00:21:49,167 together. 346 00:21:52,830 --> 00:21:54,830 So we're going to merge this list and that list. 347 00:21:54,830 --> 00:21:59,170 So let's see how this works and figure out how many comparisons 348 00:21:59,170 --> 00:22:01,630 we have to do to sort the list. 349 00:22:09,580 --> 00:22:16,140 For example, let's say we're sorting this list. 350 00:22:16,140 --> 00:22:25,260 10, 7, 23, 5, 2, 4, 3, and 9. 351 00:22:25,260 --> 00:22:29,420 So I've got eight numbers that I'm going to sort. 352 00:22:29,420 --> 00:22:35,110 So the first step is to recursively sort that list. 353 00:22:35,110 --> 00:22:44,380 And when I do that, I produce, so I'm sorting 10, 7, 23, 5. 354 00:22:44,380 --> 00:22:48,725 And that produces, let's see, I'm going to get 5, 7, 10, 23. 355 00:22:55,050 --> 00:22:59,200 And then I recursively sort these guys. 356 00:22:59,200 --> 00:23:05,430 And that produces 2, 3, 4, and 9. 357 00:23:08,500 --> 00:23:10,850 I've done comparisons, but haven't counted them 358 00:23:10,850 --> 00:23:12,640 because they're in the recursion. 359 00:23:12,640 --> 00:23:16,210 Now I'm going to use comparisons to merge the lists, 360 00:23:16,210 --> 00:23:21,740 And the way I'm going to do that is 361 00:23:21,740 --> 00:23:28,150 to look at the smallest item in each list and compare them. 362 00:23:28,150 --> 00:23:30,150 Because I know the smallest item overall 363 00:23:30,150 --> 00:23:32,220 is either this one or that one. 364 00:23:32,220 --> 00:23:36,870 And when I compare 2 to 5, 2's the smaller one, 365 00:23:36,870 --> 00:23:39,150 so I start with 2. 366 00:23:39,150 --> 00:23:42,520 And I strike it. 367 00:23:42,520 --> 00:23:47,360 What two items do I compare next to get the next smallest item? 368 00:23:47,360 --> 00:23:48,520 5 and 3. 369 00:23:48,520 --> 00:23:49,520 Smallest remaining guys. 370 00:23:49,520 --> 00:23:51,110 I get 3. 371 00:23:51,110 --> 00:23:52,960 Scratch that guy. 372 00:23:52,960 --> 00:23:55,290 And I compare 5 and 4. 373 00:23:55,290 --> 00:23:58,310 Get 4, scratch him. 374 00:23:58,310 --> 00:24:00,610 And 5 and 9. 375 00:24:00,610 --> 00:24:02,720 5 is the smallest. 376 00:24:02,720 --> 00:24:05,048 7 and 9. 377 00:24:05,048 --> 00:24:05,952 7. 378 00:24:05,952 --> 00:24:07,000 9 and 10. 379 00:24:09,700 --> 00:24:11,920 And actually now I don't really have 380 00:24:11,920 --> 00:24:13,230 to do any more comparisons. 381 00:24:13,230 --> 00:24:15,360 All I've got is one list left. 382 00:24:15,360 --> 00:24:18,580 So I can just send these guys down. 383 00:24:21,330 --> 00:24:22,906 And I produced a sorted list. 384 00:24:27,050 --> 00:24:33,410 Now, how many comparisons do I have to do in the worst case 385 00:24:33,410 --> 00:24:40,620 if this is n over 2 items here and this is n over 2 here? 386 00:24:40,620 --> 00:24:43,710 How many comparisons do I have to do 387 00:24:43,710 --> 00:24:46,080 to produce the n item sorted list after I 388 00:24:46,080 --> 00:24:49,260 have the sub list sorted? 389 00:24:49,260 --> 00:24:50,860 What is it? 390 00:24:50,860 --> 00:24:51,360 n. 391 00:24:51,360 --> 00:24:53,340 Very close. 392 00:24:53,340 --> 00:24:58,260 n minus 1 is right, because every time I 393 00:24:58,260 --> 00:25:02,420 do a comparison I pull an item and stick it in the list. 394 00:25:02,420 --> 00:25:05,020 But once I get down to the very last item, 395 00:25:05,020 --> 00:25:07,820 just one guy sitting there, I don't have to do a comparison. 396 00:25:07,820 --> 00:25:08,930 It was just the last guy. 397 00:25:08,930 --> 00:25:11,060 I know he's the largest. 398 00:25:11,060 --> 00:25:15,290 So it could go up to n minus 1 in the worst case. 399 00:25:15,290 --> 00:25:18,340 In this situation, I actually did n minus 2 400 00:25:18,340 --> 00:25:20,180 because I was left with two guys at the end, 401 00:25:20,180 --> 00:25:21,350 I just bring them down. 402 00:25:21,350 --> 00:25:24,480 But it might have been that the next largest thing to 23 403 00:25:24,480 --> 00:25:29,660 was here and then I would have done n minus 1. 404 00:25:29,660 --> 00:25:31,740 Any questions? 405 00:25:31,740 --> 00:25:32,540 Yeah. 406 00:25:32,540 --> 00:25:35,380 AUDIENCE: [INAUDIBLE]. 407 00:25:35,380 --> 00:25:37,145 Would that make it faster? 408 00:25:37,145 --> 00:25:38,770 TOM LEIGHTON: Well, that's what's going 409 00:25:38,770 --> 00:25:40,872 to happen with the recursion. 410 00:25:40,872 --> 00:25:42,580 Because the first thing we're going to do 411 00:25:42,580 --> 00:25:44,288 is when we unfold the recursion is you're 412 00:25:44,288 --> 00:25:47,610 going to take the first half, split it in half, 413 00:25:47,610 --> 00:25:50,090 recursively sort them and then merge them. 414 00:25:50,090 --> 00:25:53,020 So the answer is that's going to happen, 415 00:25:53,020 --> 00:25:56,150 but we're sort of going to hide it by the recursion. 416 00:25:56,150 --> 00:26:00,430 All right, you didn't see me produce that sorted list. 417 00:26:00,430 --> 00:26:02,510 But I would have done it by taking 418 00:26:02,510 --> 00:26:05,240 10 and 7 sorting them, 5 and 23 and sorting them 419 00:26:05,240 --> 00:26:07,680 and then merging them. 420 00:26:07,680 --> 00:26:08,787 OK, any other questions? 421 00:26:11,230 --> 00:26:12,980 All right, well let's set up the recursion 422 00:26:12,980 --> 00:26:15,510 to count how many comparisons are 423 00:26:15,510 --> 00:26:19,280 needed for the whole thing, not just the last merge phase. 424 00:26:34,200 --> 00:26:36,225 All right, so let's define. 425 00:26:39,250 --> 00:26:41,635 And instead of t sub n, I'm going to write t of n 426 00:26:41,635 --> 00:26:42,620 as a function. 427 00:26:42,620 --> 00:26:44,420 Makes no difference, but you'll see people 428 00:26:44,420 --> 00:26:46,440 do recurrences both ways. 429 00:26:46,440 --> 00:26:48,477 t of n or t sub n. 430 00:26:48,477 --> 00:26:50,268 It's going to be the number of comparisons. 431 00:26:56,180 --> 00:27:02,770 Use my merge sort to sort n numbers. 432 00:27:02,770 --> 00:27:04,853 In the worst case, worst case. 433 00:27:07,650 --> 00:27:14,270 All right, so we already know that merging 434 00:27:14,270 --> 00:27:23,810 takes n minus 1 comparisons in the worst case. 435 00:27:23,810 --> 00:27:26,215 And you always want to prepare for the worst case. 436 00:27:29,860 --> 00:27:35,010 How many comparisons are used in the recursive 437 00:27:35,010 --> 00:27:41,030 steps to sort this list and to sort that list, in general. 438 00:27:41,030 --> 00:27:48,400 tn over 2 to do this list times 2. 439 00:27:48,400 --> 00:27:50,520 Because you've got two lists of size n over 2. 440 00:27:50,520 --> 00:27:52,430 Two sub problems besides n over 2. 441 00:27:52,430 --> 00:27:55,650 Each takes t of n over 2. 442 00:27:55,650 --> 00:28:04,030 All right, so you've got 2 tn over 2 comparisons 443 00:28:04,030 --> 00:28:05,852 for the recursive sorting. 444 00:28:13,010 --> 00:28:24,850 That means that t of n equals 2 t of n over 2 plus n minus 1. 445 00:28:24,850 --> 00:28:26,960 And we should say what t of 1 is. 446 00:28:26,960 --> 00:28:28,808 What's that? 447 00:28:28,808 --> 00:28:31,390 What's the time to sort? 448 00:28:31,390 --> 00:28:31,960 0. 449 00:28:31,960 --> 00:28:34,430 You don't do any comparisons if there's one item. 450 00:28:34,430 --> 00:28:36,016 So we'll set that as our base case. 451 00:28:42,180 --> 00:28:46,710 All right, so let's find the answer. 452 00:28:46,710 --> 00:28:49,892 So the first thing to always do when you see a recurrence 453 00:28:49,892 --> 00:28:54,560 is to take a few values, compute a few values, 454 00:28:54,560 --> 00:28:57,560 and try to guess the answer. 455 00:28:57,560 --> 00:28:59,090 So what's t of 2? 456 00:29:03,020 --> 00:29:03,660 1. 457 00:29:03,660 --> 00:29:04,930 Plug it in. 458 00:29:04,930 --> 00:29:08,220 I get 2t of 1, t of 1's 0, plus 2 minus 1. 459 00:29:08,220 --> 00:29:08,740 That's 1. 460 00:29:11,530 --> 00:29:13,490 And we're only doing powers of 4 here. 461 00:29:13,490 --> 00:29:14,310 Sorry, powers of 2. 462 00:29:14,310 --> 00:29:15,720 So I go to t of 4 next. 463 00:29:15,720 --> 00:29:16,691 What's t of 4? 464 00:29:22,420 --> 00:29:24,280 5. 465 00:29:24,280 --> 00:29:28,670 Right, because I got double this is 2 plus 4 minus 1. 466 00:29:28,670 --> 00:29:30,080 That's adding 3. 467 00:29:30,080 --> 00:29:32,630 So I get 5. 468 00:29:32,630 --> 00:29:36,500 All right, let's look at t of 8. 469 00:29:36,500 --> 00:29:43,380 That's twice 5 plus 8 minus 1. 470 00:29:43,380 --> 00:29:43,880 17. 471 00:29:48,320 --> 00:29:51,360 Not so clear what to guess here. 472 00:29:51,360 --> 00:29:54,370 We can try one more. 473 00:29:54,370 --> 00:29:59,480 T of 16 equals 2 times 17 plus 16 minus 1 474 00:29:59,480 --> 00:30:07,280 equals 34 and 15 is 49. 475 00:30:07,280 --> 00:30:09,450 Is that right? 476 00:30:09,450 --> 00:30:10,782 34 and 15. 477 00:30:13,460 --> 00:30:14,496 Any guesses? 478 00:30:18,780 --> 00:30:20,729 You're not going to guess this. 479 00:30:20,729 --> 00:30:21,770 That's not going to work. 480 00:30:21,770 --> 00:30:22,994 That happens. 481 00:30:22,994 --> 00:30:25,160 And a miracle could happen, you get a divine insight 482 00:30:25,160 --> 00:30:28,730 and guess it, but I think it's not likely. 483 00:30:28,730 --> 00:30:33,180 All right, so let's go to plug and chug. 484 00:30:33,180 --> 00:30:34,104 See if that works. 485 00:30:44,370 --> 00:30:46,257 All right, so now plug and chug. 486 00:30:51,620 --> 00:30:54,270 All right, so we write down t of n 487 00:30:54,270 --> 00:30:57,310 is-- and you always write their current part last for plug 488 00:30:57,310 --> 00:30:59,830 and chug to keep it simpler. 489 00:30:59,830 --> 00:31:03,790 2 t of n over 2. 490 00:31:03,790 --> 00:31:08,310 And then I substitute, I plug in for t of n over 2. 491 00:31:08,310 --> 00:31:14,290 So I have n minus 1 plus 2 n over 2 minus 1 492 00:31:14,290 --> 00:31:19,410 plus 2 t of n over 4. 493 00:31:19,410 --> 00:31:22,010 Then I chug. 494 00:31:22,010 --> 00:31:28,710 So I have n minus 1 plus that's n minus 2. 495 00:31:28,710 --> 00:31:30,861 Plus 4 tn over 4. 496 00:31:33,950 --> 00:31:35,578 All right, then we'll plug in here. 497 00:31:41,630 --> 00:31:47,705 And I get n over 4 minus 1 plus 4 tn over 8. 498 00:31:50,510 --> 00:31:51,365 And now we chug. 499 00:31:57,180 --> 00:32:00,150 That's an n here minus a 4. 500 00:32:03,080 --> 00:32:04,580 I did something wrong. 501 00:32:04,580 --> 00:32:06,670 That should be 2. 502 00:32:06,670 --> 00:32:09,652 4 times 2 is 8 tn over 8. 503 00:32:12,590 --> 00:32:17,470 All right, can you start to guess the pattern here maybe? 504 00:32:17,470 --> 00:32:19,820 We probably should do one more to be safe. 505 00:32:19,820 --> 00:32:25,850 But you got an n minus 1 plus an n minus 2 plus an n minus 4. 506 00:32:25,850 --> 00:32:29,820 And if we do one more, what's the next term going to be? 507 00:32:29,820 --> 00:32:31,720 n minus 8. 508 00:32:31,720 --> 00:32:33,530 And this is looking pretty simple. 509 00:32:33,530 --> 00:32:39,830 So we could guess now that the pattern is n minus 1 510 00:32:39,830 --> 00:32:45,070 plus n minus 2 plus n minus 4 all the way 511 00:32:45,070 --> 00:32:49,290 out to plus n minus 2 to the i minus 1 512 00:32:49,290 --> 00:32:55,350 plus 2 to the i t n over 2 to the i. 513 00:32:55,350 --> 00:32:57,160 And now we go all the way down where 514 00:32:57,160 --> 00:33:02,480 i equals log n, base 2, and see what we get. 515 00:33:05,810 --> 00:33:11,390 1 plus n minus 2 plus all the way down n minus 2 516 00:33:11,390 --> 00:33:22,270 to the log n minus 1 plus 2 to the log n t of 1. 517 00:33:22,270 --> 00:33:25,070 What's t of 1? 518 00:33:25,070 --> 00:33:25,780 0. 519 00:33:25,780 --> 00:33:26,460 That's nice. 520 00:33:26,460 --> 00:33:28,680 That goes away. 521 00:33:28,680 --> 00:33:31,640 All right, so this is not looking so bad. 522 00:33:31,640 --> 00:33:42,120 This equals the sum i equals 0 to log n minus 1 of n minus 2 523 00:33:42,120 --> 00:33:42,768 to the i. 524 00:33:45,102 --> 00:33:46,810 All right, and this will be good practice 525 00:33:46,810 --> 00:33:49,850 maybe for towards the test. 526 00:33:49,850 --> 00:33:52,346 Well, I can split that sum up. 527 00:33:52,346 --> 00:34:02,160 i equals 0 to log n minus 1 of n minus the i equals 0 log 528 00:34:02,160 --> 00:34:04,063 n minus 1 of 2 to the i. 529 00:34:06,841 --> 00:34:07,940 Well, this is easy. 530 00:34:07,940 --> 00:34:12,639 I'm just adding n up a total of log n times. 531 00:34:12,639 --> 00:34:13,659 So I have n log n. 532 00:34:17,150 --> 00:34:19,050 This is just something the powers of 2 533 00:34:19,050 --> 00:34:20,460 which we know how to do. 534 00:34:20,460 --> 00:34:26,659 So that's going to be 2 to the log n minus 1. 535 00:34:26,659 --> 00:34:33,730 And so we get our answer is n log n minus 2 536 00:34:33,730 --> 00:34:39,710 the log n is just n minus 1 is a plus 1. 537 00:34:39,710 --> 00:34:41,072 Well, we derived an answer. 538 00:34:43,670 --> 00:34:47,060 Let's just go back and check some of the values. 539 00:34:47,060 --> 00:34:48,870 n equals 1. 540 00:34:48,870 --> 00:34:50,850 That's 0 and I get 1 minus 1. 541 00:34:50,850 --> 00:34:52,630 It works. 542 00:34:52,630 --> 00:34:55,159 n equals 2. 543 00:34:55,159 --> 00:35:00,040 2 log 2 is 2 minus 2 plus 1. 544 00:35:00,040 --> 00:35:01,360 Works. 545 00:35:01,360 --> 00:35:07,900 Plug in 4, I get 4 log 4 is 8 minus 4 is 4 plus 1 is 5. 546 00:35:07,900 --> 00:35:10,060 So it works for those examples. 547 00:35:10,060 --> 00:35:14,690 Now, to be really careful we should prove it by induction. 548 00:35:14,690 --> 00:35:17,170 Because when you do enough of these equations, 549 00:35:17,170 --> 00:35:19,920 you make mistakes like I invariably do even up here. 550 00:35:19,920 --> 00:35:21,590 And even though it's written down here, 551 00:35:21,590 --> 00:35:23,840 I'll make mistakes writing it on the board. 552 00:35:23,840 --> 00:35:26,245 And so you want to check by induction. 553 00:35:30,310 --> 00:35:31,880 OK, so this is the answer. 554 00:35:31,880 --> 00:35:36,600 This is how many comparisons are done in merge sort. 555 00:35:36,600 --> 00:35:40,550 And the nice thing is this is growing nearly linearly. 556 00:35:40,550 --> 00:35:41,810 It grows as n log n. 557 00:35:41,810 --> 00:35:44,510 And that's why this algorithm is used a lot in practice. 558 00:35:44,510 --> 00:35:46,330 Much better than comparing every item 559 00:35:46,330 --> 00:35:50,000 to every other item, which would be more like n squared steps. 560 00:35:50,000 --> 00:35:54,270 So much better than a naive sorting algorithm. 561 00:35:54,270 --> 00:35:59,060 Any questions about what we did there? 562 00:35:59,060 --> 00:36:05,240 Just one rule of thumb when you're doing the plug and chug. 563 00:36:05,240 --> 00:36:10,870 Notice that I didn't try to collapse terms along the way. 564 00:36:10,870 --> 00:36:13,470 It makes it easier sometimes to see the pattern. 565 00:36:13,470 --> 00:36:17,570 I left it as n minus 1 plus n minus 2 and then plus n minus 4 566 00:36:17,570 --> 00:36:19,350 rather than saying, oh, this is 3 n 567 00:36:19,350 --> 00:36:23,560 minus 7, which might have made it harder to guess the pattern. 568 00:36:23,560 --> 00:36:26,130 So you don't always, when you're doing the chugging, collapse 569 00:36:26,130 --> 00:36:26,810 everything down. 570 00:36:26,810 --> 00:36:28,440 You want to try and see if there are some. 571 00:36:28,440 --> 00:36:30,050 Because this is the amount of work done at each step, 572 00:36:30,050 --> 00:36:32,258 and you want to see if you can see a pattern in that. 573 00:36:37,070 --> 00:36:39,250 All right, so let's write down our two recurrences 574 00:36:39,250 --> 00:36:42,090 and sort of compare them. 575 00:36:52,870 --> 00:36:57,000 So first we did Towers of Hanoi, which was t of n 576 00:36:57,000 --> 00:37:02,380 equals 2 t of n minus 1 plus 1. 577 00:37:02,380 --> 00:37:08,455 And the answer was t of n is tilde 2 to the n. 578 00:37:08,455 --> 00:37:11,680 It was actually 2 to the n minus 1. 579 00:37:11,680 --> 00:37:19,760 Then we did merge sort, which is t of n equals 2 t of n over 2. 580 00:37:19,760 --> 00:37:21,240 Difference there. 581 00:37:21,240 --> 00:37:23,990 Plus n minus 1. 582 00:37:23,990 --> 00:37:26,970 And now the answer was t of n. 583 00:37:26,970 --> 00:37:29,540 What's the tilde for this one? 584 00:37:29,540 --> 00:37:32,600 For merge sort. 585 00:37:32,600 --> 00:37:33,600 n log n. 586 00:37:33,600 --> 00:37:36,433 Yeah, I can forget the rest if I know the tilde, n log n. 587 00:37:40,140 --> 00:37:42,360 Wow. 588 00:37:42,360 --> 00:37:45,290 These are pretty different answers, right? 589 00:37:45,290 --> 00:37:48,170 This one's nearly linear, that one's exponential. 590 00:37:50,810 --> 00:37:54,320 The recurrences looks pretty similar, 591 00:37:54,320 --> 00:37:55,970 in fact, this one sort of looks worse 592 00:37:55,970 --> 00:38:00,960 because I'm adding in n minus 1 every time instead of 1. 593 00:38:00,960 --> 00:38:05,060 But the answer's a lot better, a lot smaller. 594 00:38:05,060 --> 00:38:09,030 What's the critical difference between these recurrences? 595 00:38:09,030 --> 00:38:09,530 Yeah? 596 00:38:12,440 --> 00:38:17,201 AUDIENCE: You're dropping the argument of t a lot faster. 597 00:38:17,201 --> 00:38:18,700 You're cutting it in half every time 598 00:38:18,700 --> 00:38:19,990 instead of subtracting one. 599 00:38:19,990 --> 00:38:21,010 TOM LEIGHTON: Right. 600 00:38:21,010 --> 00:38:22,430 AUDIENCE: So it's going to go away faster. 601 00:38:22,430 --> 00:38:23,555 TOM LEIGHTON: That's right. 602 00:38:23,555 --> 00:38:26,560 Here I've got two problems of size one less. 603 00:38:26,560 --> 00:38:30,230 Here I've got two problems of half a size. 604 00:38:30,230 --> 00:38:33,450 And that makes an enormous difference in the answer. 605 00:38:33,450 --> 00:38:36,450 Even though I'm doing more work after 606 00:38:36,450 --> 00:38:39,700 did the recursive problems, much faster running 607 00:38:39,700 --> 00:38:45,540 time or answer if these are algorithms that I'm analyzing. 608 00:38:45,540 --> 00:38:47,520 All right? 609 00:38:47,520 --> 00:38:50,330 In fact, let's take a look at what 610 00:38:50,330 --> 00:38:55,050 happens if I didn't use that n minus 1, I just put a 1 there. 611 00:38:55,050 --> 00:38:59,590 Let's see what impact that n minus 1 had. 612 00:38:59,590 --> 00:39:05,080 So let's do the following recurrence. 613 00:39:05,080 --> 00:39:09,640 This one is going to be a little bit different because I'm not 614 00:39:09,640 --> 00:39:10,900 going to stick to powers of 2. 615 00:39:10,900 --> 00:39:13,130 I'm going to get a little messier, because that often 616 00:39:13,130 --> 00:39:15,130 happens in reality. 617 00:39:15,130 --> 00:39:20,690 And I'm going to define s of 1 is 0. 618 00:39:20,690 --> 00:39:29,600 s of n is s of the floor function of n over 2 619 00:39:29,600 --> 00:39:35,540 plus s of the ceiling function of n over 2 plus 1. 620 00:39:40,120 --> 00:39:45,460 Now, the ceiling function means the smallest integer 621 00:39:45,460 --> 00:39:46,947 that's at least that big. 622 00:39:46,947 --> 00:39:49,030 So you're sort of rounding up to the next integer. 623 00:40:00,260 --> 00:40:02,850 And the floor function means you're rounding down 624 00:40:02,850 --> 00:40:04,430 to the next integer. 625 00:40:04,430 --> 00:40:13,190 So it's the biggest integer less than equal to n over 2. 626 00:40:13,190 --> 00:40:14,010 All right. 627 00:40:14,010 --> 00:40:17,480 Now, if we had just powers of 2, this 628 00:40:17,480 --> 00:40:19,380 would be much easier to write. 629 00:40:19,380 --> 00:40:24,985 It would just be sn equals 2 sn over 2 plus 1. 630 00:40:24,985 --> 00:40:27,610 So it's analogous to what we're doing up there where we're just 631 00:40:27,610 --> 00:40:29,746 adding 1 instead of n minus 1. 632 00:40:29,746 --> 00:40:30,870 But I'm being careful. now. 633 00:40:30,870 --> 00:40:32,245 I'm going to make this recurrence 634 00:40:32,245 --> 00:40:35,561 work for all natural numbers. 635 00:40:35,561 --> 00:40:37,060 Because sometimes in reality, you're 636 00:40:37,060 --> 00:40:39,250 dealing with non powers of 2. 637 00:40:43,440 --> 00:40:47,580 All right, let's see if we can solve this. 638 00:40:47,580 --> 00:40:50,170 What's the first thing to try to solve this? 639 00:40:50,170 --> 00:40:50,890 What do you do? 640 00:40:56,520 --> 00:40:59,060 What's the first step when you get a recurrence? 641 00:40:59,060 --> 00:41:00,142 You try to solve. 642 00:41:03,300 --> 00:41:06,420 What method should you try? 643 00:41:06,420 --> 00:41:07,470 Guess. 644 00:41:07,470 --> 00:41:10,840 And to guess, well, you got to plug some values in. 645 00:41:10,840 --> 00:41:12,720 It's sort of hard to guess. 646 00:41:12,720 --> 00:41:15,940 I mean, looking at that, I guess nothing comes to mind other 647 00:41:15,940 --> 00:41:18,820 than panic looking at that. 648 00:41:18,820 --> 00:41:21,470 But if we plug in some values, then maybe it won't be so bad. 649 00:41:21,470 --> 00:41:25,200 So we've got s1 equals 0. 650 00:41:25,200 --> 00:41:26,026 What's s2? 651 00:41:34,040 --> 00:41:39,380 It's not 3, because this is 2 over 2 is 1, floor of 1 is 1. 652 00:41:39,380 --> 00:41:42,590 s of 1 is 0. 653 00:41:42,590 --> 00:41:45,460 Ceiling of 1 is 1 because it's an integer. 654 00:41:45,460 --> 00:41:48,630 s of 1 is 0 plus 1. 655 00:41:48,630 --> 00:41:51,400 So s of 2 is 1. 656 00:41:51,400 --> 00:41:53,530 What's s of 3? 657 00:41:53,530 --> 00:41:55,510 Yikes, all right. 658 00:41:55,510 --> 00:41:57,510 3 over 2 is 1 and 1/2. 659 00:41:57,510 --> 00:41:59,890 What's the floor of 1 and 1/2? 660 00:41:59,890 --> 00:42:00,460 1. 661 00:42:00,460 --> 00:42:01,360 s of 1 is 0. 662 00:42:01,360 --> 00:42:02,530 That's gone. 663 00:42:02,530 --> 00:42:05,090 What's the ceiling of 1 and 1/2? 664 00:42:05,090 --> 00:42:05,780 2. 665 00:42:05,780 --> 00:42:09,470 s of 2 is 1 plus 1. 666 00:42:09,470 --> 00:42:10,800 What's the answer for s of 3? 667 00:42:14,070 --> 00:42:20,120 2, because I've got 0, 1, and 1. 668 00:42:20,120 --> 00:42:21,710 s of 4. 669 00:42:21,710 --> 00:42:25,540 This one's easy because it's even. 670 00:42:25,540 --> 00:42:31,500 I get s of 2, which is 1 plus s of two which is 1 plus 1. 671 00:42:31,500 --> 00:42:33,500 So what's s of 4? 672 00:42:33,500 --> 00:42:36,040 3. 673 00:42:36,040 --> 00:42:38,880 You might try a guess. 674 00:42:38,880 --> 00:42:42,010 What would you guess? 675 00:42:42,010 --> 00:42:43,860 Yeah, n minus 1. 676 00:42:43,860 --> 00:42:51,297 So we're going to guess s of n equals n minus 1. 677 00:42:51,297 --> 00:42:52,630 And that's not so bad, actually. 678 00:42:52,630 --> 00:42:53,421 It's pretty simple. 679 00:42:53,421 --> 00:42:56,900 Looking at that mess, you'd never think of that right away. 680 00:42:56,900 --> 00:43:00,250 But once you plug in some values it's not so bad. 681 00:43:00,250 --> 00:43:02,012 All right, let's verify it. 682 00:43:05,270 --> 00:43:07,261 And we're going to verify by strong induction. 683 00:43:14,790 --> 00:43:16,720 The induction hypothesis is our guess. 684 00:43:26,704 --> 00:43:28,560 The base case is easy. 685 00:43:31,870 --> 00:43:36,820 Because I've got s of 1 is 0. 686 00:43:36,820 --> 00:43:40,160 And that equals 1 minus 1. 687 00:43:40,160 --> 00:43:41,970 So that's good. 688 00:43:41,970 --> 00:43:44,124 So let's do the induction step. 689 00:44:00,530 --> 00:44:02,340 All right, so the induction step. 690 00:44:07,690 --> 00:44:15,850 We're going to assume that p1 is true, p2 is true, 691 00:44:15,850 --> 00:44:24,128 all the way up to pn, to prove pn plus 1 is true. 692 00:44:26,820 --> 00:44:28,690 For n better than or equal to 1. 693 00:44:28,690 --> 00:44:32,880 And then we look at the n plus first term. 694 00:44:32,880 --> 00:44:41,990 That equals s of floor n plus 1 over 2 plus s of ceiling n 695 00:44:41,990 --> 00:44:47,710 plus 1 over 2 plus 1. 696 00:44:47,710 --> 00:44:52,470 All right, now I don't know what these are exactly depending on 697 00:44:52,470 --> 00:44:55,820 whether n is even a rod. 698 00:44:55,820 --> 00:45:00,070 First let me plug in using the induction hypothesis. 699 00:45:00,070 --> 00:45:06,040 I know that this is n plus 1 over 2 minus 1. 700 00:45:06,040 --> 00:45:11,850 This is ceiling n plus 1 over 2 minus 1. 701 00:45:11,850 --> 00:45:13,400 And then I carry the plus 1 down. 702 00:45:13,400 --> 00:45:15,350 That's by the induction hypothesis. 703 00:45:20,760 --> 00:45:23,600 All right, now I don't know exactly what 704 00:45:23,600 --> 00:45:26,640 either of these values is unless I know the parity of n. 705 00:45:26,640 --> 00:45:30,680 But I do know the sum of these two things. 706 00:45:30,680 --> 00:45:34,150 The floor of something plus the ceiling of that something 707 00:45:34,150 --> 00:45:37,600 is twice the something. 708 00:45:37,600 --> 00:45:42,940 All right, so this plus that equals n plus 1. 709 00:45:42,940 --> 00:45:46,740 Minus 1 minus 1 plus 1 is minus 1. 710 00:45:46,740 --> 00:45:49,670 And so in fact, we prove that s of n plus 1 711 00:45:49,670 --> 00:45:54,040 equals n plus 1 minus 1. 712 00:45:54,040 --> 00:45:58,680 So we verified the induction hypothesis. 713 00:45:58,680 --> 00:45:59,591 Yeah? 714 00:45:59,591 --> 00:46:00,507 AUDIENCE: [INAUDIBLE]. 715 00:46:08,040 --> 00:46:09,150 Taking the n [INAUDIBLE]. 716 00:46:11,877 --> 00:46:13,460 TOM LEIGHTON: Let's see, I'm not sure. 717 00:46:13,460 --> 00:46:18,282 So are you asking why I started sn plus 1 here? 718 00:46:18,282 --> 00:46:19,198 AUDIENCE: [INAUDIBLE]. 719 00:46:27,332 --> 00:46:28,540 TOM LEIGHTON: Doesn't matter. 720 00:46:28,540 --> 00:46:33,140 But the expression is s of whatever 721 00:46:33,140 --> 00:46:36,380 this is, the induction hypothesis would be, 722 00:46:36,380 --> 00:46:40,240 s of whatever's inside there is what's inside there minus 1. 723 00:46:40,240 --> 00:46:45,178 And so the induction hypothesis says s of this is that minus 1. 724 00:46:48,437 --> 00:46:49,270 Any other questions? 725 00:46:55,260 --> 00:46:57,086 So let's write that on our table up here. 726 00:47:01,950 --> 00:47:07,980 So this is sn equals s of floor n over 2 727 00:47:07,980 --> 00:47:13,926 plus s of ceiling n over 2 plus 1. 728 00:47:13,926 --> 00:47:19,020 And the answer is, sn is tilde. 729 00:47:19,020 --> 00:47:20,010 What's the tilde of? 730 00:47:23,140 --> 00:47:23,680 n. 731 00:47:23,680 --> 00:47:24,680 The answer is n minus 1. 732 00:47:24,680 --> 00:47:27,850 So it's tilde of n. 733 00:47:27,850 --> 00:47:32,310 Now, this is the same expression is this if n's a power of 2. 734 00:47:32,310 --> 00:47:37,440 I'd have 2 sn over 2 plus 1 instead of n minus 1. 735 00:47:37,440 --> 00:47:44,450 And by going from n minus 1 to 1, I got rid of the log term. 736 00:47:44,450 --> 00:47:48,040 But it's still not exponential. 737 00:47:48,040 --> 00:47:50,370 And that's because I'm cutting down the insides 738 00:47:50,370 --> 00:47:53,388 here by a factor of 2 roughly. 739 00:47:59,260 --> 00:48:03,300 All right, so all the recurrences we've seen so far 740 00:48:03,300 --> 00:48:05,540 have been pretty simple. 741 00:48:05,540 --> 00:48:08,330 If guess and verify doesn't work after seeing a few terms, 742 00:48:08,330 --> 00:48:11,750 you can use plug and chug and you can figure it out. 743 00:48:11,750 --> 00:48:15,640 The good news is, that happens a lot in practice. 744 00:48:15,640 --> 00:48:18,300 The bad news is, sometimes it doesn't and you 745 00:48:18,300 --> 00:48:19,630 can get nasty recurrences. 746 00:48:19,630 --> 00:48:27,250 So let me show you a nasty one that actually was once 747 00:48:27,250 --> 00:48:37,417 asked in 6046, I think, on a quiz with disastrous results. 748 00:48:46,340 --> 00:48:57,960 All right, so this one is t of x equals 2 t of x over 2 749 00:48:57,960 --> 00:49:08,260 plus 8/9 t of 3x over 4 plus x squared 750 00:49:08,260 --> 00:49:11,720 for x bigger and equal to 1, and otherwise it's 751 00:49:11,720 --> 00:49:14,410 0 when x is less than 1. 752 00:49:18,230 --> 00:49:21,890 Now, this is nasty for a lot of reasons. 753 00:49:21,890 --> 00:49:25,610 First, it's not just a find on the integers. 754 00:49:25,610 --> 00:49:28,690 We can make an integer version by putting floors or ceilings 755 00:49:28,690 --> 00:49:29,500 here. 756 00:49:29,500 --> 00:49:32,730 But the way you set it up now, it's defined on real numbers. 757 00:49:36,650 --> 00:49:41,150 You can, in theory, solve it with plug and chug. 758 00:49:41,150 --> 00:49:44,520 The nasty part is, you've got a bunch of things going down 759 00:49:44,520 --> 00:49:49,410 by powers of 2 and a bunch of things going down by 3/4 here. 760 00:49:49,410 --> 00:49:51,400 And so you have to keep track of all the terms 761 00:49:51,400 --> 00:49:54,179 that are decreasing, and that's a nightmare plug and chug. 762 00:49:54,179 --> 00:49:56,220 Because it's not just one thing that's unfolding, 763 00:49:56,220 --> 00:49:58,890 it's two things that are unfolding. 764 00:49:58,890 --> 00:50:01,280 So very painful. 765 00:50:01,280 --> 00:50:05,390 Now, it used to be that was the state of life. 766 00:50:05,390 --> 00:50:09,080 20 years ago, I think we started teaching this course sometime 767 00:50:09,080 --> 00:50:11,700 around then, there just was nothing else 768 00:50:11,700 --> 00:50:14,870 to say other than life's tough and sometimes you've 769 00:50:14,870 --> 00:50:18,030 got to do these things and it's a pain. 770 00:50:18,030 --> 00:50:20,140 Just had to make the best of it. 771 00:50:20,140 --> 00:50:24,730 And then in the mid-90s, one day two students 772 00:50:24,730 --> 00:50:28,900 at the American University in Beirut came by my office. 773 00:50:28,900 --> 00:50:32,590 They were named Akra and Bazzi. 774 00:50:32,590 --> 00:50:34,130 And they'd actually come to Boston 775 00:50:34,130 --> 00:50:35,530 to escape the daily bombings. 776 00:50:35,530 --> 00:50:37,900 This was when all heck was breaking loose in Beirut. 777 00:50:37,900 --> 00:50:39,139 It was a mess. 778 00:50:39,139 --> 00:50:40,680 And they come into my office and they 779 00:50:40,680 --> 00:50:44,110 claim to have a simple formula for solving 780 00:50:44,110 --> 00:50:46,460 any recurrence like this. 781 00:50:46,460 --> 00:50:51,590 In fact, you can even add a 5 t of x over 7. 782 00:50:51,590 --> 00:50:52,580 They said no problem. 783 00:50:52,580 --> 00:50:56,490 Their simple formula just will always work. 784 00:50:56,490 --> 00:51:00,100 And the general form that they claim to solve 785 00:51:00,100 --> 00:51:02,020 was the following. 786 00:51:02,020 --> 00:51:06,490 It's the general class of divide and conquer recurrences. 787 00:51:06,490 --> 00:51:08,460 So any recurrence you get from an algorithm 788 00:51:08,460 --> 00:51:12,130 where you break it into smaller pieces by a constant factor, 789 00:51:12,130 --> 00:51:14,390 they claim to solve. 790 00:51:14,390 --> 00:51:16,810 So let me define this class. 791 00:51:16,810 --> 00:51:18,290 It's a little painful. 792 00:51:28,330 --> 00:51:34,268 So a divide and conquer recurrence has the form. 793 00:51:38,450 --> 00:51:41,290 There's a simple version and the hard version. 794 00:51:41,290 --> 00:51:45,080 The simple version is it looks like that. 795 00:51:45,080 --> 00:51:47,210 You got a bunch of terms. 796 00:51:47,210 --> 00:51:50,460 Inside each term, you cut it down by a constant factor 797 00:51:50,460 --> 00:51:52,290 and then you add something at the end. 798 00:51:52,290 --> 00:51:55,020 That's the simple version of what a divide and conquer 799 00:51:55,020 --> 00:51:56,130 recurrence is. 800 00:51:56,130 --> 00:51:59,856 And you can throw in floors and ceilings and other junk too. 801 00:51:59,856 --> 00:52:01,730 Now I'm going to give you the formal version. 802 00:52:04,950 --> 00:52:11,120 So it has the form t of x equals some constant a1 times 803 00:52:11,120 --> 00:52:17,080 t of a constant less than b1 of x, plus some slop 804 00:52:17,080 --> 00:52:20,170 you're allowed to have like floors and ceilings 805 00:52:20,170 --> 00:52:22,170 and who knows what else. 806 00:52:22,170 --> 00:52:23,630 And then you get more of these. 807 00:52:23,630 --> 00:52:33,450 You get a2, t of b2x plus epsilon 2x and so forth. 808 00:52:33,450 --> 00:52:34,950 And you can have any number of these 809 00:52:34,950 --> 00:52:37,120 you want up to a constant. 810 00:52:37,120 --> 00:52:43,285 So maybe there's a sub k t b sub kx plus epsilon kx. 811 00:52:45,860 --> 00:52:51,330 Plus some function of x sitting out here, like the x squared. 812 00:52:51,330 --> 00:52:55,520 And this happens for x bigger than some fixed value, 813 00:52:55,520 --> 00:52:57,690 some constant. 814 00:52:57,690 --> 00:53:03,096 Now, the constraints are, well, the ai's are positive 815 00:53:03,096 --> 00:53:03,595 constants. 816 00:53:06,550 --> 00:53:12,650 The bi's are positive and less than 1. 817 00:53:12,650 --> 00:53:15,532 You gotta cut it down by a constant factor to be a divide 818 00:53:15,532 --> 00:53:16,490 and conquer recurrence. 819 00:53:16,490 --> 00:53:17,156 That's critical. 820 00:53:19,740 --> 00:53:21,660 k is fixed. 821 00:53:21,660 --> 00:53:22,330 It's constant. 822 00:53:25,720 --> 00:53:29,990 And these epsilon functions can't be too big. 823 00:53:29,990 --> 00:53:34,390 They can't be bigger than 0 of x over log squared of x. 824 00:53:34,390 --> 00:53:37,100 Almost as big as x, but you can't let them be too big 825 00:53:37,100 --> 00:53:42,390 or you'll not be cutting things down by a constant factor. 826 00:53:42,390 --> 00:53:44,260 Now, you don't get too hung up over this 827 00:53:44,260 --> 00:53:48,030 because you don't get too deep in the details. 828 00:53:48,030 --> 00:53:53,146 But this is that most 0 of x over log squared of x. 829 00:53:55,840 --> 00:54:00,550 And finally, the g thing has got to be polynomial. 830 00:54:00,550 --> 00:54:01,560 It can't be an exponent. 831 00:54:01,560 --> 00:54:03,720 You can't put 2 to the x here. 832 00:54:03,720 --> 00:54:05,510 And the formal way that's expressed 833 00:54:05,510 --> 00:54:11,720 is the derivative of g of x is less than x 834 00:54:11,720 --> 00:54:13,820 to the c for some constant c. 835 00:54:18,450 --> 00:54:19,020 That's it. 836 00:54:19,020 --> 00:54:20,900 So this is a mouthful. 837 00:54:20,900 --> 00:54:23,560 But like I said before, anything that looks 838 00:54:23,560 --> 00:54:26,520 like that, basically. 839 00:54:26,520 --> 00:54:28,270 As long as you don't throw in exponentials 840 00:54:28,270 --> 00:54:32,370 or do something too wild to it, it works. 841 00:54:32,370 --> 00:54:36,790 So let's see which of our guys over here are divide 842 00:54:36,790 --> 00:54:37,790 and conquer recurrences. 843 00:54:43,390 --> 00:54:44,210 How about this one? 844 00:54:44,210 --> 00:54:49,300 Towers of Hanoi, is that a divide and conquer recurrence? 845 00:54:49,300 --> 00:54:52,350 Some yes, some no. 846 00:54:52,350 --> 00:54:54,540 Who thinks no can tell me why it's not? 847 00:54:54,540 --> 00:54:55,479 Why isn't that? 848 00:54:55,479 --> 00:54:56,395 AUDIENCE: [INAUDIBLE]. 849 00:54:58,936 --> 00:55:00,060 TOM LEIGHTON: That's right. 850 00:55:00,060 --> 00:55:03,290 The recurrence problem is n minus 1 851 00:55:03,290 --> 00:55:07,300 and we've got to make it down by a constant factor here. 852 00:55:07,300 --> 00:55:09,080 The b1 is less than 1. 853 00:55:09,080 --> 00:55:10,820 It can't be equal to 1. 854 00:55:10,820 --> 00:55:14,330 And this is not big enough to make up for the x over 2 855 00:55:14,330 --> 00:55:16,600 term or something we're missing. 856 00:55:16,600 --> 00:55:19,690 So whenever you have an n minus 1 and minus 2, 857 00:55:19,690 --> 00:55:22,690 that is not divide and conquer. 858 00:55:22,690 --> 00:55:24,860 We'll discuss those next time, on Thursday, 859 00:55:24,860 --> 00:55:26,510 how to solve those. 860 00:55:26,510 --> 00:55:29,957 So that's not divide and conquer. 861 00:55:29,957 --> 00:55:30,790 What about this one? 862 00:55:30,790 --> 00:55:32,180 Is that divide and conquer here? 863 00:55:35,480 --> 00:55:36,710 Yes. 864 00:55:36,710 --> 00:55:39,750 You've got two problems of half the size 865 00:55:39,750 --> 00:55:42,060 and you're adding a linear function. 866 00:55:42,060 --> 00:55:43,310 It's not exponential. 867 00:55:43,310 --> 00:55:45,130 It's polynomial. 868 00:55:45,130 --> 00:55:47,710 That is divide and conquer. 869 00:55:47,710 --> 00:55:50,822 What about this nasty guy? 870 00:55:50,822 --> 00:55:51,946 Is that divide and conquer? 871 00:55:54,700 --> 00:55:55,940 Yeah. 872 00:55:55,940 --> 00:55:58,400 You've got basically n over 2 here. 873 00:55:58,400 --> 00:56:00,320 Two n over 2 terms now. 874 00:56:00,320 --> 00:56:03,200 This is where that epsilon thing comes in handy. 875 00:56:03,200 --> 00:56:05,600 Because epsilon 1 of n, in this case, 876 00:56:05,600 --> 00:56:13,020 is floor n over 2 minus n over 2. 877 00:56:13,020 --> 00:56:14,876 And that's, at most, 1. 878 00:56:18,180 --> 00:56:21,570 So maybe we're adding a half here or something like that. 879 00:56:21,570 --> 00:56:23,070 And we're allowed to add anything up 880 00:56:23,070 --> 00:56:27,240 to n over log squared of n to fit. 881 00:56:27,240 --> 00:56:30,580 So you can mess around with the low order terms in here. 882 00:56:30,580 --> 00:56:33,560 And adding 1 is fine as well. 883 00:56:33,560 --> 00:56:34,432 So that qualifies. 884 00:56:37,670 --> 00:56:40,830 Let's check that this thing qualifies. 885 00:56:40,830 --> 00:56:41,866 Let's check that. 886 00:56:50,570 --> 00:56:51,280 Well, yeah. 887 00:56:51,280 --> 00:56:57,840 I got 2 constant a1 t of 1/2 x. 888 00:56:57,840 --> 00:56:59,050 That's OK. 889 00:56:59,050 --> 00:57:01,450 8/9 t of 3/4 x. 890 00:57:01,450 --> 00:57:02,850 That's OK. 891 00:57:02,850 --> 00:57:04,690 My g function is a polynomial. 892 00:57:04,690 --> 00:57:06,020 The derivative is 2x. 893 00:57:06,020 --> 00:57:09,800 That's less than x to a power or of x to a power. 894 00:57:09,800 --> 00:57:10,455 So this fits. 895 00:57:14,430 --> 00:57:16,860 Any questions about the definition 896 00:57:16,860 --> 00:57:18,486 of a divide and conquer recurrence? 897 00:57:23,722 --> 00:57:25,430 And you can see why sort of they come up, 898 00:57:25,430 --> 00:57:28,800 because whenever you're solving a problem with recursive pieces 899 00:57:28,800 --> 00:57:30,780 that are a constant factor smaller, 900 00:57:30,780 --> 00:57:34,140 you're going to have something that looks like that. 901 00:57:34,140 --> 00:57:36,100 OK, so getting back to Akra and Bazzi. 902 00:57:36,100 --> 00:57:38,580 B They show up in my office. 903 00:57:38,580 --> 00:57:41,050 They claim to have solved anything like this. 904 00:57:41,050 --> 00:57:44,817 Now needless to say, I'm a little skeptical. 905 00:57:44,817 --> 00:57:47,150 I got to tell you, you're a professor at MIT, especially 906 00:57:47,150 --> 00:57:49,790 mathematics, and people are always 907 00:57:49,790 --> 00:57:54,020 coming by with proofs of the Riemann hypothesis, 908 00:57:54,020 --> 00:57:57,210 p equals np, p doesn't equal np. 909 00:57:57,210 --> 00:58:00,020 And this was a classic problem. 910 00:58:00,020 --> 00:58:01,960 Hundreds of researchers had studied 911 00:58:01,960 --> 00:58:04,960 recurrences for decades. 912 00:58:04,960 --> 00:58:06,670 And not only, it wasn't even believed 913 00:58:06,670 --> 00:58:08,792 that there was possibly a solution. 914 00:58:08,792 --> 00:58:11,250 It's not like it's one of these open questions where we all 915 00:58:11,250 --> 00:58:13,300 believe p doesn't equal np and it's just 916 00:58:13,300 --> 00:58:14,930 a matter of proving it. 917 00:58:14,930 --> 00:58:18,770 People didn't think a solution existed that would be clean. 918 00:58:18,770 --> 00:58:20,970 I mean, look at just the definition's a mess. 919 00:58:20,970 --> 00:58:25,060 And how could there possibly be a clean solution to that? 920 00:58:25,060 --> 00:58:28,020 So it's sort of like the holy grail. 921 00:58:28,020 --> 00:58:29,940 It doesn't exist, but it's fun to think about, 922 00:58:29,940 --> 00:58:35,370 that fantasy, maybe there's a solution and you could find it. 923 00:58:35,370 --> 00:58:38,380 And much less, these aren't reputable researchers 924 00:58:38,380 --> 00:58:38,980 in my office. 925 00:58:38,980 --> 00:58:41,860 These are two students from Beirut 926 00:58:41,860 --> 00:58:44,740 named Akra and Bazzi claiming they've figured out 927 00:58:44,740 --> 00:58:47,030 this thing that all these professors been teaching 928 00:58:47,030 --> 00:58:50,830 this stuff in 6046 for years said you've got to do 929 00:58:50,830 --> 00:58:53,350 plug and chug. 930 00:58:53,350 --> 00:58:56,420 Anyway, they'd been kicked out of everybody else's office, 931 00:58:56,420 --> 00:58:58,320 so I said, OK, let's do it. 932 00:58:58,320 --> 00:59:00,650 Show me the solution. 933 00:59:00,650 --> 00:59:01,930 And they did. 934 00:59:01,930 --> 00:59:04,280 And I go, holy cow. 935 00:59:04,280 --> 00:59:05,297 It's right. 936 00:59:05,297 --> 00:59:05,880 It looks good. 937 00:59:05,880 --> 00:59:07,296 Now, they had a few details wrong, 938 00:59:07,296 --> 00:59:09,560 but my goodness, the answer was correct 939 00:59:09,560 --> 00:59:12,680 and it's amazingly simple. 940 00:59:12,680 --> 00:59:15,260 So now we teach it. 941 00:59:15,260 --> 00:59:18,610 And in fact, you can write the solution 942 00:59:18,610 --> 00:59:21,800 down easier than you can write the definition of divide 943 00:59:21,800 --> 00:59:24,870 and conquer recurrence. 944 00:59:24,870 --> 00:59:25,790 Really remarkable. 945 00:59:28,870 --> 00:59:39,950 So the theorem by Akra and Bazzi in '96. 946 00:59:44,290 --> 00:59:55,240 Set p, real number, So that the sum i equals 1 to k 947 00:59:55,240 --> 01:00:00,720 of ai times bi to the p equals 1. 948 01:00:00,720 --> 01:00:04,160 So the ai's are these things here. 949 01:00:04,160 --> 01:00:07,720 The bi's are the fractions here. 950 01:00:07,720 --> 01:00:13,540 All right, find a p such that this sum is 1. 951 01:00:13,540 --> 01:00:16,920 Then the answer is this. 952 01:00:16,920 --> 01:00:21,740 Then t of x equals theta. 953 01:00:21,740 --> 01:00:24,030 We're going to use our asymptotic notation. 954 01:00:24,030 --> 01:00:29,180 x to the p plus x to the p times the integral 955 01:00:29,180 --> 01:00:33,060 from 1 to x g of u. 956 01:00:33,060 --> 01:00:37,070 G is that polynomial thing, that thing at the end. 957 01:00:37,070 --> 01:00:40,020 Over u to the p minus 1 du. 958 01:00:43,050 --> 01:00:45,460 That's it. 959 01:00:45,460 --> 01:00:47,910 This is really great news for you guys, 960 01:00:47,910 --> 01:00:49,940 especially if you go on to algorithms, 961 01:00:49,940 --> 01:00:51,606 because you don't have to deal with plug 962 01:00:51,606 --> 01:00:53,460 and chug on nasty things. 963 01:00:53,460 --> 01:00:55,896 You just solve the answer. 964 01:00:58,790 --> 01:01:03,300 Now, the proof is by guess and verify. 965 01:01:03,300 --> 01:01:04,875 And they made a really good guess. 966 01:01:07,510 --> 01:01:09,951 You wouldn't think of looking at that, oh, 967 01:01:09,951 --> 01:01:10,950 I'm going to guess this. 968 01:01:10,950 --> 01:01:13,070 Probably not. 969 01:01:13,070 --> 01:01:16,310 And there's no real numbers to try to guess this. 970 01:01:16,310 --> 01:01:18,850 It's asymptotic notation. 971 01:01:18,850 --> 01:01:19,860 But it works. 972 01:01:19,860 --> 01:01:21,640 You can verify it by induction. 973 01:01:21,640 --> 01:01:23,920 It's a little painful to verify by induction, 974 01:01:23,920 --> 01:01:24,880 but you can do it. 975 01:01:24,880 --> 01:01:26,580 We won't do it in class. 976 01:01:26,580 --> 01:01:28,660 I did try one year, and it was a disaster, 977 01:01:28,660 --> 01:01:31,400 so we don't do it in class anymore. 978 01:01:31,400 --> 01:01:33,990 But let's see some examples of how to apply this. 979 01:01:36,750 --> 01:01:42,481 So let's start with merge sort, which is the second one here. 980 01:01:42,481 --> 01:01:43,064 Let's do that. 981 01:01:53,190 --> 01:01:56,390 All right, so you got an example here. 982 01:01:56,390 --> 01:02:06,760 t of x equals 2 t of x over 2 plus n minus 1, or x minus 1, x 983 01:02:06,760 --> 01:02:07,860 now. 984 01:02:07,860 --> 01:02:10,630 And I'm not even going to worry about powers of 2 anymore. 985 01:02:10,630 --> 01:02:11,210 I don't care. 986 01:02:11,210 --> 01:02:14,210 It's going to be fine. 987 01:02:14,210 --> 01:02:17,670 All right, what is p in this case? 988 01:02:17,670 --> 01:02:18,780 I've only got one term. 989 01:02:18,780 --> 01:02:22,260 I don't even need a sum from that definition over there. 990 01:02:22,260 --> 01:02:27,400 What's the value of p for this guy? 991 01:02:27,400 --> 01:02:30,760 So I've got a1 is 2. 992 01:02:30,760 --> 01:02:33,280 b1 is 1/2. 993 01:02:33,280 --> 01:02:34,202 Just one term. 994 01:02:37,320 --> 01:02:42,900 What's the value of p such that 2 times 1/2 to the p equals 1? 995 01:02:42,900 --> 01:02:43,410 1. 996 01:02:43,410 --> 01:02:46,020 Doesn't get much simpler. 997 01:02:46,020 --> 01:02:53,460 So 2 times 1/2 to the p equals 1 implies p is 1. 998 01:02:53,460 --> 01:02:57,370 All right, now I just plug into the integral over there. 999 01:02:57,370 --> 01:02:58,476 All right, let's do that. 1000 01:03:12,370 --> 01:03:17,210 So t of x equals theta. 1001 01:03:17,210 --> 01:03:20,620 Well, x to the p is x to the 1 plus just take off 1002 01:03:20,620 --> 01:03:24,960 the 1, plus x times the integral 1 to x. 1003 01:03:24,960 --> 01:03:29,220 What's the g function? 1004 01:03:29,220 --> 01:03:31,020 What's g of x in this case? 1005 01:03:34,000 --> 01:03:35,950 x minus 1. 1006 01:03:35,950 --> 01:03:42,730 So I take the integral of g of u is just u minus 1 1007 01:03:42,730 --> 01:03:53,770 over u to the p plus 1 p minus 1. 1008 01:03:53,770 --> 01:03:56,310 Did I write the formula right? 1009 01:03:56,310 --> 01:04:00,840 Is it plus 1 or minus 1 here? 1010 01:04:00,840 --> 01:04:01,640 Did I get it? 1011 01:04:01,640 --> 01:04:02,520 p plus 1. 1012 01:04:02,520 --> 01:04:04,970 Sorry, didn't write that right. 1013 01:04:04,970 --> 01:04:07,136 p plus 1 over here. 1014 01:04:10,670 --> 01:04:14,860 All right, so u to the p plus 1 is u squared du. 1015 01:04:18,410 --> 01:04:27,580 This is theta of x plus x integral 1 over u minus 1 1016 01:04:27,580 --> 01:04:28,340 over u squared. 1017 01:04:40,650 --> 01:04:42,970 The integral of 1 over u is the log of u. 1018 01:04:45,500 --> 01:04:50,640 The integral of 1 over u squared is 1 over u, 1019 01:04:50,640 --> 01:04:53,800 but I changed the sign there. 1020 01:04:53,800 --> 01:04:54,680 1 to x. 1021 01:04:59,860 --> 01:05:08,670 All right, so I get theta of x plus x times log of x plus 1 1022 01:05:08,670 --> 01:05:12,790 over x minus log of 0. 1023 01:05:12,790 --> 01:05:15,820 That's log of 1 is nothing. 1024 01:05:15,820 --> 01:05:16,758 Minus 1. 1025 01:05:21,160 --> 01:05:27,870 And I'm doing thetas here so I can forget the low order terms. 1026 01:05:27,870 --> 01:05:32,430 So I get x log x plus 1 minus x. 1027 01:05:32,430 --> 01:05:35,400 This is the only term that survives, x log x. 1028 01:05:40,040 --> 01:05:42,860 And that's right. 1029 01:05:42,860 --> 01:05:47,956 t of n is in fact tilde n log n, so in fact, it's theta n log n. 1030 01:05:53,030 --> 01:05:56,150 Now, in this case, going through the integral and all that, 1031 01:05:56,150 --> 01:05:59,420 maybe that was harder than just guessing and verifying. 1032 01:05:59,420 --> 01:06:04,060 And by the way, guess and verify got it really the exact answer. 1033 01:06:04,060 --> 01:06:09,730 The nice thing is this works for the nasty guys too. 1034 01:06:09,730 --> 01:06:14,390 In fact, let's figure out the solution 1035 01:06:14,390 --> 01:06:17,950 to this thing, which I guarantee you is a pain 1036 01:06:17,950 --> 01:06:21,042 to do without this method. 1037 01:06:33,520 --> 01:06:37,050 The first thing to do is compute p. 1038 01:06:37,050 --> 01:06:42,610 So to do that, we need to find a p such that 2 times 1/2 1039 01:06:42,610 --> 01:06:55,270 to the p plus 8/9 times 3/4 to the p equals 1. 1040 01:06:55,270 --> 01:06:57,000 I gotta find a p for which that's true. 1041 01:06:59,700 --> 01:07:01,854 Any guesses what p works? 1042 01:07:06,140 --> 01:07:08,460 p equals 1 not going to do it. 1043 01:07:12,170 --> 01:07:14,330 p equals 2? 1044 01:07:14,330 --> 01:07:17,220 Let's try that. 1045 01:07:17,220 --> 01:07:26,000 I'd have 2 times 1/4 plus 8/9 times 9/16. 1046 01:07:26,000 --> 01:07:27,800 That looks pretty good. 1047 01:07:27,800 --> 01:07:32,310 I get 1/2 plus 1/2 equals 1. 1048 01:07:32,310 --> 01:07:33,430 So p equals 2 works. 1049 01:07:37,500 --> 01:07:39,337 All right, so let's do the integral. 1050 01:07:42,190 --> 01:07:49,640 tx is theta of x to the p plus x to the p times the integral. 1051 01:07:49,640 --> 01:07:55,170 All right, what is g of x in this thing? 1052 01:07:55,170 --> 01:07:56,670 x squared. 1053 01:07:56,670 --> 01:08:00,240 So the integral is 1 to x. g of u 1054 01:08:00,240 --> 01:08:07,580 is u squared divided by u to the 1 plus p. 1055 01:08:07,580 --> 01:08:10,276 u to the 1 plus 2 is just u cubed. 1056 01:08:15,860 --> 01:08:20,830 This is theta x squared plus x squared times, well, 1057 01:08:20,830 --> 01:08:23,140 the integral of 1 over u is just log of you. 1058 01:08:29,069 --> 01:08:32,529 And that's just log of x. 1059 01:08:32,529 --> 01:08:35,390 So the answer is, x square ln of x. 1060 01:08:35,390 --> 01:08:38,020 Done. 1061 01:08:38,020 --> 01:08:42,000 So that's pretty good what these guys did. 1062 01:08:42,000 --> 01:08:47,120 In fact, that is the correct answer. 1063 01:08:47,120 --> 01:08:50,029 So really easy to pug in. 1064 01:08:50,029 --> 01:08:55,930 Any questions on that? 1065 01:08:55,930 --> 01:08:57,274 Yeah. 1066 01:08:57,274 --> 01:09:03,069 AUDIENCE: [INAUDIBLE] or does it give the actual number? 1067 01:09:03,069 --> 01:09:05,380 TOM LEIGHTON: It doesn't give you the actual number. 1068 01:09:05,380 --> 01:09:08,870 It just gives you the asymptotic growth. 1069 01:09:08,870 --> 01:09:13,609 All you know is that the limit of tx over x squared ln of x 1070 01:09:13,609 --> 01:09:16,779 is less than infinity and bigger than 0. 1071 01:09:16,779 --> 01:09:18,760 So it's growing as this. 1072 01:09:18,760 --> 01:09:20,710 You don't know if it's 10x squared lnx 1073 01:09:20,710 --> 01:09:23,010 or a million squared lnx. 1074 01:09:23,010 --> 01:09:25,604 On the other hand, for algorithms usually 1075 01:09:25,604 --> 01:09:28,020 you're forgetting the constant factors anyway because they 1076 01:09:28,020 --> 01:09:29,826 depend on things to do with the CPU 1077 01:09:29,826 --> 01:09:31,200 you're using and stuff like that. 1078 01:09:31,200 --> 01:09:33,300 What you really care about, often, 1079 01:09:33,300 --> 01:09:37,819 when you're studying algorithms is the asymptotic growth, not 1080 01:09:37,819 --> 01:09:39,460 the actual value. 1081 01:09:39,460 --> 01:09:40,534 Yeah? 1082 01:09:40,534 --> 01:09:43,870 AUDIENCE: [INAUDIBLE] log base e. 1083 01:09:43,870 --> 01:09:45,620 And then above I see you using log, 1084 01:09:45,620 --> 01:09:46,825 which I assume is base two? 1085 01:09:46,825 --> 01:09:48,529 Does it matter? 1086 01:09:48,529 --> 01:09:51,560 TOM LEIGHTON: It does up there because I've got a tilde. 1087 01:09:51,560 --> 01:09:54,310 And in the tilde, the constant factor matters. 1088 01:09:54,310 --> 01:09:56,810 It doesn't matter in here because log 1089 01:09:56,810 --> 01:10:00,530 base two and log base E are within a constant factor, 1090 01:10:00,530 --> 01:10:03,860 namely log of 2, ln of 2 is the constant factor. 1091 01:10:03,860 --> 01:10:06,800 So logs don't matter what the base 1092 01:10:06,800 --> 01:10:09,910 is once you're inside a theta or an O or that kind of thing. 1093 01:10:09,910 --> 01:10:13,660 In a tilde, they matter because the constant factor matters. 1094 01:10:13,660 --> 01:10:14,534 Yeah. 1095 01:10:14,534 --> 01:10:15,450 AUDIENCE: [INAUDIBLE]. 1096 01:10:22,685 --> 01:10:23,476 TOM LEIGHTON: Yeah. 1097 01:10:23,476 --> 01:10:25,460 Because x squared is small compared 1098 01:10:25,460 --> 01:10:29,404 to x squared ln of x because it's all by the theta. 1099 01:10:29,404 --> 01:10:31,690 AUDIENCE: [INAUDIBLE]. 1100 01:10:31,690 --> 01:10:33,040 TOM LEIGHTON: No. 1101 01:10:33,040 --> 01:10:35,840 You could put all sorts of other stuff in here 1102 01:10:35,840 --> 01:10:38,290 and it doesn't matter because it still equals 1103 01:10:38,290 --> 01:10:39,910 theta of x squared ln of x. 1104 01:10:39,910 --> 01:10:44,380 There's no hidden meanings by having extra stuff in here. 1105 01:10:44,380 --> 01:10:45,340 That's important. 1106 01:10:45,340 --> 01:10:48,260 In this case, when you put more stuff into theta, 1107 01:10:48,260 --> 01:10:51,430 doesn't mean a thing. 1108 01:10:51,430 --> 01:10:52,380 Any other questions? 1109 01:10:52,380 --> 01:10:53,364 Yeah. 1110 01:10:53,364 --> 01:10:56,190 AUDIENCE: [INAUDIBLE]. 1111 01:10:56,190 --> 01:10:59,480 TOM LEIGHTON: Ah, good question. 1112 01:10:59,480 --> 01:11:00,915 Well, let's see. 1113 01:11:00,915 --> 01:11:03,553 It goes back to over here. 1114 01:11:13,340 --> 01:11:15,390 See, the b's are between 0 and 1. 1115 01:11:15,390 --> 01:11:19,000 By making p big enough, I drive the value down. 1116 01:11:19,000 --> 01:11:22,460 By making p be large and negative, I drag the value up. 1117 01:11:22,460 --> 01:11:23,710 So there is some value of p. 1118 01:11:23,710 --> 01:11:25,040 But that's a good question. 1119 01:11:25,040 --> 01:11:30,310 Let's see what happens when p is not so nice. 1120 01:11:30,310 --> 01:11:32,565 Because the examples I gave you, in one case p was 1 1121 01:11:32,565 --> 01:11:35,220 and one p was 2, sort of nice. 1122 01:11:35,220 --> 01:11:37,175 Let's look at a bad case there. 1123 01:11:43,130 --> 01:11:44,523 Let's look at this recurrence. 1124 01:11:50,790 --> 01:12:01,041 3 t of x over 3 plus 4 t of x over 4 plus x squared. 1125 01:12:01,041 --> 01:12:02,540 All right, so that's the recurrence. 1126 01:12:02,540 --> 01:12:04,210 And first step is to compute p. 1127 01:12:04,210 --> 01:12:07,420 And this time it's not going to be nice. 1128 01:12:07,420 --> 01:12:11,890 So I need to find a p such that 3 times 1/3 of the p plus 4 1129 01:12:11,890 --> 01:12:17,600 times 1/4 to the p equals 1. 1130 01:12:17,600 --> 01:12:20,412 And you might say, OK, let's try p equals 1. 1131 01:12:24,500 --> 01:12:29,540 I get 1 plus 1 is 2. 1132 01:12:29,540 --> 01:12:31,850 That's too big. 1133 01:12:31,850 --> 01:12:34,010 So which way do I have to go on p? 1134 01:12:34,010 --> 01:12:37,370 Do I need a bigger p or a smaller p? 1135 01:12:37,370 --> 01:12:38,230 Bigger. 1136 01:12:38,230 --> 01:12:40,800 Because I got to get from 2 down to 1. 1137 01:12:40,800 --> 01:12:44,241 So I know that p is bigger than 1. 1138 01:12:47,190 --> 01:12:49,050 All right, so the next thing to try 1139 01:12:49,050 --> 01:12:54,440 would be hoping the world is nice, p equals 2. 1140 01:12:54,440 --> 01:12:55,800 Maybe I'll do that over here. 1141 01:12:55,800 --> 01:12:56,980 Let's try p equals 2. 1142 01:13:00,250 --> 01:13:07,190 So I get three times 1/9 plus 4 times 1/16 1143 01:13:07,190 --> 01:13:12,200 and that equals a 1/3 plus 1/4. 1144 01:13:12,200 --> 01:13:15,590 That's less than 1. 1145 01:13:15,590 --> 01:13:16,300 That didn't work. 1146 01:13:16,300 --> 01:13:17,633 What do I have to do now with p? 1147 01:13:17,633 --> 01:13:20,700 Is it bigger than 2 or less than 2? 1148 01:13:20,700 --> 01:13:21,647 Less than 2. 1149 01:13:24,940 --> 01:13:26,690 That's sort of a pain. 1150 01:13:26,690 --> 01:13:30,270 P is between one and two, so it's not an integer. 1151 01:13:30,270 --> 01:13:32,400 Now, you can sort of do divide and conquer here. 1152 01:13:32,400 --> 01:13:36,300 Next I'd try p is 3/2, get my calculator out 1153 01:13:36,300 --> 01:13:37,870 and see which way it goes. 1154 01:13:37,870 --> 01:13:39,980 And you could sort of home in on it. 1155 01:13:39,980 --> 01:13:41,360 But there's something really nice 1156 01:13:41,360 --> 01:13:45,110 that happens with Akra Bazzi. 1157 01:13:45,110 --> 01:13:48,450 And that is if you ever conclude that p 1158 01:13:48,450 --> 01:13:52,700 is less than your exponent here, you 1159 01:13:52,700 --> 01:13:55,610 don't even have to compute it. 1160 01:13:55,610 --> 01:13:56,407 Let's see why. 1161 01:13:56,407 --> 01:13:57,240 Let's do an example. 1162 01:14:02,240 --> 01:14:05,870 This will happen a lot. 1163 01:14:05,870 --> 01:14:08,810 There's some p, but it's just going to be a pain to evaluate. 1164 01:14:08,810 --> 01:14:10,300 But let's see what happens here. 1165 01:14:14,226 --> 01:14:15,600 So I'm just going to try to solve 1166 01:14:15,600 --> 01:14:17,810 it, write the solution anyway. t of x 1167 01:14:17,810 --> 01:14:25,420 equals theta x to the p plus x to the p integral 1 to x. 1168 01:14:25,420 --> 01:14:27,366 What's g of x? 1169 01:14:27,366 --> 01:14:28,400 x squared. 1170 01:14:28,400 --> 01:14:33,354 So that's u squared over u 1 plus p. 1171 01:14:33,354 --> 01:14:34,770 And I still don't know what it is, 1172 01:14:34,770 --> 01:14:39,020 but I'm going to give it a try here. 1173 01:14:39,020 --> 01:14:44,150 That's theta x to the p plus x to the p. 1174 01:14:44,150 --> 01:14:51,466 Well, this is now the integral of u to the 1 minus p du. 1175 01:14:51,466 --> 01:14:53,125 And p is bigger than 1. 1176 01:14:53,125 --> 01:14:55,470 I know that. 1177 01:14:55,470 --> 01:15:02,020 This is theta x to the p plus x to the p. 1178 01:15:02,020 --> 01:15:07,880 Well, the integral here is x to the 2 minus p. 1179 01:15:07,880 --> 01:15:08,730 Up to constants. 1180 01:15:08,730 --> 01:15:10,830 I don't even care about dividing by 2 minus p, 1181 01:15:10,830 --> 01:15:13,090 because I got big theta. 1182 01:15:13,090 --> 01:15:15,920 Just put that p up here. 1183 01:15:15,920 --> 01:15:17,250 OK 1184 01:15:17,250 --> 01:15:24,270 Well, this equals theta x to the p plus cancelled there, 1185 01:15:24,270 --> 01:15:24,790 x squared. 1186 01:15:27,340 --> 01:15:30,280 And what's the answer going to be? 1187 01:15:30,280 --> 01:15:33,240 x squared, because p is less than 2. 1188 01:15:33,240 --> 01:15:36,000 So this is a small order term. 1189 01:15:36,000 --> 01:15:37,950 So I didn't even know what p was, just 1190 01:15:37,950 --> 01:15:40,020 that it was less than two. 1191 01:15:40,020 --> 01:15:41,235 And I got the answer. 1192 01:15:41,235 --> 01:15:42,110 And what do you know? 1193 01:15:42,110 --> 01:15:45,440 The answer was just this thing. 1194 01:15:45,440 --> 01:15:48,070 That's sort of nice and simple. 1195 01:15:48,070 --> 01:15:51,750 And in fact, that's a theorem. 1196 01:15:51,750 --> 01:16:11,900 Which will state, in general, if g of x equals theta x to the t, 1197 01:16:11,900 --> 01:16:13,502 for some t bigger and equal to 0. 1198 01:16:16,320 --> 01:16:21,610 And when you plug that value in and take your sum. 1199 01:16:21,610 --> 01:16:22,860 Trying to compute the p value. 1200 01:16:22,860 --> 01:16:26,320 So I take ai times bi to the t. 1201 01:16:26,320 --> 01:16:28,500 And that comes out to be less than 1 1202 01:16:28,500 --> 01:16:34,987 like it did here when I plugged in p equals 2 and I computed it 1203 01:16:34,987 --> 01:16:36,970 and it was less than 1. 1204 01:16:36,970 --> 01:16:42,940 But if that happens in general, then the answer 1205 01:16:42,940 --> 01:16:50,350 is theta g of x. 1206 01:16:50,350 --> 01:16:51,900 Plain and simple. 1207 01:16:51,900 --> 01:16:58,460 It just turns out to be the g of x term up to constant factors. 1208 01:16:58,460 --> 01:17:00,430 So you don't even have to compute it. 1209 01:17:00,430 --> 01:17:03,280 Just do that check on the power here to see 1210 01:17:03,280 --> 01:17:05,969 is it smaller than that. 1211 01:17:05,969 --> 01:17:07,010 I won't prove that there. 1212 01:17:07,010 --> 01:17:08,310 It's actually not hard to do. 1213 01:17:08,310 --> 01:17:10,980 The proof is pretty much what I did with this example, 1214 01:17:10,980 --> 01:17:12,082 so it's not too hard. 1215 01:17:15,820 --> 01:17:16,860 OK, any questions? 1216 01:17:19,550 --> 01:17:21,200 I want to show you one more thing, 1217 01:17:21,200 --> 01:17:24,060 but just make sure that's OK. 1218 01:17:24,060 --> 01:17:30,530 And this is something never to do with an asymptotic notation, 1219 01:17:30,530 --> 01:17:33,860 like we talked about last time. 1220 01:17:33,860 --> 01:17:36,840 When you're using Akra Bazzi and when you're doing occurrences 1221 01:17:36,840 --> 01:17:41,630 with algorithm, you always are using asymptotic notation. 1222 01:17:41,630 --> 01:17:44,490 Your answer is theta of something. 1223 01:17:44,490 --> 01:17:46,360 And in fact, what will start happening is 1224 01:17:46,360 --> 01:17:52,470 you'll do this stuff because the constant factors don't matter. 1225 01:17:52,470 --> 01:17:55,320 Because you're wiping them out in the theta notation, 1226 01:17:55,320 --> 01:17:56,860 what you'll see is things like this. 1227 01:17:56,860 --> 01:18:02,300 Your recurrence will be set up to be t of n is 2 t of n over 2 1228 01:18:02,300 --> 01:18:07,230 plus o of n or plus theta of n. 1229 01:18:07,230 --> 01:18:11,130 And then you'll conclude that t of n 1230 01:18:11,130 --> 01:18:15,550 is theta of n log n using Akra Bazzi 1231 01:18:15,550 --> 01:18:18,769 or whatever method you want. 1232 01:18:18,769 --> 01:18:20,310 Because it doesn't really matter what 1233 01:18:20,310 --> 01:18:26,330 the constant is on g because the constants disappear. 1234 01:18:26,330 --> 01:18:27,410 Doesn't matter. 1235 01:18:27,410 --> 01:18:31,650 And so people stop worrying about it early on. 1236 01:18:31,650 --> 01:18:34,830 All right, now here's the bad example. 1237 01:18:39,394 --> 01:18:44,230 Let me show you a false proof like last time. 1238 01:18:44,230 --> 01:18:47,342 And this time I think it'll be pretty easy to spot the flaw. 1239 01:19:01,960 --> 01:19:03,060 Theorem not. 1240 01:19:08,630 --> 01:19:16,050 If t of n equals 2 t of n over 2 plus n minus 1, 1241 01:19:16,050 --> 01:19:20,590 this is the recurrence we had for merge sort, and t of 1 1242 01:19:20,590 --> 01:19:29,118 equals 0, then I'm going to prove that t of n is o event. 1243 01:19:32,360 --> 01:19:38,150 Now, that can't be right because we've proved it's theta 1244 01:19:38,150 --> 01:19:40,050 then log n, tilde then log n. 1245 01:19:40,050 --> 01:19:42,340 And that is not o of n, right? 1246 01:19:42,340 --> 01:19:46,190 n log n grows faster than n. 1247 01:19:46,190 --> 01:19:47,190 But let's see the proof. 1248 01:19:52,920 --> 01:19:53,911 By strong induction. 1249 01:20:00,720 --> 01:20:04,800 The induction hypothesis is going 1250 01:20:04,800 --> 01:20:07,240 to be what we're trying to prove. 1251 01:20:16,770 --> 01:20:19,830 Base case. 1252 01:20:19,830 --> 01:20:21,600 n equals 1. 1253 01:20:21,600 --> 01:20:23,810 t of 1 is 0. 1254 01:20:23,810 --> 01:20:26,050 0 is surely o of 1. 1255 01:20:29,360 --> 01:20:30,873 The induction step. 1256 01:20:36,370 --> 01:20:41,730 We're going to assume p1, p2, p3 up to pn minus 1 to prove pn. 1257 01:20:55,190 --> 01:20:57,190 And let's look at p of n. 1258 01:20:57,190 --> 01:21:04,820 So I look at tn is 2 t of n over 2 plus n minus 1. 1259 01:21:04,820 --> 01:21:07,230 Induction hypothesis on n over 2 says 1260 01:21:07,230 --> 01:21:12,920 this is 2 times o of n over 2 plus n minus 1. 1261 01:21:12,920 --> 01:21:18,000 Twice o of n over 2 is o of n plus n is o of n. 1262 01:21:18,000 --> 01:21:19,780 This is o of n. 1263 01:21:19,780 --> 01:21:22,520 And I'm done. 1264 01:21:22,520 --> 01:21:25,580 I just proved that t of n is o of n. 1265 01:21:28,890 --> 01:21:30,420 What's the bad step? 1266 01:21:30,420 --> 01:21:31,050 What's the bug? 1267 01:21:33,710 --> 01:21:35,150 See the bug? 1268 01:21:35,150 --> 01:21:36,514 What is it? 1269 01:21:36,514 --> 01:21:37,430 AUDIENCE: [INAUDIBLE]. 1270 01:21:37,430 --> 01:21:38,890 TOM LEIGHTON: Induction hypothesis. 1271 01:21:38,890 --> 01:21:40,541 What's wrong with that? 1272 01:21:40,541 --> 01:21:41,457 AUDIENCE: [INAUDIBLE]. 1273 01:21:46,017 --> 01:21:46,850 TOM LEIGHTON: Right. 1274 01:21:46,850 --> 01:21:50,260 The end from the predicate cannot be in the o of n. 1275 01:21:50,260 --> 01:21:54,930 Because remember, o of n means, this statement here by itself 1276 01:21:54,930 --> 01:22:01,240 means the limit as n goes to infinity of t of n over n 1277 01:22:01,240 --> 01:22:04,160 is less than infinity. 1278 01:22:04,160 --> 01:22:10,700 But this makes no sense if I specified n in a predicate. 1279 01:22:10,700 --> 01:22:11,710 So what's the rule? 1280 01:22:11,710 --> 01:22:13,290 Same rule as last time. 1281 01:22:13,290 --> 01:22:14,170 Same bug. 1282 01:22:14,170 --> 01:22:15,075 What's the rule? 1283 01:22:18,360 --> 01:22:21,730 Never do asymptotic notation in induction hypothesis. 1284 01:22:21,730 --> 01:22:24,430 Never mix big o and a predicate. 1285 01:22:24,430 --> 01:22:25,930 Never, ever. 1286 01:22:25,930 --> 01:22:30,400 It looks so nice to do, but it is so wrong. 1287 01:22:30,400 --> 01:22:32,910 It just makes nonsense. 1288 01:22:32,910 --> 01:22:34,430 All right, very good. 1289 01:22:34,430 --> 01:22:39,650 So remember ice cream tonight, recitation optional tomorrow. 1290 01:22:39,650 --> 01:22:42,300 But study session is there.