1 00:00:00,090 --> 00:00:02,500 The following content is provided under a Creative 2 00:00:02,500 --> 00:00:04,019 Commons license. 3 00:00:04,019 --> 00:00:06,360 Your support will help MIT OpenCourseWare 4 00:00:06,360 --> 00:00:10,730 continue to offer high quality educational resources for free. 5 00:00:10,730 --> 00:00:13,340 To make a donation or view additional materials 6 00:00:13,340 --> 00:00:17,217 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:17,217 --> 00:00:17,842 at ocw.mit.edu. 8 00:00:20,872 --> 00:00:22,680 LING REN: All right, welcome everyone. 9 00:00:22,680 --> 00:00:25,832 This is 6046 presentation. 10 00:00:25,832 --> 00:00:27,540 Just make sure you're in the right place. 11 00:00:30,670 --> 00:00:32,050 My name is Ling Ren. 12 00:00:32,050 --> 00:00:36,570 I'm one of the 10 TAs for this class. 13 00:00:36,570 --> 00:00:39,860 We do have a second TA for this section. 14 00:00:39,860 --> 00:00:41,840 I think he is not here right now, 15 00:00:41,840 --> 00:00:44,920 but, basically, [INAUDIBLE] and I will 16 00:00:44,920 --> 00:00:48,840 be switching every other week. 17 00:00:48,840 --> 00:00:51,500 I want to remind you, just a heads up, 18 00:00:51,500 --> 00:00:55,500 this section is recorded for OCW purpose. 19 00:00:55,500 --> 00:01:01,110 But I think he's only recording us, the TAs. 20 00:01:01,110 --> 00:01:02,710 You're not in a camera. 21 00:01:08,040 --> 00:01:10,140 All right, so, the only purpose we are here 22 00:01:10,140 --> 00:01:13,060 is to help you learn this very interesting, and also very 23 00:01:13,060 --> 00:01:14,320 useful, class. 24 00:01:14,320 --> 00:01:17,810 So don't hesitate to ask any questions 25 00:01:17,810 --> 00:01:20,170 or give us any feedback, like whether I'm 26 00:01:20,170 --> 00:01:24,360 going too fast or too slow, whether you want us to cover 27 00:01:24,360 --> 00:01:28,370 something that's not in the posted schedule, 28 00:01:28,370 --> 00:01:32,390 or just anything we can help. 29 00:01:32,390 --> 00:01:34,930 All right, so let's get started. 30 00:01:34,930 --> 00:01:38,640 The two lectures in this week, in the first week, 31 00:01:38,640 --> 00:01:40,220 focus on divide and conquer. 32 00:01:49,610 --> 00:01:53,170 It is a class of algorithm that usually involves recursion 33 00:01:53,170 --> 00:01:55,960 in the algorithm description. 34 00:01:55,960 --> 00:02:02,220 And Professor Devadas worked through several algorithms, 35 00:02:02,220 --> 00:02:04,800 including weighted interval scheduling 36 00:02:04,800 --> 00:02:07,949 and a bunch of others, and he left several open problems. 37 00:02:20,920 --> 00:02:24,460 So we will answer those open questions in this section, 38 00:02:24,460 --> 00:02:27,810 and we'll also show you a new algorithm, 39 00:02:27,810 --> 00:02:32,080 and analyze a bunch of other algorithms. 40 00:02:32,080 --> 00:02:35,520 So just to remind everyone what weighted interval scheduling 41 00:02:35,520 --> 00:02:36,020 is. 42 00:02:40,230 --> 00:02:52,490 In this problem, we are given a bunch of requests, 43 00:02:52,490 --> 00:02:55,230 each with a start time and a finish time. 44 00:02:55,230 --> 00:03:01,480 And our goal is to find a subset of them that are compatible, 45 00:03:01,480 --> 00:03:04,750 meaning they do not overlap, and that 46 00:03:04,750 --> 00:03:07,640 have a largest combined weight. 47 00:03:07,640 --> 00:03:08,680 OK, are we clear? 48 00:03:08,680 --> 00:03:10,760 Everyone clear about that? 49 00:03:10,760 --> 00:03:14,260 So an easier case is when the problem 50 00:03:14,260 --> 00:03:18,980 is unweighted, meaning that every task has the same weight. 51 00:03:18,980 --> 00:03:20,880 In that case, we can just solve it using 52 00:03:20,880 --> 00:03:24,120 our [? Grady ?] algorithm. 53 00:03:24,120 --> 00:03:26,360 But when the problem becomes weighted, 54 00:03:26,360 --> 00:03:29,860 we have to use dynamic programming, or recursion, 55 00:03:29,860 --> 00:03:35,560 and Srini introduced a simple one, a basic version, 56 00:03:35,560 --> 00:03:36,560 in a class. 57 00:03:36,560 --> 00:03:41,395 Can someone remind us how that algorithm works? 58 00:03:41,395 --> 00:03:42,020 Any volunteers? 59 00:03:53,255 --> 00:03:54,130 AUDIENCE: [INAUDIBLE] 60 00:03:58,359 --> 00:03:59,650 LING REN: Can you speak louder? 61 00:03:59,650 --> 00:04:02,750 AUDIENCE: [INAUDIBLE] so we find that it's 62 00:04:02,750 --> 00:04:04,154 not conflicting [INAUDIBLE]. 63 00:04:09,330 --> 00:04:10,597 LING REN: So what's your name? 64 00:04:10,597 --> 00:04:11,472 AUDIENCE: [INAUDIBLE] 65 00:04:15,749 --> 00:04:17,790 LING REN: Yeah, I think the version you described 66 00:04:17,790 --> 00:04:19,360 is for the unweighted case. 67 00:04:19,360 --> 00:04:22,230 In unweighted case, we just schedule 68 00:04:22,230 --> 00:04:24,270 the 1 with the earliest finish time, 69 00:04:24,270 --> 00:04:27,040 and, then, we remove all the incompatible ones, 70 00:04:27,040 --> 00:04:28,500 and we keep going, right? 71 00:04:28,500 --> 00:04:30,990 That solved the unweighted version. 72 00:04:30,990 --> 00:04:34,495 If it's the weighted version, we need to use recursion. 73 00:04:34,495 --> 00:04:39,970 And remember, we break the problem into many sub-problems, 74 00:04:39,970 --> 00:04:43,997 and each one can potentially be an optimal solution. 75 00:04:43,997 --> 00:04:45,080 Does anyone remember that? 76 00:04:48,010 --> 00:04:49,398 Care to give it a try? 77 00:04:49,398 --> 00:04:50,862 AUDIENCE: Sure. 78 00:04:50,862 --> 00:04:53,790 So we break it into subproblems. 79 00:04:53,790 --> 00:04:57,694 We took the best solution from a certain point, 80 00:04:57,694 --> 00:05:00,622 and then we calculate that subproblem 81 00:05:00,622 --> 00:05:03,062 starting at all the different finish times. 82 00:05:03,062 --> 00:05:05,804 [INAUDIBLE] 83 00:05:05,804 --> 00:05:06,470 LING REN: Great. 84 00:05:06,470 --> 00:05:07,516 What is your name? 85 00:05:07,516 --> 00:05:10,090 AUDIENCE: [? Amin. ?] 86 00:05:10,090 --> 00:05:11,210 LING REN: [? Amin? ?] OK. 87 00:05:11,210 --> 00:05:14,700 [? Amin ?] said, let's just try every one 88 00:05:14,700 --> 00:05:18,775 as our potential first request. 89 00:05:22,250 --> 00:05:28,890 So if we request the j as our first, we get its weight. 90 00:05:28,890 --> 00:05:31,900 And then, we're going to solve a subproblem. 91 00:05:31,900 --> 00:05:33,930 So let me call the original problem. 92 00:05:33,930 --> 00:05:41,040 We did interval scheduling with all the incoming requests. 93 00:05:41,040 --> 00:05:44,480 Now, we choose j, request as our first. 94 00:05:44,480 --> 00:05:51,800 Now, we are left with a subproblem that starts 95 00:05:51,800 --> 00:05:54,250 after request to j finishes. 96 00:05:54,250 --> 00:05:59,330 So I'll write that as Rj, Where I 97 00:05:59,330 --> 00:06:06,350 define Rj to be the set of requests where their stop 98 00:06:06,350 --> 00:06:15,170 time is later than the finish time of the j-th request. 99 00:06:18,450 --> 00:06:23,180 OK, so just to repeat, we choose a request, 100 00:06:23,180 --> 00:06:26,060 has the potential to be the first request, 101 00:06:26,060 --> 00:06:29,510 and then, we look at all the requests that start after it, 102 00:06:29,510 --> 00:06:34,090 and solve a subproblem of that case. 103 00:06:34,090 --> 00:06:43,880 Then, we take a max of all the candidate we have, 104 00:06:43,880 --> 00:06:47,164 and that's going to give us the optimal solution. 105 00:06:47,164 --> 00:06:48,580 Any question about this algorithm? 106 00:06:54,980 --> 00:06:57,640 So this algorithm runs in n square time. 107 00:07:05,560 --> 00:07:09,120 Now, we're going to try to optimize that, and come up 108 00:07:09,120 --> 00:07:12,460 with a better algorithm. 109 00:07:12,460 --> 00:07:18,330 So in order to improve anything, we first 110 00:07:18,330 --> 00:07:22,650 want to identify the inefficiency in this algorithm. 111 00:07:22,650 --> 00:07:25,960 So which part in the algorithm do you 112 00:07:25,960 --> 00:07:29,191 think is inefficient, or silly, unnecessary? 113 00:07:29,191 --> 00:07:29,690 Go ahead. 114 00:07:29,690 --> 00:07:31,594 AUDIENCE: So it's inefficient to look 115 00:07:31,594 --> 00:07:36,584 through every previous subproblem, when we're trying 116 00:07:36,584 --> 00:07:42,890 to find maximum [INAUDIBLE]. 117 00:07:42,890 --> 00:07:44,390 LING REN: I was saying that we don't 118 00:07:44,390 --> 00:07:48,428 need to go through every of this case. 119 00:07:48,428 --> 00:07:51,400 AUDIENCE: Yeah, we shouldn't Well, 120 00:07:51,400 --> 00:07:58,090 we should be able to efficiently query for the right one. 121 00:07:58,090 --> 00:08:01,910 LING REN: OK, I think you are definitely correct. 122 00:08:01,910 --> 00:08:05,230 So let me just go through what this algorithm does, and it 123 00:08:05,230 --> 00:08:06,770 will be more clear. 124 00:08:06,770 --> 00:08:12,510 So what this means is, I'll choose the first request. 125 00:08:12,510 --> 00:08:14,820 I'll request 1 as my first request. 126 00:08:14,820 --> 00:08:18,090 Then, I'm going to consider only the requests that 127 00:08:18,090 --> 00:08:20,100 start after 1 finishes, right? 128 00:08:20,100 --> 00:08:22,680 That only leaves request a 5. 129 00:08:22,680 --> 00:08:25,710 Potentially some other's already been drawn there. 130 00:08:25,710 --> 00:08:28,592 That's my first candidate in that max. 131 00:08:28,592 --> 00:08:34,169 My second candidate is actually request 2 as my first request. 132 00:08:34,169 --> 00:08:38,049 Then, I have to remove request 1, because it starts too early, 133 00:08:38,049 --> 00:08:40,510 and then, I'm left with all the remaining requests. 134 00:08:40,510 --> 00:08:42,380 I'll solve that subproblem. 135 00:08:42,380 --> 00:08:43,880 That's candidate 2. 136 00:08:43,880 --> 00:08:50,190 Candidate 3, I chose request 3 as my first request, 137 00:08:50,190 --> 00:08:52,460 and then I have to remove 1 and 2, 138 00:08:52,460 --> 00:08:54,810 because they start too early. 139 00:08:54,810 --> 00:08:55,817 Oh, 4 as well. 140 00:08:55,817 --> 00:08:57,150 It also starts too early, right? 141 00:08:57,150 --> 00:08:58,750 Before 3 finishes. 142 00:08:58,750 --> 00:09:00,080 Everyone following that? 143 00:09:00,080 --> 00:09:04,170 So we're left with the remaining request. 144 00:09:04,170 --> 00:09:05,070 Is it more clear now? 145 00:09:08,790 --> 00:09:09,290 Go ahead. 146 00:09:09,290 --> 00:09:12,720 AUDIENCE: So candidate, if you start with 3, that's 147 00:09:12,720 --> 00:09:16,140 actually a subproblem of 2. 148 00:09:16,140 --> 00:09:16,640 [INAUDIBLE] 149 00:09:20,070 --> 00:09:22,462 LING REN: Great point. 150 00:09:22,462 --> 00:09:23,170 What's your name? 151 00:09:23,170 --> 00:09:24,990 AUDIENCE: Andrew. 152 00:09:24,990 --> 00:09:27,370 LING REN: Andrew said, we can potentially 153 00:09:27,370 --> 00:09:32,590 be solving many repeated subproblems, like this. 154 00:09:32,590 --> 00:09:34,370 We definitely don't want to do that. 155 00:09:34,370 --> 00:09:39,340 And that's actually the core idea, the one crisp idea 156 00:09:39,340 --> 00:09:41,720 of dynamic programming. 157 00:09:41,720 --> 00:09:43,880 Andrew, can you tell me what's the definition 158 00:09:43,880 --> 00:09:46,016 of dynamic programming? 159 00:09:46,016 --> 00:09:46,970 You remember? 160 00:09:46,970 --> 00:09:49,846 Anyone remember that? 161 00:09:49,846 --> 00:09:50,700 Go ahead. 162 00:09:50,700 --> 00:09:53,940 AUDIENCE: Just memorize subproblems, then look them up. 163 00:09:53,940 --> 00:09:54,830 LING REN: Exactly. 164 00:09:54,830 --> 00:10:00,060 So dynamic programming says, we will break a problem 165 00:10:00,060 --> 00:10:01,920 into subproblems, and subproblems 166 00:10:01,920 --> 00:10:05,370 into even more subproblems, but whenever we solve one, 167 00:10:05,370 --> 00:10:09,220 we should memorize, or just remember its result 168 00:10:09,220 --> 00:10:10,450 and store it somewhere. 169 00:10:10,450 --> 00:10:12,780 And if you need it again, we'll just retrieve it, 170 00:10:12,780 --> 00:10:14,115 without resolving the problem. 171 00:10:16,830 --> 00:10:19,240 That's definitely a great point. 172 00:10:19,240 --> 00:10:24,050 So we can analyze the complexity of this algorithm 173 00:10:24,050 --> 00:10:26,560 later, because I want to touch on this more 174 00:10:26,560 --> 00:10:28,080 efficient algorithm first. 175 00:10:28,080 --> 00:10:31,070 And we'll see that, even after Andrew's optimization, 176 00:10:31,070 --> 00:10:33,030 its runtime is n square. 177 00:10:33,030 --> 00:10:34,920 So without that observation, if we 178 00:10:34,920 --> 00:10:36,580 are solving repeated subproblems, 179 00:10:36,580 --> 00:10:40,105 these will be a lot worse than that. 180 00:10:40,105 --> 00:10:40,730 Yeah, go ahead. 181 00:10:40,730 --> 00:10:44,062 AUDIENCE: You can also trim ones that get to the same place, 182 00:10:44,062 --> 00:10:46,387 so you don't need to explore two paths. 183 00:10:46,387 --> 00:10:47,720 LING REN: Sorry, say that again? 184 00:10:47,720 --> 00:10:49,710 AUDIENCE: You don't need to explore two paths that 185 00:10:49,710 --> 00:10:50,918 [INAUDIBLE] both exploring 3. 186 00:10:50,918 --> 00:10:53,246 Once they're both looking at 3, then you only 187 00:10:53,246 --> 00:10:56,360 need to do the ones more efficient to that point. 188 00:10:56,360 --> 00:10:57,420 LING REN: OK, cool. 189 00:10:57,420 --> 00:10:59,318 Go ahead. 190 00:10:59,318 --> 00:11:00,525 Oh, same thing? 191 00:11:00,525 --> 00:11:02,316 AUDIENCE: Well, I have another one as well. 192 00:11:02,316 --> 00:11:05,020 So I think to get better than n squared, 193 00:11:05,020 --> 00:11:13,140 we need to make the observation that it's always fine to start 194 00:11:13,140 --> 00:11:15,830 a subproblem later. 195 00:11:15,830 --> 00:11:17,522 So if you've decided you're taking 196 00:11:17,522 --> 00:11:19,896 a certain sequence of intervals from your first interval, 197 00:11:19,896 --> 00:11:22,840 and then you want to see how to compute from there, 198 00:11:22,840 --> 00:11:31,708 it's always valid to start later than, maybe, you had to. 199 00:11:31,708 --> 00:11:35,575 So that means that if we can efficiently 200 00:11:35,575 --> 00:11:36,968 starting at any particular point, 201 00:11:36,968 --> 00:11:42,956 query for the maximum of any of the subproblems starting 202 00:11:42,956 --> 00:11:45,460 after that point, then we can [INAUDIBLE]. 203 00:11:45,460 --> 00:11:47,050 LING REN: OK, great. 204 00:11:47,050 --> 00:11:48,740 I think we are on the same page. 205 00:11:48,740 --> 00:11:53,420 So when I describe the steps of this algorithm, 206 00:11:53,420 --> 00:11:56,110 remember this third candidate. 207 00:11:56,110 --> 00:11:59,440 I choose this as my first, right? 208 00:11:59,440 --> 00:12:03,230 It makes zero sense, because, if I do that, 209 00:12:03,230 --> 00:12:06,410 I might as well put in 2, as well. 210 00:12:06,410 --> 00:12:10,117 Doesn't hurt, right? 211 00:12:10,117 --> 00:12:13,500 Everyone get that? 212 00:12:13,500 --> 00:12:22,120 So the idea is that we shouldn't try every possible W 213 00:12:22,120 --> 00:12:24,340 request as my first. 214 00:12:24,340 --> 00:12:27,000 Some requests are just better, more 215 00:12:27,000 --> 00:12:29,425 suited to be the first request. 216 00:12:29,425 --> 00:12:32,150 And how we're going to do that? 217 00:12:32,150 --> 00:12:34,130 So, apparently, 1 can potentially 218 00:12:34,130 --> 00:12:35,020 be the first request. 219 00:12:35,020 --> 00:12:36,780 2 can also be. 220 00:12:36,780 --> 00:12:40,350 But it doesn't make any sense for any requests 221 00:12:40,350 --> 00:12:44,110 come after that, because there's and earlier request. 222 00:12:44,110 --> 00:12:48,330 So the efficient algorithm, let's 223 00:12:48,330 --> 00:12:52,370 first sort them by their start time. 224 00:12:52,370 --> 00:12:56,600 We're going to consider the request that comes early first. 225 00:13:02,128 --> 00:13:05,350 So I have my entire problem, here. 226 00:13:08,710 --> 00:13:10,890 Now, I'm going to ask a question. 227 00:13:10,890 --> 00:13:15,750 Should I include request 1 in my solution or not? 228 00:13:15,750 --> 00:13:16,710 That's only two cases. 229 00:13:22,180 --> 00:13:28,440 So if I do not select 1 in my solution, what subproblem 230 00:13:28,440 --> 00:13:29,230 am I left with? 231 00:13:34,700 --> 00:13:37,640 Any idea? 232 00:13:37,640 --> 00:13:42,690 If I decide I will not include 1 in my solution. 233 00:13:42,690 --> 00:13:43,911 AUDIENCE: [INAUDIBLE]. 234 00:13:43,911 --> 00:13:44,910 LING REN: Yeah, exactly. 235 00:13:47,660 --> 00:13:49,410 There's no conflicts anywhere. 236 00:13:49,410 --> 00:13:52,560 I'll solve a subproblem from 2 to n. 237 00:13:52,560 --> 00:13:57,090 If I do decide to put my request 1 in a solution, 238 00:13:57,090 --> 00:13:58,610 I get this weight. 239 00:13:58,610 --> 00:14:00,910 So what's the subproblem I'm left with? 240 00:14:09,520 --> 00:14:12,330 Yeah, in this example, it's five. 241 00:14:12,330 --> 00:14:12,830 Correct. 242 00:14:12,830 --> 00:14:13,663 But more generally-- 243 00:14:16,850 --> 00:14:23,100 AUDIENCE: [INAUDIBLE] So every request that starts afterward. 244 00:14:23,100 --> 00:14:29,000 LING REN: Exactly, every request that starts after 1 finishes. 245 00:14:29,000 --> 00:14:31,720 Now, suddenly, we are not breaking the original problem 246 00:14:31,720 --> 00:14:32,990 into n subproblems. 247 00:14:32,990 --> 00:14:34,310 We only have two subproblems. 248 00:14:40,050 --> 00:14:41,880 So let me draw a recursion tree, which 249 00:14:41,880 --> 00:14:45,830 is a powerful tool in analyzing these sort of things. 250 00:14:49,800 --> 00:14:56,780 So we start with our original problem, from 1 to n, 251 00:14:56,780 --> 00:15:02,760 and we have two subproblems, 2 to n and R1 to n. 252 00:15:02,760 --> 00:15:06,800 This one, we'll also break them into subproblems. 253 00:15:06,800 --> 00:15:07,645 So what is this one? 254 00:15:12,590 --> 00:15:15,890 OK, I have my sub problem 2 to n, 255 00:15:15,890 --> 00:15:25,856 and I need to further reapply the same trick 256 00:15:25,856 --> 00:15:26,772 AUDIENCE: [INAUDIBLE]. 257 00:15:32,103 --> 00:15:35,350 The max of Wi has to be-- 258 00:15:35,350 --> 00:15:36,380 LING REN: Oh, sorry. 259 00:15:39,660 --> 00:15:41,710 These are the two cases. 260 00:15:44,590 --> 00:15:48,650 Either I do not schedule my first, or I schedule it, 261 00:15:48,650 --> 00:15:50,970 and this is my first request. 262 00:15:50,970 --> 00:15:53,550 Now, here, I'm left with this problem. 263 00:15:53,550 --> 00:15:56,880 Either I do not schedule it, or I schedule it 264 00:15:56,880 --> 00:15:58,470 as my first request. 265 00:15:58,470 --> 00:15:59,390 So what is this one? 266 00:16:05,102 --> 00:16:08,440 AUDIENCE: Probably be 3 to the n. 267 00:16:08,440 --> 00:16:10,540 LING REN: 3 to n, because now I'm 268 00:16:10,540 --> 00:16:12,560 asking the same question for 2. 269 00:16:12,560 --> 00:16:20,140 And, here, I'll have R2 to n, so on and so forth. 270 00:16:20,140 --> 00:16:27,700 Now, let me point out the big difference for this version, 271 00:16:27,700 --> 00:16:29,380 and for the basic version. 272 00:16:29,380 --> 00:16:33,820 So I'm starting with the request that starts first. 273 00:16:33,820 --> 00:16:37,940 If I do not do that, say, if here, 274 00:16:37,940 --> 00:16:42,870 I am asking the question for 5, do I schedule 5 as my first 275 00:16:42,870 --> 00:16:44,150 or not? 276 00:16:44,150 --> 00:16:47,250 Then, what happens? 277 00:16:47,250 --> 00:16:51,400 Then, these two branches do not cover all the cases, 278 00:16:51,400 --> 00:16:55,100 because I can potentially schedule it, 279 00:16:55,100 --> 00:16:59,840 but not as my first, in the optimal solution, 280 00:16:59,840 --> 00:17:03,650 if I'm asking a question for a random request. 281 00:17:03,650 --> 00:17:08,970 However, if I start with the first request, first meaning it 282 00:17:08,970 --> 00:17:11,710 starts earliest, it will either be not scheduled, 283 00:17:11,710 --> 00:17:14,040 or it will be scheduled, and as the first. 284 00:17:14,040 --> 00:17:16,956 Because it cannot be the second request in my solution. 285 00:17:20,432 --> 00:17:21,890 Any questions about this algorithm? 286 00:17:31,230 --> 00:17:34,820 OK, now, this is the algorithm. 287 00:17:34,820 --> 00:17:37,930 Let's analyze its complexity. 288 00:17:37,930 --> 00:17:41,130 So what's the overall complexity. 289 00:17:41,130 --> 00:17:42,710 when we go all the way down, solve 290 00:17:42,710 --> 00:17:45,710 the entire original problem? 291 00:17:45,710 --> 00:17:46,850 Any guesses? 292 00:17:49,480 --> 00:17:50,230 What do you think? 293 00:17:55,040 --> 00:17:57,450 What do you think? 294 00:17:57,450 --> 00:17:59,286 Go ahead. 295 00:17:59,286 --> 00:18:02,484 AUDIENCE: It should be log n, because [INAUDIBLE] sort 296 00:18:02,484 --> 00:18:07,158 the intervals, and we're going to need to, for every interval, 297 00:18:07,158 --> 00:18:13,800 find the next interval that starts after it, 298 00:18:13,800 --> 00:18:16,985 and do that for [INAUDIBLE]. 299 00:18:16,985 --> 00:18:17,610 LING REN: Cool. 300 00:18:17,610 --> 00:18:21,990 So the first step is sorting, which is n log n. 301 00:18:25,930 --> 00:18:32,890 Now, after I sort everything, the question I need to answer 302 00:18:32,890 --> 00:18:37,020 is, how many unique subproblems are there, 303 00:18:37,020 --> 00:18:38,480 in this entire recursion tree? 304 00:18:38,480 --> 00:18:42,270 So I'm not going to solve the same problem twice. 305 00:18:42,270 --> 00:18:44,840 How many unique problems exist, here? 306 00:18:47,420 --> 00:18:48,500 n of them, right? 307 00:18:48,500 --> 00:18:50,780 Just this left branch. 308 00:18:50,780 --> 00:18:54,280 All the others will be one of these. 309 00:18:54,280 --> 00:18:59,430 So I can start from the bottom of the tree and work my way up, 310 00:18:59,430 --> 00:19:04,470 and when I want to take this step, 311 00:19:04,470 --> 00:19:07,690 I'll look up the result of this in one 312 00:19:07,690 --> 00:19:10,100 of the subproblems I've already solved. 313 00:19:10,100 --> 00:19:16,972 So, actually, the recursion itself is only O of n. 314 00:19:16,972 --> 00:19:18,555 AUDIENCE: I missed when you said, what 315 00:19:18,555 --> 00:19:22,080 is the sorting referring to? 316 00:19:22,080 --> 00:19:27,960 LING REN: OK, so we need to start with the request that 317 00:19:27,960 --> 00:19:30,180 starts first, right? 318 00:19:30,180 --> 00:19:33,710 We need to decide whether we schedule it or not. 319 00:19:33,710 --> 00:19:37,400 And, then, we need to do the same thing for this request 2. 320 00:19:37,400 --> 00:19:40,880 It's the start earliest request in this subset. 321 00:19:40,880 --> 00:19:44,210 We always need to do that for all the subproblems. 322 00:19:47,890 --> 00:19:52,710 So the overall complexity is n log n. 323 00:19:56,510 --> 00:19:59,540 But if we only focus on this recursion step, 324 00:19:59,540 --> 00:20:01,770 our improvement is actually larger than that, 325 00:20:01,770 --> 00:20:10,490 because it went from n square to O of n. 326 00:20:10,490 --> 00:20:14,160 So why is the original algorithm n squared? 327 00:20:14,160 --> 00:20:17,920 I think it also has only n unique subproblems, right? 328 00:20:22,340 --> 00:20:24,460 So do you agree that the original algorithm is n 329 00:20:24,460 --> 00:20:27,430 squared, or do you think it's also O of n? 330 00:20:27,430 --> 00:20:28,810 Just focus on the recursion step. 331 00:20:33,720 --> 00:20:35,684 AUDIENCE: Didn't the original algorithm 332 00:20:35,684 --> 00:20:39,612 solve some subproblems up to n times? 333 00:20:39,612 --> 00:20:42,067 So that's why it's still of n squared, 334 00:20:42,067 --> 00:20:44,784 because it solved the same thing? 335 00:20:44,784 --> 00:20:46,200 LING REN: But assuming I do not do 336 00:20:46,200 --> 00:20:48,250 that, assuming, whenever I solve it once, 337 00:20:48,250 --> 00:20:53,100 I store the result somewhere, and I directly get it. 338 00:20:53,100 --> 00:20:55,330 Assuming I do that, what's the complexity? 339 00:21:01,006 --> 00:21:02,900 Go ahead. 340 00:21:02,900 --> 00:21:03,726 AUDIENCE: O of n. 341 00:21:03,726 --> 00:21:06,960 LING REN: You think it's O of n? 342 00:21:06,960 --> 00:21:09,230 So anyone think it's n squared? 343 00:21:09,230 --> 00:21:10,980 Because I think Srini said it's n squared. 344 00:21:10,980 --> 00:21:12,456 Go ahead. 345 00:21:12,456 --> 00:21:14,424 AUDIENCE: I think the original algorithm isn't 346 00:21:14,424 --> 00:21:16,228 [INAUDIBLE] squared, because we're still 347 00:21:16,228 --> 00:21:19,360 doing [INAUDIBLE] for every subproblem [INAUDIBLE]. 348 00:21:19,360 --> 00:21:21,620 LING REN: Exactly right. 349 00:21:21,620 --> 00:21:24,130 So, here, whenever we go up one step, 350 00:21:24,130 --> 00:21:26,420 I'm doing constant number of work. 351 00:21:26,420 --> 00:21:29,940 Just comparing two numbers and taking the max. 352 00:21:29,940 --> 00:21:31,970 However, in the original algorithm, 353 00:21:31,970 --> 00:21:36,510 which is here, whenever I want to go one step up, 354 00:21:36,510 --> 00:21:40,720 there are n branches of the tree. 355 00:21:40,720 --> 00:21:46,410 So my total amount of work is 1 plus 2 plus 3 plus n, right? 356 00:21:46,410 --> 00:21:48,940 Every step becomes harder when I go up. 357 00:21:48,940 --> 00:21:51,400 And this is n squared. 358 00:22:02,459 --> 00:22:03,500 Any questions about that? 359 00:22:10,192 --> 00:22:14,690 OK, this is so much for our weighted interval scheduling. 360 00:22:14,690 --> 00:22:16,830 Now, I'm going to transition into the next topic. 361 00:22:16,830 --> 00:22:21,180 So any question on the side, in general, 362 00:22:21,180 --> 00:22:22,395 for the scheduling problem? 363 00:22:26,021 --> 00:22:27,914 Do you have a question? 364 00:22:27,914 --> 00:22:29,210 No? 365 00:22:29,210 --> 00:22:37,490 OK, now, let's turn to a second topic of this section, which 366 00:22:37,490 --> 00:22:38,555 is Strassen algorithm. 367 00:22:42,270 --> 00:22:44,750 Strassen algorithm is an efficient algorithm 368 00:22:44,750 --> 00:22:48,970 for matrix multiplication, and matrix multiplication 369 00:22:48,970 --> 00:22:51,630 is a really useful primitive. 370 00:22:51,630 --> 00:22:56,690 It has applications in almost every area I can think of. 371 00:22:56,690 --> 00:23:00,410 Circuit simulation, climate simulation, 372 00:23:00,410 --> 00:23:03,760 and physics, basically everything. 373 00:23:03,760 --> 00:23:06,830 Now, I actually had some experience 374 00:23:06,830 --> 00:23:10,250 with matrix multiplication, because my undergrad research 375 00:23:10,250 --> 00:23:13,090 was improving matrix algorithms. 376 00:23:13,090 --> 00:23:15,040 And, actually, many matrix algorithms, 377 00:23:15,040 --> 00:23:17,840 including inversion, solving equations, 378 00:23:17,840 --> 00:23:21,090 they all use multiplication as a primitive. 379 00:23:21,090 --> 00:23:23,530 So it actually comes down to improving 380 00:23:23,530 --> 00:23:25,400 matrix multiplication. 381 00:23:25,400 --> 00:23:31,000 And I tried very hard to just optimize this basic matrix 382 00:23:31,000 --> 00:23:32,170 multiplication. 383 00:23:32,170 --> 00:23:34,360 We'll take a row, you take a column, 384 00:23:34,360 --> 00:23:39,200 and then you get your answer for this spot. 385 00:23:39,200 --> 00:23:41,690 Everyone's familiar with that, right? 386 00:23:41,690 --> 00:23:43,370 I tried very hard, but it's still 387 00:23:43,370 --> 00:23:46,980 100 x slower than the best algorithm out there. 388 00:23:46,980 --> 00:23:50,990 So I finally look it up, and I was completely mind-blown 389 00:23:50,990 --> 00:23:55,860 when I know that matrix algorithm complexity is not 390 00:23:55,860 --> 00:23:57,850 n cubic. 391 00:23:57,850 --> 00:23:59,360 It's actually smaller than that. 392 00:24:03,790 --> 00:24:06,440 Is this a surprise to you? 393 00:24:06,440 --> 00:24:07,540 Anyone expect that before? 394 00:24:12,210 --> 00:24:15,790 And the technique the more efficient algorithm uses 395 00:24:15,790 --> 00:24:18,180 is, exactly, Strassen algorithm. 396 00:24:18,180 --> 00:24:20,180 Now that we're talking about divide and conquer, 397 00:24:20,180 --> 00:24:23,770 you can guess it must be a divide and conquer algorithm. 398 00:24:23,770 --> 00:24:26,540 And, so, does anyone have an idea 399 00:24:26,540 --> 00:24:28,230 how to divide the original problem? 400 00:24:34,322 --> 00:24:35,530 Anyone want to give it a try? 401 00:24:41,490 --> 00:24:45,760 So are you familiar with tiled matrix multiplication, 402 00:24:45,760 --> 00:24:48,686 or blocked matrix multiplication? 403 00:24:48,686 --> 00:24:51,265 OK, can you tell us what that is? 404 00:24:51,265 --> 00:24:51,890 AUDIENCE: Yeah. 405 00:24:51,890 --> 00:24:54,830 You can break it into [INAUDIBLE]. 406 00:25:01,200 --> 00:25:05,890 LING REN: OK, so say this is our A and B, 407 00:25:05,890 --> 00:25:10,330 and we want C. We can break each matrix into four parts. 408 00:25:10,330 --> 00:25:16,710 I'll call this A11, A12, A21, A22. 409 00:25:16,710 --> 00:25:20,820 B11, B12, B21, B22. 410 00:25:24,750 --> 00:25:29,390 Same thing for C. 411 00:25:29,390 --> 00:25:36,880 Now, I would like someone to tell me, what is C11, 412 00:25:36,880 --> 00:25:37,485 in this case? 413 00:25:41,755 --> 00:25:42,255 Yes. 414 00:25:42,255 --> 00:25:43,338 AUDIENCE: A11 [INAUDIBLE]. 415 00:25:49,590 --> 00:25:52,300 LING REN: A12, B21, yep. 416 00:25:52,300 --> 00:26:01,150 And C12 is-- Just speak up, don't be shy. 417 00:26:01,150 --> 00:26:04,650 This is also my first section ever, in my life, 418 00:26:04,650 --> 00:26:05,619 teaching a recitation. 419 00:26:05,619 --> 00:26:06,910 I'm more nervous than you guys. 420 00:26:09,681 --> 00:26:10,180 What is C12? 421 00:26:16,288 --> 00:26:19,620 AUDIENCE: A11, B12? 422 00:26:19,620 --> 00:26:23,180 LING REN: A11, B12. 423 00:26:25,800 --> 00:26:28,152 A12, B22, right? 424 00:26:28,152 --> 00:26:31,000 So the rule is the same as before. 425 00:26:31,000 --> 00:26:32,040 Matrix multiplication. 426 00:26:32,040 --> 00:26:34,400 Compute this, we take this row, this column, 427 00:26:34,400 --> 00:26:35,655 and it gives us this. 428 00:26:35,655 --> 00:26:37,280 AUDIENCE: So the first one [INAUDIBLE]. 429 00:26:41,040 --> 00:26:42,400 LING REN: Right, thank you. 430 00:26:47,060 --> 00:26:47,680 A12, B21. 431 00:26:50,370 --> 00:26:51,680 OK. 432 00:26:51,680 --> 00:26:53,530 And, same thing, C21, we're going 433 00:26:53,530 --> 00:26:56,655 to take this row and this column. 434 00:26:56,655 --> 00:27:07,840 A21 B11 plus A22 B21. 435 00:27:07,840 --> 00:27:18,151 C11 is A21 B12 plus A22 B22. 436 00:27:18,151 --> 00:27:18,650 OK. 437 00:27:24,260 --> 00:27:26,562 And everyone understands this? 438 00:27:26,562 --> 00:27:28,090 OK, great. 439 00:27:28,090 --> 00:27:30,490 So, now, we've broken up the original problem 440 00:27:30,490 --> 00:27:32,600 into several subproblems. 441 00:27:32,600 --> 00:27:36,600 We only need to do matrix multiplication here 442 00:27:36,600 --> 00:27:37,280 eight times. 443 00:27:39,900 --> 00:27:44,520 And each of this matrix is half in size. 444 00:27:44,520 --> 00:27:47,120 If the original algorithm is n cubic, 445 00:27:47,120 --> 00:27:50,530 now each sub problem is half n cubic. 446 00:27:50,530 --> 00:27:55,860 Then we have eight of them, so the complexity is still 447 00:27:55,860 --> 00:27:57,170 n cubic. 448 00:27:57,170 --> 00:27:58,640 No improvement at all. 449 00:28:01,860 --> 00:28:05,880 So, actually, to be more precise, 450 00:28:05,880 --> 00:28:10,640 we can further break up these matrices into smaller blocks. 451 00:28:10,640 --> 00:28:13,800 So, to be precise, its complexity 452 00:28:13,800 --> 00:28:16,430 should be given by a recursion. 453 00:28:20,740 --> 00:28:24,060 Eight subproblems, each one half the size, 454 00:28:24,060 --> 00:28:27,540 plus-- can anyone tell me? 455 00:28:27,540 --> 00:28:33,560 What's the merging complexity, once I get all this? 456 00:28:38,494 --> 00:28:38,994 [INAUDIBLE] 457 00:28:44,428 --> 00:28:45,310 Go ahead. 458 00:28:45,310 --> 00:28:48,232 AUDIENCE: Is it just constant because you're adding? 459 00:28:48,232 --> 00:28:49,690 LING REN: Yeah, because I'm adding. 460 00:28:49,690 --> 00:28:52,191 But is it constant? 461 00:28:52,191 --> 00:28:53,440 I'm adding these two matrices. 462 00:28:56,000 --> 00:28:58,852 AUDIENCE: [INAUDIBLE]. 463 00:28:58,852 --> 00:29:00,160 LING REN: Pardon? 464 00:29:00,160 --> 00:29:03,060 AUDIENCE: For each level it can base case? 465 00:29:03,060 --> 00:29:05,560 LING REN: Yeah, for base case, it's, of course, constant. 466 00:29:05,560 --> 00:29:07,180 OK, so maybe I'm not clear about this. 467 00:29:07,180 --> 00:29:09,110 What's the subproblem, in this case? 468 00:29:09,110 --> 00:29:13,100 It's this multiplication operation, OK? 469 00:29:13,100 --> 00:29:17,270 And, so, these are the eight subproblems I will solve. 470 00:29:17,270 --> 00:29:20,590 After solving them, I need to add them together. 471 00:29:20,590 --> 00:29:27,520 And adding two matrices, and each is size ha;f of n. 472 00:29:27,520 --> 00:29:28,580 The complexity is-- 473 00:29:28,580 --> 00:29:30,670 AUDIENCE: n squared 2 over 2? 474 00:29:30,670 --> 00:29:34,300 LING REN: Yeah, it's n squared. 475 00:29:34,300 --> 00:29:35,770 Basically, n squared. 476 00:29:35,770 --> 00:29:40,790 So, to get a precise complexity, we should solve this recursion, 477 00:29:40,790 --> 00:29:43,530 but it will end up being the same thing as this intuition, 478 00:29:43,530 --> 00:29:46,090 n cubic. 479 00:29:46,090 --> 00:29:48,470 OK, so now, this is the magic. 480 00:29:48,470 --> 00:29:52,800 So Strassen, in 1969, came up with this algorithm. 481 00:29:55,104 --> 00:29:56,020 AUDIENCE: [INAUDIBLE]. 482 00:30:00,820 --> 00:30:05,768 LING REN: Each of these is a half n by half n matrix. 483 00:30:05,768 --> 00:30:06,684 AUDIENCE: [INAUDIBLE]. 484 00:30:10,380 --> 00:30:13,270 LING REN: So Strassen came up with this algorithm. 485 00:30:13,270 --> 00:30:20,620 He somehow defined M1 through M7, seven matrices, 486 00:30:20,620 --> 00:30:22,820 in this way. 487 00:30:22,820 --> 00:30:25,280 I can't provide any intuition, because I 488 00:30:25,280 --> 00:30:26,570 didn't come up with this. 489 00:30:26,570 --> 00:30:30,580 And, somehow, with those seven matrices, 490 00:30:30,580 --> 00:30:35,110 he can reconstruct, he can compute all the four 491 00:30:35,110 --> 00:30:38,680 submatrices in C. 492 00:30:38,680 --> 00:30:40,770 And it's not very just interesting to check it, 493 00:30:40,770 --> 00:30:43,470 because the algorithm is definitely correct. 494 00:30:43,470 --> 00:30:45,150 But let's just do one of them. 495 00:30:45,150 --> 00:30:46,740 OK, how about this one. 496 00:30:46,740 --> 00:30:49,270 So C21 is M2 plus M4. 497 00:30:49,270 --> 00:30:52,190 So M2 plus M4. 498 00:30:52,190 --> 00:30:58,830 So M2 will have A12 B11, A22 B11. 499 00:30:58,830 --> 00:31:03,920 So M4, there's A22 minus B11, so that cancels out. 500 00:31:03,920 --> 00:31:09,000 So we're left with A21 B11 plus A22 B21. 501 00:31:12,180 --> 00:31:15,370 That's the correct answer. 502 00:31:15,370 --> 00:31:18,400 So this, I guess, is a very clever algorithm. 503 00:31:18,400 --> 00:31:20,670 You have to work in that area for 10 years 504 00:31:20,670 --> 00:31:23,220 to come up with this so that's not our concern. 505 00:31:23,220 --> 00:31:27,620 Our goal is to analyze this algorithm. 506 00:31:27,620 --> 00:31:29,050 What's the complexity of it? 507 00:31:33,760 --> 00:31:37,200 So does anyone understand this recursion? 508 00:31:37,200 --> 00:31:41,400 Can someone tell me, what's the recursion for this part, 509 00:31:41,400 --> 00:31:42,610 for this Strassen algorithm? 510 00:31:47,860 --> 00:31:57,280 We have the original problem, and we have some-- go ahead. 511 00:31:57,280 --> 00:32:00,703 AUDIENCE: So since each of the M 1 through 7 512 00:32:00,703 --> 00:32:03,148 only require one multiplication, you'll 513 00:32:03,148 --> 00:32:09,749 need to solve seven subproblems, so 7T n over 2 plus O n 514 00:32:09,749 --> 00:32:11,630 squared. 515 00:32:11,630 --> 00:32:13,130 LING REN: That's absolutely correct. 516 00:32:16,220 --> 00:32:18,040 Everyone gets this? 517 00:32:18,040 --> 00:32:22,730 So what Strassen did, is he came up with the seven matrices. 518 00:32:22,730 --> 00:32:27,600 Each one requires only one multiplication. 519 00:32:27,600 --> 00:32:31,340 So we have seven subproblems, instead of eight, 520 00:32:31,340 --> 00:32:34,810 and that's going to give us a benefit, an improvement. 521 00:32:38,290 --> 00:32:41,370 So the question now becomes, how do I solve this recursion? 522 00:32:41,370 --> 00:32:44,000 Given this recursion, how do I know its complexity? 523 00:32:44,000 --> 00:32:45,150 And same question there. 524 00:32:49,160 --> 00:32:50,368 Anyone want to give it a try? 525 00:32:54,200 --> 00:33:14,070 So that's going to be covered in a third topic, which 526 00:33:14,070 --> 00:33:15,307 is Master Theorem. 527 00:33:21,770 --> 00:33:25,840 So Master Theorem does exactly that. 528 00:33:25,840 --> 00:33:38,930 All it does is, given the recursion, a and T of n 529 00:33:38,930 --> 00:33:47,130 over b, plus some work for merging, 530 00:33:47,130 --> 00:33:51,700 where a and b are constants, it directly tell you what T of n 531 00:33:51,700 --> 00:33:55,810 is, in some cases. 532 00:33:55,810 --> 00:33:59,640 So I'll first write the formula. 533 00:33:59,640 --> 00:34:01,990 Master Theorem actually has three cases. 534 00:34:01,990 --> 00:34:07,440 The first case, fn, is order n raised 535 00:34:07,440 --> 00:34:11,354 to c, where c is less than log b of a. 536 00:34:11,354 --> 00:34:21,690 Then, Master Theorem says, it's complexity is theta log b of a. 537 00:34:25,570 --> 00:34:37,335 Second case, fn is theta nc log K, where c is, 538 00:34:37,335 --> 00:34:45,240 log b is equal to b of a, then it's complexity is n 539 00:34:45,240 --> 00:34:50,024 raised to c log K plus 1 n. 540 00:34:50,024 --> 00:34:51,690 You don't necessarily have to copy them, 541 00:34:51,690 --> 00:34:55,520 because you just find them anywhere. 542 00:34:55,520 --> 00:34:58,160 And the third case, you can imagine, 543 00:34:58,160 --> 00:35:03,205 it's the only remaining case, which is, fn is large, 544 00:35:03,205 --> 00:35:10,085 it's omega n raised to c, where c is greater than log b of a. 545 00:35:10,085 --> 00:35:13,845 Then, Master Theorem says, its complexity, the complexity 546 00:35:13,845 --> 00:35:19,630 of Tn, is theta fn. 547 00:35:19,630 --> 00:35:23,810 So, intuitively, if fn is not too much work, 548 00:35:23,810 --> 00:35:26,170 then it's basically this recursion, 549 00:35:26,170 --> 00:35:27,800 what recursion gives you. 550 00:35:27,800 --> 00:35:33,330 If fn dominates, fn is the biggest component, 551 00:35:33,330 --> 00:35:35,746 then Tn is, roughly, on the order of fn, 552 00:35:35,746 --> 00:35:37,120 and there's a case in the middle. 553 00:35:45,600 --> 00:35:48,000 Now, let's see why that is the case. 554 00:35:48,000 --> 00:35:49,600 I'll only cover one case, here. 555 00:35:59,200 --> 00:36:01,200 So, again, we are going to draw a recursion 556 00:36:01,200 --> 00:36:04,300 tree, because that is very useful in all the recursion 557 00:36:04,300 --> 00:36:06,390 problems. 558 00:36:06,390 --> 00:36:10,556 So we start with a problem of size n, 559 00:36:10,556 --> 00:36:14,080 and we break them into problem size n over b. 560 00:36:22,560 --> 00:36:23,802 So on and so forth. 561 00:36:23,802 --> 00:36:25,260 What's the size of this subproblem? 562 00:36:31,770 --> 00:36:36,500 So that recursion represents a class of recursive algorithm. 563 00:36:36,500 --> 00:36:39,236 Every time it breaks the problem, 564 00:36:39,236 --> 00:36:40,860 it reduced the problem size by a factor 565 00:36:40,860 --> 00:36:43,015 of b, so what do I have here? 566 00:36:45,630 --> 00:36:46,404 Go ahead. 567 00:36:46,404 --> 00:36:48,180 AUDIENCE: [INAUDIBLE] 568 00:36:48,180 --> 00:36:52,510 LING REN: n over b squared. 569 00:36:52,510 --> 00:36:53,760 So on and so forth. 570 00:36:53,760 --> 00:36:59,640 So what is a, in this graph? 571 00:36:59,640 --> 00:37:01,950 AUDIENCE: 3. 572 00:37:01,950 --> 00:37:03,650 LING REN: 3, think so? 573 00:37:03,650 --> 00:37:04,322 3. 574 00:37:04,322 --> 00:37:08,490 a is just a branching factor of this tree. 575 00:37:08,490 --> 00:37:09,980 I keep going. 576 00:37:09,980 --> 00:37:14,330 Finally, I will reach my base case. 577 00:37:14,330 --> 00:37:19,060 So my next question is, after how many levels of recursion 578 00:37:19,060 --> 00:37:22,340 will I reach a base case of size 1? 579 00:37:31,498 --> 00:37:32,462 Go ahead. 580 00:37:32,462 --> 00:37:34,872 AUDIENCE: Log b of [INAUDIBLE]? 581 00:37:34,872 --> 00:37:45,370 LING REN: Log b of-- OK, so, here 582 00:37:45,370 --> 00:37:49,780 is n over b, n over b square, next one n over b cubic, 583 00:37:49,780 --> 00:37:50,630 so on and so forth. 584 00:37:50,630 --> 00:37:54,370 So, say, this last level it Tth level, 585 00:37:54,370 --> 00:38:01,930 than the problems size is n over-- we raise to T, 586 00:38:01,930 --> 00:38:08,510 and we want that to be a constant, so what is T? 587 00:38:13,440 --> 00:38:15,420 AUDIENCE: Log b on n. 588 00:38:15,420 --> 00:38:20,276 LING REN: Log b of n. 589 00:38:20,276 --> 00:38:24,320 Now, this is the recursion tree, and we have that fn 590 00:38:24,320 --> 00:38:26,350 among the emerging work to do. 591 00:38:26,350 --> 00:38:32,300 So, here, we have to do fn work, to merge these a results 592 00:38:32,300 --> 00:38:36,940 to the solution of our problem, n. 593 00:38:36,940 --> 00:38:42,040 Then, we have f- what's the emerging work for this level, 594 00:38:42,040 --> 00:38:44,190 for this part of the tree? 595 00:38:47,160 --> 00:38:48,750 This is my problem size. 596 00:38:51,708 --> 00:38:54,670 AUDIENCE: [INAUDIBLE] 597 00:38:54,670 --> 00:38:55,890 LING REN: n over b, right. 598 00:38:55,890 --> 00:38:57,650 And we have a of them. 599 00:38:57,650 --> 00:39:01,590 OK, so on and so forth. 600 00:39:01,590 --> 00:39:05,920 So, then, we know what is Tn. 601 00:39:05,920 --> 00:39:09,260 Let's just enumerate all the work we have to do. 602 00:39:09,260 --> 00:39:13,470 So on the first level, we have to do fn. 603 00:39:13,470 --> 00:39:20,460 OK, on second level, af n of b-- sorry, n over b. 604 00:39:20,460 --> 00:39:21,570 And what's the next level? 605 00:39:25,802 --> 00:39:27,010 We have how many subproblems? 606 00:39:30,080 --> 00:39:31,007 Speak louder. 607 00:39:31,007 --> 00:39:32,200 AUDIENCE: a squared. 608 00:39:32,200 --> 00:39:35,210 LING REN: a square subproblems, and each of them 609 00:39:35,210 --> 00:39:43,570 is n over b squared. 610 00:39:43,570 --> 00:39:49,172 And, finally, I reach my last level. 611 00:39:49,172 --> 00:39:54,300 They are all base cases, so I have a raised to T of them, 612 00:39:54,300 --> 00:39:57,370 because I defined T to be my depth of the tree. 613 00:39:57,370 --> 00:39:59,690 And each of them is T of 1. 614 00:40:02,630 --> 00:40:05,770 OK, so that's Tn. 615 00:40:05,770 --> 00:40:08,590 I'm not entirely happy with this formula, 616 00:40:08,590 --> 00:40:11,850 because I have this beautiful pattern, here, 617 00:40:11,850 --> 00:40:14,540 except for that last guy. 618 00:40:14,540 --> 00:40:19,910 It's add one a and divide one b, blah, blah, blah. 619 00:40:19,910 --> 00:40:24,580 So I'm going to change this T into f. 620 00:40:24,580 --> 00:40:26,570 Can I do that? 621 00:40:26,570 --> 00:40:28,610 Because it's [INAUDIBLE] is the same, right? 622 00:40:28,610 --> 00:40:31,230 T1 is a constant, f1 is also constant. 623 00:40:31,230 --> 00:40:41,140 Then, I get my beautiful form, where it's a sum from i 624 00:40:41,140 --> 00:40:44,880 equals 0 to T. What's in the sum? 625 00:40:50,226 --> 00:40:51,254 Go ahead. 626 00:40:51,254 --> 00:40:52,170 AUDIENCE: [INAUDIBLE]. 627 00:40:55,086 --> 00:40:59,166 LING REN: a raised to i, f of n over b1. 628 00:41:02,022 --> 00:41:03,190 Everyone gets that? 629 00:41:05,970 --> 00:41:10,080 Now, you can roughly see why we have three cases. 630 00:41:10,080 --> 00:41:12,750 So let me deal with the first case. 631 00:41:12,750 --> 00:41:21,177 The first case says, fn is order n raised to c. 632 00:41:21,177 --> 00:41:22,010 What does that mean? 633 00:41:25,200 --> 00:41:30,940 It means this guy here is sigma a raised to n, 634 00:41:30,940 --> 00:41:37,120 then this is what's in my f raised to c. 635 00:41:37,120 --> 00:41:40,250 There should be order, here, but everything has order before it, 636 00:41:40,250 --> 00:41:42,120 so I just omit that. 637 00:41:42,120 --> 00:41:46,310 So it actually should be this, OK? 638 00:41:46,310 --> 00:41:50,890 So this, because n raised to c is actually 639 00:41:50,890 --> 00:41:53,380 independent of the sum, I can pull it out. 640 00:41:58,300 --> 00:42:00,640 And what am I left with? 641 00:42:07,940 --> 00:42:11,000 Is that correct? 642 00:42:11,000 --> 00:42:15,190 Now, this is a sum of geometric sequence. 643 00:42:15,190 --> 00:42:17,480 We know how to solve that, but we 644 00:42:17,480 --> 00:42:21,210 need to check whether this ratio is greater or larger than 1, 645 00:42:21,210 --> 00:42:22,760 or if it's equal to 1. 646 00:42:22,760 --> 00:42:25,260 And what is this ratio? 647 00:42:25,260 --> 00:42:26,500 The case tells us. 648 00:42:26,500 --> 00:42:30,160 So c is less than log b of a. 649 00:42:30,160 --> 00:42:34,240 That means b raised to c is less than b 650 00:42:34,240 --> 00:42:41,830 raised to this guy, which is a, right? 651 00:42:41,830 --> 00:42:48,150 So we know our denominator is smaller than or numerator. 652 00:42:48,150 --> 00:42:52,470 So this is an increasing sequence, right? 653 00:42:52,470 --> 00:42:55,770 So what we have is n raised to c, 654 00:42:55,770 --> 00:43:01,530 then that thing raised to t minus 1 655 00:43:01,530 --> 00:43:04,030 divided by this thing minus 1. 656 00:43:04,030 --> 00:43:06,151 But they are all constants. 657 00:43:06,151 --> 00:43:09,250 Are everyone familiar with this formula. 658 00:43:09,250 --> 00:43:12,350 of geometric sequence? 659 00:43:12,350 --> 00:43:14,140 OK, so that's what we have. 660 00:43:18,830 --> 00:43:23,480 Next, we have t equals log b of n. 661 00:43:23,480 --> 00:43:33,120 That means b raised to t is n, correct? 662 00:43:33,120 --> 00:43:37,770 So, then, b raised to t is n, then we have raised to c, 663 00:43:37,770 --> 00:43:39,610 they cancel out. 664 00:43:39,610 --> 00:43:41,980 What do we have? 665 00:43:41,980 --> 00:43:44,020 I want to make sure everyone's following. 666 00:43:52,146 --> 00:43:53,590 AUDIENCE: is it a over t? 667 00:43:53,590 --> 00:43:55,150 LING REN: A raised to t. 668 00:43:59,730 --> 00:44:00,650 No questions? 669 00:44:00,650 --> 00:44:02,570 AUDIENCE: Can you do that stuff again? 670 00:44:02,570 --> 00:44:05,990 LING REN: OK, let me do that again. 671 00:44:05,990 --> 00:44:09,340 It's actually a raised to t, and then 672 00:44:09,340 --> 00:44:13,816 n over bt raised to c, right? 673 00:44:13,816 --> 00:44:15,940 AUDIENCE: How did you get from the line above that? 674 00:44:15,940 --> 00:44:17,190 LING REN: This one to here? 675 00:44:17,190 --> 00:44:17,815 AUDIENCE: Yeah. 676 00:44:17,815 --> 00:44:20,100 LING REN: Oh, it's the sum of geometric sequence, 677 00:44:20,100 --> 00:44:25,620 so if I have 1 plus q plus q squared, all the way to qt, 678 00:44:25,620 --> 00:44:29,870 it's qt plus 1, I guess-- or t, I 679 00:44:29,870 --> 00:44:35,530 don't remember very well-- minus 1, then q minus 1. 680 00:44:35,530 --> 00:44:38,620 I guess this should be t plus 1. 681 00:44:38,620 --> 00:44:40,050 So this is what we're doing. 682 00:44:40,050 --> 00:44:43,635 So this is our q minus 1, and divided by q minus 1, 683 00:44:43,635 --> 00:44:45,760 they are all constants, so I don't care about them. 684 00:44:48,820 --> 00:44:52,430 So a raised plus this thing raised to c, 685 00:44:52,430 --> 00:44:56,190 but we said that b raised to t is equal to n, 686 00:44:56,190 --> 00:45:01,710 so we're just left with a raised to t. 687 00:45:01,710 --> 00:45:06,700 And what is t? 688 00:45:06,700 --> 00:45:09,820 t is log b of n. 689 00:45:09,820 --> 00:45:17,820 I can write it as log a of n over log a of b. 690 00:45:17,820 --> 00:45:20,260 Everyone familiar with that? 691 00:45:20,260 --> 00:45:27,279 That means log a of n, log b of a. 692 00:45:27,279 --> 00:45:28,320 This is when I flip them. 693 00:45:31,660 --> 00:45:33,700 So this, what is that? 694 00:45:36,430 --> 00:45:37,547 AUDIENCE: n. 695 00:45:37,547 --> 00:45:38,380 LING REN: That is n. 696 00:45:42,930 --> 00:45:44,280 OK, we're done. 697 00:45:47,190 --> 00:45:48,160 Are we? 698 00:45:58,840 --> 00:46:01,820 Not exactly, because I have an order here, right? 699 00:46:01,820 --> 00:46:04,260 So everything is ordered. 700 00:46:04,260 --> 00:46:07,620 If you only care about order, big O, then it's fine. 701 00:46:07,620 --> 00:46:09,840 But that theorem says theta, so you 702 00:46:09,840 --> 00:46:13,119 have to prove it the other way, that it's no less than that. 703 00:46:13,119 --> 00:46:14,160 I'm not going to do that. 704 00:46:14,160 --> 00:46:16,970 It's not very hard. 705 00:46:16,970 --> 00:46:20,970 Next, I'm going to apply this theorem to the two 706 00:46:20,970 --> 00:46:23,780 problems we left here. 707 00:46:23,780 --> 00:46:29,570 So let's apply Master Theorem to this recurrence. 708 00:46:29,570 --> 00:46:31,623 I think you are still looking at that side. 709 00:46:35,970 --> 00:46:39,260 So what is the a, b, c for this? 710 00:46:44,180 --> 00:46:47,700 a is 8, b is 2, right? 711 00:46:47,700 --> 00:46:53,150 And c is 2. 712 00:46:53,150 --> 00:46:59,945 Log b of a is 3, so that's which case of the Master Theorem? 713 00:47:03,910 --> 00:47:06,410 So the theorem says it should be n raised 714 00:47:06,410 --> 00:47:10,510 to log b of a, which is 3, OK? 715 00:47:13,550 --> 00:47:20,530 Now, what we have here, can I remind someone 716 00:47:20,530 --> 00:47:21,353 to do that for me? 717 00:47:27,029 --> 00:47:28,070 Want to give it a try? 718 00:47:34,860 --> 00:47:37,300 Go ahead. 719 00:47:37,300 --> 00:47:45,660 AUDIENCE: OK, so we have a equals 7, b equals 2, 720 00:47:45,660 --> 00:47:49,300 and c equals 2, like before. 721 00:47:49,300 --> 00:47:57,282 And, now, we want to see whether c equals 2 is less than log b 722 00:47:57,282 --> 00:48:00,865 of a, which is log 2 of 7. 723 00:48:06,396 --> 00:48:08,140 Pretty sure that's still the case. 724 00:48:08,140 --> 00:48:13,660 So we should just get n to the log 2 of 7. 725 00:48:13,660 --> 00:48:15,180 LING REN: Yeah, exactly. 726 00:48:15,180 --> 00:48:16,384 So, yeah. 727 00:48:16,384 --> 00:48:18,550 Yeah, thank you. let's give him a round of applause. 728 00:48:22,640 --> 00:48:26,480 So log 2 of 7 is definitely greater than 2. 729 00:48:26,480 --> 00:48:27,070 Why? 730 00:48:27,070 --> 00:48:29,880 Because log 2 of 4 is 2, right? 731 00:48:29,880 --> 00:48:38,030 So this happens to be n raised to 2.80, and many other digits, 732 00:48:38,030 --> 00:48:42,450 but it's less than n cubic. 733 00:48:42,450 --> 00:48:48,200 And, just for knowledge purpose, this is no longer the best. 734 00:48:48,200 --> 00:48:50,300 It was the best, when it was proposed, 735 00:48:50,300 --> 00:48:52,970 and, well, researchers in that area 736 00:48:52,970 --> 00:48:56,590 have got it down to n raised to 2.35. 737 00:48:56,590 --> 00:48:59,654 I think 2.37, first, then 2.35. 738 00:48:59,654 --> 00:49:01,570 I'm not following the literature very closely, 739 00:49:01,570 --> 00:49:06,010 so maybe it's 2.34 now. 740 00:49:06,010 --> 00:49:08,575 So I should have one more thing to cover, 741 00:49:08,575 --> 00:49:10,200 but I think we are running out of time. 742 00:49:10,200 --> 00:49:11,340 Sorry about that. 743 00:49:11,340 --> 00:49:12,490 I can post it. 744 00:49:12,490 --> 00:49:15,490 So the last thing we should do is, 745 00:49:15,490 --> 00:49:17,490 remember, we have this medium finding algorithm, 746 00:49:17,490 --> 00:49:22,260 where we have a recurrence, which is, I think, 747 00:49:22,260 --> 00:49:30,470 10 over 5, some ceiling, and then plus this [INAUDIBLE] 748 00:49:30,470 --> 00:49:33,760 n plus theta n. 749 00:49:33,760 --> 00:49:38,780 And we want to solve this recursion, 750 00:49:38,780 --> 00:49:41,180 but we cannot apply Master Theorem. 751 00:49:41,180 --> 00:49:43,430 Apparently, it's not the right form. 752 00:49:43,430 --> 00:49:45,340 So when Master Theorem doesn't apply, 753 00:49:45,340 --> 00:49:47,389 we have to study it case by case. 754 00:49:47,389 --> 00:49:48,930 Let me see if I have time to do that. 755 00:49:54,970 --> 00:49:56,430 OK, I think I probably do. 756 00:50:01,640 --> 00:50:04,180 To solve that case, first, can someone 757 00:50:04,180 --> 00:50:06,180 tell me, what's the definition of theta? 758 00:50:06,180 --> 00:50:08,620 We have to go back to the definition to solve that. 759 00:50:11,180 --> 00:50:12,720 What does theta even mean? 760 00:50:12,720 --> 00:50:18,140 So if I say fn is theta n, what do I really mean? 761 00:50:21,764 --> 00:50:22,490 Go ahead. 762 00:50:22,490 --> 00:50:26,442 AUDIENCE: It's tightly bounded, so you can move the [INAUDIBLE] 763 00:50:26,442 --> 00:50:27,930 to either side. 764 00:50:27,930 --> 00:50:33,470 LING REN: So it means I can find some K1 and K2 such 765 00:50:33,470 --> 00:50:38,680 that this holds when n gets sufficiently large. 766 00:50:38,680 --> 00:50:41,030 OK, so, now, we're going to do induction. 767 00:50:49,490 --> 00:50:55,530 Assuming, for all the small n less than capital N, 768 00:50:55,530 --> 00:51:00,360 my Tn is bounded by this K2 and K1 thing. 769 00:51:03,360 --> 00:51:10,660 Then, my next step is, T of this capital N would be bounded. 770 00:51:10,660 --> 00:51:12,885 I'll do the right side first. 771 00:51:12,885 --> 00:51:28,220 It will be bounded by K2 n5 ceiling, plus K2. 772 00:51:28,220 --> 00:51:33,740 [INAUDIBLE] in a second term, n plus another theta n. 773 00:51:33,740 --> 00:51:36,860 So, we know, that means it's bounded by some other number. 774 00:51:36,860 --> 00:51:39,110 I'll say A2, then. 775 00:51:39,110 --> 00:51:41,910 That's the definition of theta n. 776 00:51:41,910 --> 00:51:46,950 Then, I want this to be-- sorry, all of them 777 00:51:46,950 --> 00:51:51,880 should be capital N. Capital N, Capital N. I want this to be 778 00:51:51,880 --> 00:52:01,140 smaller than K2 capital N. 779 00:52:01,140 --> 00:52:03,120 So let me redo this first step. 780 00:52:03,120 --> 00:52:15,930 This is roughly 5 of K2 plus 7 over 10 K2, plus A2 of n, 781 00:52:15,930 --> 00:52:18,910 plus a bunch of constants that I don't care. 782 00:52:18,910 --> 00:52:24,321 I want it to be smaller than K2 of n. 783 00:52:24,321 --> 00:52:25,640 Can I reach that? 784 00:52:25,640 --> 00:52:27,050 Of course I can, right? 785 00:52:27,050 --> 00:52:32,140 If I select a K2 to be greater than all we have here, 786 00:52:32,140 --> 00:52:33,030 what is this? 787 00:52:33,030 --> 00:52:40,930 This is 9 over 10 K2 plus A2, right? 788 00:52:40,930 --> 00:52:47,800 So if I select the K2 to be greater than 10 times 789 00:52:47,800 --> 00:52:53,160 A2-- is everyone following that? 790 00:52:53,160 --> 00:52:59,020 When n is sufficiently large, Tn should be bounded by A2 n. 791 00:53:01,700 --> 00:53:04,890 That's the induction. 792 00:53:04,890 --> 00:53:08,110 I am assuming, when n is smaller than capital N, 793 00:53:08,110 --> 00:53:09,030 I have solved them. 794 00:53:09,030 --> 00:53:13,030 So I can use these two, and I solve the next step. 795 00:53:13,030 --> 00:53:15,400 So there's the other side, which is very similar. 796 00:53:15,400 --> 00:53:18,640 I'm not going to go through that. 797 00:53:18,640 --> 00:53:22,760 All right, so that's all for today. 798 00:53:22,760 --> 00:53:25,320 And just to quickly recap, we went 799 00:53:25,320 --> 00:53:30,450 through the weighted interval scheduling, 800 00:53:30,450 --> 00:53:33,240 and the Strassen algorithm, Master Theorem and applying 801 00:53:33,240 --> 00:53:39,350 Master Theorem, and that case study of a new recursion. 802 00:53:39,350 --> 00:53:42,210 OK, thanks, everyone, for coming.