1 00:00:00,070 --> 00:00:01,770 The following content is provided 2 00:00:01,770 --> 00:00:04,000 under a Creative Commons license. 3 00:00:04,000 --> 00:00:06,860 Your support will help MIT OpenCourseWare continue 4 00:00:06,860 --> 00:00:10,720 to offer high quality educational resources for free. 5 00:00:10,720 --> 00:00:13,340 To make a donation or view additional materials 6 00:00:13,340 --> 00:00:17,207 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:17,207 --> 00:00:17,832 at ocw.mit.edu. 8 00:00:21,310 --> 00:00:23,400 VICTOR COSTAN: Hi, everyone. 9 00:00:23,400 --> 00:00:25,480 I am Victor. 10 00:00:25,480 --> 00:00:26,730 My full name is Victor Costan. 11 00:00:26,730 --> 00:00:28,610 I'm a third year Ph.D. student. 12 00:00:28,610 --> 00:00:33,150 I'm in CSAIL, and I work in systems. 13 00:00:33,150 --> 00:00:36,610 I do mostly security and distributed systems. 14 00:00:36,610 --> 00:00:39,675 And I'll be your algorithm TA for this term. 15 00:00:39,675 --> 00:00:40,290 Yay. 16 00:00:40,290 --> 00:00:42,040 How many people went to lecture yesterday? 17 00:00:45,340 --> 00:00:45,840 All right. 18 00:00:45,840 --> 00:00:48,620 Are there any parts of the lecture that were particularly 19 00:00:48,620 --> 00:00:51,610 unclear that you guys want to go over? 20 00:00:51,610 --> 00:00:56,410 So that I schedule what I talk about in such a way 21 00:00:56,410 --> 00:00:58,580 that I'll cover that more than the parts 22 00:00:58,580 --> 00:01:01,900 that are boring and obvious. 23 00:01:01,900 --> 00:01:05,874 AUDIENCE: Can you explain complexity in equations, 24 00:01:05,874 --> 00:01:08,210 because I took 042 like 3 years ago. 25 00:01:08,210 --> 00:01:08,960 VICTOR COSTAN: OK. 26 00:01:17,710 --> 00:01:19,410 What else? 27 00:01:19,410 --> 00:01:19,980 Thank you. 28 00:01:24,950 --> 00:01:26,938 AUDIENCE: Maybe some basic outlines for, like, 29 00:01:26,938 --> 00:01:29,873 writing proofs relevant to like the [INAUDIBLE]. 30 00:01:29,873 --> 00:01:30,914 VICTOR COSTAN: All right. 31 00:01:30,914 --> 00:01:32,902 AUDIENCE: Sort of methodology or something. 32 00:01:40,601 --> 00:01:41,351 VICTOR COSTAN: OK. 33 00:01:45,327 --> 00:01:45,910 Anything else? 34 00:01:51,622 --> 00:01:53,050 OK. 35 00:01:53,050 --> 00:01:56,460 We're going to get started then. 36 00:01:56,460 --> 00:02:00,990 So the hardest, and I think most important of the whole course, 37 00:02:00,990 --> 00:02:06,290 is learning how to deal with asymptotic complexity. 38 00:02:06,290 --> 00:02:09,370 So asymptotic complexity is this very nice tool 39 00:02:09,370 --> 00:02:12,020 where you have a function that looks 40 00:02:12,020 --> 00:02:18,142 like this plus-- what'll I have here? 41 00:02:18,142 --> 00:02:22,516 10 plus sin x. 42 00:02:32,240 --> 00:02:35,774 So you have this guy, which is quite long and horrible 43 00:02:35,774 --> 00:02:37,190 and would make your life miserable 44 00:02:37,190 --> 00:02:39,750 if you'd have to reason about it. 45 00:02:39,750 --> 00:02:49,760 And you can say that this is theta of n squared. 46 00:02:49,760 --> 00:02:50,360 x, sorry. 47 00:02:55,010 --> 00:02:58,320 So everything that's here goes away. 48 00:02:58,320 --> 00:03:00,270 And this constant factor here goes away, 49 00:03:00,270 --> 00:03:03,640 and it's nice and small function. 50 00:03:03,640 --> 00:03:06,040 Why is that the case, first of all? 51 00:03:06,040 --> 00:03:11,360 If I plot this function-- and since I don't have my projector 52 00:03:11,360 --> 00:03:13,430 you'll have to take my word for it-- 53 00:03:13,430 --> 00:03:16,670 I will get something that looks like this. 54 00:03:19,950 --> 00:03:22,500 If I plug this function, I will get 55 00:03:22,500 --> 00:03:25,560 something that looks like this. 56 00:03:25,560 --> 00:03:29,800 This is a scale from 0 to 200. 57 00:03:29,800 --> 00:03:38,260 However, if I make my scale 0 to 20,000-- 58 00:03:38,260 --> 00:03:42,250 Let's say 200,000 to make sure that is going to be correct. 59 00:03:42,250 --> 00:03:44,910 x squared looks like this. 60 00:03:44,910 --> 00:03:52,267 And this function looks like this. 61 00:03:52,267 --> 00:03:54,100 And as you make the scale bigger and bigger, 62 00:03:54,100 --> 00:03:57,290 you can't even see this noise anymore. 63 00:03:57,290 --> 00:04:01,000 The reason is that all the noise is in this term. 64 00:04:01,000 --> 00:04:04,020 In the x to the power of one fifth term. 65 00:04:04,020 --> 00:04:09,720 And as n-- pretend this is n, x, n same thing-- 66 00:04:09,720 --> 00:04:14,590 so as the input to the function gets bigger and bigger, 67 00:04:14,590 --> 00:04:17,589 this grows at a much faster rate than this. 68 00:04:17,589 --> 00:04:21,220 So if I don't care about this, how the function grows, 69 00:04:21,220 --> 00:04:25,870 I can ignore anything aside from the dominant term. 70 00:04:28,829 --> 00:04:30,370 Do people remember this from lecture? 71 00:04:30,370 --> 00:04:31,255 Does this make sense? 72 00:04:34,140 --> 00:04:34,640 OK. 73 00:04:34,640 --> 00:04:35,806 If it doesn't, that's great. 74 00:04:35,806 --> 00:04:39,230 It means that I'm not wasting time covering it. 75 00:04:39,230 --> 00:04:41,639 So we learned about three symbols, 76 00:04:41,639 --> 00:04:43,180 or we will learn about three symbols, 77 00:04:43,180 --> 00:04:46,620 when talking about asymptotic complexity. 78 00:04:46,620 --> 00:04:52,790 The three symbols are theta, O, and omega. 79 00:04:52,790 --> 00:04:58,630 Let's say this complicated function is g. 80 00:04:58,630 --> 00:05:00,840 And this nice simple function is f. 81 00:05:07,330 --> 00:05:15,180 If I say that g of x is theta of f of x, that means two things. 82 00:05:15,180 --> 00:05:20,140 It means I have a lower bound, like I showed here, 83 00:05:20,140 --> 00:05:23,950 and the lower bound looks something like x squared. 84 00:05:23,950 --> 00:05:27,370 And it means that I also have a higher-- an upper bound for it 85 00:05:27,370 --> 00:05:29,390 that also looks like x squared. 86 00:05:29,390 --> 00:05:31,390 So the upper bound would be something like this. 87 00:05:34,110 --> 00:05:37,660 And if I say this, then I'm making a promise 88 00:05:37,660 --> 00:05:39,790 that the function stays between the lower 89 00:05:39,790 --> 00:05:41,530 bound and the upper bound, and the bounds 90 00:05:41,530 --> 00:05:43,250 look sort of the same. 91 00:05:43,250 --> 00:05:46,270 I can't really have x squared here, and x squared here. 92 00:05:46,270 --> 00:05:47,380 Right? 93 00:05:47,380 --> 00:05:49,140 That would be impossible. 94 00:05:49,140 --> 00:05:51,050 So that's why I said they look the same. 95 00:05:51,050 --> 00:05:53,090 They're not the same. 96 00:05:53,090 --> 00:05:55,320 And they're not the same, because they're 97 00:05:55,320 --> 00:05:58,040 allowed to differ by a constant factor. 98 00:05:58,040 --> 00:06:04,730 So if this is f, and say this is f prime, f of x 99 00:06:04,730 --> 00:06:07,740 might be x squared. 100 00:06:07,740 --> 00:06:11,710 And we know for sure that this is going to be lower than this. 101 00:06:11,710 --> 00:06:20,800 And f prime of x might be 1.2 times x squared. 102 00:06:20,800 --> 00:06:22,860 And because I have a 1.1 here-- and I 103 00:06:22,860 --> 00:06:26,400 know that these get smaller and smaller in relation to this 104 00:06:26,400 --> 00:06:29,640 as the function-- as the inputs to the function get bigger-- 105 00:06:29,640 --> 00:06:32,620 I know that this is going to be the dominating factor. 106 00:06:32,620 --> 00:06:34,940 And because this is bigger than this, 107 00:06:34,940 --> 00:06:36,950 this function is going to eventually 108 00:06:36,950 --> 00:06:40,060 be larger than this function. 109 00:06:40,060 --> 00:06:43,330 So I have a bound from above, and I have a bound from below. 110 00:06:43,330 --> 00:06:45,370 And the bounds look almost the same. 111 00:06:45,370 --> 00:06:47,740 They're only different by a constant factor. 112 00:06:47,740 --> 00:06:50,670 This is theta. 113 00:06:50,670 --> 00:06:52,400 To make things spicier-- by the way, 114 00:06:52,400 --> 00:06:56,370 theta is really hard to pronounce as opposed to O, 115 00:06:56,370 --> 00:07:00,950 so we usually say and write O instead of theta. 116 00:07:00,950 --> 00:07:03,730 We'll talk about what O really means in a bit. 117 00:07:03,730 --> 00:07:06,740 But if you see O in a lot of places, including our notes, 118 00:07:06,740 --> 00:07:09,780 know that most of the time we actually mean theta. 119 00:07:09,780 --> 00:07:11,286 And sorry for that. 120 00:07:11,286 --> 00:07:13,827 AUDIENCE: When do we you actually mean O? 121 00:07:13,827 --> 00:07:15,910 VICTOR COSTAN: You can ask us, and we'll tell you. 122 00:07:15,910 --> 00:07:18,050 [LAUGHTER] 123 00:07:18,050 --> 00:07:20,250 Fortunately, it doesn't a matter too much. 124 00:07:20,250 --> 00:07:22,696 So we're trying to get an idea of how the functions look 125 00:07:22,696 --> 00:07:23,820 like in the bigger picture. 126 00:07:23,820 --> 00:07:25,690 So most of the time it doesn't matter. 127 00:07:25,690 --> 00:07:28,080 I'll give you an example though of where it matters, 128 00:07:28,080 --> 00:07:30,900 and how it could matter. 129 00:07:30,900 --> 00:07:38,215 So suppose I have another function that looks like this. 130 00:07:40,880 --> 00:07:42,850 So this is kind of ugly, too. 131 00:07:42,850 --> 00:07:56,000 This function is g of x, say x times 1 plus sin of x. 132 00:08:00,140 --> 00:08:02,730 So I can't say anything about the lower bound of this. 133 00:08:02,730 --> 00:08:07,750 This is going to hit 0 at an infinite number of values. 134 00:08:07,750 --> 00:08:08,250 Right? 135 00:08:08,250 --> 00:08:11,670 So I can't say that-- I can't have a nice lower bound for it. 136 00:08:11,670 --> 00:08:13,680 The lower bound for it is 0. 137 00:08:13,680 --> 00:08:15,780 But I'm pretty sure that it's never 138 00:08:15,780 --> 00:08:17,380 going to be bigger than 2x. 139 00:08:17,380 --> 00:08:17,880 Right? 140 00:08:17,880 --> 00:08:19,870 And that's still worth something. 141 00:08:19,870 --> 00:08:22,552 So there's this other function here 142 00:08:22,552 --> 00:08:23,760 that's supposed to be a line. 143 00:08:23,760 --> 00:08:26,370 Pretend it's a line. 144 00:08:26,370 --> 00:08:28,780 f of x equals 2x. 145 00:08:28,780 --> 00:08:32,409 And I know for sure that this is never 146 00:08:32,409 --> 00:08:34,419 going to exceed these, this. 147 00:08:34,419 --> 00:08:36,809 And in that case, where I only have one upper bound, 148 00:08:36,809 --> 00:08:41,799 and I don't have a good lower bound, I say that g of x 149 00:08:41,799 --> 00:08:44,670 is order of f of x. 150 00:08:44,670 --> 00:08:48,230 And this is where-- This is what O really means. 151 00:08:48,230 --> 00:08:50,230 It means that I only have an upper bound and not 152 00:08:50,230 --> 00:08:52,920 a lower bound. 153 00:08:52,920 --> 00:08:55,750 In most of the cases when you're coding something, 154 00:08:55,750 --> 00:08:59,350 you don't really care how well your program 155 00:08:59,350 --> 00:09:00,650 can run if you're really lucky. 156 00:09:00,650 --> 00:09:02,358 What you really care about is how does it 157 00:09:02,358 --> 00:09:03,620 run in the worst case? 158 00:09:03,620 --> 00:09:05,897 Or how does it run most of the time? 159 00:09:05,897 --> 00:09:08,230 So suppose you're Google, and you're writing your search 160 00:09:08,230 --> 00:09:10,890 engine, and you have to return the result 161 00:09:10,890 --> 00:09:12,990 to the user in 200 milliseconds. 162 00:09:12,990 --> 00:09:15,290 Because they did a study that shows that if you return 163 00:09:15,290 --> 00:09:17,750 the result in 400 milliseconds, they 164 00:09:17,750 --> 00:09:19,810 lose some number of users, which means 165 00:09:19,810 --> 00:09:22,130 they lose some number of billions a year. 166 00:09:22,130 --> 00:09:24,550 So to them a guarantee that their algorithms complete 167 00:09:24,550 --> 00:09:26,820 in 200 milliseconds is pretty important, 168 00:09:26,820 --> 00:09:29,040 because it means a ton of money. 169 00:09:29,040 --> 00:09:31,770 So if you have a running time that looks like this, 170 00:09:31,770 --> 00:09:35,280 that's OK, as long as this is 200 milliseconds 171 00:09:35,280 --> 00:09:38,900 for the number of pages that Google indexes. 172 00:09:38,900 --> 00:09:39,775 Does that make sense? 173 00:09:43,030 --> 00:09:43,530 OK. 174 00:09:43,530 --> 00:09:45,270 No answer means yes. 175 00:09:45,270 --> 00:09:47,620 So if it doesn't make sense, please say something. 176 00:09:52,240 --> 00:09:53,960 OK, I will draw another function. 177 00:09:53,960 --> 00:09:54,460 Again-- 178 00:09:54,460 --> 00:09:54,950 AUDIENCE: Victor? 179 00:09:54,950 --> 00:09:55,790 VICTOR COSTAN: --wavy. 180 00:09:55,790 --> 00:09:56,290 Yes? 181 00:09:56,290 --> 00:09:59,910 AUDIENCE: It says x times 1 plus something [INAUDIBLE]. 182 00:09:59,910 --> 00:10:00,630 So what? 183 00:10:00,630 --> 00:10:00,950 VICTOR COSTAN: So it's-- 184 00:10:00,950 --> 00:10:01,866 AUDIENCE: Oh it's sin. 185 00:10:01,866 --> 00:10:03,283 VICTOR COSTAN: 1 plus sin of x. 186 00:10:03,283 --> 00:10:03,782 It-- 187 00:10:03,782 --> 00:10:05,000 AUDIENCE: It looks like [INAUDIBLE]. 188 00:10:05,000 --> 00:10:06,090 VICTOR COSTAN: Yeah. 189 00:10:06,090 --> 00:10:07,784 Sorry, my handwriting is not very good. 190 00:10:07,784 --> 00:10:08,534 AUDIENCE: It's OK. 191 00:10:08,534 --> 00:10:09,360 Mine isn't either. 192 00:10:09,360 --> 00:10:10,860 VICTOR COSTAN: And it doesn't really 193 00:10:10,860 --> 00:10:13,170 matter what it is, it just needs to be sin 194 00:10:13,170 --> 00:10:14,639 so that it looks wavy like this. 195 00:10:14,639 --> 00:10:15,180 AUDIENCE: OK. 196 00:10:15,180 --> 00:10:16,010 VICTOR COSTAN: So that I can make 197 00:10:16,010 --> 00:10:17,380 something that looks complicated. 198 00:10:17,380 --> 00:10:17,872 OK. 199 00:10:17,872 --> 00:10:18,364 AUDIENCE: Got it. 200 00:10:18,364 --> 00:10:19,840 AUDIENCE: I have a question, too. 201 00:10:19,840 --> 00:10:23,649 In the first example, is it 1.2 times x squared? 202 00:10:23,649 --> 00:10:24,440 VICTOR COSTAN: Yes. 203 00:10:24,440 --> 00:10:26,720 I wanted something that's higher than 1.1. 204 00:10:26,720 --> 00:10:28,080 AUDIENCE: It's like [INAUDIBLE]. 205 00:10:28,080 --> 00:10:28,871 VICTOR COSTAN: Yes. 206 00:10:32,450 --> 00:10:32,950 OK. 207 00:10:32,950 --> 00:10:35,334 Now that I got a break, does anyone else 208 00:10:35,334 --> 00:10:36,250 want to ask questions? 209 00:10:40,480 --> 00:10:41,900 OK. 210 00:10:41,900 --> 00:10:44,828 So I'm going to draw another function. 211 00:10:44,828 --> 00:10:47,340 This one looks like this. 212 00:10:47,340 --> 00:10:58,176 And this is f of x equals 1 plus sin of x. 213 00:10:58,176 --> 00:11:02,110 I have to use these signs to make them look like that. 214 00:11:02,110 --> 00:11:05,880 Times x to the power of-- What do I have there? 215 00:11:05,880 --> 00:11:13,180 1 5 plus x to the power of one fourth. 216 00:11:13,180 --> 00:11:16,510 So for this function I know for sure that it is greater than x 217 00:11:16,510 --> 00:11:17,910 to the power of one fourth. 218 00:11:17,910 --> 00:11:18,710 Right? 219 00:11:18,710 --> 00:11:20,640 So if this is-- 220 00:11:26,770 --> 00:11:29,216 I know for sure that it's greater than that. 221 00:11:29,216 --> 00:11:32,440 Also if I look at this, and I know that sin is never 222 00:11:32,440 --> 00:11:35,100 bigger than 1, I know this whole thing is going to be, 223 00:11:35,100 --> 00:11:36,470 at most, 2. 224 00:11:36,470 --> 00:11:38,020 So if I have another function that 225 00:11:38,020 --> 00:11:46,690 says f prime of x equals, say, 3 times x to the power of 1.5, 226 00:11:46,690 --> 00:11:51,210 I know that it's going to bound it from above. 227 00:11:51,210 --> 00:11:53,480 So I have two bounds. 228 00:11:53,480 --> 00:11:58,470 So it sort of looks like this case. 229 00:11:58,470 --> 00:12:01,540 Except when I use theta, I have to make sure 230 00:12:01,540 --> 00:12:04,220 that the bounds are very similar. 231 00:12:04,220 --> 00:12:06,690 They can only differ by a constant factor. 232 00:12:06,690 --> 00:12:10,290 In this case my bounds differ by the exponent 233 00:12:10,290 --> 00:12:12,330 here, which is more than a constant factor, 234 00:12:12,330 --> 00:12:14,810 so they're not the same. 235 00:12:14,810 --> 00:12:20,360 So if I take my function-- which by the way is g-- 236 00:12:20,360 --> 00:12:30,580 if I want to say that g to the x is theta of something, I can't. 237 00:12:30,580 --> 00:12:32,490 Because no matter what I put in here, 238 00:12:32,490 --> 00:12:35,790 it's not going to be correct. 239 00:12:35,790 --> 00:12:39,290 The lower bound is this, the upper bound is this. 240 00:12:39,290 --> 00:12:42,520 But what I can do is I can say that I know for sure that it 241 00:12:42,520 --> 00:12:46,740 is theta of x to the power one fourth. 242 00:12:46,740 --> 00:12:48,490 And by the way I can also say that it 243 00:12:48,490 --> 00:12:52,240 is order of x to the power of one fifth. 244 00:12:55,180 --> 00:12:58,100 So if you have weird running times that looks like this, 245 00:12:58,100 --> 00:13:04,600 then you can only use O and omega. 246 00:13:04,600 --> 00:13:06,220 And with that being said, we will 247 00:13:06,220 --> 00:13:09,210 use O for pretty much the rest of this course. 248 00:13:09,210 --> 00:13:12,610 And for most of the cases we'll use it to mean theta. 249 00:13:16,960 --> 00:13:20,110 Oh, by the way, these formulas look ugly and complicated, 250 00:13:20,110 --> 00:13:24,390 because I wanted the graphs to look complicated to illustrate 251 00:13:24,390 --> 00:13:27,440 the idea that asymptotic notation makes 252 00:13:27,440 --> 00:13:29,941 things look simpler. 253 00:13:29,941 --> 00:13:32,440 In this course the running times that you'll be dealing with 254 00:13:32,440 --> 00:13:35,130 are all look like this. 255 00:13:35,130 --> 00:13:45,800 They will either be order 1, order log n, order n, order n 256 00:13:45,800 --> 00:13:50,550 log n, n ordered n squared. 257 00:13:50,550 --> 00:13:53,450 You might have one or two cases where you get something else, 258 00:13:53,450 --> 00:13:56,040 but most of the time if you have something else 259 00:13:56,040 --> 00:13:59,730 coming out of your occurrence, it means you're doing it wrong. 260 00:13:59,730 --> 00:14:01,081 So nice tip to have. 261 00:14:01,081 --> 00:14:03,080 And you can see that I already started doing it. 262 00:14:03,080 --> 00:14:06,580 I'm already using O instead of theta. 263 00:14:06,580 --> 00:14:08,670 So I guess I'm getting into the habit. 264 00:14:11,664 --> 00:14:15,575 AUDIENCE: So why did you leave the theta one blank for g of x? 265 00:14:15,575 --> 00:14:16,950 VICTOR COSTAN: Because whatever I 266 00:14:16,950 --> 00:14:18,790 put here is going to be wrong. 267 00:14:18,790 --> 00:14:22,250 If I put x to the power of one fifth, 268 00:14:22,250 --> 00:14:25,380 this is wrong, because I can have an upper bound that 269 00:14:25,380 --> 00:14:28,020 has x to the power of one fifth in it. 270 00:14:28,020 --> 00:14:30,300 But I can't have a lower bound like that, 271 00:14:30,300 --> 00:14:33,030 because this function hits x to the power of one 272 00:14:33,030 --> 00:14:35,880 fourth in infinitely many points. 273 00:14:35,880 --> 00:14:38,910 If I try to do x to the power of one fourth, same deal. 274 00:14:38,910 --> 00:14:41,610 Lower bound yes, upper bound no. 275 00:14:41,610 --> 00:14:43,670 So whatever I put in here is going to be wrong. 276 00:14:46,220 --> 00:14:47,030 Thank you. 277 00:14:47,030 --> 00:14:48,698 Good question. 278 00:14:48,698 --> 00:14:49,198 Yes? 279 00:14:49,198 --> 00:14:52,109 AUDIENCE: Is that 1.4 or one fourth? 280 00:14:52,109 --> 00:14:52,900 VICTOR COSTAN: 1.4. 281 00:14:52,900 --> 00:14:54,851 They are 1.4 and 1.5 282 00:14:54,851 --> 00:14:55,392 AUDIENCE: OK. 283 00:14:55,392 --> 00:14:57,609 Because I thought you were saying 1/4 and I was-- 284 00:14:57,609 --> 00:14:58,650 VICTOR COSTAN: Oh, sorry. 285 00:14:58,650 --> 00:14:59,944 1 4, 1.4. 286 00:14:59,944 --> 00:15:05,591 1.4, 1.5. 287 00:15:05,591 --> 00:15:06,090 OK. 288 00:15:06,090 --> 00:15:06,881 Are we good so far? 289 00:15:23,180 --> 00:15:23,680 OK. 290 00:15:23,680 --> 00:15:25,304 Let's go through a couple of exercises. 291 00:15:25,304 --> 00:15:27,377 So I'll give you a few functions-- Yes? 292 00:15:27,377 --> 00:15:29,210 AUDIENCE: Do you make omega-- Is that-- It's 293 00:15:29,210 --> 00:15:31,800 a lower bound already without multiplying by any constant, 294 00:15:31,800 --> 00:15:32,480 or something? 295 00:15:32,480 --> 00:15:33,990 Is that what it is? 296 00:15:33,990 --> 00:15:36,190 VICTOR COSTAN: So for all of these, 297 00:15:36,190 --> 00:15:41,400 what you put here is the simple, is the simple form. 298 00:15:41,400 --> 00:15:44,090 So you're lower bound can be this function multiplied 299 00:15:44,090 --> 00:15:45,300 by any constant factor. 300 00:15:45,300 --> 00:15:47,915 AUDIENCE: That-- So omega means lower bound? 301 00:15:47,915 --> 00:15:49,540 VICTOR COSTAN: Omega means lower bound, 302 00:15:49,540 --> 00:15:52,570 O means upper bound, theta means both. 303 00:15:58,380 --> 00:16:00,930 OK. 304 00:16:00,930 --> 00:16:04,860 Any other things to clear up before I ask questions? 305 00:16:04,860 --> 00:16:06,530 So it's your turn before it's my turn. 306 00:16:09,671 --> 00:16:10,170 All right. 307 00:16:10,170 --> 00:16:12,000 I'm going to go over a few functions, 308 00:16:12,000 --> 00:16:14,220 and I want you guys to tell me what-- 309 00:16:14,220 --> 00:16:17,520 to help me write them in asymptotic notation. 310 00:16:17,520 --> 00:16:22,230 So I will have f of n. 311 00:16:22,230 --> 00:16:27,530 And I want to know it's theta of what? 312 00:16:27,530 --> 00:16:32,680 So let's start with f of n equals 10 to the 80. 313 00:16:32,680 --> 00:16:34,780 Does anyone know why 10 to the 80? 314 00:16:37,141 --> 00:16:37,640 Yes. 315 00:16:37,640 --> 00:16:40,300 AUDIENCE: That'd be O of 1. 316 00:16:40,300 --> 00:16:41,050 VICTOR COSTAN: OK. 317 00:16:41,050 --> 00:16:42,435 That's the correct answer. 318 00:16:42,435 --> 00:16:43,560 You 're going ahead though. 319 00:16:43,560 --> 00:16:48,750 I was wondering if-- I was wondering if you know why? 320 00:16:48,750 --> 00:16:50,424 Why did I choose this? 321 00:16:50,424 --> 00:16:51,840 Does anyone know why I chose this? 322 00:16:51,840 --> 00:16:52,340 Yes? 323 00:16:52,340 --> 00:16:53,814 AUDIENCE: Is that [INAUDIBLE]? 324 00:16:53,814 --> 00:16:55,230 VICTOR COSTAN: I think that's 100. 325 00:16:55,230 --> 00:16:56,099 AUDIENCE: Oh, OK. 326 00:16:56,099 --> 00:16:57,890 VICTOR COSTAN: You're getting close though. 327 00:16:57,890 --> 00:16:58,787 It's something. 328 00:16:58,787 --> 00:17:00,245 AUDIENCE: It's a really big number. 329 00:17:00,245 --> 00:17:02,311 VICTOR COSTAN: It's a really big number. 330 00:17:02,311 --> 00:17:02,810 All right. 331 00:17:02,810 --> 00:17:04,190 So it's a really big number. 332 00:17:04,190 --> 00:17:07,440 That's the estimated number of atoms in the universe. 333 00:17:07,440 --> 00:17:09,859 So realistically any quantity that you 334 00:17:09,859 --> 00:17:12,260 deal with that will represent the real world, 335 00:17:12,260 --> 00:17:15,839 is not going to be bigger than this. 336 00:17:15,839 --> 00:17:18,079 So if you want to get all philosophical, 337 00:17:18,079 --> 00:17:20,849 you can say that for any algorithm that you write 338 00:17:20,849 --> 00:17:22,720 in any realistic input the running time 339 00:17:22,720 --> 00:17:24,280 will be theta of 1. 340 00:17:24,280 --> 00:17:26,849 But if you do this, we're not going to be happy with you 341 00:17:26,849 --> 00:17:27,968 on exams. 342 00:17:27,968 --> 00:17:30,320 [LAUGHTER] 343 00:17:30,320 --> 00:17:32,800 So let's choose another one. 344 00:17:32,800 --> 00:17:35,460 20 n to the power of 7. 345 00:17:39,100 --> 00:17:40,740 What's this an asymptotic notation? 346 00:17:40,740 --> 00:17:41,501 And why? 347 00:17:49,990 --> 00:17:50,490 Yes? 348 00:17:50,490 --> 00:17:54,550 AUDIENCE: Order O of n to the seventh. 349 00:17:54,550 --> 00:17:56,360 VICTOR COSTAN: OK. 350 00:17:56,360 --> 00:17:59,720 Of that same thing, n to the seventh. 351 00:17:59,720 --> 00:18:00,220 Thank you. 352 00:18:00,220 --> 00:18:01,722 Why is that? 353 00:18:01,722 --> 00:18:04,949 AUDIENCE: Because if you distribute the exponent 7, 354 00:18:04,949 --> 00:18:07,290 20 to the seventh is a constant. 355 00:18:07,290 --> 00:18:08,570 VICTOR COSTAN: Excellent. 356 00:18:08,570 --> 00:18:11,160 20 to the 7 time n to the 7, this 357 00:18:11,160 --> 00:18:12,600 is a constant so it goes away. 358 00:18:18,710 --> 00:18:19,330 OK. 359 00:18:19,330 --> 00:18:21,420 And before I do another one, I want 360 00:18:21,420 --> 00:18:24,780 to give you guys some help and remind you about logs. 361 00:18:24,780 --> 00:18:27,240 How many people remember the laws 362 00:18:27,240 --> 00:18:29,860 of working with logs from algebra or some math class? 363 00:18:32,240 --> 00:18:32,740 All right. 364 00:18:32,740 --> 00:18:36,510 You guys will be helping me write them for everyone else. 365 00:18:36,510 --> 00:18:40,960 So if I have log of n to the 100, 366 00:18:40,960 --> 00:18:44,640 and I want to get to the theta notation for it, what is it 367 00:18:44,640 --> 00:18:47,190 and why? 368 00:18:47,190 --> 00:18:48,330 AUDIENCE: Log n. 369 00:18:48,330 --> 00:18:49,080 VICTOR COSTAN: OK. 370 00:18:49,080 --> 00:18:49,580 Excellent. 371 00:18:53,920 --> 00:18:55,746 Why is that the case? 372 00:18:55,746 --> 00:18:57,950 AUDIENCE: It's 100 log n. 373 00:18:57,950 --> 00:18:58,700 VICTOR COSTAN: OK. 374 00:18:58,700 --> 00:19:00,380 So when I have an exponent here, I 375 00:19:00,380 --> 00:19:03,430 can pull it out in front of the log as a constant factor. 376 00:19:03,430 --> 00:19:10,471 So this is the same thing as 100 log n. 377 00:19:10,471 --> 00:19:10,970 OK. 378 00:19:10,970 --> 00:19:19,810 Now if I have the logarithm to the base using base 5 of n-- 379 00:19:19,810 --> 00:19:20,640 How about this? 380 00:19:26,412 --> 00:19:27,855 AUDIENCE: [INAUDIBLE] multiply. 381 00:19:27,855 --> 00:19:29,780 That's just log n? 382 00:19:29,780 --> 00:19:30,610 VICTOR COSTAN: OK. 383 00:19:30,610 --> 00:19:32,560 So this is-- can you say that again? 384 00:19:32,560 --> 00:19:33,530 AUDIENCE: Log n. 385 00:19:33,530 --> 00:19:34,280 VICTOR COSTAN: OK. 386 00:19:34,280 --> 00:19:39,260 So this is theta of log n because it is-- 387 00:19:39,260 --> 00:19:41,360 AUDIENCE: Log n over log 5. 388 00:19:41,360 --> 00:19:48,160 VICTOR COSTAN: So that's log n over log 5. 389 00:19:48,160 --> 00:19:51,610 So exponents inside the logs turn into constant factors, 390 00:19:51,610 --> 00:19:55,330 bases turn into constant factors. 391 00:19:55,330 --> 00:19:58,390 So if you have an ugly expression inside a logarithm, 392 00:19:58,390 --> 00:20:00,020 you can, most of the time, simplify it 393 00:20:00,020 --> 00:20:01,061 to something really nice. 394 00:20:01,061 --> 00:20:04,390 And this contributes to the fact that these 395 00:20:04,390 --> 00:20:07,764 are the only functions that you will see when the answer is 396 00:20:07,764 --> 00:20:09,180 cleaned up in asymptotic notation. 397 00:20:13,941 --> 00:20:14,440 OK. 398 00:20:14,440 --> 00:20:14,660 Yes? 399 00:20:14,660 --> 00:20:16,201 AUDIENCE: So when you're writing log, 400 00:20:16,201 --> 00:20:18,452 are you assuming the second base then? 401 00:20:18,452 --> 00:20:20,660 VICTOR COSTAN: Yes, there is a certain base for logs. 402 00:20:20,660 --> 00:20:22,021 We're all computer scientists. 403 00:20:22,021 --> 00:20:22,520 Right? 404 00:20:22,520 --> 00:20:25,213 So what's the natural base for our logarithms? 405 00:20:25,213 --> 00:20:26,099 AUDIENCE: 2. 406 00:20:26,099 --> 00:20:27,430 2D. 407 00:20:27,430 --> 00:20:28,350 VICTOR COSTAN: OK. 408 00:20:28,350 --> 00:20:29,880 Sweet. 409 00:20:29,880 --> 00:20:31,486 Math major? 410 00:20:31,486 --> 00:20:33,780 AUDIENCE: No, you know I always heard natural base-- 411 00:20:33,780 --> 00:20:34,530 VICTOR COSTAN: OK. 412 00:20:34,530 --> 00:20:35,800 Yeah, it's a math joke. 413 00:20:35,800 --> 00:20:39,580 So for everyone else a natural logarithm is in base E. 414 00:20:39,580 --> 00:20:42,490 For us CS people, a natural base is 2, 415 00:20:42,490 --> 00:20:46,580 because that's how many values you have for a bit. 416 00:20:46,580 --> 00:20:54,820 So log means log n is to log base 2 of n. 417 00:20:57,437 --> 00:20:59,020 The good news is that most of the time 418 00:20:59,020 --> 00:21:00,644 we're working with asymptotic notation, 419 00:21:00,644 --> 00:21:01,840 so it doesn't really matter. 420 00:21:01,840 --> 00:21:04,670 We can communicate with our math friends, 421 00:21:04,670 --> 00:21:08,371 and we don't have to talk about this. 422 00:21:08,371 --> 00:21:08,870 OK. 423 00:21:08,870 --> 00:21:12,060 I need to get another complicated example. 424 00:21:12,060 --> 00:21:13,590 And my memory fails me. 425 00:21:17,518 --> 00:21:18,018 OK. 426 00:21:22,450 --> 00:21:27,200 So log-- See, I'm using Ln to mean the math 427 00:21:27,200 --> 00:21:28,850 natural logarithm. 428 00:21:28,850 --> 00:21:33,075 So we still have that somewhere in here. 429 00:21:33,075 --> 00:21:39,570 Of log n to the base 100. 430 00:21:39,570 --> 00:21:42,300 They made me add all these parentheses in the last section 431 00:21:42,300 --> 00:21:45,110 to be clear, so I'm adding them now from the start. 432 00:21:45,110 --> 00:21:46,790 Who wants to help me clean this up? 433 00:21:49,800 --> 00:21:50,496 Yes? 434 00:21:50,496 --> 00:21:51,799 AUDIENCE: Log, log n. 435 00:21:51,799 --> 00:21:52,840 VICTOR COSTAN: All right. 436 00:21:52,840 --> 00:21:55,940 Wow, one step. 437 00:21:55,940 --> 00:22:03,200 Log, log n OK. 438 00:22:03,200 --> 00:22:05,931 Can you tell me why, very quickly? 439 00:22:05,931 --> 00:22:08,347 AUDIENCE: So we can simplify out the base and the exponent 440 00:22:08,347 --> 00:22:08,847 in there. 441 00:22:08,847 --> 00:22:09,664 VICTOR COSTAN: OK. 442 00:22:09,664 --> 00:22:10,330 AUDIENCE: Then-- 443 00:22:10,330 --> 00:22:11,538 VICTOR COSTAN: Base goes out. 444 00:22:11,538 --> 00:22:12,880 Exponent goes out. 445 00:22:12,880 --> 00:22:16,177 AUDIENCE: Log, log of something is another [INAUDIBLE]. 446 00:22:16,177 --> 00:22:17,260 It's not exactly up there. 447 00:22:17,260 --> 00:22:17,745 But can you-- 448 00:22:17,745 --> 00:22:18,536 VICTOR COSTAN: Yep. 449 00:22:18,536 --> 00:22:20,170 AUDIENCE: --occassionally [INAUDIBLE]. 450 00:22:20,170 --> 00:22:21,470 VICTOR COSTAN: Yep. 451 00:22:21,470 --> 00:22:23,680 What does a theory student say when they're drowning? 452 00:22:26,720 --> 00:22:28,090 Log, log, log, log. 453 00:22:28,090 --> 00:22:31,210 Some of you are laughing already, because know this. 454 00:22:31,210 --> 00:22:31,710 OK. 455 00:22:31,710 --> 00:22:33,900 What if I want to make this a little bit harder? 456 00:22:33,900 --> 00:22:45,752 And I say log [INAUDIBLE] 5 of log n to the 100. 457 00:22:45,752 --> 00:22:47,210 You have to do a bit more work now, 458 00:22:47,210 --> 00:22:49,001 you can't just use the tricks on the right. 459 00:22:53,470 --> 00:22:55,980 Anyone else? 460 00:22:55,980 --> 00:22:56,490 Yes? 461 00:22:56,490 --> 00:22:57,930 AUDIENCE: It's still log, log n. 462 00:22:57,930 --> 00:22:58,680 VICTOR COSTAN: OK. 463 00:22:58,680 --> 00:23:00,546 It is still log, log n. 464 00:23:00,546 --> 00:23:01,670 But for a different reason. 465 00:23:01,670 --> 00:23:03,294 Right? 466 00:23:03,294 --> 00:23:04,650 AUDIENCE: [INAUDIBLE]. 467 00:23:04,650 --> 00:23:06,010 VICTOR COSTAN: Almost. 468 00:23:06,010 --> 00:23:06,690 OK. 469 00:23:06,690 --> 00:23:07,510 Maybe not. 470 00:23:07,510 --> 00:23:12,270 So the base still goes out just like before. 471 00:23:12,270 --> 00:23:14,920 But this guy now goes out here. 472 00:23:14,920 --> 00:23:21,100 So I will have this be, sort of like-- Notice 473 00:23:21,100 --> 00:23:22,670 that I'm not saying equal, I'm saying 474 00:23:22,670 --> 00:23:24,590 like because I got rid of the base. 475 00:23:24,590 --> 00:23:39,790 So it's like 100 times log n, which is log 100 plus log, 476 00:23:39,790 --> 00:23:40,780 log n. 477 00:23:43,750 --> 00:23:47,040 And this is a constant factor, this grows as n grows. 478 00:23:47,040 --> 00:23:49,740 So this is going to become incredibly small 479 00:23:49,740 --> 00:23:52,960 as n becomes larger and larger, so I can get rid of it. 480 00:23:55,980 --> 00:23:56,480 OK? 481 00:23:59,720 --> 00:24:00,730 OK. 482 00:24:00,730 --> 00:24:03,010 One more, and this one's going to be less boring. 483 00:24:03,010 --> 00:24:06,979 AUDIENCE: [INAUDIBLE] out into a separate term-- the log of 100? 484 00:24:06,979 --> 00:24:09,046 Going from it in the last step, how 485 00:24:09,046 --> 00:24:12,090 are you able to cross that out? 486 00:24:12,090 --> 00:24:15,910 VICTOR COSTAN: If my memory-- If my memory works, I think, 487 00:24:15,910 --> 00:24:21,017 log of a b is log a plus log b. 488 00:24:21,017 --> 00:24:21,725 AUDIENCE: Gotcha. 489 00:24:21,725 --> 00:24:22,965 VICTOR COSTAN: Am I right guys? 490 00:24:22,965 --> 00:24:23,465 Sweet. 491 00:24:29,620 --> 00:24:30,770 Good question. 492 00:24:30,770 --> 00:24:33,010 I wasn't sure, but now that you asked, 493 00:24:33,010 --> 00:24:35,340 everyone else helped me make sure it's right. 494 00:24:43,790 --> 00:24:44,290 OK. 495 00:24:44,290 --> 00:24:45,706 I'm erasing so much, because we're 496 00:24:45,706 --> 00:24:47,940 going to need more room for this. 497 00:24:47,940 --> 00:24:51,090 So I have a log again, but this time what I have inside the log 498 00:24:51,090 --> 00:24:56,790 is n choose n over 2. 499 00:24:56,790 --> 00:24:59,730 Does any brave soul want to come up here and solve this for me? 500 00:25:08,135 --> 00:25:08,635 No? 501 00:25:12,320 --> 00:25:14,963 You guys are going to make me write for the entire hour? 502 00:25:14,963 --> 00:25:16,895 [LAUGHTER] 503 00:25:19,793 --> 00:25:23,160 VICTOR COSTAN: I'll offer help. 504 00:25:23,160 --> 00:25:25,405 AUDIENCE: It's n-- [INAUDIBLE] 505 00:25:25,405 --> 00:25:26,530 VICTOR COSTAN: So this is-- 506 00:25:26,530 --> 00:25:28,609 AUDIENCE: Is that n factorial over -- 507 00:25:28,609 --> 00:25:29,400 VICTOR COSTAN: Yep. 508 00:25:29,400 --> 00:25:31,980 AUDIENCE: --n divided by 2 factorial? 509 00:25:31,980 --> 00:25:37,250 VICTOR COSTAN: n factorial over n divided by 2 factorial, 510 00:25:37,250 --> 00:25:40,120 times n divided by 2 factorial. 511 00:25:40,120 --> 00:25:40,620 Close. 512 00:25:43,250 --> 00:25:46,170 And I'll give up another hint, which 513 00:25:46,170 --> 00:25:49,950 is that n factorial can be approximated 514 00:25:49,950 --> 00:25:56,420 as square root of 2pi n multiplied by n over E divided 515 00:25:56,420 --> 00:25:58,070 by n to the power of n. 516 00:25:58,070 --> 00:25:59,870 But I think this will make you guys not 517 00:25:59,870 --> 00:26:01,460 want to come up and solve this. 518 00:26:01,460 --> 00:26:02,018 Yes? 519 00:26:02,018 --> 00:26:03,101 AUDIENCE: OK. [INAUDIBLE]. 520 00:26:05,970 --> 00:26:09,120 VICTOR COSTAN: I was-- Is that it? 521 00:26:09,120 --> 00:26:10,490 Is that not it? 522 00:26:10,490 --> 00:26:12,170 Did I get it wrong? 523 00:26:12,170 --> 00:26:12,886 OK. 524 00:26:12,886 --> 00:26:16,540 AUDIENCE: [INAUDIBLE] minus n. 525 00:26:16,540 --> 00:26:18,890 VICTOR COSTAN: I think that's an asymptotic. 526 00:26:18,890 --> 00:26:22,260 That's what we'll-- I think we'll get to that. 527 00:26:22,260 --> 00:26:23,775 But factorials grow exponentially. 528 00:26:26,450 --> 00:26:30,201 So it should be something like this. 529 00:26:30,201 --> 00:26:30,700 OK. 530 00:26:30,700 --> 00:26:33,210 So we have that nice approximation. 531 00:26:33,210 --> 00:26:35,680 Does anyone want to work it out, or will I have to do it? 532 00:26:38,774 --> 00:26:41,350 I'll keep that in mind when I'm grading your homework. 533 00:26:41,350 --> 00:26:42,480 [LAUGHTER] 534 00:26:42,480 --> 00:26:43,230 VICTOR COSTAN: OK. 535 00:26:43,230 --> 00:26:53,390 So it's log of square root 2 pi n multiplied by n over 2 536 00:26:53,390 --> 00:26:55,750 to the power of n. 537 00:26:55,750 --> 00:26:58,140 I'm going to put these two together. 538 00:26:58,140 --> 00:27:06,720 So 2 pi n over-- There's an n over 2 there, right? 539 00:27:06,720 --> 00:27:16,780 So it's n over 2E times n over 2 to the power of 2. 540 00:27:22,990 --> 00:27:27,330 OK, I can get rid of this guy, because it's a constant. 541 00:27:27,330 --> 00:27:31,805 I can get-- No, I can't get rid of this guy yet. 542 00:27:31,805 --> 00:27:35,605 AUDIENCE: In the denominator shouldn't it be 2pi n over 2 543 00:27:35,605 --> 00:27:37,980 under the radical? 544 00:27:37,980 --> 00:27:39,450 VICTOR COSTAN: Yep. 545 00:27:39,450 --> 00:27:42,330 2pi n over 2, so this simplifies. 546 00:27:42,330 --> 00:27:43,230 OK. 547 00:27:43,230 --> 00:27:44,750 Thank you. 548 00:27:44,750 --> 00:27:46,250 AUDIENCE: Square at the bottom term, 549 00:27:46,250 --> 00:27:49,100 you can get rid of the square root on the end-- 550 00:27:49,100 --> 00:27:50,050 VICTOR COSTAN: Yep. 551 00:27:50,050 --> 00:27:53,830 AUDIENCE: --and then cancel the some of the exponents. 552 00:27:53,830 --> 00:27:54,580 VICTOR COSTAN: OK. 553 00:27:54,580 --> 00:27:55,746 Do you want to come help me? 554 00:27:55,746 --> 00:27:56,491 AUDIENCE: Sure. 555 00:27:56,491 --> 00:27:57,740 VICTOR COSTAN: Thanks so much. 556 00:28:05,884 --> 00:28:06,384 STUDENT: OK. 557 00:28:06,384 --> 00:28:09,680 So we can distribute this exponent here to cancel this. 558 00:28:09,680 --> 00:28:19,898 And then we get square root of pi n times n over 2. 559 00:28:19,898 --> 00:28:23,167 And it'd be n-- sorry about my handwriting. 560 00:28:23,167 --> 00:28:25,000 VICTOR COSTAN: I think it's better than mine 561 00:28:25,000 --> 00:28:27,760 by quite a bit. 562 00:28:27,760 --> 00:28:32,485 STUDENT: Square root of pi n. 563 00:28:32,485 --> 00:28:34,960 And this is square root squared, so that cancels . 564 00:28:40,405 --> 00:28:44,365 Times n over E to the n. 565 00:28:49,315 --> 00:28:51,702 AUDIENCE: Do you need parenthesis around n over 2E? 566 00:28:51,702 --> 00:28:52,285 STUDENT: Yeah. 567 00:28:52,285 --> 00:28:54,265 That's probably good. 568 00:28:54,265 --> 00:28:58,450 So, since this is n over E to the n, and n over 2E n. 569 00:28:58,450 --> 00:29:01,210 Well since they're to the n, both the 570 00:29:01,210 --> 00:29:05,140 n we can give them a single exponent and cancel stuff. 571 00:29:05,140 --> 00:29:13,910 So we just get square root of pi n to the times 2 572 00:29:13,910 --> 00:29:18,520 the n, over pi n. 573 00:29:22,892 --> 00:29:30,666 Can do some more cancelling and just get 2 the n over-- 574 00:29:30,666 --> 00:29:36,995 Going to simplify this to the n over the square root of n. 575 00:29:36,995 --> 00:29:39,920 We can take out the pi, because it's in our natural logarithm. 576 00:29:39,920 --> 00:29:41,660 It's a constant factor. 577 00:29:41,660 --> 00:29:54,980 And this is log of-- and then-- Since it's a log of, 578 00:29:54,980 --> 00:29:56,960 it's just log of 2 to the n. 579 00:29:56,960 --> 00:30:09,236 So it's-- say since it's log you can turn it into log of 2 580 00:30:09,236 --> 00:30:13,070 to the n minus log of square root of n. 581 00:30:13,070 --> 00:30:15,180 Log of 2 to the n is a much greater complexity 582 00:30:15,180 --> 00:30:17,270 than square root of n, so we can-- 583 00:30:17,270 --> 00:30:18,173 VICTOR COSTAN: Sweet. 584 00:30:18,173 --> 00:30:20,290 Thank you. 585 00:30:20,290 --> 00:30:21,140 So see? 586 00:30:21,140 --> 00:30:24,510 Complicated formula-- Even more complicated formula 587 00:30:24,510 --> 00:30:26,370 came out to be one of the numbers 588 00:30:26,370 --> 00:30:28,260 that I promised it would come out to. 589 00:30:28,260 --> 00:30:29,470 Thanks for the help. 590 00:30:29,470 --> 00:30:33,320 So you can get rid of the pi as soon as you want to. 591 00:30:33,320 --> 00:30:36,120 And this is the last step. 592 00:30:36,120 --> 00:30:37,550 I just want to say it again, just 593 00:30:37,550 --> 00:30:39,020 make sure everyone heard it. 594 00:30:39,020 --> 00:30:42,780 So once you get here, you get this, 595 00:30:42,780 --> 00:30:48,310 order n minus log of square root of n, 596 00:30:48,310 --> 00:31:07,130 which is-- This is-- This is order n minus one half log n. 597 00:31:07,130 --> 00:31:09,880 And the log n is much smaller than n, 598 00:31:09,880 --> 00:31:12,377 so this goes away completely. 599 00:31:12,377 --> 00:31:13,960 And you get this as your final answer. 600 00:31:20,450 --> 00:31:20,950 OK. 601 00:31:20,950 --> 00:31:21,824 Does this make sense? 602 00:31:27,230 --> 00:31:28,420 Any questions? 603 00:31:28,420 --> 00:31:30,320 Anyone? 604 00:31:30,320 --> 00:31:34,210 How many people are still paying attention honestly? 605 00:31:34,210 --> 00:31:35,130 Wow. 606 00:31:35,130 --> 00:31:36,532 Nice. 607 00:31:36,532 --> 00:31:37,990 You should come to all my sections. 608 00:31:59,890 --> 00:32:00,390 OK. 609 00:32:00,390 --> 00:32:02,410 Let's talk about something we covered in class. 610 00:32:02,410 --> 00:32:03,820 Let's talk about peak finding. 611 00:32:03,820 --> 00:32:05,560 And we'll start with 1D peak finding, 612 00:32:05,560 --> 00:32:08,000 and then we'll go to 2D peak finding. 613 00:32:08,000 --> 00:32:11,316 So does anyone remember the pseudocode for the faster 1D 614 00:32:11,316 --> 00:32:12,440 peak finding by any chance? 615 00:32:16,560 --> 00:32:18,660 ID peak finding. 616 00:32:22,220 --> 00:32:24,270 I think everyone said they went to lecture, 617 00:32:24,270 --> 00:32:26,580 so I don't have to explain what peak finding is, right? 618 00:32:30,460 --> 00:32:30,960 OK. 619 00:32:30,960 --> 00:32:32,870 So that's go for it. 620 00:32:32,870 --> 00:32:36,320 So suppose I have a few numbers-- 621 00:32:36,320 --> 00:32:48,370 3-- let me write this one-- 3, 4, 5, 6, 7, 6, 5, 4. 622 00:32:48,370 --> 00:32:52,890 So the peak finding problem says there's an array of numbers, 623 00:32:52,890 --> 00:32:54,610 and I want to find the local peak. 624 00:32:54,610 --> 00:32:56,720 And the local peak is a number that's 625 00:32:56,720 --> 00:33:01,040 surrounded by numbers that are smaller or equal to it. 626 00:33:01,040 --> 00:33:05,720 So there shouldn't be any larger numbers than it. 627 00:33:05,720 --> 00:33:10,355 And I want to find a peak as quickly as I can. 628 00:33:10,355 --> 00:33:12,480 So what's the faster algorithm that we talked about 629 00:33:12,480 --> 00:33:12,980 in lecture? 630 00:33:17,730 --> 00:33:18,370 Yes? 631 00:33:18,370 --> 00:33:18,870 OK. 632 00:33:18,870 --> 00:33:22,378 You didn't answer, right, as much, at least so. 633 00:33:22,378 --> 00:33:23,390 AUDIENCE: Binary search? 634 00:33:23,390 --> 00:33:24,140 VICTOR COSTAN: OK. 635 00:33:24,140 --> 00:33:25,840 It's exactly like binary search. 636 00:33:25,840 --> 00:33:27,270 Yep. 637 00:33:27,270 --> 00:33:29,740 Do you wanna help me write the pseudocode? 638 00:33:29,740 --> 00:33:31,089 You can tell me what it is. 639 00:33:31,089 --> 00:33:31,714 AUDIENCE: Sure. 640 00:33:31,714 --> 00:33:34,290 So, I guess well start in the middle. 641 00:33:34,290 --> 00:33:35,040 VICTOR COSTAN: OK. 642 00:33:37,550 --> 00:33:40,310 Start in the middle. 643 00:33:43,930 --> 00:33:46,334 If you-- If you write this on your Pset 644 00:33:46,334 --> 00:33:48,250 you might want to be a little bit more formal. 645 00:33:48,250 --> 00:33:49,410 Someone asked about proofs. 646 00:33:49,410 --> 00:33:51,730 I would say something like, start 647 00:33:51,730 --> 00:33:54,500 by looking at the middle element of the array. 648 00:33:54,500 --> 00:33:57,530 Same thing though. 649 00:33:57,530 --> 00:33:58,640 OK. 650 00:33:58,640 --> 00:34:02,160 So let's pretend this is the middle. 651 00:34:02,160 --> 00:34:04,480 Now it is. 652 00:34:04,480 --> 00:34:05,568 What do I do? 653 00:34:05,568 --> 00:34:09,870 AUDIENCE: Then you can check one side in the middle. 654 00:34:09,870 --> 00:34:11,720 So it's [INAUDIBLE] the right side. 655 00:34:11,720 --> 00:34:12,469 VICTOR COSTAN: OK. 656 00:34:12,469 --> 00:34:13,982 I'm looking at the right side. 657 00:34:13,982 --> 00:34:16,250 I see a 7 there, so I know for sure 658 00:34:16,250 --> 00:34:17,760 that this is not a local peak. 659 00:34:17,760 --> 00:34:19,389 Right? 660 00:34:19,389 --> 00:34:21,468 What do I do now? 661 00:34:21,468 --> 00:34:26,704 AUDIENCE: So now you can start over, and using 7. 662 00:34:26,704 --> 00:34:27,659 VICTOR COSTAN: OK. 663 00:34:27,659 --> 00:34:29,560 What if-- This would be small. 664 00:34:29,560 --> 00:34:32,900 What if I would look at 5 first? 665 00:34:32,900 --> 00:34:35,860 Or you should I look at the other side too? 666 00:34:35,860 --> 00:34:38,483 AUDIENCE: Well you don't have to in this case, but if you-- 667 00:34:38,483 --> 00:34:40,897 But if 7 were smaller then you would 668 00:34:40,897 --> 00:34:42,769 go and look at the other side, and check 669 00:34:42,769 --> 00:34:44,110 to see if 5 is also smaller. 670 00:34:44,110 --> 00:34:46,580 In that case you've already found the peak. 671 00:34:46,580 --> 00:34:47,330 VICTOR COSTAN: OK. 672 00:34:47,330 --> 00:34:51,469 So I have to look at both sides and make sure-- if none of them 673 00:34:51,469 --> 00:34:52,929 is smaller, than I-- I'm sorry. 674 00:34:52,929 --> 00:34:54,989 If none of them is bigger, I found the peak. 675 00:34:54,989 --> 00:35:00,340 If any of them is bigger then I can go whichever way I want. 676 00:35:00,340 --> 00:35:03,990 So I saw my 7, I'm going to cut this in half. 677 00:35:03,990 --> 00:35:05,652 And now I look at this. 678 00:35:05,652 --> 00:35:06,610 You were saying, right? 679 00:35:10,650 --> 00:35:13,140 So look at the neighbors-- By the way, 680 00:35:13,140 --> 00:35:14,700 can anyone understand my handwriting? 681 00:35:14,700 --> 00:35:17,830 Is it even worth bothering to write? 682 00:35:17,830 --> 00:35:19,090 AUDIENCE: It's pretty good. 683 00:35:19,090 --> 00:35:20,160 VICTOR COSTAN: OK. 684 00:35:20,160 --> 00:35:21,060 I see three nods. 685 00:35:21,060 --> 00:35:24,181 I'm going to write for those three people. 686 00:35:24,181 --> 00:35:25,680 AUDIENCE: It's just we haven't taken 687 00:35:25,680 --> 00:35:27,858 any cursive in about 12 years. 688 00:35:27,858 --> 00:35:29,830 So-- 689 00:35:29,830 --> 00:35:30,580 VICTOR COSTAN: OK. 690 00:35:30,580 --> 00:35:31,560 Thanks. 691 00:35:31,560 --> 00:35:33,590 Not feeling well at all. 692 00:35:33,590 --> 00:35:37,346 OK look at neighbors, figure out if you're a peak. 693 00:35:37,346 --> 00:35:39,470 You're one of the people that said you can read it, 694 00:35:39,470 --> 00:35:42,420 so now there's only two people that can read my handwriting 695 00:35:42,420 --> 00:35:43,140 and that I like. 696 00:35:43,140 --> 00:35:45,180 [LAUGHTER] 697 00:35:45,180 --> 00:35:47,110 VICTOR COSTAN: Look at the neighbors. 698 00:35:47,110 --> 00:35:48,540 See if local peak. 699 00:35:52,280 --> 00:35:58,160 And if not, figure out which way to go in. 700 00:36:01,080 --> 00:36:03,850 And I'll say if not recurse. 701 00:36:06,740 --> 00:36:09,930 So what's the running time for this? 702 00:36:09,930 --> 00:36:12,250 Suppose I have n elements to start with, 703 00:36:12,250 --> 00:36:13,660 that's my array size. 704 00:36:13,660 --> 00:36:19,584 I will use t of n as my running time. 705 00:36:19,584 --> 00:36:22,000 So to figure out the running time for the whole algorithm, 706 00:36:22,000 --> 00:36:23,890 I have to look at the running time for each step 707 00:36:23,890 --> 00:36:24,660 and add them up. 708 00:36:24,660 --> 00:36:26,100 Right? 709 00:36:26,100 --> 00:36:29,010 What's the running time for the first step? 710 00:36:29,010 --> 00:36:29,940 Looking in the middle. 711 00:36:34,350 --> 00:36:37,780 AUDIENCE: [INAUDIBLE] the middle elements. 712 00:36:37,780 --> 00:36:38,520 VICTOR COSTAN: 1. 713 00:36:38,520 --> 00:36:39,020 1. 714 00:36:39,020 --> 00:36:40,610 I would not say 0. 715 00:36:40,610 --> 00:36:41,900 I still have to look at it. 716 00:36:41,900 --> 00:36:46,940 Or to be on the safe side you say 1, theta of 1. 717 00:36:46,940 --> 00:36:49,530 It's a constant. 718 00:36:49,530 --> 00:36:50,030 OK. 719 00:36:50,030 --> 00:36:51,530 Now I have to look at the neighbors, 720 00:36:51,530 --> 00:36:54,150 and I have to see if one of them is bigger or not. 721 00:36:54,150 --> 00:36:56,030 How about that? 722 00:36:56,030 --> 00:36:57,118 How much time is left? 723 00:37:00,050 --> 00:37:01,440 I heard the constant. 724 00:37:01,440 --> 00:37:02,670 OK. 725 00:37:02,670 --> 00:37:05,430 Very soft voice, but I heard a constant there, 726 00:37:05,430 --> 00:37:07,820 so I'm going to take it. 727 00:37:07,820 --> 00:37:11,254 So this guy is a constant. 728 00:37:11,254 --> 00:37:12,670 If it's a local, if you come down. 729 00:37:12,670 --> 00:37:15,750 If it's not a local peak, what's the running time for this? 730 00:37:22,146 --> 00:37:25,750 n over 2, since-- 731 00:37:25,750 --> 00:37:28,250 AUDIENCE: If you FOIL that you divide by [INAUDIBLE], right? 732 00:37:28,250 --> 00:37:29,180 VICTOR COSTAN: Yep. 733 00:37:29,180 --> 00:37:30,810 So I'm looking at this guy. 734 00:37:30,810 --> 00:37:32,916 It's clearly not a peak. 735 00:37:32,916 --> 00:37:34,540 So I'm going to look on the right side, 736 00:37:34,540 --> 00:37:37,330 because I see a 7 here that's bigger than a 6. 737 00:37:37,330 --> 00:37:40,590 So now I have the same problem, except my array size just 738 00:37:40,590 --> 00:37:42,290 dropped to half. 739 00:37:42,290 --> 00:37:46,390 So same problem, because I already have a name for it, 740 00:37:46,390 --> 00:37:50,110 except the input size is different. 741 00:37:50,110 --> 00:37:56,130 So the running time for this entire step is theta of 1 742 00:37:56,130 --> 00:37:59,540 plus T of n over 2. 743 00:37:59,540 --> 00:38:01,650 So the running time for the whole thing 744 00:38:01,650 --> 00:38:07,770 is-- I should have said theta of 1 here, sorry. 745 00:38:07,770 --> 00:38:13,890 So theta of 1 plus theta of 1 plus T of n over 2. 746 00:38:13,890 --> 00:38:22,460 So this is theta 1 plus T of n over 2. 747 00:38:22,460 --> 00:38:24,330 So I wrote out my pseudocode, looked 748 00:38:24,330 --> 00:38:26,460 at the running time for each step, added them up. 749 00:38:26,460 --> 00:38:30,630 Now I have a recurrence, and I have to solve it. 750 00:38:30,630 --> 00:38:32,460 Let's solve it. 751 00:38:32,460 --> 00:38:34,450 We sort of glossed over that in lectures, 752 00:38:34,450 --> 00:38:36,980 so we're going to do it now. 753 00:38:46,150 --> 00:38:52,620 So T of n equals theta of 1 plus T of n over 2. 754 00:38:52,620 --> 00:38:56,220 I'm going to expand this using the definition. 755 00:38:56,220 --> 00:39:01,840 And theta of one is long, so I'm going to call this c instead. 756 00:39:01,840 --> 00:39:04,060 It's a constant, right? 757 00:39:04,060 --> 00:39:09,510 So c plus-- expanding this-- c plus T of n over 4. 758 00:39:09,510 --> 00:39:11,080 I'm going to expand it again. 759 00:39:16,670 --> 00:39:25,530 And again-- Anyone getting bored of this? 760 00:39:25,530 --> 00:39:31,020 So if I expand it a lot of times, I'm going to get i times 761 00:39:31,020 --> 00:39:35,912 c, plus T of n divided by 2 to the n. 762 00:39:35,912 --> 00:39:37,370 And you can expand it as many times 763 00:39:37,370 --> 00:39:39,120 as you want until we see the pattern, 764 00:39:39,120 --> 00:39:41,950 but this is the pattern. 765 00:39:41,950 --> 00:39:43,200 So I have a recurrence. 766 00:39:43,200 --> 00:39:45,770 What else do I need in order to get a function? 767 00:39:45,770 --> 00:39:47,336 I need a base case. 768 00:39:47,336 --> 00:39:49,710 So the problem size gets smaller and smaller and smaller, 769 00:39:49,710 --> 00:39:52,130 but at some point it becomes so small 770 00:39:52,130 --> 00:39:55,080 that I can solve it just by looking at it. 771 00:39:55,080 --> 00:39:58,930 What's that case? 772 00:39:58,930 --> 00:40:00,400 AUDIENCE: [INAUDIBLE] 773 00:40:00,400 --> 00:40:01,150 VICTOR COSTAN: OK. 774 00:40:01,150 --> 00:40:04,250 So if I have an array of one element, it's a peak. 775 00:40:04,250 --> 00:40:04,910 Right? 776 00:40:04,910 --> 00:40:05,410 Job done. 777 00:40:08,210 --> 00:40:15,970 So T of 1 is a constant. 778 00:40:15,970 --> 00:40:22,350 So I want to write this in such a way that this becomes this. 779 00:40:22,350 --> 00:40:29,680 So I want to figure out an i such that-- Yeah. 780 00:40:29,680 --> 00:40:36,770 T of n divided by 2 to the i equals T of 1. 781 00:40:36,770 --> 00:40:38,780 And the easiest way of doing that is 782 00:40:38,780 --> 00:40:42,660 to say that n divided by 2 to the i is 1. 783 00:40:42,660 --> 00:40:48,770 So i is the inverse of an exponential-- Anyone? 784 00:40:52,534 --> 00:40:53,450 AUDIENCE: [INAUDIBLE]. 785 00:40:53,450 --> 00:40:54,950 VICTOR COSTAN: Logarithm, thank you. 786 00:40:57,570 --> 00:41:01,470 Log n. 787 00:41:01,470 --> 00:41:05,250 So I have i, and I'm going to plug it back in here. 788 00:41:05,250 --> 00:41:15,120 And I get constant times log n, plus T of 1, which I know 789 00:41:15,120 --> 00:41:16,390 is theta of 1. 790 00:41:19,560 --> 00:41:22,670 I'm going to make this theta of 1 again. 791 00:41:22,670 --> 00:41:28,900 So theta of 1 times log n, plus theta of 1. 792 00:41:28,900 --> 00:41:30,870 And I know that log n dominates a constant, 793 00:41:30,870 --> 00:41:35,760 so this is going to be theta of log n. 794 00:41:40,655 --> 00:41:41,155 Solved. 795 00:41:45,260 --> 00:41:45,760 OK. 796 00:41:45,760 --> 00:41:48,130 How are we doing? 797 00:41:48,130 --> 00:41:48,750 Yes? 798 00:41:48,750 --> 00:41:51,435 AUDIENCE: In the second to last from the bottom, 799 00:41:51,435 --> 00:41:55,604 I don't understand [INAUDIBLE] of c times log n 800 00:41:55,604 --> 00:41:58,019 replace i by log n. 801 00:41:58,019 --> 00:42:02,350 In the second part of the equation, how did you-- 802 00:42:02,350 --> 00:42:03,100 VICTOR COSTAN: OK. 803 00:42:03,100 --> 00:42:06,430 So in the jump from here to here, where I said 804 00:42:06,430 --> 00:42:08,862 i equals log n-- Right? 805 00:42:08,862 --> 00:42:10,510 AUDIENCE: Yep. 806 00:42:10,510 --> 00:42:11,660 VICTOR COSTAN: OK. 807 00:42:11,660 --> 00:42:14,990 So i becomes log n, c stays c. 808 00:42:14,990 --> 00:42:17,750 You're wondering about this part, right? 809 00:42:17,750 --> 00:42:21,400 So if i is log n, then 2 to the i is n. 810 00:42:21,400 --> 00:42:25,660 So I have T of 1, which I have here. 811 00:42:25,660 --> 00:42:29,150 So I want this to have this happen. 812 00:42:29,150 --> 00:42:34,726 When I'm solving for i here, I set this guy to my base case, 813 00:42:34,726 --> 00:42:36,100 so that I can get this to happen. 814 00:42:39,030 --> 00:42:42,094 The idea here is that you're expanding the recursion 815 00:42:42,094 --> 00:42:43,760 forever, and you know that at some point 816 00:42:43,760 --> 00:42:44,890 you're going to have to stop. 817 00:42:44,890 --> 00:42:45,540 When do you stop? 818 00:42:45,540 --> 00:42:46,630 You stop at the base case. 819 00:42:46,630 --> 00:42:49,208 So you force this to become the base case. 820 00:42:54,576 --> 00:42:56,395 OK? 821 00:42:56,395 --> 00:42:56,895 OK. 822 00:43:02,440 --> 00:43:04,650 Let's do 2D peak finding. 823 00:43:04,650 --> 00:43:06,040 Let's talk about it a little bit. 824 00:43:17,350 --> 00:43:20,420 So 2D peak finding is exactly like 1D peak finding, 825 00:43:20,420 --> 00:43:27,020 except you have a 2D matrix instead of a 1D array. 826 00:43:27,020 --> 00:43:28,610 Let's see if I can do this right. 827 00:43:49,980 --> 00:43:53,210 So suppose this is our matrix. 828 00:43:53,210 --> 00:43:57,910 And suppose these are the rows. 829 00:43:57,910 --> 00:44:05,600 Suppose we have m rows and n columns. 830 00:44:22,940 --> 00:44:26,580 Can anyone remind me what the psuedocode for 2D 831 00:44:26,580 --> 00:44:29,260 peak finding that we talked about in lecture? 832 00:44:29,260 --> 00:44:30,550 So we had two attempts. 833 00:44:30,550 --> 00:44:32,110 The first one was a clear failure. 834 00:44:32,110 --> 00:44:33,440 It didn't work. 835 00:44:33,440 --> 00:44:35,350 The second one worked. 836 00:44:35,350 --> 00:44:38,140 And we talked about its running time a little bit. 837 00:44:38,140 --> 00:44:40,390 And then [INAUDIBLE] asked you to think of a variation 838 00:44:40,390 --> 00:44:41,630 to the second one. 839 00:44:41,630 --> 00:44:45,720 So let's do those straight up second version 840 00:44:45,720 --> 00:44:48,490 of the pseudocode without the variation. 841 00:44:48,490 --> 00:44:49,570 So 2D peak finding. 842 00:45:00,434 --> 00:45:01,350 What is the algorithm? 843 00:45:07,681 --> 00:45:08,180 All right. 844 00:45:08,180 --> 00:45:11,220 One person was not asleep in my section at the end of lecture. 845 00:45:11,220 --> 00:45:13,105 I'm proud. 846 00:45:13,105 --> 00:45:13,646 AUDIENCE: OK. 847 00:45:13,646 --> 00:45:15,128 Well, Which one of us? 848 00:45:15,128 --> 00:45:16,634 We both spoke at the same time. 849 00:45:16,634 --> 00:45:18,675 VICTOR COSTAN: You had your hand up first though. 850 00:45:18,675 --> 00:45:21,590 AUDIENCE: So you just pick a middle of the column. 851 00:45:21,590 --> 00:45:23,448 VICTOR COSTAN: OK. 852 00:45:23,448 --> 00:45:24,876 And then you find the peak there. 853 00:45:29,170 --> 00:45:32,292 AUDIENCE: And then you shift the neighbors, I think. 854 00:45:32,292 --> 00:45:42,686 VICTOR COSTAN: Find peak neighbors. 855 00:45:42,686 --> 00:45:46,630 AUDIENCE: Neighbors in the same room? 856 00:45:46,630 --> 00:45:48,409 AUDIENCE: Yeah, yeah. 857 00:45:48,409 --> 00:45:50,450 VICTOR COSTAN: That's the horizontal line, right? 858 00:45:50,450 --> 00:45:50,950 OK. 859 00:45:50,950 --> 00:45:55,788 So what do you mean by find the peak here? 860 00:45:55,788 --> 00:45:57,600 AUDIENCE: Like within the column? 861 00:45:57,600 --> 00:45:58,391 VICTOR COSTAN: Yes. 862 00:45:58,391 --> 00:46:01,290 So find the peak using what we talked about before? 863 00:46:01,290 --> 00:46:02,240 AUDIENCE: Yeah. 864 00:46:02,240 --> 00:46:02,990 VICTOR COSTAN: OK. 865 00:46:05,600 --> 00:46:10,770 Find 1D peak. 866 00:46:15,570 --> 00:46:16,650 So you're thinking ahead. 867 00:46:16,650 --> 00:46:18,750 You're thinking of the variation that [INAUDIBLE] 868 00:46:18,750 --> 00:46:20,601 asked you guys to think about at home. 869 00:46:20,601 --> 00:46:21,850 I'm glad you thought about it. 870 00:46:21,850 --> 00:46:23,060 That's good. 871 00:46:23,060 --> 00:46:25,169 But what's the one that we had in lecture? 872 00:46:25,169 --> 00:46:26,710 I don't want to think about-- I don't 873 00:46:26,710 --> 00:46:28,251 want to give you guys the answer yet. 874 00:46:28,251 --> 00:46:32,110 So what did we say we were going to do in lecture? 875 00:46:32,110 --> 00:46:32,767 Yes? 876 00:46:32,767 --> 00:46:35,202 AUDIENCE: I think we just picked a spot, 877 00:46:35,202 --> 00:46:37,130 checked the neighbors and kept walking uphill. 878 00:46:37,130 --> 00:46:38,856 That makes sense. 879 00:46:38,856 --> 00:46:41,590 Every time we found a neighbor that was bigger, 880 00:46:41,590 --> 00:46:42,910 we moved to that spot. 881 00:46:42,910 --> 00:46:44,490 Checked its neighbors. 882 00:46:44,490 --> 00:46:45,990 VICTOR COSTAN: So this is the-- This 883 00:46:45,990 --> 00:46:48,430 is the first algorithm that's slow. 884 00:46:48,430 --> 00:46:49,026 It works. 885 00:46:49,026 --> 00:46:50,650 You're going to get the correct answer. 886 00:46:50,650 --> 00:46:54,530 But it's slow because if you have some handily crafted 887 00:46:54,530 --> 00:46:58,570 matrix you can go like this, and you're basically 888 00:46:58,570 --> 00:47:01,400 going through the whole matrix. 889 00:47:01,400 --> 00:47:01,917 Yes? 890 00:47:01,917 --> 00:47:04,000 AUDIENCE: I think there's like an algorithm that's 891 00:47:04,000 --> 00:47:06,462 find a-- look through the column for its maximum, 892 00:47:06,462 --> 00:47:08,300 and then look to either side. 893 00:47:08,300 --> 00:47:09,100 VICTOR COSTAN: OK. 894 00:47:09,100 --> 00:47:10,940 So instead of finding a 1D peak, we're 895 00:47:10,940 --> 00:47:12,825 going to find the maximum value. 896 00:47:16,161 --> 00:47:16,660 OK. 897 00:47:16,660 --> 00:47:20,480 And this is correct, and let's look at its running time. 898 00:47:25,034 --> 00:47:26,200 Does anyone want to help me? 899 00:47:26,200 --> 00:47:32,050 So this is T of n m. 900 00:47:32,050 --> 00:47:32,550 Right? 901 00:47:32,550 --> 00:47:33,549 Then rows, then columns. 902 00:47:36,280 --> 00:47:39,050 So what's the running time for the first step? 903 00:47:39,050 --> 00:47:39,870 Come on. 904 00:47:39,870 --> 00:47:41,030 This is the easy one. 905 00:47:41,030 --> 00:47:42,390 Yes? 906 00:47:42,390 --> 00:47:42,950 The constant. 907 00:47:42,950 --> 00:47:43,450 Perfect. 908 00:47:46,180 --> 00:47:47,740 OK. 909 00:47:47,740 --> 00:47:49,650 What's the running time for finding 910 00:47:49,650 --> 00:47:52,087 a maximum amount of n numbers? 911 00:47:52,087 --> 00:47:53,460 AUDIENCE: n. 912 00:47:53,460 --> 00:47:56,000 VICTOR COSTAN: OK. 913 00:47:56,000 --> 00:47:57,830 Perfect. 914 00:47:57,830 --> 00:48:01,820 And suppose we do that here? 915 00:48:01,820 --> 00:48:03,950 Suppose these rows don't exist. 916 00:48:03,950 --> 00:48:06,130 Suppose this is my entire matrix. 917 00:48:06,130 --> 00:48:09,230 And I'm here, and I found the maximum. 918 00:48:09,230 --> 00:48:10,180 What do I do? 919 00:48:10,180 --> 00:48:12,200 I look left, I look right. 920 00:48:12,200 --> 00:48:14,450 If both of them are smaller, then it 921 00:48:14,450 --> 00:48:16,260 means I found my peak, because I know 922 00:48:16,260 --> 00:48:18,960 that number was a maximum in my column. 923 00:48:18,960 --> 00:48:23,290 Now if that's not the case, if I find a number that's bigger, 924 00:48:23,290 --> 00:48:25,330 I'll go in that direction. 925 00:48:25,330 --> 00:48:28,290 Does that sound reasonable? 926 00:48:28,290 --> 00:48:28,790 OK. 927 00:48:28,790 --> 00:48:34,660 So maximum for my column, and I'm going to go right. 928 00:48:34,660 --> 00:48:41,020 So check neighbors, and I have the same two conditions there. 929 00:48:41,020 --> 00:48:42,325 If it's a peak, then I'm happy. 930 00:48:45,100 --> 00:48:48,430 Otherwise recurse. 931 00:48:48,430 --> 00:48:50,850 So what's the running time if I have to recurse? 932 00:48:50,850 --> 00:48:54,660 How does the new problem look like? 933 00:48:54,660 --> 00:48:55,800 It's an identical problem. 934 00:48:55,800 --> 00:49:01,450 I still have a matrix, except-- What changed? 935 00:49:01,450 --> 00:49:07,840 So suppose I got here-- and this is the middle of my matrix-- 936 00:49:07,840 --> 00:49:09,810 and I see that this guy is bigger, 937 00:49:09,810 --> 00:49:12,950 so I'm going to recurse to the right side. 938 00:49:12,950 --> 00:49:14,725 What happened to my problem size? 939 00:49:17,590 --> 00:49:18,090 OK? 940 00:49:18,090 --> 00:49:20,090 AUDIENCE: Sorry I'm not answering your question, 941 00:49:20,090 --> 00:49:23,874 but shouldn't find x be theta of m, not n? 942 00:49:23,874 --> 00:49:24,708 Is that [INAUDIBLE]? 943 00:49:24,708 --> 00:49:25,790 VICTOR COSTAN: Let me see. 944 00:49:25,790 --> 00:49:26,710 AUDIENCE: I can't tell 945 00:49:26,710 --> 00:49:28,710 VICTOR COSTAN: You are right. 946 00:49:28,710 --> 00:49:31,380 Thank you for paying attention. 947 00:49:31,380 --> 00:49:32,920 Theta m. 948 00:49:32,920 --> 00:49:33,480 Sweet. 949 00:49:33,480 --> 00:49:34,980 That would have tripped me up later. 950 00:49:34,980 --> 00:49:35,650 Thanks for saving me. 951 00:49:35,650 --> 00:49:36,098 AUDIENCE: OK. 952 00:49:36,098 --> 00:49:37,181 So what was your question? 953 00:49:37,181 --> 00:49:38,126 Sorry. 954 00:49:38,126 --> 00:49:40,500 VICTOR COSTAN: So the question is, if I have to recurse-- 955 00:49:40,500 --> 00:49:41,460 AUDIENCE: Uh-huh. 956 00:49:41,460 --> 00:49:43,168 VICTOR COSTAN: --what's the running time? 957 00:49:46,800 --> 00:49:51,100 So I'm-- If I have to recurse, I'm in this array that was 958 00:49:51,100 --> 00:49:55,500 originally n by m, and I'm only going to look at the right 959 00:49:55,500 --> 00:49:56,000 side. 960 00:50:00,310 --> 00:50:06,110 How many these-- how many rows do I have in the [INAUDIBLE]? 961 00:50:06,110 --> 00:50:08,584 Start to the easy one. 962 00:50:08,584 --> 00:50:09,299 AUDIENCE: m. 963 00:50:09,299 --> 00:50:10,340 VICTOR COSTAN: All right. 964 00:50:10,340 --> 00:50:13,010 Same as before. 965 00:50:13,010 --> 00:50:14,340 How many columns? 966 00:50:14,340 --> 00:50:15,210 AUDIENCE: n over 2. 967 00:50:15,210 --> 00:50:16,210 VICTOR COSTAN: n over 2. 968 00:50:18,820 --> 00:50:20,300 Sweet. 969 00:50:20,300 --> 00:50:24,090 So there recursion for this is theta of 1 970 00:50:24,090 --> 00:50:32,830 plus theta of m plus T of n over 2 and m. 971 00:50:35,350 --> 00:50:38,870 I know that a constant can be absorbed by this other guy, 972 00:50:38,870 --> 00:50:47,122 so it's theta of m plus T of n over 2 m. 973 00:50:47,122 --> 00:50:48,080 How would I solve this? 974 00:50:54,370 --> 00:50:55,870 What's the first thing that we need? 975 00:50:58,380 --> 00:50:58,880 Yes? 976 00:50:58,880 --> 00:51:00,620 AUDIENCE: Need to figure out the base case. 977 00:51:00,620 --> 00:51:01,180 VICTOR COSTAN: All right. 978 00:51:01,180 --> 00:51:01,540 Excellent. 979 00:51:01,540 --> 00:51:01,950 Base case. 980 00:51:01,950 --> 00:51:03,658 What's the best-- What was the base case? 981 00:51:08,400 --> 00:51:10,866 AUDIENCE: T of 1 comma T of n-- m. 982 00:51:10,866 --> 00:51:11,597 Sorry. 983 00:51:11,597 --> 00:51:12,430 VICTOR COSTAN: Nice. 984 00:51:12,430 --> 00:51:14,850 AUDIENCE: Equal to O of m. 985 00:51:17,290 --> 00:51:18,040 VICTOR COSTAN: OK. 986 00:51:21,260 --> 00:51:23,020 Sweet. 987 00:51:23,020 --> 00:51:26,180 So if I only have one column, then I 988 00:51:26,180 --> 00:51:28,770 can do-- I can find the maximum and I know for sure 989 00:51:28,770 --> 00:51:29,880 that's going to be a peak. 990 00:51:29,880 --> 00:51:31,060 Job done. 991 00:51:31,060 --> 00:51:32,990 That's my base case. 992 00:51:32,990 --> 00:51:35,680 So I'm going to reduce the problem size in half, 993 00:51:35,680 --> 00:51:37,240 until I get to a single column. 994 00:51:37,240 --> 00:51:40,800 Then I find the peak there, and I'm happy. 995 00:51:40,800 --> 00:51:42,298 How would I solve this recurrence? 996 00:51:46,552 --> 00:51:48,010 It's a one minute answer, because I 997 00:51:48,010 --> 00:51:51,245 think that's exactly how much time we have, as a hint. 998 00:51:57,680 --> 00:51:59,486 Come on guys, this looks familiar. 999 00:51:59,486 --> 00:51:59,985 Yes? 1000 00:51:59,985 --> 00:52:01,905 AUDIENCE:Going to be something like the recurrence over 1001 00:52:01,905 --> 00:52:02,390 there-- 1002 00:52:02,390 --> 00:52:03,348 VICTOR COSTAN: Exactly. 1003 00:52:03,348 --> 00:52:04,782 AUDIENCE: [INAUDIBLE] of just n. 1004 00:52:04,782 --> 00:52:05,740 VICTOR COSTAN: Exactly. 1005 00:52:05,740 --> 00:52:07,230 m never changes, right? 1006 00:52:07,230 --> 00:52:09,990 So this is basically a function of n. 1007 00:52:09,990 --> 00:52:13,280 So if m never changes, and I pretend that it's a constant, 1008 00:52:13,280 --> 00:52:15,950 this looks exactly like the recursion-- the recurrence 1009 00:52:15,950 --> 00:52:17,340 that we solve before. 1010 00:52:17,340 --> 00:52:22,920 T of n is a constant, plus T of n over 2. 1011 00:52:26,000 --> 00:52:29,770 So I'm going to get that constant, 1012 00:52:29,770 --> 00:52:35,380 multiply it by log n, which is the solution that I had before. 1013 00:52:35,380 --> 00:52:42,211 So the running time is theta of m log n. 1014 00:52:42,211 --> 00:52:42,710 All right. 1015 00:52:42,710 --> 00:52:46,250 I think I have 30 seconds, so what if I would choose-- What 1016 00:52:46,250 --> 00:52:48,870 if I would try to do this algorithm instead? 1017 00:52:48,870 --> 00:52:50,740 It's another one minute question. 1018 00:52:50,740 --> 00:52:51,740 So I guess I'm cheating. 1019 00:52:51,740 --> 00:52:53,280 I'm taking time away. 1020 00:52:53,280 --> 00:52:54,220 Sorry. 1021 00:52:54,220 --> 00:52:56,800 So what if I would be finding a 1D 1022 00:52:56,800 --> 00:52:58,557 peak instead of finding a maximum? 1023 00:53:01,315 --> 00:53:03,940 I'm looking at the running time, because if the running time is 1024 00:53:03,940 --> 00:53:05,648 going to be the same, it's not even worth 1025 00:53:05,648 --> 00:53:07,400 thinking if it's correct or not. 1026 00:53:07,400 --> 00:53:08,950 If it's a more complicated algorithm 1027 00:53:08,950 --> 00:53:11,350 with the same running time, why care about it? 1028 00:53:11,350 --> 00:53:13,901 So is the running time going to be better? 1029 00:53:13,901 --> 00:53:14,400 OK. 1030 00:53:14,400 --> 00:53:15,900 I heard a nod. 1031 00:53:15,900 --> 00:53:17,094 Why? 1032 00:53:17,094 --> 00:53:19,589 AUDIENCE: Won't it be log m times log n? 1033 00:53:19,589 --> 00:53:20,630 VICTOR COSTAN: All right. 1034 00:53:20,630 --> 00:53:23,240 So 1D peak is exactly what we have there. 1035 00:53:23,240 --> 00:53:24,380 Thank you. 1036 00:53:24,380 --> 00:53:29,230 1D peak is exactly like you said, log m. 1037 00:53:29,230 --> 00:53:32,480 So this constant here becomes this constant here, 1038 00:53:32,480 --> 00:53:35,550 so the algorithm would be a lot faster. 1039 00:53:35,550 --> 00:53:38,660 As an order of magnitude idea, if your m and n are one million 1040 00:53:38,660 --> 00:53:40,900 each, then log n is 20. 1041 00:53:40,900 --> 00:53:43,290 So that's a 50 time max improvement. 1042 00:53:43,290 --> 00:53:46,900 And these are real input sizes you might get. 1043 00:53:46,900 --> 00:53:47,520 And that's it. 1044 00:53:47,520 --> 00:53:49,530 Thank you guys.