Recitation 5: Recursion Trees, Binary Search Trees

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

PROFESSOR: If you guys want me to cover anything in particular, is there anything you didn't understand in lecture? In the last section, I covered the recursion trees because they will be on the Pset, and people said they were a bit unclear, so we can do that and cover less of the stuff that I have here. Or if there's anything else, you can tell me what you want. So there I cover recursion trees because someone said, hey, can you go over that again? Is there any pain points? No? OK.

So then I'm going to give you the same choice that I gave to people last time, and that is we can go over recursion trees again, but if I do that, then I won't have time to go over the code for deleting a node from a binary search tree. So we'll go through the theory and you guys will have to go through the code on your own. But instead, we'll go over recursion trees again and remember how you solve a recurrence using recursion trees. The alternative is we don't do that and we complete the deletions part.

AUDIENCE: I feel like covering deletions, since we didn't do that in lecture, that would probably be more helpful.

PROFESSOR: Let's take a vote. Who wants to do deletions in painstaking detail? So deletions and not recursion? Who wants to do recursion trees and not deletion?

AUDIENCE: It's about equal.

PROFESSOR: It's equal and nobody cares. I'm really sad.

AUDIENCE: Let's do both in half detail.

PROFESSOR: OK, sure. Who remembers merge sort? What does merge sort do really quick?

AUDIENCE: It takes some sort of unsorted array, splits it in half, and then continually splits it, and then once it finally gets to the point where you have arrays of two elements, then it sorts them, and then sorts those, and then sorts those. It's a fun thing. And then it merges [INAUDIBLE].

PROFESSOR: That's so much code. I don't like to write much code because for every line of code that you write, you might have a bug in it, so I like to write less code. So the way I do it is when I get to an array size of one element, I know it's already sorted.

So merge sort. You have an array, it's unsorted. Split it into two halves, call merge sort on each half, assume that magically, they're going to come back sorted, and then you merge the sorted halves. How much time does merging take? OK. So the recursion for the running time of merge sort?

AUDIENCE: Why does it take n time? Just too large?

AUDIENCE: Isn't it the finger thing where you take each element, and you're like, this one, is that greater or less than, then you put it in the array. So you get--

PROFESSOR: Please take my word for it that it's order n.

AUDIENCE: I'll explain it and then I'll be confused.

PROFESSOR: OK, so order n. What's the recursion? Don't give me the solution because then I can't do the trees anymore, so give me the recursion before it's solved. Give me the recurrence formula. So it starts with T of N, right?

AUDIENCE: It starts with N over 2 plus N, I think.

PROFESSOR: Perfect. So you take the array, you split it into two, you call merge sort on the two halves of the arrays. So you call merge sort twice. That's why you have a 2 here. The 2 matters. Without it, you get a different answer. And when you call it, the arrays that you give it are half the size, and then merge takes order and time. Splitting depends on what you're using to store your arrays. Can be constant time or it can be order N. So the time won't change because of split.

How do we solve this recurrence? The recursion tree method says that we're going to draw a call graph. So we start out with a call to merge sort with an array of size N. Then it's going to call merge sort again, but after the array is split. So it's going to call merge sort twice, size is N over 2. This guy gets an array of N over 2, calls merge sort. Two arrays, sizes N over 4, N over 4. This does the same.

So this goes on forever and ever and ever until at some point we reach our base case. So we're going to have a bunch of calls here where the array size is? What's our base case? 1. Excellent. So this is the call graph for merge sort, and let's put the base case here so we know what we're talking about. T of 1 is theta 1.

Now inside the nodes, we're going to put the cost for each call without counting the sub-call, so the children here. That's this guy here, except instead of order N, I will write CN. Remember how sometimes we use CN instead of the order of notation? The reason we do that is if I put in the asymptotic notation, then we're going to be tempted to sum them up. You're allowed to sum terms using asymptotic notation as long as there's a finite number of them, but here, it turns out there's an infinite number of them. Also, if you go this way, you can never go wrong. You always get the right answer, so that's why we switch from order N to CN.

In order to merge sort an array of size N, we're going to merge sort two arrays of size N over 2 and then spend CN time on doing the merge. What are the costs here? To sort an array of N over 2, what's the cost outside the cost to merge?

AUDIENCE: C of N over 2.

PROFESSOR: Perfect. C times N over 2. C times N over 2. How about here?

AUDIENCE: C times N over 4.

PROFESSOR: Perfect. CN over 4. My nodes are really ugly. I should have drawn them like this from the beginning. CN over 4. There you go. How about down here?

AUDIENCE: C of N over 2 to the i.

PROFESSOR: You're going on step ahead. We'll do that right next.

AUDIENCE: C of N over log N, right? Because they're log N levels, so--

PROFESSOR: Let's not worry about the number of levels. You're ruining my steps. I was going to get to that two steps after this.

AUDIENCE: Is it just C?

PROFESSOR: Yep. So array size is 1, right? So the cost is C. C, C, C, C. OK, you guys got it if you're thinking of levels already. The next thing I want to do is I want to figure out how many levels I have in this tree. Why do I care about that?

The answer for T of N is the sum of all these costs in here because the cost of merge sorting an array of size N is the merge sort plus the costs for sorting the two arrays. And the nodes here keep track of all the time spent in recursive sub-calls, so if we can add up everything up, we have the answer to T of N. It turns out the easiest way to do that is to sum up the cost at each level because the costs are this guy copied over here. For a level, they tend to be the same, so it's reasonably easy to add them up, except in order to be able to add those up, you have to know how many levels you have.

So how do I know how many levels I have? Someone already told me log N. How do I get to that log N? So when I get to the bottommost level, the number has to be 1, the number next to the node, because that's my base case. When I have a one element array, it's sorted, I'm done. I return.

So I can say that for each level, the number next to the node is something as a function of L. Here, I'm going to say that this is N over 1, which is N over 2 to the 0 power. And this is N over 2, so it's N over 2 to the first power. This is N over 2 to the second, and so on and so forth. It might not be obvious if you only have two levels.

I don't want to draw a lot on the board because I don't have a lot of space and I'd get my nodes all messed into each other. If it takes more than two levels to see the pattern, go for it. Expand for three levels, four levels, five levels, whatever it takes to get it right on a Pset or on a test. So you see the pattern, then you write the formula for the node size at the level.

And assuming this pattern holds, we see that the size of a node at level l, the size is 2 N over 2 to the l minus 1. Fair enough? You can say N over 2 to the l, and forget that there's a minus 1, and then the asymptotics will save you, so it's no big deal, but this is the real number.

So that means that at the bottommost level, at level l, this size is going to be 1. N over 2 to the l minus 1 equals 1. So now this is an equation, so I can solve for l. I pull this on the right side, N equals 2 to the l minus 1, so l minus 1 equals-- anyone? The inverse of an exponential?

AUDIENCE: I wasn't paying attention. Sorry.

AUDIENCE: Log N.

PROFESSOR: The inverse of an exponential is a logarithm. Keep that in mind for solving 6.006 problems. l minus 1 is log N so l is log n plus 1, roughly log n. I could use log n plus 1 and go through the math. It's a bit more painful and, because we're using asymptotics, it doesn't really matter.

So now we know how many levels we have. Let's see what's the cost at the level. So all the calls at a certain level, what's the sum of the costs? For this level, what's the cost? CN. And That was the easy question. Just the root, right?

How about this level? Because I have two nodes, the cost in each node is CN over 2. How about this level? Four levels, each level CN over 4. How about the bottom level?

AUDIENCE: CN.

PROFESSOR: Why is it CN?

AUDIENCE: Because there are N arrays of size 1.

PROFESSOR: N arrays of size 1. Excellent. A cute argument I heard once is you start out with N, you split it into N over 2 and N over 2. Then you split this guy in N over 4, N over 4, so this is like conservation of mass. If you start with N and here, you don't end up with N total, then you lost some element somewhere on the way.

So CN. CN, CN, CN, CN. I think I see a pattern. I think it's reasonable to say that for every level, it's CN. And if you write the proof, you can prove that by using math instead of waving hands. So CN times the number of levels, right? The answer for this guy is C of N is CN times l. What's l?

AUDIENCE: N log N.

PROFESSOR: Roughly. OK So order of N log N. C becomes order of, l is order of log N, N stays the same. Any questions? Are people getting it or did I confuse you even more?

AUDIENCE: We got it.

PROFESSOR: OK, sweet. Thank you for the encouragement. So this gets you through problem one of Pset 2. So in this case, the tree is nicely balanced. The cost at each level is the same. When [INAUDIBLE] talked about recursion trees in lectures, he showed two more trees, one where pretty much all the cost was up here-- the cost of the children was negligible-- and one tree where all the cost was concentrated here, so the cost of all the inner nodes was negligible and the leaves were doing all the real work. So don't be scared if your costs aren't the same. Just sum them up and you'll get to the right answer.

Now I'm going to talk about binary search trees, except I will make a five minute general talk about data structures before I do that. So we use the term "data structures." I think we covered it well, and I want to give you a couple of tips for dealing with them on Pset 1.

A data structure is a bunch of algorithms that help you store and then retrieve information. You have two types of algorithms. You have queries, and you have updates.

You start out with an empty data structure, like an empty binary search tree or an empty list, and then you throw some data at it. That's when you update it. Then you ask it some questions, and that's when you query it. Then maybe you throw more data at it, so you do more updates, and you ask more questions, so you do more queries. What are the queries and the updates for the binary search trees that we talked about in lecture?

AUDIENCE: A query would be like, what's your right child, what's your left child?

PROFESSOR: So that's for a node.

AUDIENCE: What are you looking for?

PROFESSOR: I'm looking for something for the entire infrastructure. So for the entire tree, what's a question that you would ask the tree?

PROFESSOR: Max.

PROFESSOR: OK. Min.

AUDIENCE: Next larger.

PROFESSOR: Next larger. Are you looking at the nodes?

AUDIENCE: Is there an are you balanced question?

PROFESSOR: Well, I would say that the most popular operation in a binary search tree is Search, which looks for-- we call it Find in the code because most code implementations call it Find nowadays, but binary search tree. What are you going to do in it? You search for a value. That's why it has the Search in binary search.

So queries are operations where you ask questions to the data structure and it doesn't change. How about updates? What did we learn for updates?

AUDIENCE: Insert.

PROFESSOR: Excellent. So Insert was covered in lecture, and we're doing Delete today. So data structures have this property that's called the representation invariant, RI, or Rep Invariant. Actually, before I get there, the rep invariant says that the data in the data structures is organized in this way, and as long as it's organized in this way, the data structure functions correctly. Can someone guess for a sorted array what's the representation invariant?

AUDIENCE: It can mean sorted.

PROFESSOR: Yeah. A sorted array should be sorted. Sounds like a very good rep invariant. So the elements should be stored an array. Every element should be smaller than any element after it. And as long as the rep invariant holds, so as long as elements are stored in the right way in the data structure, the queries will return the right results. If the rep invariant doesn't hold, then God knows what's going to happen. What can you do in a storage array as long as the rep invariant holds? Sorted array. What's the reason why I would have a sorted array? What can I do that's fast in a sorted array?

AUDIENCE: Min and Max.

PROFESSOR: I can do that very fast. That's good. What's the running time?

AUDIENCE: A constant.

PROFESSOR: Perfect. Min you look at the beginning, Max you look at the end. Yes?

AUDIENCE: Binary search.

PROFESSOR: Binary search. That's the other reason for that. So binary search runs in order log N time, doesn't have to look at most of the array, tells you whether an element is there are not. Now, what if the array is unsorted? Will the algorithm work? It might say something isn't there when it actually is there. You can do binary search on a non-sorted array. So if the rep invariant doesn't hold, your queries might give you a wrong answer. How about updates? How do you search something in a sorted list?

AUDIENCE: You find where it should go and you move everything.

PROFESSOR: Yep. So you have to move everything, make room for it, and put it there so that the array is still sorted at the end. You can't just append things at the end, even though that would be faster and lazier and less code. When you do an update to a data structure, you have to make sure that the rep invariant still holds at the end. Sort of a correctness proof for an update algorithm says that if the rep invariant holds at the beginning, the rep invariant is guaranteed to hold at the end.

Why do we care about this rep invariant stuff? Suppose you have a problem, say on the next Pset, that asks you to find the place that's slow in your code and then speed it up. And suppose you recognize the data structure there, and you say that's inefficient, and you want to implement another data structure that would be more efficient.

You're going to implement it. You might have bugs in an update. How do you find the bugs? Queries give you the wrong answers. You might do queries a long time after you do updates, and you're not going to know which update failed.

The right way to do this is you implement the method called Check RI-- that's what I call it-- so check the representation invariant. And that method walks through the entire data structure and make sure that the rep invariant holds, and if it doesn't, it raises an exception because you know that whatever you try to do from there is not going to work, so there's no reason to keep going.

So at the end of every update, you add a call to this Check RI method until you're sure that your code is correct. And after you're done debugging your code, you remove this method and you submit the code. Why do I want to remove the method? It might be painfully slow and inefficient, much slower than the actual queries and updates.

For example, let's take a heap. Do people remember heaps from lecture? What's the query for a heap? Say you have a max heap. What's a query?

AUDIENCE: Where's the max?

PROFESSOR: OK, cool. So for a max heap, a query would be max. Running time?

AUDIENCE: Constant.

PROFESSOR: Perfect. Constant. What do you do? Look at the top?

AUDIENCE: Yeah, exactly.

PROFESSOR: OK. Sweet. So what are the two popular updates in a max heap?

AUDIENCE: There would be Insert as well.

PROFESSOR: OK. Insert. And did we teach you general delete? Usually Extract Max is simpler. That's all you need. What's the running time for Insert? Do people remember heaps?

AUDIENCE: I think it was per N, but I'm not completely sure.

PROFESSOR: Anyone else? It's not. Life would be bad if it would be N.

AUDIENCE: N squared?

PROFESSOR: No. It's better than N, so you guys are doing a binary search over the few running times that I gave you earlier.

AUDIENCE: [INAUDIBLE] add to the N, and then you compare your neighbor, and then you [INAUDIBLE].

AUDIENCE: If it's an array, there isn't--

PROFESSOR: So conceptually, a heap looks like this. And yeah, it becomes an array eventually, but let's look at it this way. It is a full binary tree. Binary tree means that each node has at most two children, and full means that every level except for the last level is completely populated. So every internal node has exactly two children, and in here, every node except for some nodes and then some nodes after it will not have. Everything to the left is fully populated, and then at some point, you stop having children.

It turns out that this is easy to store in an array, but I will not go over that. Instead, I want to go over inserting. What's the rep invariant for a heap?

AUDIENCE: The max in the top, right? Well, for max heap, and then the two children are less than the next node.

PROFESSOR: All right. So the guy here has to be bigger than these guys, then the guy here has to be bigger than these guys, and so on and so forth. And if you use induction, you can prove that if this is bigger than this, it has to be bigger than these guys, and bigger than these guys, and bigger than everything, and it's a max. That's the reason why we have that rep invariant.

So the way we insert a node is we add it at the bottom, the only place where we could add it. And then if this guy is bigger than this guy, the rep invariant is violated, so we swap them in order to fix that. Now the guy is here. If this is bigger than this, we do another swap. If this is bigger than this, we do another swap. So you're going to go from the bottom of the heap potentially all the way up to the root. So the running time of insert is order of the height of the heap.

Now, the heap is a full binary tree. I said "full." I keep saying "full." The reason I care about full is that the full binary tree is guaranteed to have a height of log N. It's always log N, where N is the number of nodes. So inserting in a heap takes log N.

AUDIENCE: I have a question. Didn't they say that because it's in an array, then to find it-- oh no, I guess because you can still do the swaps.

PROFESSOR: You can still do the swaps when you have it serialized in an array. You know that given an item's index, the parent is that index divided by 2. So you add an element at the end of the array, and then you know what the parent is, and then you keep swapping and swapping and swapping towards the [INAUDIBLE].

AUDIENCE: You don't ever have to put it in and shift everything over. You're only swapping.

PROFESSOR: Yep. You only swap. That's important. Thanks for asking. That's important. So log N. Extract max, take my word for it, also log N. What's the running time for checking the invariant in a heap? So to make sure that this guy is a heap, if I had numbers here, what would you have to do?

AUDIENCE: You'd have to look at every node.

PROFESSOR: Yep. So running time?

AUDIENCE: Theta of N.

PROFESSOR: Yep. So if I'm going to submit code for a heap where the operations are our order of log N, or order 1, but then each of these calls Check RI, that's going to be painfully slow because I'm making the updates be order N instead of log N. So you're putting Check RI calls in every update. You debug your code. When you make sure it's correct, you remove those, and then you submit the Pset. Make sense? Sweet. And we looked a little bit at heaps, which is good.

Binary search trees. So a binary tree is a tree where every node has at most two children. When we code this up, we represent a node as a Python object, and for a node, we keep track of the left child, of the right child, parent, and then this is a hollow tree. It's not very useful. This becomes useful when you start putting keys in the nodes so that you can find them and do other things with them. So each node has a key.

Let me draw a binary search tree. Can people see this? So this is a binary tree. Can someone say something a bit more specific about it?

AUDIENCE: It's unbalanced.

PROFESSOR: OK. It's imbalanced. So that means that finding things all the way at the bottom is going to be expensive. What else? So I said it's a binary tree. Give me something more specific.

So binary tree just means that every node has two children. There's a bit more structure in this guy. So if I look at the root, if I look at 23, all the nodes to the left are smaller. All the nodes to the right are bigger. Now, if I look at 8, all the nodes to the left are smaller, all the nodes to the right are greater.

This additional rep invariant defines a binary search tree. This is what we talked about in class. BST. Why would I want to have this rep invariant? It sounds like a pain to maintain nodes with all these ordering constraints. What's the advantage of doing that?

AUDIENCE: Search is fast.

PROFESSOR: Yep. Search is fast. How do I do search?

If you're looking for 42 or for 16, you'd be like, oh, it's less than 23. I'll get on this path.

PROFESSOR: So start at the root, compare my key to the root. If it's smaller, go left. If it's bigger, go right. Then keep doing that until I arrive somewhere or until I arrive at a dead end if I'm looking for 14. This is a lot like binary search. Binary search in an array, you look at the middle. If your key is smaller, go left. If your key is bigger, then go right.

Let's look at the code for a little bit. Look at the BST Node Class, and you'll see that it has the fields that we have up here. And look at the Find method, and this is pretty much the binary search code. Lines 8 and 9 have the return condition when you're happy and you found the key, and then line 10 compares the key that you're looking for with the key in the node that you're at, and then lines 11, 14, 16, and 19 are pretty much copy pasted, except one of them deals with the left case, the other one deals with the right case. What is the running time for Find?

AUDIENCE: Wouldn't it be log N, right?

PROFESSOR: I wish. If this is all you have to do to get log N, then I would have to write a lot less code. So not quite log N. We will have to go through next lecture to get to log N. Until then, what's the running time?

AUDIENCE: Order h.

PROFESSOR: Yep. So you told me at the beginning it's unbalanced.

AUDIENCE: Yeah.

PROFESSOR: So then it's not going to be fast. OK, so order h. The reason why we care about h, and the reason we don't say order N, is because next lecture after we learn how to balance a tree, there's some magic that you can do to these binary search trees to guarantee that the height is order of log N. And then we'll go through all the running times that we have and replace h with log N.

Now, it happens that in this case, if you would have told me order N, I couldn't argue with you because worst case, searches are order N. Can someone give me a binary search tree that exposes this degenerate case? Yes?

AUDIENCE: If it's completely unbalanced and every node is greater than the parent nodes.

PROFESSOR: So give me some inserts that create it.

AUDIENCE: Insert 5.

PROFESSOR: 5.

AUDIENCE: Insert 10.

PROFESSOR: 10.

AUDIENCE: Insert 15.

PROFESSOR: 15.

AUDIENCE: Insert 20.

PROFESSOR: Yep. And I could keep going. I could say, 1, 2, 3, 4, 5. I could say 5, 10, 15. As long as these keep growing, this is basically going to be a list, so searching is order N. This is a degenerate case.

Turns out it doesn't happen too often in practice. If you have random data, the height will be roughly log N. But in order to avoid those degenerate cases, we'll be doing balanced trees later on. So we covered Find. We know it's order h. How do you insert, really quickly?

AUDIENCE: Do you mean in searching when it's balanced or unbalanced?

PROFESSOR: This guy. So the trees look exactly the same. If it's balanced, it's going to look more like that than like this. Actually, this is balanced. This is perfectly unbalanced. This is somewhere in the middle. If it's balanced, it's just going to look more like this, but it's still a binary search tree. How would you insert a node? Yes?

AUDIENCE: Can't you start at the root and find your way down, and then the first open child that you see that's applicable to your element, state it then?

PROFESSOR: Yep. So if I wanted to insert 14, which way do I go?

AUDIENCE: So you'd look at 23, and you'd say, it's less than 23, go left. You'd look at 8. You'd say, it's greater than 8. You'd go right. Look at 16. You'd say it's less, so you go left. 15, it's less. Then you have an open spot so you stick it there.

PROFESSOR: Excellent. Thank you. Yes?

AUDIENCE: I have a question. What if we want to insert 5? Then--

PROFESSOR: So if you want to insert who?

AUDIENCE: 5. Or actually no, we can't. I'm thinking, is there any case in which need to move a node?

PROFESSOR: How would you insert 5? Let's see. What would you do for 5?

AUDIENCE: For 5, then we'd insert it to the right of 4, right?

PROFESSOR: Smaller, smaller, greater, 5. Right?

AUDIENCE: So there would be no case in which we'd need to swap nodes or something?

PROFESSOR: No. You're thinking ahead. We'll talk about that a little later when we get to deleting. As long as you follow a path in the tree, the path that finding would get you to, as soon as you hit a dead end, that's where your node belongs. Because you know next time you're going to search for it, the search is going to follow that path and find the node. Yes?

AUDIENCE: If you have values are the same, like two nodes at the same number, does it matter which side you put it on?

PROFESSOR: You don't.

AUDIENCE: Oh, I see. It's more like you would only have four 1's in the tree.

PROFESSOR: Yes. So if you're trying to store keys and values, then what you'd have to do if you want to allow multiple values for the same key is you have a linked list going off of this, which node becomes an array of values aside from the key. Smart question. Thank you. That trips you up every time you do actual code, so that's the right question to ask yourself when you're implementing this. Will I have duplicates? How do I handle them? We don't. We take the easy way out.

So if you look at Insert, on the next page, you will see that the code is pretty much the Find code copy pasted, except when Self Left is None or Self Right is None, instead of returning, it creates a new node. Does that make sense to people? All right.

So Delete is going to be the hardest operation for today. Before we do that, let's do a warm up operation. Let's say I want to implement Find Next Larger, also called Successor in some implementations. So I have a node. Say I have node 8, and I want to find the next key in the tree that's strictly larger than 8 but smaller than anything else. So if I would take these nodes and write them down in order, I want to find the element that would go right after it. How do I do that? Don't cheat. Don't look at the code, or make my life easier and do searches.

AUDIENCE: Go down one to the right, and you try to get down left as far as you can.

PROFESSOR: OK. Very good. So I have a node, and it has some subtree here, so I can go to the right here, I can go all the way left. We have an operation that does this, and it's called Min for a tree. In order to find the minimum in a binary search tree, you keep going left. For example, in this case, you get 4, which is good.

So the way you would code this up is if you have Min, you go to the right if you can, and then you call Min on the subtree. And you can see that lines 3 and 4 do exactly that. Good guess. But you can line 1 says case one, so you have the right answer for one case.

Now we have to handle more difficult cases. What if instead, I go down a bunch of nodes, and I want to find the successor for this guy, for example, and there's nothing here. What do I do? So if I want to find the successor for 8, what do I do? Sorry. It has an answer. What if I want to find the successor for 4?

AUDIENCE: Go up one.

PROFESSOR: OK. Go up one. Why does that work?

AUDIENCE: You know it's going to be greater.

PROFESSOR: So I'm going up right. So I know that everything here is guaranteed to be smaller, everything here is guaranteed to be greater than this guy. This guy is up right, so this is guaranteed to be greater than this, and everything here is guaranteed to be greater than this, and so on and so forth for the entire tree. So if I go up right, I'm happy. I definitely found my answer. Now, what if I have something that looks like this, and I want to find the successor for this guy?

AUDIENCE: There is none.

PROFESSOR: In this case, there is none if there's nothing else here. What if I have this, but then I have this? So I came down this way.

AUDIENCE: Are you saying you're calling on that last node?

PROFESSOR: Yep.

AUDIENCE: Find the larger? I guess you'd just trace back up.

PROFESSOR: And where do I stop?

AUDIENCE: It affects the tree, so you go up one from there. You don't stop there.

PROFESSOR: Why can't I stop here?

AUDIENCE: Because you know that that-- not necessarily.

AUDIENCE: You know that everything in that long branch right there is less than that node [INAUDIBLE].

PROFESSOR: This is to the left of this guy, so this guy has to be greater than everything here, and then you can repeat the argument that we had before. So here, we could stop right away because we could branch left. In this case, you have to go up until you're able to go left and up. If you get to the root, then what happened? Then we're in this case, and you have no successor.

So take a look at the code. The next larger, lines 1 through 9. Case two, 6 through 8, does exactly that. If I can't go to my right and find the tree there, then I go up through my parent chain, and as long as I have to go up to the left, so as long as I'm the right child of my parent, I have to keep going. The moment I find the parent where I'm the left child, I stop. That's my successor.

What if I would have to find the predecessor instead? So the element that's smaller than me but bigger than everything else in the tree. What would I do?

AUDIENCE: It's just the opposite.

PROFESSOR: Just the opposite. So how do I do the opposite?

AUDIENCE: You can take the max of the left side tree, or traverse up, and if that's less than--

PROFESSOR: OK, so if I have a left subtree, fine. Call max on it and get the rightmost node there. If not, I go up, and when do I stop? When I go left or right?

AUDIENCE: You'd have to go right. Is that right?

PROFESSOR: Yep. So last time, in this case, when I was going up, if I was going left, I had to keep going, and the moment I went right, I was happy and I stopped. What if I want to find the predecessor? It's the opposite, right? So I will go this way, and the moment I can go this way, I'm done. How do you do this in code? Slightly tricky. Just slightly, I promise.

AUDIENCE: [INAUDIBLE].

PROFESSOR: It's hard. What I would do is copy paste the code, replace "left" with "right" everywhere, and replace "min" with "max." You get it done. So we talked about how the tree is symmetric, right? So every time, instead of saying "left," you say "right," and instead of saying "min," you say "max." That's how you do this.

How do we do deletions? So suppose I'm in this tree and I want to delete 15. What do I do?

AUDIENCE: Kill it.

PROFESSOR: Kill it. Very good. What if I want to delete 16? What do I do?

AUDIENCE: You need to put 15 where 16 is.

PROFESSOR: OK. So I would put 15 here. So I had 16. Suppose I have a big tree here. Actually, let's go for an easier case. Let's say I have this tree here. So you're here, you have a big tree here, you don't have anything here, and you want to delete this guy.

AUDIENCE: You know that everything less than the top node is going to be less than it, so you can just move that up.

PROFESSOR: Everything less than this guy is also going to be less than this guy. So you're saying move the whole tree up.

AUDIENCE: Yep.

PROFESSOR: So the way we do that is we'd take this node's left link and make it point here, and take this guy's parent link and make it point here, and this guy sort of goes away. So we have two cases for deleting. We have if you're a leaf, we'll take you out. Sorry. I got confused. If you have one child and that child is in the same direction as your parent, then you can do this. What if you have one child, but it's a zigzag like this? What do you do?

AUDIENCE: It's still greater than, so you do the same thing.

PROFESSOR: Exactly. Same thing. Just change this guy, change this guy, and I'm happy. So it doesn't matter if you have a zigzag or a straight line. It might help you think about it to convince yourself that the code is correct, but in the end, you do the same thing. Now, what if I want to delete node 8? So what if I have a nasty case where I want to delete this guy and it has children both on the left and on the right?

AUDIENCE: You have to take 8, compare it to its parent and compare it to its right child, and see which one is greater in order to figure out which node gets replaced in its spot.

PROFESSOR: OK. So there is replacing that's going to happen. The answer is really tricky. I always forget this when coding. Try to understand it, and if it doesn't work, refer to the textbook. When you forget it, because you will, refer to the textbook or to the internet.

So what you do is I can't just magically replace this node with one of the subtrees, but we talked right before this about Next Greater, so finding a node's successor. If this node has both a left subtree and a right subtree, then I know that if I call Find Successor on it, I'm going to go somewhere inside here, and I'm going to find a node somewhere in here all the way to the left that is this guy's successor.

So what I'm going to do is I'm going to delete this node instead, and then I'm going to take its key and put it up here. So if I want to delete 8, what I do is I find its successor, then I delete it, then I take the 15 that was here-- you can see it, right? It's still there. Put it here.

So the reason this works is that everything here is greater than this guy. Everything here is smaller than this guy. This is the next node that's greater than this guy, but everything else is bigger than it, right, because we wanted it to be a successor. So if I take this value and I put it up here, everything in here is still going to be greater than it. This is a successor of this guy, so everything here is still going to be smaller than the successor.

Great. In order to do a delete, I find the successor, and then I call Delete on it. How do I know that this will end? How do I know that I'm not going to go into a loop that runs forever?

AUDIENCE: Because it's not--

AUDIENCE: It's acyclic, right?

PROFESSOR: OK. First answer, good. Eventually, worst case, I'm going to get to the maximum, and then not going on have to delete the successor anymore.

Now, another thing to note here is that if this guy is the successor of this guy, it can't have anything on the left, because if it would, then whatever is down here has to be bigger than this, and whatever's to the left of this node has to be smaller than this. But we said that this is the successor of this, so there's nothing here.

So this will be one of the easy cases that we talked about. The successor either has no kids, or it has only one child, only one subtree. So then I can delete it using one of the easy cases. So in fact, worst case that happens in a delete is my node has two subtrees. Then I find the successor that's only going to have one subtree, I change my links there, and I'm done. What is the running time for Delete?

AUDIENCE: Is it order h, because you should do it all the way down to the bottom of the tree, right?

PROFESSOR: You have the right answer. Let's see why it's order h. It has to be order h, right? Otherwise, the tree would be too slow. If it's order N, then it's bad. So why would Delete be order h? This was a heap, right, so I can't use this. I'm going to write "delete" here again.

So the first thing you do is you have to search for the key, right? That's order h. Now, if it's a happy case, if it's case one or two, you change some links and you're done. What's the time for that?

AUDIENCE: Constant.

PROFESSOR: Constant. So happy case, order h for sure. Now sad case. If you have two children, what do you have to do after you realize that you have two subtrees?

AUDIENCE: Find the successor.

PROFESSOR: OK. What's the running time for finding a successor?

AUDIENCE: Order h.

PROFESSOR: Order h. Once I find the successor, what do I do? Call Delete on that, and what happens? It's a happy case or a sad case?

AUDIENCE: It's a happy case.

PROFESSOR: Happy case, a few links get swapped, constant time. So worst case, order h plus order h. Order h. So insertions are order h, deletions are order h.

AUDIENCE: The first one. Because the second one is from finding the successor. What is the first one for?

PROFESSOR: Finding the node for a key in the tree. So if I say Delete 8, then you have to find 8. If I give you the node, then you don't have that. Good question. It's a good question. Thank you. So that's insertion. That's deletion.

Let's look at the code for Delete. Looks kind of long. So lines through 21, happy case or sad case? Try to do it by looking at the "if" instead of looking at the comments. So lines through 21 for Delete.

AUDIENCE: On this tree? Which tree, because there are two deletes?

PROFESSOR: Oh really? Sorry. Why do we have two deletes?

AUDIENCE: There's BST Delete and then there's BST Node Delete.

PROFESSOR: So BST Delete. Finds the node, and then calls Delete on the node. And then if the node is a tree's root, then it updates the tree's root. So let's look at the nodes delete. Oh, I see. I think I was looking at the wrong one. Thank you. My Delete was much longer than yours. So lines 3 through 12, happy case or sad case? Look at the "if" on line 3 and tell me, what case is it going for?

AUDIENCE: [INAUDIBLE].

PROFESSOR: If it doesn't have a left child or it doesn't have a right child, is that the happy case or the sad case?

AUDIENCE: Happy.

PROFESSOR: Happy case. So lines 4 through 12 handle the happy case. Lines 14 through 16 handle the sad case. Do lines 14 through 16 make sense? Find the successor, then swap the keys, then delete that successor.

Now, lines 4 through 11 are pretty much what we talked about here, except I can't draw arrows on the board and instead I have to change left and right links. Line 4 has to see if we're a left child or a right child, and then lines 5 through 7 and 9 through 11 are pretty much copy paste, swap left with right. And they changed the links like we changed them here. Do we have any questions on Deletes?

AUDIENCE: So if the successor had a right child, then all you do, you just do the workaround thing where you just--

PROFESSOR: Yep. So the case that it doesn't have two children. As long as it doesn't have both children, you're in the happy case and you can do some link swapping. Are you guys burned out already? Fair enough.

I left a part out. What I left out is how to augment a binary tree. So binary trees by default can answer the question, what's the minimum node in a tree in order h. You go all the way to the left, you find the minimum. That's the minimum.

It turns out that if you make a node a little bit fatter, so if instead of storing, say, 23 in this node, I store 23, and I store the fact that the minimum in my left subtree is 4, then it turns out that I can answer the question in constant time, what's the minimum? Oh gee, if you store the minimum here, of course you can retrieve it in constant time, right?

The hard part is, how do you handle insertions and updates in the same time? So the idea is that if I have a node and I have a function here, say the minimum of everything, if I have two children, here they're 15 and 42, and say the minimum in this tree is 4 and the minimum in this tree is. So if I already computed the function for these guys, how do I compute the function for this?

AUDIENCE: [INAUDIBLE] and compare it?

PROFESSOR: Yep. Take the minimum of these two guys, right? There are some special cases if you don't have a child. If you don't have a left child, then you're the minimum. But you write down those special cases, and you can compute this in how much time?

AUDIENCE: Order h, right?

PROFESSOR: What if I already computed the answer for the children? How much time does it take to compute the answer for a single node?

AUDIENCE: Constant.

PROFESSOR: Constant. OK.

AUDIENCE: For a tree, though.

PROFESSOR: For a tree, it's order h. Yeah. You're getting ahead. You're rushing me. You're not letting me finish.

AUDIENCE: Are you saying that we store the minimum value?

PROFESSOR: So for every--

AUDIENCE: Each node has a field that says what the minimum value is in that tree.

PROFESSOR: Yep, exactly. So for each node, what's the minimum in the subtree. So if I add a node here, suppose I add three and I had my minimums, what changed? This subtree changed, this subtree changed, this subtree changed, and then this subtree changed.

So I have to update the minimums here, here, here, here. Nothing else changed. Outside the path where I did the Insert, nothing changed, so I don't have to update anything. So what I do is after the Insert, I go back up and I re-compute the values. So here, I'll have 3. I go back up 3, 3, 3.

AUDIENCE: You could when you're passing down, though. When you're going down that column, you can just compare it on the way down. You don't have to go back up, right?

PROFESSOR: Yep. So the advantage of doing it the way I'm saying it is that you can have other functions instead of minimum. As long as you can compute the function inside the parent in constant time using the function from the children, it makes sense to compute the function on the children first. There's an obvious function that I can't tell you because that's on the Pset, but when you see the next Pset, you'll see what I mean.

So if you have a function where you know the result for the children and you can compute the result for the parent in constant time, then after you do the Insert, you go up on the path and you re-compute the function. When you delete, what do you do?

AUDIENCE: Same thing.

PROFESSOR: Same thing. If this goes away, then this subtree changed, and then if there would be something else here, then this subtree changed, but nothing else changed. So whenever you do an Insert or a Delete, all you have to do is go back up the path to the parent and re-compute the function that you're trying to compute. And that's tree augmentation. Does this make sense somewhat?

That's it. So what you'll find in lecture notes is a harder way of doing it that works for minimum, but what I told you works for everything. So don't tell people I told you how to do this for everything. Sure nobody's going to know.