1B: Procedures and Processes; Substitution Model

• Download English-US transcript (PDF)

[MUSIC PLAYING BY J.S. BACH]

PROFESSOR: Hi. You've seen that the job of a programmer is to design processes that accomplish particular goals, such as finding the square roots of numbers or other sorts of things you might want to do. We haven't introduced anything else yet. Of course, the way in which a programmer does this is by constructing spells, which are constructed out of procedures and expressions. And that these spells are somehow direct a process to accomplish the goal that was intended by the programmer. In order for the programmer to do this effectively, he has to understand the relationship between the particular things that he writes, these particular spells, and the behavior of the process that he's attempting to control.

So what we're doing this lecture is attempt to establish that connection in as clear a way as possible. What we will particularly do is understand how particular patterns of procedures and expressions cause particular patterns of execution, particular behaviors from the processes.

Let's get down to that. I'm going to start with a very simple program. This is a program to compute the sum of the squares of two numbers. And we'll define the sum of the squares of x and y to be the sum of the square of x-- I'm going to write it that way-- and the square of y where the square of x is the product of x and x. Now, supposing I were to say something to this, like, to the system after having defined these things, of the form, the sum of the squares of three and four, I am hoping that I will get out a 25. Because the square of three is nine, and the square of four is 16, and 25 is the sum of those.

But how does that happen? If we're going to understand processes and how we control them, then we have to have a mapping from the mechanisms of this procedure into the way in which these processes behave. What we're going to have is a formal, or semi-formal, mechanical model whereby you understand how a machine could, in fact, in principle, do this. Whether or not the actual machine really does what I'm about to tell you is completely irrelevant at this moment.

In fact, this is an engineering model in the same way that, electrical resistor, we write down a model v equals i r, it's approximately true. It's not really true. If I put up current through the resistor it goes boom. So the voltage is not always proportional to the current, but for some purposes the model is appropriate. In particular, the model we're going to describe right now, which I call the substitution model, is the simplest model that we have for understanding how procedures work and how processes work. How procedures yield processes. And that substitution model will be accurate for most of the things we'll be dealing with in the next few days. But eventually, it will become impossible to sustain the illusion that that's the way the machine works, and we'll go to other more specific and particular models that will show more detail.

OK, well, the first thing, of course, is we say, what are the things we have here? We have some cryptic symbols. And these cryptic symbols are made out of pieces. There are kinds of expressions. So let's write down here the kinds of expressions there are. And we have-- and so far I see things like numbers. I see things like symbols like that.

We have seen things before like lambda expressions, but they're not here. I'm going to leave them out. Lambda expressions, we'll worry about them later. Things like definitions. Things like conditionals. And finally, things like combinations.

These kinds of expressions are-- I'll worry about later-- these are special forms. There are particular rules for each of these. I'm going to tell you, however, the rules for doing a general case. How does one evaluate a combination? Because, in fact, over here, all I really have are combinations and some symbols and numbers. And the simple things like a number, well, it will evaluate to itself.

In the model I will have for you, the symbols will disappear. They won't be there at the time when you need them, when you need to get at them. So the only thing I really have to explain to you is, how do we evaluate combinations?

OK, let's see. So first I want to get the first slide. Here is the rule for evaluating an application. What we have is a rule that says, to evaluate a combination, there are two parts, three parts to the rule. The combination has several parts. It has operators and it has operands. The operator returns into a procedure. If we evaluate the operator, we will get a procedure. And you saw, for example, how I'll type at the machine and out came compound procedure something or other.

And the operands produce arguments. Once we've gotten the operator evaluated to get a procedure, and the argument is evaluated to get argument-- the operand's value to get arguments-- we apply the procedure to these arguments by copying the body of the procedure, which is the expression that the procedure is defined in terms of. What is it supposed to do? Substituting the argument supplied for the formal parameters of the procedure, the formal parameters being the names defined by the declaration of the procedure. Then we evaluate the resulting new body, the body resulting from copying the old body with the substitutions made.

It's a very simple rule, and we're going to do it very formally for a little while. Because for the next few lectures, what I want you to do is to say, if I don't understand something, if I don't understand something, be very mechanical and do this.

So let's see. Let's consider a particular evaluation, the one we were talking about before. The sum of the squares of three and three. What does that mean? It says, take-- well, I could find out what's on the square-- it's some procedure, and I'm not going to worry about the representation, and I'm not going to write it on the blackboard for you. And I have that three represents some number, but if I have to repeat that number, I can't tell you the number. The number itself is some abstract thing. There's a numeral which represents it, which I'll call three, and I'll use that in my substitution.

And four is also a number. I'm going to substitute three for x and four for y in the body of this procedure that you see over here. Here's the body of the procedure. It corresponds to this combination, which is an addition.

So what that reduces to, as a reduction step, we call it, is the sum of the square of three and the square of four. Now, what's the next step I have to do here? I say, well, I have to evaluate this. According to my rule, which you just saw on that overhead or slide, what we had was that we have to evaluate the operands-- and here are the operands, here's one and here's the next operand-- and how we have to evaluate procedure. The order doesn't matter. And then we're going to apply the procedure, which is plus, and magically somehow that's going to produce the answer. I'm not to open up plus and look inside of it.

However, in order to evaluate the operand, let's pick some arbitrary order and do them. I'm going to go from right to left. Well, in order to evaluate this operand, I have to evaluate the parts of it by the same rule. And the parts are I have to find out what square is-- it's some procedure, which has a formal parameter x. And also, I have an operand which is four, which I have to substitute for x in the body of square.

So the next step is basically to say that this is the sum of the square of three and the product of four and four. Of course, I could open up asterisk if I liked-- the multiplication operation-- but I'm not going to do that. I'm going to consider that primitive. And, of course, at any level of detail, if you look inside this machine, you're going to find that there's multiple levels below that that you don't know about.

But one of the things we have to learn how to do is ignore details. The key to understanding complicated things is to know what not to look at and what not compute and what not to think. So we're going to stop this one here and say, oh, yes, this is the product of two things. We're going to do it now.

So this is nothing more than the sum of the square of three and 16. And now I have another thing I have to evaluate, but that square of three, well, it's the same thing. That's the sum of the product of three and three and 16, which is the sum of nine and 16, which is 25.

So now you see the basic method of doing substitutions. And I warn you that this is not a perfect description of what the computer does. But it's a good enough description for the problems that we're going to have in the next few lectures that you should think about this religiously. And this is how the machine works for now. Later we'll get more detailed.

Now, of course, I made a specific choice of the order of evaluation here. There are other possibilities. If we go back to the telestrator here and look at the substitution rule, we see that I evaluated the operator to get the procedures, and I evaluated the operands to get the arguments first, before I do the application. It's entirely possible, and there are alternate rules called normal order evaluation whereby you can do the substitution of the expressions which are the operands for the formal parameters inside the body first. And you'll get also the same answer.

But right now, for concreteness, and because this is the way our machine really does it, I'm going to give you this rule, which has a particular order. But that order is to some extent arbitrary, too. In the long run, there are some reasons why you might pick one order or another, and we'll get to that later in the subject.

OK, well now the only other thing I have to tell you about just to understand what's going on is let's look at the rule for conditionals. Conditionals are very simple, and I'd like to examine this. A conditional is something that is if-- there's also cond, of course-- but I'm going to give names to the parts of the expression. There's a predicate, which is a thing that is either true or false. And there's a consequent, which is the thing you do if the predicate is true. And there's an alternative, which is the thing you do if the predicate is false.

It's important, by the way, to get names for, to get names for, the parts of things, or the parts of expressions. One of the things that every sorcerer will tell you is if you have the name of a spirit, you have power over it. So you have to learn these names so that we can discuss these things.

So here we have a predicate, a consequent, and an alternative. And, using such words, we see that an if expression, the problems you evaluate to the predicate expression, if that yields true, then you then go on to evaluate the consequent. Otherwise, you evaluate the alternative expression.

So I'd like to illustrate that now in the context of a particular little program. Going to write down a program which we're going to see many times. This is the sum of x and y done by what's called Peano arithmetic, which is all we're doing is incrementing and decrementing. And we're going to see this for a little bit. It's a very important program.

If x equals zero, then the result is y. Otherwise, this is the sum of the decrement of x and the increment of y. We're going to look at this a lot more in the future.

Let's look at the overhead. So here we have this procedure, and we're going to look at how we do the substitutions, the sequence of substitutions. Well, I'm going to try and add together three and four. Well, using the first rule that I showed you, we substitute three for x and four four y in the body of this procedure. The body of the procedure is the thing that begins with if and finishes over here. So what we get is, of course, if three is zero, then the result is four. Otherwise, it's the sum of the decrement of three and the increment of four.

But I'm not going to worry about these yet because three is not zero. So the answer is not four. Therefore, this if reduces to an evaluation of the expression, the sum to the decrement of three and the increment of four. Continuing with my evaluation, the increment I presume to be primitive, and so I get a five there. OK, and then the decrement is also primitive, and I get a two. And so I change the problem into a simpler problem. Instead of adding three to four, I'm adding two to five. The reason why this is a simpler problem is because I'm counting down on x, and eventually, then, x will be zero.

So, so much for the substitution rule. In general, I'm not going to write down intermediate steps when using substitutions having to do with ifs, because they just expand things to become complicated. What we will be doing is saying, oh, yes, the sum of three and four results in the sum of two and five and reduces to the sum of two and five, which, in fact, reduces to the sum of one and six, which reduces to the sum of zero and seven over here, which reduces to a seven. That's what we're going to be seeing.

Are there any questions for the first segment yet? Yes?

STUDENT: You're using one plus and minus one plus. Are those primitive operations?

PROFESSOR: Yes. One of the things you're going to be seeing in this subject is I'm going to, without thinking about it, introduce more and more primitive operations. There's presumably some large library of primitive operations somewhere. But it doesn't matter that they're primitive-- there may be some manual that lists them all. If I tell you what they do, you say, oh, yes, I know what they do. So one of them is the decrementor-- minus one plus-- and the other operation is increment, which is one plus.

Thank you. That's the end of the first segment.

[MUSIC PLAYING BY J.S. BACH]

PROFESSOR: Now that we have a reasonably mechanical way of understanding how a program made out of procedures and expressions evolves a process, I'd like to develop some intuition about how particular programs evolve particular processes, what the shapes of programs have to be in order to get particular shaped processes. This is a question about, really, pre-visualizing.

That's a word from photography. I used to be interested in photography a lot, and one of the things you discover when you start trying to learn about photography is that you say, gee, I'd like to be a creative photographer. Now, I know the rules, I push buttons, and I adjust the aperture and things like that. But the key to being a creative person, partly, is to be able to do analysis at some level. To say, how do I know what it is that I'm going to get on the film before I push the button. Can I imagine in my mind the resulting image very precisely and clearly as a consequence of the particular framing, of the aperture I choose, of the focus, and things like that?

That's part of the art of doing this sort of thing. And learning a lot of that involves things like test strips. You take very simple images that have varying degrees of density in them, for example, and examine what those look like on a piece of paper when you print them out. You find out what is the range of contrasts that you can actually see. And what, in a real scene, would correspond to the various levels and zones that you have of density in an image.

Well, today I want to look at some very particular test strips, and I suppose one of them I see here is up on the telestrator, so we should switch to that. There's a very important, very important pair of programs for understanding what's going on in the evolution of a process by the execution of a program. What we have here are two procedures that are almost identical. Almost no difference between them at all. It's a few characters that distinguish them. These are two ways of adding numbers together.

The first one, which you see here, the first one is the sum of two numbers-- just what we did before-- is, if the first one is zero, it's the answer of the second one. Otherwise, it's the sum of the decrement of the first and the increment of the second. And you may think of that as having two piles. And the way I'm adding these numbers together to make a third pile is by moving marbles from one to the other. Nothing more than that. And eventually, when I run out of one, then the other is the sum.

However, the second procedure here doesn't do it that way. It says if the first number is zero, then the answer is the second. Otherwise, it's the increment of the sum of the decrement of the first number and the second. So what this says is add together the decrement of the first number and the second-- a simpler problem, no doubt-- and then change that result to increment it.

And so this means that if you think about this in terms of piles, it means I'm holding in my hand the things to be added later. And then I'm going to add them in. As I slowly decrease one pile to zero, I've got what's left here, and then I'm going to add them back. Two different ways of adding. The nice thing about these two programs is that they're almost identical. The only thing is where I put the increment. A couple of characters moved around.

Now I want to understand the kind of behavior we're going to get from each of these programs. Just to get them firmly in your mind-- I usually don't want to be this careful-- but just to get them firmly in your mind, I'm going to write the programs again on the blackboard, and then I'm going to evolve a process. And you're going to see what happens. We're going to look at the shape of the process as a consequence of the program.

So the program we started with is this: the sum of x and y says if x is zero, then the result is y. Otherwise, it's the sum of the decrement of x and the increment of y.

Now, supposing we wish to do this addition of three and four, the sum of three and four, well, what is that? It says that I have to substitute the arguments for the formal parameters in the body. I'm doing that in my mind. And I say, oh, yes, three is substituted for x, but three is not zero, so I'm going to go directly to this part and write down the simplified consequent here. Because I'm really interested in the behavior of addition.

Well, what is that? That therefore turns into the sum of two and five. In other words, I've reduced this problem to this problem. Then I reduce this problem to the sum of one and six, and then, going around again once, I get the sum of zero and seven. And that's one where x equals zero so the result is y, and so I write down here a seven. So this is the behavior of the process evolved by trying to add together three and four with this program.

For the other program, which is over here, I will define the sum of x and y. And what is it? If x is zero, then the result is y-- almost the same-- otherwise the increment of the sum of the decrement of x and y. No. I don't have my balancer in front of me.

OK, well, let's do it now. The sum of three and four. Well, this is actually a little more interesting. Of course, three is not zero as before, so that results in the increment of the sum of the decrement of x, which is two and four, which is the increment of the sum of one and-- whoops: the increment of the increment.

What I have to do now is compute what this means. I have to evaluate this. Or what that is, the result of substituting two and four for x and y here. But that is the increment of the sum of one and four, which is-- well, now I have to expand this. Ah, but that's the increment of the increment of the increment of the sum of zero and four.

Ah, but now I'm beginning to find things I can do. The increment of the increment of the increment of-- well, the sum of zero and four is four. The increment of four is five. So this is the increment of the increment of five, which is the increment of six, which is seven. Two different ways of computing sums.

Now, let's see. These processes have very different shapes. I want you to feel these shapes. It's the feeling for the shapes that matters. What's some things we can see about this? Well, somehow this is sort of straight. It goes this way-- straight. This right edge doesn't vary particularly in size.

Whereas this one, I see that this thing gets bigger and then it gets smaller. So I don't know what that means yet, but what are we seeing? We're seeing here that somehow these increments are expanding out and then contracting back. I'm building up a bunch of them to do later. I can't do them now. There's things to be deferred.

Well, let's see. I can imagine an abstract machine. There's some physical machine, perhaps, that could be built to do it, which, in fact, executes these programs exactly as I tell you, substituting character strings in like this. Such a machine, the number of such steps is an approximation of the amount of time it takes. So this way is time. And the width of the thing is how much I have to remember in order to continue the process. And this much is space. And what we see here is a process that takes a time which is proportional to the argument x. Because if I made x larger by one, then I'd had an extra line.

So this is a process which is space-- sorry-- time. The time of this process is what we say order of x. That means it is proportional to x by some constant of proportionality, and I'm not particularly interested in what the constant is. The other thing we see here is that the amount of space this takes up is constant, it's proportional to one. So the space complexity of this is order of one.

We have a name for such a process. Such a process is called an iteration. And what matters here is not that some particular machine I designed here and talked to you about and called a substitution machine or whatever-- substitution model-- managed to do this in constant space. What really matters is this tells us a bound. Any machine could do this in constant space. This algorithm represented by this procedure is executable in constant space.

Now, of course, the model is ignoring some things, standard sorts of things. Like numbers that are bigger take up more space and so on. But that's a level of abstraction at which I'm cutting off. How do you represent numbers? I'm considering every number to be the same size. And numbers grow slowly for the amount of space they take up and their size.

Now, this algorithm is different in its complexity. As we can see here, this algorithm has a time complexity which is also proportional to the input argument x. That's because if I were to add one to three, if I made a larger problem, which is larger by one here, then I'd add a line at the top and I'd add a line at the bottom. And the fact that it's a constant amount, like this is twice as many lines as that, is not interesting at the level of detail I'm talking about right now.

So this is a time complexity order of the input argument x. And space complexity, well, this is more interesting. I happen to have some overhead, which you see over here, which is constant approximately. Constant overhead. But then I have something which increases and decreases and is proportional to the input argument x. The input argument x is three. That's why there are three deferred increments sitting around here. See? So the space complexity here is also order x.

And this kind of process, named for the kind of process, this is a recursion. A linear recursion, I will call it, because of the fact that it's proportional to the input argument in both time and space. This could have been a linear iteration.

So then what's the essence of this matter? This matter isn't so obvious. Maybe there are other models by which we can describe the differences between iterative and recursive processes. Because this is hard now. Remember, we have-- those are both recursive definitions. What we're seeing there are both recursive definitions, definitions that refer to the thing being defined in the definition. But they lead to different shape processes. There's nothing special about the fact that the definition is recursive that leads to a recursive process.

OK. Let's think of another model. I'm going to talk to you about bureaucracy. Bureaucracy is sort of interesting. Here we see on a slide an iteration. An iteration is sort of a fun kind of process.

Imagine that there's a fellow called GJS-- that stands for me-- and he's got a problem: he wants to add together three and four. This fella here wants to add together three and four. Well, the way he's going to do it-- he's lazy-- is he's going to find somebody else to help him do it. They way he finds someone else to-- he finds someone else to help him do it and says, well, give me the answer to three and four and return the result to me. He makes a little piece of paper and says, here, here's a piece of paper-- you go ahead and solve this problem and give the result back to me.

And this guy, of course, is lazy, too. He doesn't want to see this piece of paper again. He says, oh, yes, produce a new problem, which is the sum of two ad five, and return the result back to GJS. I don't want to see it again. This guy does not want to see this piece of paper. And then this fellow makes a new problem, which is the addition of the sum of one and six, and he give it to this fella and says, produce that answer and returned it to GJS. And that produces a problem, which is to add together zero and seven, and give the result to GJS. This fella finally just says, oh, yeah, the answer is seven, and sends it back to GJS. That's what an iteration is.

By contrast, a recursion is a slightly different kind of process. This one involves more bureaucracy. It keeps more people busy. It keeps more people employed. Perhaps it's better for that reason.

But here it is: I want the answer to the problem three and four. So I make a piece of paper that says, give the result back to me. Give it to this fella. This fellow says, oh, yes, I will remember that I have to add later, and I want to get the answer the problem two plus four, give that one to Harry, and have the results sent back to me-- I'm Joe. When the answer comes back from Harry, which is a six, I will then do the increment and give that seven back to GJS. So there are more pieces of paper outstanding in the recursive process than the iteration.

There's another way to think about what an iteration is and the difference between an iteration and a recursion. You see, the question is, how much stuff is under the table? If I were to stop-- supposing I were to kill this computer right now, OK? And at this point I lose the state of affairs, well, I could continue the computation from this point but everything I need to continue the computation is in the valuables that were defined in the procedure that the programmer wrote for me. An iteration is a system that has all of its state in explicit variables.

Whereas the recursion is not quite the same. If I were to lose this pile of junk over here, and all I was left with was the sum of one and four, that's not enough information to continue the process of computing out the seven from the original problem of adding together three of four. Besides the information that's in the variables of the formal parameters of the program, there is also information under the table belonging to the computer, which is what things have been deferred for later.

And, of course, there's a physical analogy to this, which is in differential equations, for example, when we talk about something like drawing a circle. Try to draw a circle, you make that out of a differential equation which says the change in my state as a function of my current state. So if my current state corresponds to particular values of y and x, then I can compute from them a derivative which says how the state must change.

And, in fact, you can see this was a circle because if I happen to be, say, at this place over here, at one, zero, for example, on this graph, then it means that the derivative of y is x, which we see over here. That's one, so I'm going up. And the derivative of x is minus y, which means I'm going backwards. I'm actually doing nothing at this point, then I start going backwards as y increases.

So that's how you make a circle. And the interesting thing to see is a little program that will draw a circle by this method. Actually, this won't draw a circle because it's a forward oil or integrator and will eventually spiral out and all that. But it'll draw a circle for a while before it starts spiraling.

However, what we see here is two state variables, x and y. And there's an iteration that says, in order to circle, given an x and y, what I want is to circle with the next values of x and y being the old value of x decrement by y times dt where dt is the time step and the old value of y being implemented by x times dt, giving me the new values of x and y.

So now you have a feeling for at least two different kinds of processes that can be evolved by almost the same program. And with a little bit of perturbation analysis like this, how you change a program a little bit and see how the process changes, that's how we get some intuition. Pretty soon we're going to use that intuition to build big, hairy, complicated systems.

Thank you.

[MUSIC PLAYING BY J.S. BACH]

PROFESSOR: Well, you've just seen a simple perturbational analysis of some programs. I took a program that was very similar to another program and looked at them both and saw how they evolved processes. I want to show you some variety by showing you some other processes and shapes they may have. Again, we're going to take very simple things, programs that you wouldn't want to ever write. They would be probably the worst way of computing some of the things we're going to compute. But I'm just going to show you these things for the purpose of feeling out how to program represents itself as the rule for the evolution of a process.

So let's consider a fun thing, the Fibonacci numbers. You probably know about the Fibonacci numbers. Somebody, I can't remember who, was interested in the growth of piles of rabbits. And for some reason or other, the piles of rabbits tend to grow exponentially, as we know.

And we have a nice model for this process, is that we start with two numbers, zero and one. And then every number after this is the sum of the two previous. So we have here a one. Then the sum of these two is two. The sum of those two is three. The sum of those two is five. The sum of those two is eight. The sum of those two is 13. This is 21. 34. 55. Et cetera.

If we start numbering these numbers, say this is the zeroth one, the first one, the second one, the third one, the fourth one, et cetera. This is the 10th one, the 10th Fibonacci number. These numbers grow very fast. Just like rabbits. Why rabbits grow this way I'm not going to hazard a guess.

Now, I'm going to try to write for you the very simplest program that computes Fibonacci numbers. What I want is a program that, given an n, will produce for me Fibonacci event. OK? I'll write it right here. I want the Fibonacci of n, which means the-- this is the n, and this is Fibonacci of n. And here's the story. If n is less than two, then the result is n. Because that's what these are. That's how you start it up. Otherwise, the result is the sum of Fib of n minus one and the Fibonacci number, n minus two.

So this is a very simple, direct specification of the description of Fibonacci numbers that I gave you when I introduced those numbers. It represents the recurrence relation in the simplest possible way.

Now, how do we use such a thing? Let's draw this process. Let's figure out what this does. Let's consider something very simple by computing Fibonacci of four. To compute Fibonacci of four, what do I do?

Well, it says I have-- it's not less than two. Therefore it's the sum of two things. Well, in order to compute that I have to compute, then, Fibonacci of three and Fibonacci of two. In order to compute Fibonacci of three, I have to compute Fibonacci of two and Fibonacci of one. In order to compute Fibonacci of two, I have to compute Fibonacci of one and Fibonacci of zero. In order to compute Fibonacci of one, well, the answer is one. That's from the base case of this recursion. And in order to compute Fibonacci of one, well, that answer is zero, from the same base. And here is a one. And Fibonacci of two is really the sum of Fibonacci of one. And Fib of zero, in order to compute that, I get a one, and here I've got a zero.

I've built a tree. Now, we can observe some things about this tree. We can see why this is an extremely bad way to compute Fibonacci numbers. Because in order to compute Fibonacci of four, I had to compute Fibonacci of two's sub-tree twice.

In fact, in order way to add one more, supposing I want to do Fibonacci of five, what I really have to do then is compute Fibonacci of four plus Fibonacci of three. But Fibonacci of three's sub-tree has already been built. This is a prescription for a process that's exponential in time. To add one, I have to multiply by something because I take a proportion of the existing thing and add it to itself to add one more step. So this is a thing whose time complexity is order of-- actually, it turns out to be Fibonacci-- of n.

There's a thing that grows exactly at Fibonacci numbers. It's a horrible thing. You wouldn't want to do it. The reason why the time has to grow that way is because we're presuming in the model-- the substitution model that I gave you, which I'm not doing formally here, I sort of now spit it out in a simple way-- but presuming that everything is done sequentially. That every one of these nodes in this tree has to be examined. And so since the number of nodes in this tree grows exponentially, because I add a proportion of the existing nodes to the nodes I already have to add one, then I know I've got an exponential explosion here.

Now, let's see if we can think of how much space this takes up. Well, it's not so bad. It depends on how much we have to remember in order to continue this thing running. Well, that's not so hard. It says, gee, in order to know where I am in this tree, I have to have a path back to the root. In other words, in order to-- let's consider the path I would have to execute this.

I'd say, oh, yes, I'm going to go down here. I don't care which direction I go. I have to do this. I have to then do this. I have to traverse this tree in a sort of funny way. I'm going to walk this nice little path. I come back to here. Well, I've got to remember where I'm going to be next. I've got to keep that in mind. So I have to know what I've done. I have to know what's left. In order to compute Fibonacci of four, at some point I'm going to have to be down here. And I have to remember that I have to go back and then go back to here to do an addition. And then go back to here to do an addition to something I haven't touched yet.

The amount of space that takes up is the path, the longest path. How long it is. And that grows as n. So the space-- because that's the length of the deepest line through the tree-- the space is order of n. It's a pretty bad process.

Now, one thing I want to see from this is a feeling of what's going on here. Why are there-- how is this program related to this process? Well, what are we seeing here? There really are only two sorts of things this program does. This program consists of two rules, if you will. One rule that says Fibonacci of n is this sum that you see over here, which is a node that's shaped like this. It says that I break up something into two parts. Under some condition over here that n is greater than two, then the node breaks up into two parts. Less than two. No. Greater than two. Yes.

The other possibility is that I have a reduction that looks like this. And that's this case. If it's less than two, the answer is n itself.

So what we're seeing here is that the process that got built locally at every place is an instance of this rule. Here's one instance of the rule. Here is another instance of the rule.

And the reason why people think of programming as being hard, of course, is because you're writing down a general rule, which is going to be used for lots of instances, that a particular instance-- it's going to control each particular instance for you. You've got to write down something that's a general in terms of variables, and you have to think of all the things that could possibly fit in those variables, and all those have to lead to the process you want to work. Locally, you have to break up your process into things that can be represented in terms of these very specific local rules.

Well, let's see. Fibonaccis are, of course, not much fun. Yes, they are. You get something called the golden ratio, and we may even see a lot of that some time.

Well, let's talk about another thing. There's a famous game called the Towers of Hanoi, because I want to teach you how to think about these recursively.

The problem is this one: I have a bunch of disks, I have a bunch of spikes, and it's rumored that somewhere in the Orient there is a 64-high tower, and the job of various monks or something is to move these spikes in some complicated pattern so eventually-- these disks-- so eventually I moved all of the disks from one spike to the other. And if it's 64 high, and it's going to take two to the 64th moves, then it's a long time. They claim that the universe ends when this is done.

Well, let's see. The way in which you would construct a recursive process is by wishful thinking. You have to believe.

So, the idea. Supposing I want to move this pile from here to here, from spike one to spike two, well, that's not so hard. See, supposing somehow, by some magic-- because I've got a simpler problem-- I move a three-high pile to here-- I can only move one disk at a time, so identifying how I did it. But supposing I could do that, well, then I could just pick up this disk and move it here. And now I have a simple problem. I have to move a three-high tower to here, which is no problem. So by two moves of a three high tower plus one move of a single object, I can move the tower from here to here.

Now, whether or not-- this is not obvious in any deep way that this works. And why? Now, why is it the case that I can presume, maybe, that I can move the three-high tower? Well, the answer is because I'm always counting down, and eventually I get down to zero-high tower, and a zero-high tower requires no moves.

So let's write the algorithm for that. Very easy. I'm going to label these towers with numbers, but it doesn't matter what they're labelled with. And the problem is to move an n-high tower from a spike called From to a spike called To with a particular spike called Spare. That's what we're going to do.

Using the algorithm I informally described to you, move of a n-high tower from From to To with a Spare. Well, I've got two cases, and this is a case analysis, just like it is in all the other things we've done.

If n is zero, then-- I'm going to put out some answers-- Done, we'll say. I don't know what that means. Because we'll never use that answer for anything. We're going to do these moves. Else. I'm going to do a move. Move a tower of height less than n, the decrement of n height. Now, I'm going to move it to the Spare tower. The whole idea now is to move this from here to here, to the Spare tower-- so from From to Spare-- using To as a spare tower.

Later, somewhere later, I'm going to move that same n-high tower, after I've done this. Going to move that same n minus one-high tower from the Spare tower to the To tower using the From tower as my spare. So the Spare tower to the To tower using the From as the spare.

All I have to do now is when I've gotten it in this condition, between these two moves of a whole tower-- I've got it into that condition-- now I just have to move one disk. So I'm going to say that some things are printing a move and I don't care how it works. From the To.

Now, you see the reason why I'm bringing this up at this moment is this is an almost identical program to this one in some sense. It's not computing the same mathematical quantity, it's not exactly the same tree, but it's going to produce a tree. The general way of making these moves is going to lead to an exponential tree.

Well, let's do this four-high. I have my little crib sheet here otherwise I get confused. Well, what I'm going to put in is the question of move a tower of height four from one to spike two using spike three as a spare. That's all I'm really going to do. You know, let's just do it. I'm not going to worry about writing out the traits of this. You can do that yourself because it's very simple.

I'm going to move disk one to disk three. And how do I get to move disk one to disk three? How do I know that? Well, I suppose I have to look at the trace a little bit. What am I doing here? Well, and this is not-- n is not zero. So I'm going to look down here. This is going to require doing two moves. I'm only going to look at the first one. It's going to require moving-- why do I have move tower? It makes it harder for me to move. I'm going to move a three-high tower from the from place, which is four, to the spare, which is two, using three as my-- no, using from--

STUDENT: [INAUDIBLE PHRASE].

PROFESSOR: Yes. I'm sorry.

From two-- from one to three using two as my spare. That's right. And then there's another move over here afterwards. So now I say, oh, yes, that requires me moving a two-high tower from one to two using three as a spare. And so, are the same, and that's going to require me moving and one-high tower from one to three using two as a spare. Well, and then there's lots of other things to be done.

So I move my one-high tower from one to three using two as a spare, which I didn't do anything with. Well, this thing just proceeds very simply. I move this from one to two. And I move this disk from three to two. And I don't really want to do it, but I move from one to three. Then I move two to one. Then I move two to three. Then one to three. One to two. Three to two. Three to one. This all got worked out beforehand, of course. Two to one. Three to two. One to three.

STUDENT: [INAUDIBLE PHRASE].

PROFESSOR: Oh, one to three. Excuse me. Thank you.

One to two. And then three to two. Whew.

Now what I'd like you to think about, you just saw a recursive algorithm for doing this, and it takes exponential time, of course. Now, I don't know if there's any algorithm that doesn't take exponential time-- it has to. As I'm doing one operation-- I can only move one thing at a time-- there's no algorithm that's not going to take exponential time.

But can you write an iterative algorithm rather than a recursive algorithm for doing this? One of the sort of little things I like to think about. Can you write one that, in fact, doesn't break this problem into two sub-problems the way I described, but rather proceeds a step at a time using a more local rule? That might be fun.

Thank you so much for the third segment.

Are there questions?

STUDENT: [INAUDIBLE] a way to reduce a tree or recursion problem, how do you save the immediate work you have done in computing the Fibonacci number?

PROFESSOR: Oh, well, in fact, one of the ways to do is what you just said. You said, I save the intermediate work. OK? Well, let me tell you-- this, again, we'll see later-- but suppose it's the case that anytime I compute anything, any one of these Fibonacci numbers, I remember the table that takes only linear time to look up the answer. Then if I ever see it again, instead of doing the expansional tree, I look it up. I've just transformed my problem into a problem that's much simpler.

Now, of course, there are the way to do this, as well. That one's called memoization, and you'll see it sometime later in this term. But I suppose there's a very simple linear time, and, in fact, iterative model for computing Fibonaccis, and that's another thing you should sit down and work out. That's important. It's important to see how to do this. I want you to practice.