Linear Approximation/Newton's Method

Related Resources

Lecture summary and Practice problems (PDF)

• Download English-US transcript (PDF)

GILBERT STRANG: OK, what I want to do today is show you two different ways that derivatives are used. In one of them, the problem is to find a close approximation to the value f at a point x. f of x. The second application is to solve an equation, where often-- and I use a different letter, capital F, just because it's a different function and there will be different examples for this one. So I just chose capital F to keep them separate. This is the problem of solving an equation. And Newton had an idea and it's survived all these centuries and it's still the good way.

OK, what's this based on, both of them? That's the point. They're both based on the same idea. Suppose at a point, which is near the x we want, or near the solution to this problem, at some point, let me call it a, suppose we know the slope, the derivative, at that point. So I'm using f prime for the derivative. At that point, well, we know what the definition is. But I'm also supposing that we've got that number. And now I want to use this knowledge of the slope at that nearby point a to come close to the solution, the f of x, or the x there.

OK, well you remember this is delta f divided by delta x. You recognize that before we take the limit this is two nearby points, delta x apart. Their values, the values of f at those points. It's the change in f divided by the change in x-- that's what the derivative is-- as one point approaches the point a where we know the slope.

Here's the idea. I'm just going to erase that stuff. Well, now I won't have an equal sign anymore. I'll have an approximately equal. So the slope-- this is delta f over delta x-- is not the same as df dx, which is this. But if x is close to a this will normally be close to the correct slope, the instant slope at the point. I'm just going to use this approximation to find a good approximation for f of x. So in this application on the left I know the x and I want to find what is f at that point?

So I look at this. I multiply up by x minus a. I move the f of a to the other side. And what do I have? I still have an approximation sign. So when I move the f of a to the other side, there it is. And then, the other part is x minus a times f prime at a. That's my formula. Let me just talk about that formula for a minute and then give examples.

What that says is that if I want to find the value of f at a nearby point, a good thing to do is use this linear approximation. I use the word "linear" because the graph of that is a line. I'm following a straight line instead of following the curved graph. That's the message of today's lecture. Follow the line. So it's a line. It starts out at the correct point at x equal a. It has the corrects slope f prime of a. And if I don't go too far that line won't be too far from the curve. Good. You'll see it now in examples. Let me get the corresponding idea here.

Now I have to remember to use capital F. So let me create the formula that helps to solve, approximately solves, this equation. What's the difference?

Now x is what I'm looking for. x is what I'm looking for and F of x is what I know. I know it's to be 0. So I'm going to use this with capital F. I know F of x should be 0 and x is the unknown. So can I just move that equation around again to get an equation? I'll bring the x minus a up as I did before. And I have to use an approximation sign as always. The F of x is 0, so I have minus F of a, and then I'm dividing by F prime of a. There you have it. That is Newton's insight. Newton and then somebody named Raphson helped out, made it work for general functions. And once again, I forgot. I should be using capital F there. Because in the right side of the board I'm calling the function capital F. This is the little movement of x away from a, which will bring us closer to the answer. You'll see is this formula work in a picture. So I'm ready for an example starting with approximation.

So here's my problem. Here's my example. Find the square root of-- so my problem is going to be find the square root or approximate square root of 9. I'm going to shift away from 9 a little bit. 9.06.

OK, how does that fit this example? My function is the square root function, x to the 1/2. It's derivative, f prime. I know the derivative of that is 1/2 x to one lower power minus 1/2. So that's 1 over 2 square root of x. Good. I know my function. I know its derivative.

Now I pick a point a, which is close and easy. So close, I'm looking for a point a, which is near 9.06 and has a nice square root. Well, 9. So I'll choose a to be 9. The correct value f at the point a is the square root of 9. 3. That's the point. We can evaluate the function easily at 9. And f prime at a is 1 over 2 square root of a. 9. Square root of 9. So what do I have for that? That's the easy number, 3. That's 1/6.

In other words, I know what's happening at x equal 9. And that's a particular point I'm calling a and I'm working from there.

Then what does my approximation say? It says that f at this nearby point. So x is 9.06. This is the x where I want to know the square root. So this is saying that the square root of 9.06-- that's my f of x-- is approximately f at a. That's the square root of 9. That's the 3. So far reasonable. That's the approximation I started with just using the 9. But now I'm improving it by the difference between x minus 3.

Oh, what is x? So I'm plugging in x is 9.06. That's where I really want the square root. That's my x. Minus a, which is the 9. Times f prime, which we figured out as 1/6. That's the linear approximation following this line to this number, which is what? 3 plus-- That's 0.06 divided by 6. That's 3. What do I have there? This difference is 0.06. Divided by 6 and its 0.01. That's the approximation. Closer than 3. That comes from following the line. Let me draw a graph to show you what I mean by following the line.

Here is my square root function. The square root looks something like that. And here is the point x equals 9, where I know that the height is 3. What else do I know?

Here is 9.06, a little further over, and I'm looking for that point. I'm looking for the square root of 9.06. So this was 9 here and this was 9.06. And how am I getting close to that point?

Well, I'm not going to follow the curve to do square root of 9.06 exactly, any more than your calculator or computer does. I'm going to follow this line. So that line is the tangent line. It's the line that goes through the right thing at a with the right slope. So you can see that by following the line that gave me the 3. And then here is the little-- you see what I'm-- I'm missing by a very small amount, practically too small to see. I'm picking the point on the line and that was this across, this delta x was the 0.06. The little tiny bit here is the 0.06. And this delta f is the little piece that I added on the 0.01. How did that come from? It came from the fact that the correct slope is 1/6.

If I go over 0.06, I should go up 0.01. So you see what I'm doing? I'm taking that point as my close approximation to the square root. Closer than 3 to the square root of 9.06. OK. That's a first example. I'll give a second one.

But first, I'd like to give an example that's like this one of Newton's method. May I change now to Newton's method? So I want to create an equation. And actually, I want it to solve the same problem. So I'm going to take my function to be x squared minus 9.06 and I'll set that to 0. I'm just keeping my two examples close because the answer-- in both problems, I'm looking for the square root of 9.06. Of course, that's the solution to this equation, the square root of 9.06.

OK again, I pick a point a close to the solution. And again, I'm going to take a to be 3. So 3 is close to the correct solution. The correct x is the square root of 9.06. But I'm starting close to it.

OK, at that point I figure out F of a. Newton wants to know F of a, the value. And Newton also wants to know the slope. So the value at a is-- 3 squared is 9. 9 minus 9.06 is minus 0.06.

F prime of a is-- what's the derivative of F? Capital F. Well, the derivative is certainly 2x. And at the point a it's 6. This is the 2x. 2a. This is the 2a because I'm evaluating the slope at the point a. Actually, if you want a picture, it's quite interesting to see the picture. Let me graph this f of x now. What does my function f of x look like?

x squared. That's a parabola going up. It starts below here. It starts somewhere down here and it curves up like so. So this is the point I want. There is the square root of 9.06. I've graphed a function x squared minus 9.06. That's the point I'd like to find. I'm not expecting to find it exactly.

What I do is I have a nearby point 3. At 3 I do know the exact value and I know the slope. Ha. Even my bad art is telling me what's going to happen.

Let me write that 3 better. a equals 3. This is where I know what's happening. I know the value of F. It's actually pretty small, but it's negative. I know the value of the slope. So this is like I've blown up this picture here. This F is the negative 0.06. Is the value of F. The slope is 6. And what's the x? What is my improved guess at the solution?

My improved guess, I don't follow that curve because that would be perfect. But curves are hard to follow. I'll follow the straight line and that's my next x. My better x. You see, that's a lot better. Let me do the numbers here and you'll see that it's a lot better.

What is the x that Newton's method gives us? I'm using Newton's formula here. So Newton's formula says that x minus the a is 3. And Newton's formula is going to take equal to get an approximate x. Minus F of a. So what's minus F of a? That will be plus 0.06. Divided by F prime of a, the slope, which was 6. What do I get? I did 0.01.

Actually, the two examples, of course, are parallel. In a way, the graph of that square root function kind of got just flipped to this graph of the x squared function. The square root function and the x squared function are just sort of inverses to each other. Their graphs flip and so now the slope is 6 instead of 1/6. And do you see that that distance, this is the x minus a distance that has to be 0.01. That's what I concluded.

I go across 1 if I'm going up/down by 6 because the slope is 6. OK, good.

Now, how close am I? Well, I can't resist asking. Suppose I multiply 3.01 by 3.01. Am I close to the 9.06 that was my whole goal? And how close? Of course, I'm not going to be exact. If I do that multiplication I get 301, 903. Combined I get 9 and the decimal put it in there, 0601. So by that method or by that method I ended up with 3.01 as much closer to the square root of 9.06.

You see when I square 3.01, it slightly overshot. It slightly overshot the 9.06. This little error there, this is the overshoot error. And when I square it, it's way out in the ten thousandths place, the fourth decimal place. OK. So those are the two parallel examples.

May I go back to the linear approximation? Because I think another example's appropriate there. And then I'll come back and give another Newton example. So that's my plan here. Two examples of each.

So those examples were parallel, the next two examples will be a little different. So let me show you example two. So linear approximation example two.

So I want-- OK, I'm going to look for something that I don't know exactly. Let me take e to the 0.01. What's the value of e to the power of 0.01?

So the function is e to the x. And I'm looking at x is 0.01. That's what I want. I am not going to get an exact number here. I'm going to follow the tangent line. I'm going to get a close number. So where shall I start?

I'll start with a number that's close to that and where I do you know the correct e. And a number close to that is take-- I'll choose a to be 0. That's close to 0.01. Then f of a is e to the 0 power. So that's 1.

Now for the straight line approximation, I also need the correct slope. Correct slope at a? Well, I do know the slope of this. f prime at 0. That's my a. f prime at 0. 0 is my a. Well, I know the derivative of e to the x. That's one thing I like. It's e to the x. So at x equals 0, again I get a 1 for the derivative, which is the same, I get e to the 0. I get also a 1.

So now I know what's happening at a equals 0. I want to know approximately what's happening at the nearby point, 0.01. e to the 0.01, e to the x. This is the e to the x. That's my function. And I'm only going to get it approximately, is the value at this known point. The value at the known point 0, the exact exponential is 1. Plus x minus a times the slope, the corrects slope, at this not quite perfect point, 0. And the correct slope is 1. And of course, a was 0. So you see I'm using all the facts at a to get an approximate fact at x. And what have I got here? I've just got 1 plus x. You know, in a way, that's perfect. Because it shows what the linear approximation is doing.

You remember the series for e to the x? My correct function is e to the x. My approximate function is 1 plus x. What's the connection? You remember that e to the x, the series for e to the x started out 1 plus x plus 1/2 x squared, 1/6 x cubed. Those are the guys, those are the higher order corrections that following the line misses. Those are the parts where the function, the curve e to the x, has left the line.

But if I don't go to far--So this is 1.01. x is 0.01. That's my approximation. 1 plus x. That's the thing to notice. That linear approximations and you could come back to that-- the formula for any f and any a. Linear approximations are just like those power series. Just like the e to the x equal 1 plus x plus 1/2 x squared plus so on. Except we cut them off after just the constant term and the linear term. That's what this linear approximation is about. And you might say, what's the next term? And of course we know that the next term is 1/2 x squared and you could ask, what's the next term in this?

In the general case, actually let me tell you the next term. Next would be-- but we're not using it. Would be the 1/2. It would be the x minus a squared and it would be the second derivative at a.

If we kept it that's what we would keep. OK, good.

You saw the main point of linear approximation stop at the linear term.

Finally, I go back to an example of Newton's method. A second example of Newton's method.

So I've been thinking, what should I do? Let me use Newton's method the way it's really used. The way you use Newton's method is you do this to come close to the solution. And then, you do it again. You do it again starting at 3.01. So I planned to do just the same thing for the next step of Newton's method, except the a. I'm still aiming to solve this same equation, but I'm going to get closer than 3.01. I got closer than 3 to 3.01. Now I'm going to restart there. a is now going to be 3.01. I need to compute F of a for Newton's method. So I have to do 3.01 squared and take away that to see how wrong I am. HA. We did 3.01 squared. Actually, right there.

So if I take away the 9.06, the F of a is-- well, that was the whole point. That it was pretty darn close. But nothing compared to the closeness we're going to get at the second term. And what's F prime of a?

Take the derivative 2x. At the point a it's 2a. And a is now 3.01. So the slope is 6.02. You see what I'm doing. I'm just moving over to this point, which that has become now the a in the second try. a in the second cycle of Newton's method. This is how Newton's method really is used. And now let's find the new x, the highly improved x, better than 3.01.

So Newton's method says x, the new x, minus a. I'm just using Newton's formula. x minus the a is supposed to be minus the F of a. So that's 0.0001 divided by F prime of a, 6.02. This is the delta x you could say. This is the little correction. It's negative. It means that we need to pull back a little. And you see that. We slightly, slightly overshot by following the tangent line. The curve went up a little across 0 a little before the tangent line. This is extremely close. So now this gives me the new x right here, 3.01 minus this tiny little bit. And so that's what the calculator will do. I hope you'll do it on a calculator.

Just make the calculator take that quantity, then make it square it. Then just go through this. Find the new x. Square it. Subtract 9.06. Let's see how close it is. I believe that the error-- I don't know what it is. That's more than I can do in my head, squaring that 3.01 minus this little tiny bit. But I am confident that the error, the x new squared. This is the formula for x new, the second cycle of Newton's method. I think that minus the 9.06, I don't know-- can I just put a bunch of zeros? Somebody will want me to put in a 1 here, so I'll put in a 1. I bet it's way out there. So Newton's method is really a terrific success. Follow the line, then follow the next tangent line. Then follow the next one and you home in very, very quickly on the exact answer. You get more and more decimal places correct. OK, that's two uses of calculus coming from the same idea. The same idea delta f over delta x.

In one case, it was f that we didn't know. In this case, it was x that we didn't know. In both cases that formula gives a terrific and a terrifically simple and close approximation to the exact answer. Good. Thank you very much.

ANNOUNCER: This has been a production of MIT OpenCourseWare and Gilbert Strang.

Funding for this video was provided by the Lord Foundation.

To help OCW continue to provide free and open access to MIT courses, please make a donation at ocw.mit.edu/donate.

Free Downloads

Video

• iTunes U (MP4 - 69MB)
• Internet Archive (MP4 - 69MB)

Caption

• English-US (SRT)