Lecture 10: Renewals and the Strong Law of Large Numbers

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

PROFESSOR: OK. Today we're all through with Markov chains, or at least with finite state Markov chains. And we're going on to renewal processes. As part of that, we will spend a good deal of time talking about the strong law of large numbers, and convergence with probability one.

The idea of convergence with probability one, at least to me is by far the most difficult part of the course. It's very abstract mathematically. It looks like it's simple, and it's one of those things you start to think you understand it, and then at a certain point, you realize that you don't. And this has been happening to me for 20 years now.

I keep thinking I really understand this idea of convergence with probability one, and then I see some strange example again. And I say there's something very peculiar about this whole idea. And I'm going to illustrate that you at the end of the lecture today.

But for the most part, I will be talking not so much about renewal processes, but this set of mathematical issues that we have to understand in order to be able to look at renewal processes in the simplest way.

One of the funny things about the strong law of large numbers and how it gets applied to renewal processes is that although the idea of convergence with probability one is sticky and strange, once you understand it, it is one of the most easy things to use there is. And therefore, once you become comfortable with it, you can use it to do things which would be very hard to do in any other way.

And because of that, most people feel they understand it better than they actually do. And that's the reason why it sometimes crops up when you're least expecting it, and you find there's something very peculiar. OK, so let's start out by talking a little bit about renewal processes. And then talking about this convergence, and the strong law of large numbers, and what it does to all of this.

This is just review. We talked about arrival processes when we started talking about Poisson processes. Renewal processes are a special kind of arrival processes, and Poisson processes are a special kind of renewal process. So this is something you're already sort of familiar with.

All of arrival processes, we will tend to treat in one of three equivalent ways, which is the same thing we did with Poisson processes. A stochastic process, we said, is a family of random variables. But in this case, we always view it as three families of random variables, which are all related. And all of which define the other.

And you jump back and forth from looking at one to looking at the other, which is, as you saw with Poisson processes, you really want to do this, because if you stick to only one way of looking at it, you really only pick up about a quarter, or a half of the picture. OK.

So this one picture gives us a relationship between the arrival epochs of an arrival process, the inter-arrival intervals, the x1, x2, x3, and the counting process, n of t, and whichever one you use, you use the one which is easiest, for whatever you plan to do.

For defining a renewal process, the easy thing to do is to look at the inter-arrival intervals, because the definition of a renewal process is it's an arrival process for which the interrenewals are independent and identically distributed. So any process where the arrivals, inter-arrivals have that property are IID.

OK, renewal processes are characterized, and the name comes from the idea that you start over at each interval. This idea of starting over is something that we talk about more later on. And it's a little bit strange, and a little bit fishy. It's like with a Poisson process, you look at different intervals, and they're independent of each other.

And we sort of know what that means by now. OK, you look at the arrival epochs for a Poisson process. Are they independent of each other? Of course not. The arrival epochs are the sums of the inter-arrival intervals. The inter-arrival intervals are the things that are independent. And the arrival epochs are the sums of inter-arrival intervals.

If you know that the first arrival epoch takes 10 times longer than its mean, then that the second arrival epoch is going to be kind of long, too. It's got to be at least 10 times as long as the mean of the inter-arrival epochs, because each arrival epoch is a sum of these inter-arrival epochs. It's the inter-arrival epochs that are independent.

So when you say that the one interval is independent of the other, yes, you know exactly what you mean. And the idea is very simple. It's the same idea here. But then you start to think you understand this, and you start to use it in a funny way. And suddenly you're starting to say that the arrival epochs are independent from one time to the other, which they certainly aren't.

What renewal theory does is it lets you treat the gross characteristics of a process in a very simple and straightforward way. So you're breaking up the process into two sets of views about it. One is the long term behavior, which you treat by renewal theory, and you use this one exotic theory in a simple and straightforward way for every different process, for every different renewal process you look at.

And then you have this usually incredibly complicated kind of thing in the inside of each arrival epoch. And the nice thing about renewal theory is it lets you look at that complicated thing without worrying about what's going on outside. So the local characteristics can be studied without worrying about the long term interactions.

One example of this, and one of the reasons we are now looking at Markov chains before we look at renewal processes is that a Markov chain is one of the nicest examples there is of a renewal process, when you look at it in the right way.

If you have a recurrent Markov chain, then the interval from one time entering a particularly recurrent state until the next time you enter that recurrent state is a renewal. So we look at the sequence of times at which we enter this one given state. Enter state one over here. We enter state one again over here. We enter state one again, and so forth.

We're ignoring everything that goes on between entries to state one. But every time you enter state 1, you're in the same situation as you were the last time you entered state one. You're in the same situation, in the sense that the inter-arrivals from state one to state one again are independent of what they were before.

In other words, when you enter state one, your successive state transitions from there are the same as they were before. So it's the same situation as we saw with Poisson processes, and it's the same kind of renewal where when you talk about renewal, you have to be very careful about what it is that's a renewable. Once you're careful about it, it's clear what's going on.

One of the things we're going to find out now is one of the things that we failed to point out before when we talked about finite state and Markov chains. One of the most interesting characteristics is the expected amount of time from one entry to a recurrent state until the next time you enter that recurrent state is 1 over pi sub i, where pi sub i is a steady state probability of that steady state.

Namely, we didn't do that. It's a little tricky to do that in terms of Markov chains it's almost trivial to do it in terms of renewal processes. And what's more, when we do it in terms of renewal processes, you will see that it's obvious, and you will never forget it.

If we did it in terms of Markov chains, it would be some long, tedious derivation, and you'd get this nice answer, and you say, why did that nice answer occur? And you wouldn't have any idea. When you look at renewal processes, it's obvious why it happens. And we'll see why that is very soon.

Also, after we finish renewal processes, the next thing we're going to do is to talk about accountable state Markov chains. Markov chains with accountable, infinitely countable set of states. If you don't have a background in renewal theory when you start to look at that, you get very confused. So renewal theory will give us the right tool to look at those more complicated Markov chains.

OK. So we carry from Markov chains with accountably infinite state space comes largely from renewal process. So yes, we'll be interested in understanding that. OK. Another example is GTM queue. The text talked a little bit, and we might have talked in class a little bit about this strange notation a queueing theorist used. There are always at least three letters separated by slashes to talk about what kind of queue you're talking about. The first letter describes the arrival process for the queue.

G means it's a general rival process, which doesn't really mean it's a general arrival process. It means the arrival process is renewal. Namely, it says the arrival process is IID, inter-arrivals. But you don't know what their distribution is. You would call that M if you meant a Poisson process, which would mean memory lists, inter-arrivals.

The second G stands for the service time distribution. Again, we assume that no matter how many servers you have, no matter how the servers work, the time to serve one user is independent of the time to serve other users. But that the distribution of that time has a general distribution. It would be M as you meant a memory list distribution, which would mean exponential distribution.

Finally, the thing at the end says we're talking about IQ with M servers. So the point here is we're talking about a relatively complicated thing. Can you talk about this in terms of renewals? Yes, you can, but it's not quite obvious how to do it. You would think that the obvious way of viewing a complicated queue like this is to look at what happens from one busy period to the next busy period.

You would think the busy periods would be independent of each other. But they're not quite. Suppose you finish one busy period, and when you finish the one busy period, one customer has just finished being served. But at that point, you're in the middle of the waiting for the next customer to arrive.

And as that's a general distribution, the amount of time you have to wait for that next customer to arrive depends on a whole lot of things in the previous interval. So how can you talk about renewals here? You talk about renewables by waiting until that next arrival comes.

When that next arrival comes to terminate the idle period between busy periods, at that time you're in the same state that you were in when the whole thing started before. When you had the first arrival come in. And at that point, you had one a rival there being served you go through some long complicated thing. Eventually the busy period is over. Eventually, then, another arrival comes in.

And presto, at that point, you're statistically back where you started. You're statistically back where you started in terms of all inter-arrival times at that point. And we will have to, even though it's intuitively obvious that those things are independent of each other, we're really going to have to sort that out a little bit, because you come upon many situations where this is not obvious.

So if you don't know how to sort it out when it is obvious, you're not going to know how to sort it out when it's not obvious. But anyway, that's another example of where we have renewals. OK. We want to talk about convergence now. This idea of convergence with probability one. It's based on the idea of numbers converging to some limit.

And I'm always puzzled about how much to talk about this, because all of you, when you first study calculus, talk about limits. Most of you, if you're engineers, when you talk about calculus, it goes in one ear and it goes out the other ear, because you don't have to understand this very much. Because all the things you deal with, the limits exist very nicely, and there's no problem. So you can ignore it.

And then you hear about these epsilons and deltas, and I do the same thing. I can deal with an epsilon, but as soon as you have an epsilon and a delta, I go into orbit. I have no idea what's going on anymore until I sit down and think about it very, very carefully. Fortunately, when we have a sequence of numbers, we only have an epsilon. We don't have a delta. So things are a little bit simpler.

I should warn you, though, that you can't let this go in one ear and out the other ear, because at this point, we are using the convergence of numbers to be able to talk about convergence of random variables, and convergence of random variables is indeed not a simple topic. Convergence of numbers is a simple topic made complicated by mathematicians.

Any good mathematician, when they hear me say this will be furious. Because in fact, when you think about what they've done, they've taken something which is simple but looks complicated, and they've turned it into something which looks complicated in another way, but is really the simplest way to deal with it. So let's do that and be done with it, and then we can start using it for random variables.

A sequence, b1, b2, b3, so forth of real numbers. Real numbers are complex numbers that doesn't make any difference, is said to converge to a limit, b. If for each real epsilon greater than zero, is there an integer M such that bn minus b is less than or equal to epsilon for all n greater than or equal to n? Now, how many people can look at that and understand it? Be honest. Good.

Some of you can. How many people look at that, and their mind just, ah! How many people are in that category? I am. But if I'm the only one, that's good. OK. There's an equivalent way to talk about this. A sequence of numbers, real or complex, is said to converge to limit b.

If for each integer k greater than zero, there's an integer m of k, such that bn minus b is less than or equal to 1 over k for all n greater than or equal to m. OK. And the argument there is pick any epsilon you want to, no matter how small. And then you pick a k, such that 1 over k is less than or equal to epsilon.

According to this definition, bn minus b less than or equal to 1 over k ensures that you have this condition up here that we're talking about. When bn minus b is less than or equal to 1 over k, then also bn minus b is less than or equal to epsilon. In other words, when you look at this, you're starting to see what this definition really means. Here, you don't really care about all epsilon.

All you care about is that this holds true for small enough epsilon. And the trouble is there's no way to specify a small enough epsilon. So the only way we can do this is to say for all epsilon. But what the argument is is if you can assert this statement for a sequence of smaller and smaller values of epsilon, that's all you need.

Because as soon as this is true for one value of epsilon, it's true for all smaller values of epsilon. Now, let me show you a picture which, unfortunately, there's a kind of a complicated picture. It's the picture that says what that argument was really talking about. So if you don't understand the picture, you were kidding yourself when you said you thought you understood what the definition said.

So what the picture says, it's in terms of this 1 over k business. It says if you have a sequence of numbers, b1, b2, b3, excuse me for insulting you by talking about something so trivial. But believe me, as soon as we start talking about random variables, this trivial thing mixed with so many other things will start to become less trivial, and you really need to understand what this is saying.

So we're saying if we have a sequence, b1, b2, b3, b4, b5 and so forth, what that second idea of convergence says is that there's an M1 which says that for all n greater than or equal to M1, b4, b5, b6, b7 minus b all lies within this limit here between b plus 1 and b minus 1.

There's a number M2, which says that as soon as you get bigger than M of 2, all these numbers lie between these two limits. There's a number M3, which says all of these numbers lie between these limits. So it's saying that, it's essentially saying that you can a pipe, and as n increases, you squeeze this pipe gradually down.

You don't know how fast you can squeeze it down when you're talking about convergence. You might have something that converges very slowly, and then M1 will be way out here. M2 will be way over there. M3 will be off on the other side of Vassar Street, and so forth. But there always is such an M1, M2, and M3, which says these numbers are getting closer and closer to b.

And they're staying closer and closer to b. An example, which we'll come back to, where you don't have convergence is the following kind of thing. b1 is equal to 3/4, in this case. b5 is equal to 3/4. b25 is equal to 3/4. b5 to the third is equal to 1. And so forth. These values at which b sub n is equal to 3/4, b is equal to little b plus 3/4 get more and more rare.

So in some sense, this sequence here where b2 up to b4 is zero. b6 up to b24 is zero and so forth. This is some kind of convergence, also. But it's not what anyone would call convergence. I mean, as far as numbers are concerned, there's only one kind of convergence that people ever talk about, and it's this kind of convergence here.

This, although these numbers are getting close to b in some sense. That's not viewed as convergence. So here, even though almost all the numbers are close to b, they don't stay close to b, in a sense. They always pop up at some place in the future, and that destroys the whole idea of convergence. It destroys most theorems about convergence.

That's an example where you don't have convergence. OK, random variables are really a lot more complicated than numbers. I mean, a random variable is a function from the sample space to real numbers. All of you know that's not really what a random variable is.

All of you know that a random variable is a number that wiggles around a little bit, rather than being fixed at what you ordinarily think of a number as being, right? Since that's a very imprecise notion, and the precise notion is very complicated, to build up your intuition about this, you have to really think hard about what convergence of random variables means.

For convergence and distribution, it's not the random variables, but the distribution function of the random variables that converge. In other words, in the distribution function of z sub n, where you have a sequence of random variables, z1, z2, z3 and so forth, the distribution function evaluated at each real value z converges for each z in the case where the distribution function of this final convergent random variable is continuous.

We all studied that. We know what that means now. For convergence and probability, we talked about convergence and probability in two ways. One with an epsilon and a delta. And then saying for every epsilon and delta that isn't big enough, something happens. And then we saw that it was a little easier to describe. It was a little easier to drive describe by saying the convergence in probability.

These distribution functions have to converge to a unit step. And that's enough. They converge to a unit steps at every z except where the step is. We talked about that. For convergence with probability one, and this is the thing we want to talk about today, this is the one that sounds so easy, and which is really tricky. I don't want to scare you about this.

If you're not scared about it to start with, I don't want to scare you. But I would like to convince you that if you think you understand it and you haven't spent a lot of time thinking about it, you're probably due for a rude awakening at some point. So for convergence with probability one, the set of sample paths that converge has probability one.

In other words, the sequence Y1, Y2 converges to zero with probability one. And now I'm going to talk about converging to zero rather than converging to some random variable. Because if you're interested in a sequence of random variables Z1, Z2 that converge to some other random variable Z, you can get rid of a lot of the complication by just saying, let's define a random variable y sub n, which is equal to z sub n minus c.

And then what we're interested in is do these random variables y sub n converged to 0. We can forget about what it's converging to, and only worry about it converging to 0. OK, so when we do that, this sequence of random variables converges to 0 with probability 1. If the probability of the set of sample points for which the sample path converges to 0. If that set of sample paths has probability 1-- namely, for almost everything in the space, for almost everything in its peculiar sense of probability-- if that holds true, then you say you have convergence with probability 1.

Now, that looks straightforward, and I hope it is. You can memorize it or do whatever you want to do with it. We're going to go on now and prove an important theorem about convergence with probability 1. I'm going to give a proof here in class that's a little more detailed than the proof I give in the notes. I don't like to give proofs in class. I think it's a lousy idea because when you're studying a proof, you have to go at your own pace.

But the problem is, I know that students-- and I was once a student myself, and I'm still a student. If I see a proof, I will only look at enough of it to say, ah, I get the idea of it. And then I will stop. And for this one, you need a little more than the idea of it because it's something we're going to build on all the time. So I want to go through this proof carefully. And I hope that most of you will follow most of it. And the parts of it that you don't follow, I hope you'll go back and think about it, because this is really important.

OK, so the theorem says, let this sequence of random variables satisfy the expected value of the magnitude of Y sub n, the sum from n equals 1 to infinity of this is less than infinity.

As usual there's a misprint there. The sum, the expected value of Yn, the bracket should be there. It's supposed to be less than infinity. Let me write that down. The sum from n equals 1 to infinity of expected value of the magnitude of Y sub n is less than infinity. So it's a finite sum. So we're talking about these Yn's when we start talking about the strong law of large numbers. Yn is going to be something like the sum from n equals 1 to m divided by m. In other words, it's going to be the sample average, or something like that. And these sample averages, if you have a mean 0, are going to get small.

The question is, when you sum all of these things that are getting small, do you still get something which is small? When you're dealing with the weak law of large numbers, it's not necessary that that sum gets small. It's only necessary that each of the terms get small. Here we're saying, let's assume also that this sum gets small.

OK, so we want to prove that under this condition, all of these sequences with probability 1 converge to 0, the individual sequences converge.

OK, so let's go through the proof now. And as I say, I won't do this to you very often. But I think for this one, it's sort of necessary.

OK, so first we'll use the Markov inequality. And I'm dealing with a finite value of m here. The probability that the sum of a finite set of these Y sub n's is greater than alpha is less than or equal to the expected value of that random variable. Namely, this random variable here. Sum from n equals 1 to m of magnitude of Y sub n. That's just a random variable. And the probability that that random variable is greater than alpha is less than or equal to the expected value of that random variable divided by alpha.

OK, well now, this quantity here is increasing in Y sub n. The magnitude of Y sub n is a non-negative quantity. You take the expectation of a non-negative quantity, if it has an expectation, which we're assuming here for this to be less infinity, all of these things have to have expectations. So as we increase m, this gets bigger and bigger. So this quantity here is going to be less than or equal to the sum from n equals 1 to infinity of expected value of Y sub n divided by alpha.

What I'm being careful about here is all of the things that happen when you go from finite m to infinite m. And I'm using what you know about finite m, and then being very careful about going to infinite m. And I'm going to try to explain why as we do it. But here, it's straightforward. The expected value of a finite sum is equal to the finite sum of an expected value.

When you go to the limit, m goes to infinity, you don't know whether these expected values exist or not. You're sort of confused on both sides of this equation. So we're sticking to finite values here.

Then, we're taking this quantity, going to the limit as m goes to infinity. This quantity has to get bigger and bigger as m goes to infinity, so this quantity has to be less than or equal to this.

This now, for a given alpha, is just a number. It's nothing more than a number, so we can deal with this pretty easily as we make alpha big enough. But for most of the argument, we're going to view alpha as being fixed.

OK, so now the probability that this sum, finite sum is greater than alpha, is less than or equal to this. This was the thing we just finished proving on the other page. This is less than or equal to that. That's what I repeated, so I'm not cheating you at all here.

Now, it's a pain to write that down all the time. So let's let the set, A sub m, be the set of sample points such that its finite sum of Y sub n of omega is greater than alpha. This is a random-- for each value of omega, this is just a number. The sum of the magnitude of Y sub n is a random variable. It takes on a numerical value for every omega in the sample space. So A sub m is the set of points in the sample space for which this quantity here is bigger than alpha.

So we can rewrite this now as just the probability of A sub m. So this is equivalent to saying, the probability of A sub m is less than or equal to this number here. For a fixed alpha, this is a number. This is something which can vary with m.

Since these numbers here now, now we're dealing with a sample space, which is a little strange. We're talking about sample points and we're saying, this number here, this magnitude of Y sub n at a particular sample point omega is greater than or equal to 0.

Therefore, a sub m is a subset of A sub m plus 1. In other words, as m gets larger and larger here, m here gets larger and larger. Therefore, this sum here gets larger and larger. Therefore, the set of omega for which this increasing sum is greater than alpha gets bigger and bigger. And that's the thing that we're saying here, A sub m is included in A sub m plus 1 for m greater than or equal to 1.

OK, so the left side of this quantity here, as a function of m, is a non-decreasing bounded sequence of real numbers. Yes, the probability of something is just a real number. A probability is a number. So this quantity here is a real number. It's a real number which is non-decreasing, so it keeps moving upward.

What I'm trying to do now is now, I went to the limit over here. I want to go to the limit here. And so I have a sequence of numbers in m. This sequence of numbers is non-decreasing. So it's moving up. Every one of those quantities is bounded by this quantity here. So I have an increasing sequence of real numbers, which is bounded on the top. What happens?

When you have a sequence of real numbers which is bounded-- I have a slide to prove this, but I'm not going to prove it because it's tedious.

Here we have this probability which I'm calling A sub m probability of A sub m. Here I have the probability of A sub m plus 1, and so forth. Here I have this limit up here. All of this sequence of numbers, there's an infinite sequence of them. They're all non-decreasing. They're all bounded by this number here. And what happens?

Well, either we go up to there as a limit or else we stop sometime earlier as a limit. I should prove this, but it's something we use all the time. It's a sequence of increasing or non-decreasing numbers. If it's bounded by something, it has to have a finite limit. The limit is less than or equal to this quantity. It might be strictly less, but the limit has to exist. And the limit has to be less than or equal to b.

OK, that's what we're saying here. When we go to this limit, this limit of the probability of A sub m is less than or equal to this number here.

OK, if I use this property of nesting intervals, when you have A sub 1 nested inside of A sub 2, nested inside of A sub 3, what we'd like to go do is go to this limit. The limit, unfortunately, doesn't make any sense in general. With this property of the axioms, it's equation number 9 in chapter 1 says that we can do something that's almost as good.

What it says that as we go to this limit here, what we get is that this limit is the probability of this infinite union. That's equal to the limit as m goes to infinity of probability of A sub m.

OK, look up equation 9, and you'll see that's exactly what it says. If you think this is obvious, it's not. It ain't obvious at all because it's not even clear that this-- well, nothing very much about this union is clear. We know that this union must be a measurable set. It must have a probability. We don't know much more about it. But anyway, that property tells us that this is true.

OK, so where we are at this point. I don't think I've skip something, have I? Oh, no, that's the thing I didn't want to talk about.

OK, so A sub m is a set of omega which satisfy this for finite m. The probability of this union is then the union of all of these quantities over all m. And this is less than or equal to this bound that we had.

OK, so I even hate giving proofs of this sort because it's a set of simple ideas. To track down every one of them is difficult. The text doesn't track down every one of them. And that's what I'm trying to do here.

We have two possibilities here, and we're looking at this limit here. This limiting sum, which for each omega is just a sequence, a non-decreasing sequence of real numbers. So one possibility is that this sequence of real numbers is bigger than alpha. The other possibility is that it's less than or equal to alpha. If it's less than or equal to alpha, then every one of these numbers is less than or equal to alpha and omega has to be not in this union here.

If the sum is bigger than alpha, then one of the elements in this set is bigger than alpha and omega is in this set. So what all of that says, and you're just going to have to look at that because it's not-- it's one of these tedious arguments. So the probability of omega such that this sum is greater than alpha is less than or equal to this number here.

At this point, we have made a major change in what we're doing. Before we were talking about numbers like probabilities, numbers like expected values. Here, suddenly, we are talking about sample points. We're talking about the probability of a set of sample points, such that the sum is greater than alpha. Yes?

AUDIENCE: I understand how if the whole sum if less than or equal than alpha then every element is. But did you say that if it's greater than alpha, then at least one element is greater than alpha? Why is that?

PROFESSOR: Well, because either the sum is less than or equal to alpha or it's greater than alpha. And if it's less than or equal to alpha, then omega is not in this set. So the alternative is that omega has to be in this set. Except the other way of looking at it is if you have a sequence of numbers, which is approaching a limit, and that limit is bigger than alpha, then one of the terms has to be bigger than alpha. Yes?

AUDIENCE: I think the confusion is between the partial sums and the terms of the sum. That's what he's confusing. Does that make sense? He's saying instead of each partial sum, not each term in the sum.

PROFESSOR: Yes. Except I don't see how that answers the question. Except the point here is, if each partial sum is less than or equal to alpha, then the limit has to be less than or equal to alpha. That's what I was saying on the other page. If you have a sequence of numbers, which has an upper bound on them, then you have to have a limit. And that limit has to be less than or equal to alpha. So that's this case here. We have a sum of numbers as we're going to the limit as m gets larger and larger, these partial sums have to go to a limit. The partial sums are all less than or equal to alpha. Then the infinite sum is less than or equal to alpha, and omega is not in this set here. And otherwise, it is.

OK, if I talk more about it, I'll get more confused. So I think the slides are clear.

Now, if we look at the case where alpha is greater than or equal to this sum, and we take the complement of the set, the probability of the set of omega for which this sum is less than or equal to alpha has-- oh, let's forget about this for the moment.

If I take the complement of this set, the probability of the set of omega, such that the sum is less than or equal to alpha, is greater than 1 minus this expected value here.

Now I'm saying, let's look at the case where alpha is big enough that it's greater than this number here. So this probability is greater than 1 minus this number. So if the sum is less than or equal to alpha for any given omega, then this quantity here converges.

Now I'm talking about sample sequences. I'm saying I have an increasing sequence of numbers corresponding to one particular sample point. This increasing set of numbers is less than or equal. Each element of it is less than or equal to alpha, so the limit of it is less than or equal to alpha. And what that says is the limit of Y sub n of omega, this has to be equal to 0 for that sample point. This is all the sample point argument.

And what that says then is the probability of omega, such that this limit here is equal to 0, that's this quantity here, which is the same as this quantity, which has to be greater than this quantity. This implies this. Therefore, the probability of this has to be bigger than this probability here.

Now, if we let alpha go to infinity, what that says is this quantity goes to 0 and the probability of the set of omega, such that this limit is equal to 0, is equal to 1.

I think if I try to spend 20 more minutes talking about that in more detail, it won't get any clearer. It is one of these very tedious arguments where you have to sit down and follow it step by step.

I wrote the steps that's very carefully. And at this point, I have to leave it as it is. But the theorem has been proven, at least in what's written, if not in what I've said.

OK, let's look at an example of this now. Let's look at the example where these random variables Y sub n for n greater than or equal to 1, have this following property. It's almost the same as the sequence of numbers I talked about before. But what I'm going to do now is-- these are not IID random variables. If they're IID random variables, you're never going to talk about the sum being finite. Sum of the expected values being finite.

How they behave is that for one less than or equal to 5, you pick one of these random variables in here and make it equal to 1. And all the rest are equal to 0. From 5 to 25, you pick one of the random variables, make it equal to 1, and all the others are equal to 0. You choose randomly in here. From 25 to 125, you pick one random variable, set it equal to 1. All the other random variables, set it equal to 0, and so forth forever after.

OK, so what does that say for the sample points? If I look at any particular sample point, what I find is that there's one occurrence of a sample value equal to 1 from here to here. There's exactly one that's equal to 1 from here to here. There's exactly one that's equal to 1 from here to way out here at 125, and so forth.

This is not a sequence of sample values which converges because it keeps popping up to 1 at all these values. So for every omega, Yn of omega is 1 for some n in this interval. For every j and it's 0 elsewhere. This Yn of omega doesn't converge for omega. So the probability that that sequence converges is not 1, it's 0. So this is a particularly awful example. This is a sequence of random variables, which does not converge with probability 1.

At the same time, the expected value of Y sub n is 1 over 5 to the j plus 1 minus 5 to the j. That's the probability that you pick that particular n for a random variable to be equal to 1. It's equal to this for 5 to the j less than or equal to n, less than 5 to the j plus 1.

When you add up all of these things, when you add up expected value of Yn equal to that over this interval, you get 1. When you add it up over the next interval, which is much, much bigger, you get 1 again. When you add it up over the next interval, you get 1 again. So the expected value of the sum-- the sum of the expected value of the Y sub n's is equal to infinity. And what you wind up with then is that this sequence does not converge--

This says the theorem doesn't apply at all. This says that the Y sub n of omega does not converge for any sample function at all. This says that according to the theorem, it doesn't have to converge. I mean, when you look at an example after working very hard to prove a theorem, you would like to find that if the conditions of the theorem are satisfied what the theorem says is satisfied also.

Here, the conditions are not satisfied. And you also don't have convergence with probability 1. You do have convergence in probability, however. So this gives you a nice example of where you have a sequence of random variables that converges in probability. It converges in probability because as n gets larger and larger, the probability that Y sub n is going to be equal to anything other than 0 gets very, very small. So the limit as n goes to infinity of the probability that Y sub n is greater than epsilon-- for any epsilon greater than 0, this probability is equal to 0 for all epsilon. So this quantity does converge in probability. It does not converge with probability 1. It's the simplest example I know of where you don't have convergence with probability 1 and you do have convergence in probability.

How about if you're looking at a sequence of sample averages.

Suppose you're looking at S sub n over n where S sub n is a sum of IID random variables. Can you find an example there where when you have a-- can you find an example where this sequence S sub n over n does converge in probability, but does not converge with probability 1?

Unfortunately, that's very hard to do. And the reason is the main theorem, which we will never get around to proving here is that if you have a random variable x, and the expected value of the magnitude of x is finite, then the strong law of large numbers holds.

Also, the weak law of large numbers holds, which says that you're not going to find an example where one holds and the other doesn't hold. So you have to go to strange things like this in order to get these examples that you're looking at.

OK, let's now go from convergence in probability 1 to applying this to the sequence of random variables where Y sub n is now equal to the sum of n IID random variable divided by n. Namely, it's the sample average, and we're looking at the limit as n goes to infinity of this sample average.

What's the probability of the set of sample points for which this sample path converges to X bar? And the theorem says that this quantity is equal to 1 if the expected value of X is less than infinity. We're not going to prove that, but what we are going to prove is that if the expected value of the fourth moment of X is finite, then we're going to prove that this theorem is true.

OK, when we write this from now on, we will sometimes get more terse. And instead of writing the probability of an omega in the set of sample points such that this limit for a sample point is equal to X bar, this whole thing is equal to 1. We can sometimes write it as the probability that this limit, which is now a limit of Sn of omega over n is equal to X bar. But that's equal to 1.

Some people write it even more tersely as the limit of S sub n over n is equal to X bar with probability 1. This is a very strange statement here because this-- I mean, what you're saying with this statement is not that this limit is equal to X bar with probability 1. It's saying, with probability 1, this limit here exists for a sample point, and that limit is equal to X bar.

The thing which makes the strong law of large numbers difficult is not proving that the limit has a particular value. If there is a limit, it's always easy to find what the value is. The thing which is difficult is figuring out whether it has a limit or not. So this statement is fine for people who understand what it says, but it's kind of confusing otherwise. Still more tersely, people talk about it as Sn over n goes to limit X bar with probability 1. This is probably an even better way to say it than this is because this is-- I mean, this says that there's something strange in the limit here. But I would suggest that you write it this way until you get used to what it's saying. Because then, when you write it this way, you realize that what you're talking about is the limit over individual sample points rather than some kind of more general limit. And convergence with probability 1 is always that sort of convergence.

OK, this strong law and the idea of convergence with probability 1 is really pretty different from the other forms of convergence. In the sense that it focuses directly on sample paths. The other forms of convergence focus on things like the sequence of expected values, or where the sequence of probabilities, or sequences of numbers, which are the things you're used to dealing with. Here you're dealing directly with sample points, and it makes it more difficult to talk about the rate of convergence as n approaches infinity.

You can't talk about the rate of convergence here as n approaches infinity. If you have any n less than infinity, if you're only looking at a finite sequence, you have no way of saying whether any of the sample values over that sequence are going to converge or whether they're not going to converge, because you don't know what the rest of them are. So talking about a rate of convergence with respect to the strong law of large numbers doesn't make any sense.

It's connected directly to the standard notion of a convergence of a sequence of numbers when you look at those numbers applied to a sample path. This is what gives the strong law of large numbers its power, the fact that it's related to this standard idea of convergence.

The standard idea of convergence is what the whole theory of analysis is built on. And there are some very powerful things you can do with analysis. And it's because convergence is defined the way that it is.

When we talk about the strong law of large numbers, we are locked into that particular notion of convergence. And therefore, it's going to have a lot of power. We will see this as soon as we start talking about renewal theory. And in fact, we'll see it in the proof of the strong law that we're going to go through.

Most of the heavy lifting with the strong law of large numbers has been done by the analysis of convergence with probability 1. The hard thing is this theorem we've just proven. And that's tricky. And I apologize for getting a little confused about it as we went through it, and not explaining all the steps completely. But as I said, it's hard to follow proofs in real time anyway. But all of that is done now.

How do we go through the strong law of large numbers now if we accept this convergence with probability 1? Well, it turns out to be pretty easy. We're going to assume that the expected value of the fourth moment of this underlying random variable is less than infinity. So let's look at the expected value of the sum of n random variables taken to the fourth power.

OK, so what is that? It's the expected value of S sub n times S sub n times S sub n times S sub n. Sub n is the sum of Xi from 1 to n. It's also this. It's also this. It's also this. So the expected value of S to the n fourth is the expected value of this entire product here. I should have a big bracket around all of that.

If I multiply all of these terms out, each of these terms goes from 1 to n, what I'm going to get is the sum from 1 to n. Sum over j from 1 to n. Sum over k from 1 to n. And a sum over l from 1 to n. So I'm going to have the expected value of X sub i times X sub j times X sub k times X sub l. Let's review what this is.

X sub i is the random variable for the i-th of these X's. I have n X's-- X1, X2, X3, up to X sub n. What I'm trying to find is the expected value of this sum to the fourth power.

When you look at the sum of something, if I look at the sum of numbers, [INAUDIBLE] of a sub i, times the sum of b sub i, I write it as j.

If i just do this, what's it equal to? It's equal to the sum over i and j of a sub i times a sub j. I'm doing exactly the same thing here, but I'm taking the expected value of it. That's a finite sum. the expected value of the sum is equal to the sum of the expected values.

So if I look at any particular value of X-- of this first X here. Suppose I look at i equals 1 . I suppose I look at the expected value of X1 times-- and I'll make this anything other than 1. I'll make this anything other than 1, and this anything other than 1.

For example, suppose I'm trying to find the expected value of X1 times X2 times X10 times X3. OK, what is that?

Since X1, X2, X3 are all independent of each other, the expected value of X1 times the expected value of all these other things is the expected value of X1 conditional on the values of these other quantities. And then I average over all the other quantities.

Now, if these are independent random variables, the expected value of this given the values of these other quantities is just the expected value of X1. I'm dealing with a case where the expected value of X is equal to 0. Assuming X bar equals 0.

So when I pick i equal to 1 and all of these equal to something other than 1, this expected value is equal to 0. That's a whole bunch of expected values because that includes j equals 2 to n, k equals 2 to n, and X sub l equals 2 to n.

Now, I can do this for X sub i equals 2, X sub i equals 3, and so forth. If i is different from j, and k, and l, this expected value is equal to 0. And the same thing if X sub j is different than all the others. The expected value is equal to 0. So how can I get anything that's nonzero?

Well, if I look at X sub 1 times X sub 1 times X sub 1 times X sub 1, that gives me expected value of X to the fourth. That's not 0, presumably. And I have n terms like that.

Well, I'm getting down to here. What we have is two kinds of nonzero terms. One of them is where i is equal to j is equal to k is equal to l. And then we have X sub i to the fourth power. And we're assuming that's some finite quantity. That's the basic assumption we're using here, expected value of X fourth is less than infinity.

What other kinds of things can we have? Well, if i is equal to j, and if k is equal to l, then I have the expected value of Xi squared times expected value of Xi squared Xk squared. What is that?

Xi squared is independent of Xk squared because i is unequal to k. These are independent random variables. So I have the expected value of Xi squared is what? It's just a variance of X. This quantity here is the variance of X also. So I have the variance of Xi squared, which is squared. So I have sigma to the fourth power. So those are the only terms that I have for this second kind of nonzero term where Xi-- excuse me. not Xi is equal to Xj. That's not what we're talking about. Where i is equal to j. Namely, we have a sum where i runs from 1 to n, where j runs from 1 to n, k runs from 1 to n, and l runs from 1 to n.

What we're looking at is, for what values of i, j, k, and l is this quantity not equal to 0? We're saying that if i is equal to j is equal to k is equal to l, then for all of those terms, we have the expected value of X fourth. For all terms in which i is equal to j and k is equal to l, for all of those terms, we have the expected value of X sub i quantity squared.

Now, how many of those terms are there?

Well, x sub i can be any one of n terms. x sub j can be any one of how many terms? It can't be equal. i is equal to j, how many things can k be? It can't be equal to i because then we would wind up with X sub i to the fourth power. So we're looking at n minus 1 possible values for k, n possible values for i. So there are n times n minus 1 of those terms.

I can also have-- let me write in this way. Times Xk Xl equals k. I can have those terms. I can also have Xi Xj unequal to i. Xk equal to k and Xl equal to i. I can have terms like this. And that gives me a sigma fourth term also.

I can also have Xi Xj unequal to i. k can be equal to i and l can be equal to j. So I really have three kinds of terms. I have three times n times n minus 1 times the expected value of X squared, this quantity squared. So that's the total value of expected value of S sub n to the fourth. It's the n terms for which i is equal to j is equal to k is equal to l plus the 3n times n minus 1 terms in which we have two pairs of equal terms. So we have that quantity here.

Now, expected value of X fourth is the second moment of the random variable X squared. So the expected value of X squared squared is the mean of X squared squared. And that's less than or equal to the variance of X squared, which is this quantity. The expected value of Sn fourth is-- well, actually it's less than or equal to 3n squared times the expected value of X fourth. And blah, blah, blah, until we get to 3 times the expected value of X fourth times the sum from n equals 1 to infinity of 1 over n squared.

Now, is that quantity finite or is it infinite? Well, let's talk of three different ways of showing that this sum is going to be finite.

One of the ways is that this is an approximation, a crude approximation, of the integral from 1 to infinity of 1 over X squared. You know that that integral is finite.

Another way of doing it is you already know that if you take 1 over n times 1 over n plus 1, you know how to sum that. That sum is finite. You can bound this by that.

And the other way of doing it is simply to know that the sum of 1 over n squared is finite.

So what this says is that the expected value of S sub n fourth over n fourth is less than infinity. That says that the probability of the set of omega for which S to the fourth over n fourth is equal to 0 is equal to 1. in other words, it's saying that S to the fourth over omega over n fourth converges to 0. That's not quite what we want, is it?

But the set of sample points for which this quantity converges has probability 1. And here is where you see the real power of the strong law of large numbers. Because if these numbers converge to 0 with probability 1, what happens to the set of numbers is Sn to the fourth of omega divided by n to the fourth, this limit-- if this was equal to 0, then what is the limit as n approaches infinity of Sn of omega over n to the fourth?

If I take the fourth root of this, I get this. If this quantity is converging to 0, the fourth root of this also has to be converging to 0 on a sample path basis of the fact that this converges means that this converges also.

Now, you see if you were dealing with convergence in probability or something like that, you couldn't play this funny game. And the ability to play this game is really what makes convergence in probability a powerful concept. You can do all sorts of strange things with it. And we'll talk about that next time. But that's why all of this works. So that's what says that the probability of the set of omega for which the limits of Sn of omega over n equals 0 equals 1.

Now, let's look at the strange aspect of what we've just done. And this is where things get very peculiar. Let's look at the Bernoulli case, which by now we all understand. So we consider a Bernoulli process, all moments of X exist. Moment-generating functions of X exist. X is about as well-behaved as you can expect because it only has the values 1 or 0. So it's very nice.

The expected value of X is going to be equal to p in this case. The set of sample paths for which Sn of omega over n is equal to p has probability 1. In other words, with probability 1, when you look at a sample path and you look at the whole thing from n equals 1 off to infinity, and you take the limit of that sample path as n goes to infinity, what you get is p. And the probability that you get p is equal to 1.

Well, now, the thing that's disturbing is, if you look at another Bernoulli process where the probability of the 1 is p prime instead of p. What happens then? With probability 1, you get convergence of Sn of omega over n, but the convergence is to p prime instead of to p.

The events in these two spaces are exactly the same. We've changed the probability measure, but we've kept all the events the same. And by changing the probability measure, we have changed one set of probability 1 into a set of probability 0. And we changed another set of probability 0 into set of probability 1. So we have two different events here.

On one probability measure, this event has probability 1. On the other one, it has probability 0. They're both very nice, very well-behaved probabilistic situations. So that's a little disturbing.

But then you say, you can pick p in an uncountably infinite number of ways. And for each way you count p, you have uncountably many events. Excuse me, for each value of p, you have one event of probability 1 for that p. So as you go through this uncountable number of events, you go through this uncountable number of p's, you have an uncountable number of events, each of which has probability 1 for its own p.

And now the set of sequences that converge is, in fact, a rather peculiar sequence to start with. So if you look at all the other things that are going to happen, there are an awful lot of those events also. So what is happening here is that these events that we're talking about are indeed very, very peculiar events. I mean, all the mathematics works out. The mathematics is fine. There's no doubt about it.

In fact, the mathematics of probability theory was worked out. People like Kolmogorov went to great efforts to make sure that all of this worked out. But then he wound up with this peculiar kind of situation here. And that's what happens when you go to an infinite number of random variables. And it's ugly, but that's the way it is. So that what I'm arguing here is that when you go from finite m to infinite n, and you start interchanging limits, and you start taking limits without much care and you start doing all the things that you would like to do, thinking that infinite n is sort of the same as finite n.

In most places in probability, you can do that and you can away with it. As soon as you start dealing with the strong law of large numbers, you suddenly really have to start being careful about this. So from now on, we have to be just a little bit careful about interchanging limits, interchanging summation and integration, interchanging all sorts of things, as soon as we have an infinite number of random variables. So that's a care that we have to worry about from here on.

OK, thank you.