1 00:00:00,530 --> 00:00:02,960 The following content is provided under a Creative 2 00:00:02,960 --> 00:00:04,370 Commons license. 3 00:00:04,370 --> 00:00:07,410 Your support will help MIT OpenCourseWare continue to 4 00:00:07,410 --> 00:00:11,060 offer high quality educational resources for free. 5 00:00:11,060 --> 00:00:13,960 To make a donation or view additional materials from 6 00:00:13,960 --> 00:00:19,790 hundreds of MIT courses, visit MIT OpenCourseWare at 7 00:00:19,790 --> 00:00:21,040 ocw.mit.edu. 8 00:00:22,600 --> 00:00:27,030 PROFESSOR: OK, today we are moving on with renewal 9 00:00:27,030 --> 00:00:39,150 processes, and we're going to finish up talking a little 10 00:00:39,150 --> 00:00:44,080 more about residual life, and we're going to generalize it 11 00:00:44,080 --> 00:00:49,440 to time averages for arbitrary renewal processes. 12 00:00:49,440 --> 00:00:53,590 And then finally we're going into the topic of stopping 13 00:00:53,590 --> 00:00:56,660 trials, stopping times, optional stopping times-- 14 00:00:56,660 --> 00:00:59,290 people call them different things-- 15 00:00:59,290 --> 00:01:00,540 and Wald's equality. 16 00:01:03,660 --> 00:01:07,140 This is, for some reason or other, normally a very tricky 17 00:01:07,140 --> 00:01:09,840 thing to understand. 18 00:01:09,840 --> 00:01:15,140 I think finally after many years I understand it. 19 00:01:15,140 --> 00:01:18,700 And I hope that I can make it easy for you to understand it, 20 00:01:18,700 --> 00:01:23,940 so some of the confusions that often come in won't come in. 21 00:01:23,940 --> 00:01:27,120 And then we're going to end up talking about stopping when 22 00:01:27,120 --> 00:01:33,590 you're ahead, which is a gambling strategy which says 23 00:01:33,590 --> 00:01:37,940 if you're playing a fair game you play until you're $1.00 24 00:01:37,940 --> 00:01:40,440 ahead, and then you stop. 25 00:01:40,440 --> 00:01:42,670 And you show that with probability 1 you will 26 00:01:42,670 --> 00:01:46,700 eventually become $1.00 ahead, so you have a way to beat the 27 00:01:46,700 --> 00:01:50,100 bank, or beat the casino. 28 00:01:50,100 --> 00:01:53,500 But of course casinos don't give you even odds, but you 29 00:01:53,500 --> 00:01:58,550 have a way to win any time you get even odds. 30 00:01:58,550 --> 00:02:00,510 We'll find that there's something wrong with that, but 31 00:02:00,510 --> 00:02:06,600 it is an interesting topic which illustrates Wald's 32 00:02:06,600 --> 00:02:09,729 equality in an interesting way. 33 00:02:09,729 --> 00:02:12,930 So let's start out by reviewing a little bit. 34 00:02:17,270 --> 00:02:21,630 We first talked last time about convergence with 35 00:02:21,630 --> 00:02:24,880 probability 1, and we weren't talking about the strong law 36 00:02:24,880 --> 00:02:28,500 then, we were just talking about a sequence of random 37 00:02:28,500 --> 00:02:32,760 variables and what it means for them to converge with 38 00:02:32,760 --> 00:02:34,480 probability 1. 39 00:02:34,480 --> 00:02:40,440 And the theorem is that if you have a sequence of random 40 00:02:40,440 --> 00:02:46,260 variables, z sub n, they converge to some number alpha. 41 00:02:46,260 --> 00:02:48,540 This is slightly different than the theorem we stated 42 00:02:48,540 --> 00:02:53,640 last time, but it's trivially the same. 43 00:02:53,640 --> 00:02:56,810 In other words, it's the probability of the set of 44 00:02:56,810 --> 00:02:58,000 sample points. 45 00:02:58,000 --> 00:03:01,760 A sample point, now, is something which runs from time 46 00:03:01,760 --> 00:03:03,800 0 to time infinity. 47 00:03:03,800 --> 00:03:07,670 It's something that covers the whole process, covers anything 48 00:03:07,670 --> 00:03:09,670 else you might want to talk about also. 49 00:03:09,670 --> 00:03:15,460 It's what you do when you go from a real situation to a 50 00:03:15,460 --> 00:03:19,610 model, and mathematically when you go to a model you put in 51 00:03:19,610 --> 00:03:20,670 everything you're going to be 52 00:03:20,670 --> 00:03:23,270 interested in at the beginning. 53 00:03:23,270 --> 00:03:30,260 So, it converges to some alpha probability 1 if the set of 54 00:03:30,260 --> 00:03:34,520 sequences for which the statement holds has 55 00:03:34,520 --> 00:03:40,350 probability 1, and this statement is that the sample 56 00:03:40,350 --> 00:03:43,050 paths converge to alpha. 57 00:03:43,050 --> 00:03:46,300 The set of sample paths has converged to alpha as 58 00:03:46,300 --> 00:03:47,280 probability 1. 59 00:03:47,280 --> 00:03:50,640 So, it's a statement not like most of the statements in 60 00:03:50,640 --> 00:03:54,200 probability where you start out talking about finite 61 00:03:54,200 --> 00:03:56,750 length, talking about probabilities, and 62 00:03:56,750 --> 00:03:59,110 expectation, and so forth. 63 00:03:59,110 --> 00:04:02,260 And then you go to the limit with these things you're used 64 00:04:02,260 --> 00:04:05,220 to talking about, which are easier to deal with. 65 00:04:05,220 --> 00:04:09,060 Here it's a statement about sample paths 66 00:04:09,060 --> 00:04:11,280 over the whole sequence. 67 00:04:11,280 --> 00:04:14,580 You break them into two categories. 68 00:04:14,580 --> 00:04:17,480 Implicitly we have shown that when you break them into two 69 00:04:17,480 --> 00:04:22,220 categories those two categories in fact have 70 00:04:22,220 --> 00:04:23,060 probabilities. 71 00:04:23,060 --> 00:04:28,540 They are events, and one of them has probability 1. 72 00:04:28,540 --> 00:04:34,750 For renewal processes, if we have a renewal process we can 73 00:04:34,750 --> 00:04:40,030 use this theorem to talk about the sample averages. 74 00:04:40,030 --> 00:04:45,970 In other words, in a renewal process we have a sequence of 75 00:04:45,970 --> 00:04:51,800 IID random variables, x1, x2, x3, and so forth. 76 00:04:51,800 --> 00:04:56,410 We have the sample sums, the sum of the time 1, the sum of 77 00:04:56,410 --> 00:05:01,920 the time 2, sum of the time 3, and so forth. 78 00:05:01,920 --> 00:05:06,330 And the strong law of large numbers, which comes pretty 79 00:05:06,330 --> 00:05:10,540 much directly out of this convergence of probability 1 80 00:05:10,540 --> 00:05:16,190 theorem, says that the probability is the set of 81 00:05:16,190 --> 00:05:24,530 sample points for which the limit of the sample average, 82 00:05:24,530 --> 00:05:26,990 namely sn of omega over n. 83 00:05:26,990 --> 00:05:31,500 You take the sample average over all lengths for one 84 00:05:31,500 --> 00:05:32,850 sample point. 85 00:05:32,850 --> 00:05:35,380 That goes to a limit, that's this convergence with 86 00:05:35,380 --> 00:05:40,180 probability 1, the probability that you get convergence is 87 00:05:40,180 --> 00:05:45,740 equal to 1, which is what with probability 1 means. 88 00:05:56,710 --> 00:05:58,480 Then we use the theorem on the top. 89 00:06:03,700 --> 00:06:05,360 Let's reset a little bit. 90 00:06:05,360 --> 00:06:08,690 This theorem on the top is more than just a theorem about 91 00:06:08,690 --> 00:06:12,280 convergence with probability 1. 92 00:06:12,280 --> 00:06:15,400 It's the reason why convergence with probability 1 93 00:06:15,400 --> 00:06:17,110 is so useful. 94 00:06:17,110 --> 00:06:22,400 It says if you have some function f of x, that function 95 00:06:22,400 --> 00:06:27,950 is continuous at this point alpha. 96 00:06:27,950 --> 00:06:32,020 You have this limit here, then the probability of the set of 97 00:06:32,020 --> 00:06:36,440 omega for which the function applied to each of these 98 00:06:36,440 --> 00:06:44,318 sample points as equal to alpha is also one. 99 00:06:44,318 --> 00:06:46,640 AUDIENCE: [INAUDIBLE]. 100 00:06:46,640 --> 00:06:48,230 PROFESSOR: It should be f of alpha. 101 00:06:58,680 --> 00:06:59,940 Yes, absolutely. 102 00:06:59,940 --> 00:07:01,190 Sorry about that. 103 00:07:11,110 --> 00:07:14,340 This is f of alpha right there. 104 00:07:14,340 --> 00:07:16,610 So that probability is equal to 1. 105 00:07:19,540 --> 00:07:22,130 I'm finally getting my bearings back on this. 106 00:07:25,910 --> 00:07:31,550 So if I use this theorem here, where I use f of x as equal to 107 00:07:31,550 --> 00:07:37,810 1 over x, then what it says for a renewal process, which 108 00:07:37,810 --> 00:07:44,160 has inter-renewals x1, x2, and so forth, with a inter-renewal 109 00:07:44,160 --> 00:07:49,920 process the expected inter-renewal time, which 110 00:07:49,920 --> 00:07:52,830 might be infinite in general, but it has to 111 00:07:52,830 --> 00:07:54,730 be bigger than 0. 112 00:07:54,730 --> 00:08:02,960 It has to be bigger than 0 because renewals in 0 time are 113 00:08:02,960 --> 00:08:05,890 not possible by definition. 114 00:08:05,890 --> 00:08:10,670 And you've done a problem in the homework, or you will do a 115 00:08:10,670 --> 00:08:13,930 problem in the homework, whether you've done it or not, 116 00:08:13,930 --> 00:08:19,042 I don't know, but when you do it, it will in fact show that 117 00:08:19,042 --> 00:08:23,390 the expected value of x has to be greater than 0. 118 00:08:23,390 --> 00:08:26,930 We're also assuming that it's less than infinity. 119 00:08:26,930 --> 00:08:35,039 So, at that point we can use this theorem which says that 120 00:08:35,039 --> 00:08:41,260 when you look at, not sn of omega over n, but the inverse 121 00:08:41,260 --> 00:08:45,850 of that, n over sn of omega, the limit of that is going to 122 00:08:45,850 --> 00:08:52,730 be equal to f of x-bar which is 1 over x-bar. 123 00:08:52,730 --> 00:08:56,220 So we have this limit here which is equal to 1 with 124 00:08:56,220 --> 00:08:58,230 probability 1. 125 00:08:58,230 --> 00:09:03,610 This is almost the strong law for renewal processes, and the 126 00:09:03,610 --> 00:09:10,560 reason why it's not takes a couple of pages in the text. 127 00:09:10,560 --> 00:09:15,040 What I'm going to try to argue here is that the couple of 128 00:09:15,040 --> 00:09:24,880 pages, while you need to do mathematically, you can see 129 00:09:24,880 --> 00:09:27,240 why when you do it you can see what answer 130 00:09:27,240 --> 00:09:28,440 you're going to get. 131 00:09:28,440 --> 00:09:30,290 And the argument is the following. 132 00:09:30,290 --> 00:09:32,910 We talked about this last time a little bit. 133 00:09:32,910 --> 00:09:38,420 If you take an arbitrary time t then you look at n of t for 134 00:09:38,420 --> 00:09:42,860 that value of t, that's the number of arrivals you've had 135 00:09:42,860 --> 00:09:46,370 in this renewal process up until time t. 136 00:09:46,370 --> 00:09:50,460 And then you compare this point down here, which is the 137 00:09:50,460 --> 00:09:55,280 time of the last arrival before time t, and this point 138 00:09:55,280 --> 00:10:00,320 here, which is the time of the next arrival after time t, and 139 00:10:00,320 --> 00:10:04,390 you look at the slopes here, this slope here 140 00:10:04,390 --> 00:10:07,090 is n of t over t. 141 00:10:07,090 --> 00:10:08,520 That's the thing we're interested in. 142 00:10:15,380 --> 00:10:18,520 That expression there is the time average of the number of 143 00:10:18,520 --> 00:10:23,830 renewals within time t, and what we're interested in is 144 00:10:23,830 --> 00:10:27,210 what this becomes as t goes to infinity. 145 00:10:27,210 --> 00:10:31,920 That's squeezed between this slope here, which is n of t 146 00:10:31,920 --> 00:10:35,240 over s sub n of t. 147 00:10:35,240 --> 00:10:38,760 This is s sub n of t. 148 00:10:38,760 --> 00:10:45,420 And this point here is n of t, and this lower band here, 149 00:10:45,420 --> 00:10:50,680 which is n of t divided by s sub n of t plus 1. 150 00:10:50,680 --> 00:10:57,480 Now, all this argument is devoted to proving two things, 151 00:10:57,480 --> 00:11:01,040 which are almost obvious and therefore I'm just going to 152 00:11:01,040 --> 00:11:03,350 wave my hands about it. 153 00:11:03,350 --> 00:11:08,450 What this says is when t gets very, very large, n of t 154 00:11:08,450 --> 00:11:10,140 becomes very, very large also. 155 00:11:12,770 --> 00:11:16,830 When n of t becomes very, very large, the difference between 156 00:11:16,830 --> 00:11:20,250 this slope and this slope doesn't make a hill of beans 157 00:11:20,250 --> 00:11:23,670 worth of difference, and therefore those two, that 158 00:11:23,670 --> 00:11:27,320 upper band and that lower band become the same. 159 00:11:27,320 --> 00:11:33,180 So with the argument as t gets large, n of t gets large, and 160 00:11:33,180 --> 00:11:36,450 these two slopes become the same, you suddenly have the 161 00:11:36,450 --> 00:11:44,900 statement that if you have a renewal process and the 162 00:11:44,900 --> 00:11:48,830 inter-renewal time is less than infinity, it has to be 163 00:11:48,830 --> 00:11:50,520 greater than 0. 164 00:11:50,520 --> 00:11:57,950 Then on a sample path basis the sample paths, the number 165 00:11:57,950 --> 00:12:04,960 of arrivals per unit time, converges to 1 over x-bar. 166 00:12:04,960 --> 00:12:09,370 That's what this previous picture was saying to you. 167 00:12:09,370 --> 00:12:19,210 It was saying that n over t over t, which is the number of 168 00:12:19,210 --> 00:12:24,300 arrivals per unit time t, as t becomes very, very large on a 169 00:12:24,300 --> 00:12:28,170 sample path basis that becomes equal to 1 over x-bar. 170 00:12:33,700 --> 00:12:38,190 So, it says that the rate of renewals over the infinite 171 00:12:38,190 --> 00:12:43,330 time horizon is 1 over x-bar with probability 1. 172 00:12:43,330 --> 00:12:46,530 And what probability 1 statement again is saying that 173 00:12:46,530 --> 00:12:50,090 the set of sample paths for which this is true has 174 00:12:50,090 --> 00:12:51,960 probability 1. 175 00:12:51,960 --> 00:12:55,430 This also implies the weak law for renewals, which says that 176 00:12:55,430 --> 00:12:59,420 the limit as t approaches infinity of the probability 177 00:12:59,420 --> 00:13:03,970 that n of t over t minus 1 over x-bar is greater than 178 00:13:03,970 --> 00:13:07,520 epsilon goes to 0. 179 00:13:07,520 --> 00:13:10,180 This, surprisingly enough, is a statement that 180 00:13:10,180 --> 00:13:12,460 you hardly ever see. 181 00:13:12,460 --> 00:13:16,330 You see a bunch of other renewal theorems, which this 182 00:13:16,330 --> 00:13:20,180 theorem up here is probably the most important. 183 00:13:20,180 --> 00:13:23,500 There's something called an elementary renewal theorem, 184 00:13:23,500 --> 00:13:26,610 which is sort of a trivial statement but 185 00:13:26,610 --> 00:13:28,860 kind of hard to prove. 186 00:13:28,860 --> 00:13:31,840 And there's something called Blackwell's theorem, which 187 00:13:31,840 --> 00:13:35,760 talks about a local period of time after time gets very, 188 00:13:35,760 --> 00:13:36,400 very large. 189 00:13:36,400 --> 00:13:40,640 It's talking about the number of renewals in some tiny 190 00:13:40,640 --> 00:13:46,700 interval and it's proving things about stationarity over 191 00:13:46,700 --> 00:13:48,830 that local time interval. 192 00:13:48,830 --> 00:13:53,830 And we'll talk about that later, but this is sort of a 193 00:13:53,830 --> 00:14:00,000 nice result, and that's what it is. 194 00:14:00,000 --> 00:14:03,370 Let's review residual life a little bit. 195 00:14:03,370 --> 00:14:09,870 The residual life of a renewal process at time t, the 196 00:14:09,870 --> 00:14:14,560 residual life is the time you have to wait until the next 197 00:14:14,560 --> 00:14:17,480 renewal, until the next arrival. 198 00:14:17,480 --> 00:14:21,990 So residual life means, how long does it take? 199 00:14:21,990 --> 00:14:24,710 If you think of these renewals as something that comes along 200 00:14:24,710 --> 00:14:30,370 and kills everybody, then this residual life is the amount of 201 00:14:30,370 --> 00:14:33,230 life that people still have left. 202 00:14:33,230 --> 00:14:37,770 So, if you look at what that is for a particular sample 203 00:14:37,770 --> 00:14:41,080 path, I have a particular sample path drawn here, 204 00:14:41,080 --> 00:14:44,080 completely arbitrary. 205 00:14:44,080 --> 00:14:48,490 When I look at the first arrival time here, s1 of 206 00:14:48,490 --> 00:14:52,410 omega, and I look at all of the time before that, the 207 00:14:52,410 --> 00:14:57,350 amount of time you have to wait until this arrival. 208 00:14:57,350 --> 00:15:01,650 Remember this is one sample point. 209 00:15:04,480 --> 00:15:08,690 We're not thinking in terms of an observer who comes in and 210 00:15:08,690 --> 00:15:11,410 doesn't know what's going to happen in the future. 211 00:15:11,410 --> 00:15:12,990 We're looking at one sample point. 212 00:15:12,990 --> 00:15:14,920 We know the whole sample point. 213 00:15:14,920 --> 00:15:18,356 And what we're doing we know this, we know this, we know 214 00:15:18,356 --> 00:15:24,610 this, and all we're doing here saying for each time t along 215 00:15:24,610 --> 00:15:29,120 this infinite path we're just plotting how long it is until 216 00:15:29,120 --> 00:15:30,620 the next arrival. 217 00:15:30,620 --> 00:15:34,260 And that's just a line with slope minus 1 until the next 218 00:15:34,260 --> 00:15:35,660 arrival occurs. 219 00:15:35,660 --> 00:15:38,860 After the next arrival and you start waiting for the arrival 220 00:15:38,860 --> 00:15:41,640 after that, after that when you wait for the 221 00:15:41,640 --> 00:15:43,220 arrival after that. 222 00:15:43,220 --> 00:15:49,570 So on a sample path basis, the residual life is just this 223 00:15:49,570 --> 00:15:55,580 sequence of isosceles triangles here. 224 00:15:55,580 --> 00:16:01,490 So we looked at that, we said, if we look at the time from n 225 00:16:01,490 --> 00:16:07,950 equals 1 up until the number of arrivals at time t, the 226 00:16:07,950 --> 00:16:13,240 number of triangles we have is n of t omega. 227 00:16:13,240 --> 00:16:20,130 The size of the area of each triangle is xi squared over 2. 228 00:16:24,830 --> 00:16:28,610 The size of each triangle is xi squared over 2, and when we 229 00:16:28,610 --> 00:16:31,800 get all done we want to take the average of it so we divide 230 00:16:31,800 --> 00:16:33,620 each of these by t. 231 00:16:33,620 --> 00:16:37,570 This quantity here is less than or equal to the integral 232 00:16:37,570 --> 00:16:43,750 of y of t over t, and that's less than or equal to the same 233 00:16:43,750 --> 00:16:48,700 sum where if we look at a particular time of t this sum 234 00:16:48,700 --> 00:16:52,500 here is summing everything up to this point. 235 00:16:52,500 --> 00:16:57,500 This sum here is summing everything up to this point. 236 00:16:57,500 --> 00:17:01,710 And as t goes to infinity, this one little triangle in 237 00:17:01,710 --> 00:17:05,410 here, even though this is the biggest triangle there, it 238 00:17:05,410 --> 00:17:07,010 still doesn't make any difference. 239 00:17:07,010 --> 00:17:09,790 As t goes to infinity, that washes out. 240 00:17:09,790 --> 00:17:12,410 You have to show that, of course, but that's what we 241 00:17:12,410 --> 00:17:13,660 showed last time. 242 00:17:16,079 --> 00:17:22,119 When we go to the limit we find that this time average 243 00:17:22,119 --> 00:17:26,450 residual life is equal to the expected value of x squared 244 00:17:26,450 --> 00:17:29,150 over twice the main effect. 245 00:17:29,150 --> 00:17:31,980 Now, this can't be infinite. 246 00:17:31,980 --> 00:17:38,710 You can have a finite expected in a renewal time and you can 247 00:17:38,710 --> 00:17:42,500 still have an infinite second moment. 248 00:17:42,500 --> 00:17:49,570 If you look at the example we talked about last time where 249 00:17:49,570 --> 00:17:55,250 the inter-renewal time at two possible values, either 250 00:17:55,250 --> 00:17:59,100 epsilon or 1 over epsilon, in other words, it was either 251 00:17:59,100 --> 00:18:04,280 enormous or it was very, very small, what we found out there 252 00:18:04,280 --> 00:18:07,090 was that some of these inter-renewal intervals, a 253 00:18:07,090 --> 00:18:10,540 very small fraction of them, but a very small fraction of 254 00:18:10,540 --> 00:18:14,570 the inter-renewal intervals were enormously large. 255 00:18:14,570 --> 00:18:24,150 And because they were enormously large, the time 256 00:18:24,150 --> 00:18:29,510 average residual life turned out to be enormous also. 257 00:18:29,510 --> 00:18:32,000 You think of what happens as epsilon 258 00:18:32,000 --> 00:18:35,100 get smaller and smaller. 259 00:18:35,100 --> 00:18:40,600 You can see intuitively why it makes sense that when you have 260 00:18:40,600 --> 00:18:46,910 humongously long inter-renewal times, and you have this x 261 00:18:46,910 --> 00:18:52,240 squared that occurs, because of this triangle here, I think 262 00:18:52,240 --> 00:18:57,150 is possible to see why this quantity here can't become 263 00:18:57,150 --> 00:19:03,010 infinite if you had this situation of a very long 264 00:19:03,010 --> 00:19:06,070 tailed distributions for the inter-renewal time. 265 00:19:06,070 --> 00:19:12,320 You have enormously long residual waits then, and you 266 00:19:12,320 --> 00:19:19,240 have them with high probability, because if you 267 00:19:19,240 --> 00:19:26,060 come into this process at some arbitrary time you're somewhat 268 00:19:26,060 --> 00:19:29,770 more likely to wind up in one of these large intervals than 269 00:19:29,770 --> 00:19:33,500 one of the small intervals, and that's what causes all the 270 00:19:33,500 --> 00:19:34,750 trouble here. 271 00:19:38,250 --> 00:19:40,600 There are similar examples here. 272 00:19:40,600 --> 00:19:47,360 You can look at the age of a process, z of t is defined as 273 00:19:47,360 --> 00:19:53,160 the interval between t and the previous arrival. 274 00:19:53,160 --> 00:19:59,240 So, if we look at sum time t, the age at that point is the 275 00:19:59,240 --> 00:20:03,090 amount of time back to the previous arrival that goes up 276 00:20:03,090 --> 00:20:07,760 like a triangle, drops as soon as we get the next arrival. 277 00:20:07,760 --> 00:20:08,940 It's exactly the same. 278 00:20:08,940 --> 00:20:11,850 It's just the same thing looking at it backwards 279 00:20:11,850 --> 00:20:17,910 instead of forwards, so the answer is the same also. 280 00:20:17,910 --> 00:20:22,240 If you look at something called duration, the question 281 00:20:22,240 --> 00:20:27,760 here is at a particular time t. 282 00:20:27,760 --> 00:20:31,390 If we take the difference between the next arrival and 283 00:20:31,390 --> 00:20:36,260 the previous arrival, that's called the duration of time t. 284 00:20:36,260 --> 00:20:37,650 How big is that? 285 00:20:37,650 --> 00:20:41,590 Well, obviously this is the same problem as that, and the 286 00:20:41,590 --> 00:20:45,270 residual life and in fact, that's exactly the sum of the 287 00:20:45,270 --> 00:20:47,760 residual life and the age. 288 00:20:47,760 --> 00:20:51,990 So, it's not surprising that the time average-- 289 00:20:51,990 --> 00:20:54,180 oh my. 290 00:20:54,180 --> 00:20:55,990 There should be a 2 down there. 291 00:21:31,340 --> 00:21:35,200 So it's exactly the same situation as we had before 292 00:21:35,200 --> 00:21:38,290 when we were talking about residual life. 293 00:21:41,250 --> 00:21:44,380 Now we'd like to generalize this, and I hope the 294 00:21:44,380 --> 00:21:49,860 generalization is almost obvious at this point. 295 00:21:49,860 --> 00:21:55,650 These three quantities here are all examples of assigning 296 00:21:55,650 --> 00:21:58,430 rewards to renewal processes. 297 00:21:58,430 --> 00:22:04,040 And the reward at any time t, when I'm trying to do this, is 298 00:22:04,040 --> 00:22:07,890 restricted to be a function of the inter-renewal period 299 00:22:07,890 --> 00:22:08,860 containing t. 300 00:22:08,860 --> 00:22:14,110 In other words, when you try to define what the reward is 301 00:22:14,110 --> 00:22:17,940 at a given time t, it's a function only of what's going 302 00:22:17,940 --> 00:22:21,650 on in a renewal period containing t. 303 00:22:24,290 --> 00:22:27,860 Now, in its simplest form, I want to go 304 00:22:27,860 --> 00:22:29,780 make this even simpler. 305 00:22:29,780 --> 00:22:35,050 In its simplest form r of t is restricted to be a function of 306 00:22:35,050 --> 00:22:39,020 the age of time t and the duration of time t. 307 00:22:39,020 --> 00:22:42,620 In other words, you try to say what's the reward in a given 308 00:22:42,620 --> 00:22:45,430 time t, and it's the same as these three 309 00:22:45,430 --> 00:22:47,280 examples we looked at. 310 00:22:47,280 --> 00:22:52,550 It's some function of how far back do you have to go to the 311 00:22:52,550 --> 00:22:56,590 previous arrival, and how far forward do you have to go on 312 00:22:56,590 --> 00:22:59,040 until the next arrival. 313 00:22:59,040 --> 00:23:02,610 You can make this a function if you want to of any of the 314 00:23:02,610 --> 00:23:07,260 three quantities residual life, age, and duration. 315 00:23:07,260 --> 00:23:11,590 It seems to be intuitively a little simpler to talk about 316 00:23:11,590 --> 00:23:16,342 age and duration as opposed to the other quantities. 317 00:23:16,342 --> 00:23:20,730 So the time average for sample path of r of t is found by 318 00:23:20,730 --> 00:23:23,790 analogy to residual life. 319 00:23:23,790 --> 00:23:26,100 That's the way I'm going to do it here in class. 320 00:23:26,100 --> 00:23:28,140 The notes do a little more careful job 321 00:23:28,140 --> 00:23:30,610 than just by analogy. 322 00:23:30,610 --> 00:23:34,310 But you start with the n-th inter-renewal interval and you 323 00:23:34,310 --> 00:23:39,920 say, what is the aggregate reward I get in the n-th 324 00:23:39,920 --> 00:23:40,790 inter-renewal interval? 325 00:23:40,790 --> 00:23:43,810 It's a random variable, obviously. 326 00:23:43,810 --> 00:23:54,730 And what it is is the integral from the previous arrival up 327 00:23:54,730 --> 00:23:59,190 to the next arrival of r of t and omega dt. 328 00:24:05,130 --> 00:24:11,000 This is very strange, because when we looked at this before 329 00:24:11,000 --> 00:24:15,750 we talked about n of t and we talked about s sub n of t plus 330 00:24:15,750 --> 00:24:22,720 1 and s sub n of t, and suddenly n of t plus 1 has 331 00:24:22,720 --> 00:24:28,690 turned into n, and n of t has turned into n minus 1. 332 00:24:28,690 --> 00:24:31,210 What has happened? 333 00:24:31,210 --> 00:24:34,280 Well, this gives you a clue. 334 00:24:34,280 --> 00:24:37,730 And it's partly how we define the intervals. 335 00:24:37,730 --> 00:24:44,010 Interval 1 goes from 0 to s1 and z of t equals t, interval 336 00:24:44,010 --> 00:24:48,590 n z of t is t minus s sub n minus 1. 337 00:24:48,590 --> 00:24:53,240 Let me go back and show you the original figure here and I 338 00:24:53,240 --> 00:24:54,595 think it will make it clear. 339 00:25:06,090 --> 00:25:11,010 Now, the first arrival comes at this time here. 340 00:25:11,010 --> 00:25:15,560 The first interval we're talking about goes from 0 to 341 00:25:15,560 --> 00:25:17,530 time s1 of omega. 342 00:25:21,080 --> 00:25:25,000 What we're interested in, in this first interval going from 343 00:25:25,000 --> 00:25:31,000 0 to s1 of omega, is this interarrival time, which is 344 00:25:31,000 --> 00:25:32,850 the first interarrival time. 345 00:25:32,850 --> 00:25:36,840 This first interval is connected to x1 and it goes 346 00:25:36,840 --> 00:25:42,250 from s sub 0, which is just at 0 up to s1. 347 00:25:42,250 --> 00:25:47,890 The second arrival time goes from s1 up to s2. 348 00:25:47,890 --> 00:25:53,400 When we look at n of t, n of t is a sum t here in this 349 00:25:53,400 --> 00:25:57,510 interval here, and this is s. 350 00:26:03,760 --> 00:26:05,790 Am I going to get totally confused writing 351 00:26:05,790 --> 00:26:07,905 this or aren't I? 352 00:26:19,600 --> 00:26:22,120 I think I might get totally confused, so I'm not going to 353 00:26:22,120 --> 00:26:23,370 try to write it. 354 00:26:29,000 --> 00:26:30,950 Because when I try to write it I'm trying to make the 355 00:26:30,950 --> 00:26:35,470 comparison between n of t and n of t plus 1, which are the 356 00:26:35,470 --> 00:26:39,080 things up here. 357 00:26:39,080 --> 00:26:44,100 But the quantities here with the renewal periods are, this 358 00:26:44,100 --> 00:26:47,150 is the first inter-renewal period, this is the second 359 00:26:47,150 --> 00:26:48,730 inter-renewal period. 360 00:26:48,730 --> 00:26:54,910 The second inter-renewal period goes from s sub 2 back 361 00:26:54,910 --> 00:26:56,280 to s sub 1. 362 00:26:56,280 --> 00:27:02,570 The n-th goes from s sub n back to s sub n minus 1. 363 00:27:02,570 --> 00:27:05,190 So that's just the way it is when you count this way. 364 00:27:05,190 --> 00:27:09,040 If you count it the other way you would get even more 365 00:27:09,040 --> 00:27:14,030 confused because then in the interval x sub n the 366 00:27:14,030 --> 00:27:17,580 inter-renewal time would be x sub n plus 1 and 367 00:27:17,580 --> 00:27:19,090 that would be awful. 368 00:27:19,090 --> 00:27:21,460 So, this is the way we have to do it. 369 00:27:30,310 --> 00:27:33,770 Let's look at what happens with the expected 370 00:27:33,770 --> 00:27:37,195 inter-renewal time, then, as a time average. 371 00:27:42,400 --> 00:27:45,990 And you can think of this as a being for a sample time. 372 00:27:45,990 --> 00:27:49,700 When you have an expression which is valid for a sample 373 00:27:49,700 --> 00:27:53,810 time, it's also valid for a random variable, because it 374 00:27:53,810 --> 00:27:58,080 maps every sample point into some value. 375 00:27:58,080 --> 00:28:01,620 So, I'll just call it here r sub n, which is the amount of 376 00:28:01,620 --> 00:28:07,410 reward that you pick up in the n-th inter-renewal period, so 377 00:28:07,410 --> 00:28:14,830 it's the integral from the n minus first arrival up to the 378 00:28:14,830 --> 00:28:19,460 n-th arrival, and we're integrating over r of t over 379 00:28:19,460 --> 00:28:21,010 this whole interval. 380 00:28:21,010 --> 00:28:28,250 So r of t is a function of the age and of the duration. 381 00:28:28,250 --> 00:28:37,940 The age is t minus s sub n minus 1, and the duration is 382 00:28:37,940 --> 00:28:39,660 just x of n. 383 00:28:39,660 --> 00:28:43,410 Now that's a little weird also because before we were talking 384 00:28:43,410 --> 00:28:47,710 about the duration as being statistically very, very 385 00:28:47,710 --> 00:28:52,260 different from the inter-renewal period. 386 00:28:52,260 --> 00:28:54,200 But it wasn't. 387 00:28:54,200 --> 00:29:04,310 If you look on a sample path basis, this duration of the 388 00:29:04,310 --> 00:29:10,680 n-th inter-renewal period is exactly equal to x sub n of 389 00:29:10,680 --> 00:29:19,110 omega, which is, for that particular sample path, it's 390 00:29:19,110 --> 00:29:22,700 the value of that inter-renewal interval. 391 00:29:22,700 --> 00:29:26,950 So when we integrate this quantity now, we want to 392 00:29:26,950 --> 00:29:29,460 integrate it over dt. 393 00:29:29,460 --> 00:29:30,460 What do we get? 394 00:29:30,460 --> 00:29:36,210 Well, we've gotten rid of this t here, we just have the n-th 395 00:29:36,210 --> 00:29:37,750 duration here. 396 00:29:37,750 --> 00:29:44,010 The only t appears over here, and this is r of a difference 397 00:29:44,010 --> 00:29:47,370 between t and s sub n minus 1. 398 00:29:47,370 --> 00:29:51,510 So, we want to do a change of variables, we want to let z be 399 00:29:51,510 --> 00:29:55,570 t minus s sub n minus 1, and then we integrate z 400 00:29:55,570 --> 00:29:58,480 from 0 to x sub n. 401 00:29:58,480 --> 00:30:02,230 And now just imagine that you put omegas in all of these 402 00:30:02,230 --> 00:30:05,220 things to let you see that you're actually dealing with 403 00:30:05,220 --> 00:30:08,900 one sample function, and when you leave the omegas out 404 00:30:08,900 --> 00:30:11,760 you're just dealing with the random variables that arise 405 00:30:11,760 --> 00:30:14,580 because of doing that. 406 00:30:14,580 --> 00:30:23,290 So when we do this integration, what we get is 407 00:30:23,290 --> 00:30:29,090 the integral of r of z with this fixed x sub n integrated 408 00:30:29,090 --> 00:30:30,280 against dz. 409 00:30:30,280 --> 00:30:33,810 This is a function only of the random variable x sub n. 410 00:30:33,810 --> 00:30:35,060 Strange. 411 00:30:35,060 --> 00:30:37,490 That's the expected value of r sub n. 412 00:30:40,340 --> 00:30:43,290 This unfortunately can't be reduced any further, this is 413 00:30:43,290 --> 00:30:45,040 just what it is. 414 00:30:45,040 --> 00:30:50,480 You have to find out what is the integral of r of zx 415 00:30:50,480 --> 00:30:55,300 integrated over z, and then we have to take the expected 416 00:30:55,300 --> 00:30:57,830 value over x. 417 00:30:57,830 --> 00:31:00,950 If the expectation exists, you can write this in a simple 418 00:31:00,950 --> 00:31:09,370 way, because this quantity here integrated 419 00:31:09,370 --> 00:31:11,140 is just r sub n. 420 00:31:11,140 --> 00:31:15,880 And then you take the expected value of r sub n, and you 421 00:31:15,880 --> 00:31:18,590 divide by x-bar down here. 422 00:31:21,950 --> 00:31:27,770 This quantity here is just the expected value of the integral 423 00:31:27,770 --> 00:31:30,240 of r of zx dz. 424 00:31:30,240 --> 00:31:32,980 And that integral there is just the expected 425 00:31:32,980 --> 00:31:36,410 value of r sub n. 426 00:31:36,410 --> 00:31:40,020 So, if you look over the entire sample path from 0 to 427 00:31:40,020 --> 00:31:49,940 infinity, what you get then is that the time average reward 428 00:31:49,940 --> 00:31:53,560 is just equal to the expected value of r sub n 429 00:31:53,560 --> 00:31:55,000 divided by x squared. 430 00:31:59,060 --> 00:32:00,270 Oops. 431 00:32:00,270 --> 00:32:01,520 Lost the example. 432 00:32:03,970 --> 00:32:08,580 Suppose we want to find the k-th moment of the age. 433 00:32:08,580 --> 00:32:10,090 Simple thing to try to find. 434 00:32:12,930 --> 00:32:16,670 I want to do that because it's simple. 435 00:32:16,670 --> 00:32:20,560 If you're looking at the k-th moment of the age looked at as 436 00:32:20,560 --> 00:32:29,130 a time average, what it is is the reward at time t, it's the 437 00:32:29,130 --> 00:32:32,490 age at time t to the k-th power. 438 00:32:32,490 --> 00:32:38,640 We want to take the age to the k-th power at time t, and then 439 00:32:38,640 --> 00:32:43,550 integrate over rt and divide by the final value 440 00:32:43,550 --> 00:32:45,990 and go to the limit. 441 00:32:45,990 --> 00:32:50,460 So the expected value of this k-th moment over 1 442 00:32:50,460 --> 00:32:56,620 inter-renewal period is this integral here, z to the k dz 443 00:32:56,620 --> 00:33:01,160 times dF sub x of x. 444 00:33:01,160 --> 00:33:06,060 The only place x comes in this integral is in the final point 445 00:33:06,060 --> 00:33:09,780 of the integration where we are integrating up to the 446 00:33:09,780 --> 00:33:14,650 point x, at which the duration ends, and then we're taking 447 00:33:14,650 --> 00:33:16,250 the expected value of this. 448 00:33:16,250 --> 00:33:20,420 So, we integrate this, the integral of the z to the k 449 00:33:20,420 --> 00:33:29,080 from 0 to x is just x to the k plus 1 divided by k. 450 00:33:29,080 --> 00:33:31,200 So when you take that integral and you look at 451 00:33:31,200 --> 00:33:33,280 this, what is this? 452 00:33:33,280 --> 00:33:37,620 It's the 1 over k times the expected value of the random 453 00:33:37,620 --> 00:33:40,710 variable, x to the k plus first power. 454 00:33:40,710 --> 00:33:46,660 That's just this taking the expected value of it, which is 455 00:33:46,660 --> 00:33:49,490 you take the expected value by integrating times 456 00:33:49,490 --> 00:33:50,980 the s sub x of x. 457 00:33:50,980 --> 00:33:51,520 Yes? 458 00:33:51,520 --> 00:33:53,440 AUDIENCE: [INAUDIBLE] 459 00:33:53,440 --> 00:33:54,690 divide by k plus 1? 460 00:33:59,160 --> 00:34:00,410 PROFESSOR: Yes. 461 00:34:03,180 --> 00:34:04,480 I'm sorry. 462 00:34:07,050 --> 00:34:11,719 My evil twin brother is speaking this morning, and 463 00:34:11,719 --> 00:34:14,560 this should be a k plus 1 here. 464 00:34:14,560 --> 00:34:18,670 This should be a k plus 1, and this should be a k plus 1. 465 00:34:18,670 --> 00:34:21,670 And if you look at the example of the first moment, which is 466 00:34:21,670 --> 00:34:24,840 the first thing we looked at today, namely finding the 467 00:34:24,840 --> 00:34:29,840 expected value of the age, what we find out is that it's 468 00:34:29,840 --> 00:34:37,199 expected value of x squared divided by 2 times x-bar. 469 00:34:37,199 --> 00:34:41,960 So with k equal to 1 we get the expected value of x 470 00:34:41,960 --> 00:34:46,070 squared over 2 times x-bar. 471 00:34:46,070 --> 00:34:48,690 It's even worse because I left a 2 out when we were talking 472 00:34:48,690 --> 00:34:51,690 about the expected value of age. 473 00:34:51,690 --> 00:34:55,659 And to make things even worse, when I did this I went back to 474 00:34:55,659 --> 00:34:59,590 check whether it was the right answer, and of course the two 475 00:34:59,590 --> 00:35:01,590 mistakes cancelled each other out. 476 00:35:01,590 --> 00:35:03,250 So, sorry about that. 477 00:35:07,150 --> 00:35:10,800 The thing I really wanted to talk about today is stopping 478 00:35:10,800 --> 00:35:14,960 trials for stochastic processes, because that's a 479 00:35:14,960 --> 00:35:21,220 topic which has always caused confusion in this class. 480 00:35:21,220 --> 00:35:24,600 If you look at any of the textbooks on stochastic 481 00:35:24,600 --> 00:35:28,130 processes, it causes even more confusion. 482 00:35:28,130 --> 00:35:32,890 People talk about it, and then they talk about it again, and 483 00:35:32,890 --> 00:35:36,410 they say what we said before wasn't right, and then they 484 00:35:36,410 --> 00:35:39,110 talk about it again, they say what we said before that 485 00:35:39,110 --> 00:35:41,710 wasn't right, and it goes on and on and on. 486 00:35:41,710 --> 00:35:43,660 And you never know what's going on. 487 00:35:47,920 --> 00:35:52,570 Very often instead of looking at an entire stochastic 488 00:35:52,570 --> 00:35:55,900 process over the infinite interval, you'd like to look 489 00:35:55,900 --> 00:35:57,380 at a finite interval. 490 00:35:57,380 --> 00:36:00,490 You'd like to look at the interval going from 0 up to 491 00:36:00,490 --> 00:36:02,130 some time t. 492 00:36:02,130 --> 00:36:09,280 But in almost a majority of those cases, the time t that 493 00:36:09,280 --> 00:36:13,430 you want to look at is not some fixed time but some 494 00:36:13,430 --> 00:36:18,990 random time which depends on something which is happening. 495 00:36:18,990 --> 00:36:26,680 And when you want to look at what's going on here, it's 496 00:36:26,680 --> 00:36:31,560 always tricky to visualize this because what you have is 497 00:36:31,560 --> 00:36:34,600 t becomes a random variable. 498 00:36:34,600 --> 00:36:39,730 That random variable is a function of the sample values 499 00:36:39,730 --> 00:36:43,390 that you have up until time t. 500 00:36:43,390 --> 00:36:47,560 But that seems like a circular argument, t a random variable 501 00:36:47,560 --> 00:36:50,110 and it depends on t. 502 00:36:52,620 --> 00:36:55,640 Things are not supposed to depend on themselves, because 503 00:36:55,640 --> 00:37:00,060 when they depend on themselves you have trouble understanding 504 00:37:00,060 --> 00:37:01,930 what's going on. 505 00:37:01,930 --> 00:37:05,450 So, we will sort that out. 506 00:37:05,450 --> 00:37:07,470 In sorting it out we're only going to look at 507 00:37:07,470 --> 00:37:09,610 discrete-time processes. 508 00:37:09,610 --> 00:37:13,830 So, we'll only look at sequences of random variables. 509 00:37:13,830 --> 00:37:17,670 We will not assume that these random variables are IID for 510 00:37:17,670 --> 00:37:22,070 this argument here, because we want to look at a much wider 511 00:37:22,070 --> 00:37:22,950 range of things. 512 00:37:22,950 --> 00:37:28,180 So we have some arbitrary sequence of random variables, 513 00:37:28,180 --> 00:37:33,090 and the idea is you want to sit there looking at this 514 00:37:33,090 --> 00:37:40,800 evolution of the sequence of random variables, and in fact, 515 00:37:40,800 --> 00:37:44,630 you want to sit there looking at the sample 516 00:37:44,630 --> 00:37:46,150 path of this evolution. 517 00:37:46,150 --> 00:37:49,830 You want to look at x1, and you want to look at x2, and 518 00:37:49,830 --> 00:37:52,830 you want to look at x3, and so forth. 519 00:37:52,830 --> 00:37:56,410 And at each point along the line in terms of what you've 520 00:37:56,410 --> 00:37:59,160 seen already, you want to decide whether you're going to 521 00:37:59,160 --> 00:38:02,370 stop or whether you're going to continue at that point. 522 00:38:10,380 --> 00:38:14,670 And if at that point you develop a rule for what you're 523 00:38:14,670 --> 00:38:19,380 going to do with each point you call that a stopping rule, 524 00:38:19,380 --> 00:38:22,240 so that you have a rule that tells you when you want to 525 00:38:22,240 --> 00:38:24,750 stop looking at these things. 526 00:38:24,750 --> 00:38:29,540 Let me give you some idea of the generality of this. 527 00:38:29,540 --> 00:38:34,080 Often when we look at queuing systems we are going to have 528 00:38:34,080 --> 00:38:40,310 an arrival process where arrivals keep coming in, 529 00:38:40,310 --> 00:38:45,040 things get serviced one after the other, and one of the very 530 00:38:45,040 --> 00:38:49,670 interesting things is whether the system ever empties out or 531 00:38:49,670 --> 00:38:53,140 whether the queue just keeps building up forever. 532 00:38:53,140 --> 00:38:58,160 Well, an interesting stopping rule then is when the queue 533 00:38:58,160 --> 00:39:00,480 empties out. 534 00:39:00,480 --> 00:39:04,430 We will see that an even more interesting stopping rule is 535 00:39:04,430 --> 00:39:09,310 starting with some arbitrary first arrival at time 0. 536 00:39:09,310 --> 00:39:14,600 Let's look for the time until another arrival occurs which 537 00:39:14,600 --> 00:39:19,580 starts another busy period. 538 00:39:19,580 --> 00:39:24,180 These stopping times, then, are going to be critical in 539 00:39:24,180 --> 00:39:27,030 analyzing things like queuing systems. 540 00:39:27,030 --> 00:39:31,080 They're also going to become critical in trying to analyze 541 00:39:31,080 --> 00:39:34,890 these renewal systems that we're already talking about. 542 00:39:34,890 --> 00:39:40,260 They will become critical to trying to take a time average 543 00:39:40,260 --> 00:39:45,920 view about renewal processes instead of a, not a time 544 00:39:45,920 --> 00:39:49,690 average, but an ensemble average viewpoint looking at 545 00:39:49,690 --> 00:39:54,020 particular times t as opposed to taking a time average. 546 00:39:54,020 --> 00:39:56,180 So we're going to look at these things in 547 00:39:56,180 --> 00:39:57,430 many different ways. 548 00:40:02,070 --> 00:40:06,920 In order to do that, since you're looking at a sample 549 00:40:06,920 --> 00:40:11,840 function, you're observing it on each 550 00:40:11,840 --> 00:40:14,910 arrival of this process. 551 00:40:23,460 --> 00:40:27,080 At each observation of a sample value of a random 552 00:40:27,080 --> 00:40:30,730 variable, you decide whether you want to stop or not. 553 00:40:30,730 --> 00:40:37,220 So a sensible way to deal with this is to look over all time, 554 00:40:37,220 --> 00:40:43,290 define a random variable J, which describes when this 555 00:40:43,290 --> 00:40:44,830 sequence is to be stopped. 556 00:40:44,830 --> 00:40:49,470 So for each sample value x1 of omega, x2 of omega, x3 of 557 00:40:49,470 --> 00:40:53,690 omega, and so forth, if your rule is to stop the first time 558 00:40:53,690 --> 00:41:00,630 you see a sample value which is equal to 0, then J is a 559 00:41:00,630 --> 00:41:05,790 integer random variable whose value is the first n at which 560 00:41:05,790 --> 00:41:10,450 is x of n is equal to 0 and many other examples. 561 00:41:10,450 --> 00:41:16,190 So we try a 1 then, x1 of omega is observed. 562 00:41:16,190 --> 00:41:20,120 A decision is made based on x1 of omega 563 00:41:20,120 --> 00:41:23,560 whether or not to stop. 564 00:41:23,560 --> 00:41:27,550 If we stop, J of omega equals 1. 565 00:41:27,550 --> 00:41:31,910 If we don't stop, J of omega is bigger than 1, but we don't 566 00:41:31,910 --> 00:41:33,510 know what it is yet. 567 00:41:33,510 --> 00:41:40,440 At trial two, if we haven't stopped yet, you observe x2 of 568 00:41:40,440 --> 00:41:44,710 omega, the second random variable, second sample value, 569 00:41:44,710 --> 00:41:49,240 you make a decision again based on x1 of omega and x2 of 570 00:41:49,240 --> 00:41:51,770 omega whether or not to stop. 571 00:41:51,770 --> 00:41:54,790 If you stop, J of omega is equal to 2. 572 00:41:54,790 --> 00:42:00,090 So you can visualize doing this on each trial, you don't 573 00:42:00,090 --> 00:42:02,790 have to worry about whether you've already stopped, you 574 00:42:02,790 --> 00:42:06,370 just have a rule that says, I want to stop at this point if 575 00:42:06,370 --> 00:42:08,240 I haven't stopped before. 576 00:42:08,240 --> 00:42:14,925 So the stopping event at time n is that you stop either if 577 00:42:14,925 --> 00:42:20,780 your rule tells you to stop or you stop before. 578 00:42:20,780 --> 00:42:23,990 And if you stop before then you're obviously stopped. 579 00:42:23,990 --> 00:42:27,700 At each trial n if stopping has not yet occurred x of n is 580 00:42:27,700 --> 00:42:32,600 observed and the decision based on x1 to xn is made. 581 00:42:32,600 --> 00:42:37,060 If you stop, then J omega is equal to n. 582 00:42:37,060 --> 00:42:45,410 So for each sample path J of n is the time or the trial at 583 00:42:45,410 --> 00:42:46,660 which you stop. 584 00:42:50,850 --> 00:42:54,890 So, we're going to define a stopping trial, or stopping 585 00:42:54,890 --> 00:42:59,980 time, J for a sequence of random variables. 586 00:42:59,980 --> 00:43:03,610 You're going to define this to be a positive integer-valued 587 00:43:03,610 --> 00:43:08,130 random variable that has to be positive, because if you're 588 00:43:08,130 --> 00:43:11,290 going to stop before you observe anything that's not a 589 00:43:11,290 --> 00:43:13,430 very interesting thing. 590 00:43:13,430 --> 00:43:17,190 So you always observe at least one thing, then you decide 591 00:43:17,190 --> 00:43:20,120 whether you want to stop or you proceed, so forth. 592 00:43:20,120 --> 00:43:22,690 So it's a positive integer-valued random 593 00:43:22,690 --> 00:43:30,900 variable, you always stop at an integer trial, and for each 594 00:43:30,900 --> 00:43:36,000 n greater than or equal to 1 the indicator random variable 595 00:43:36,000 --> 00:43:42,520 the indicator of the event J equals n is a function of what 596 00:43:42,520 --> 00:43:44,835 you have observed up until that point. 597 00:43:47,880 --> 00:43:50,850 This part is a really critical part of it. 598 00:43:50,850 --> 00:43:56,700 The decision to stop at a time n has to be a function only of 599 00:43:56,700 --> 00:43:59,010 what you have already observed. 600 00:43:59,010 --> 00:44:03,290 You're not allowed as an observer to peek ahead a 601 00:44:03,290 --> 00:44:06,900 little bit and then decide on the basis of what you see in 602 00:44:06,900 --> 00:44:12,310 the future whether you want to stop at this time or not. 603 00:44:12,310 --> 00:44:15,570 One example in the notes is that you're playing poker with 604 00:44:15,570 --> 00:44:23,280 somebody, and you make a bet and the other person wins. 605 00:44:23,280 --> 00:44:26,150 You make the bet on the basis of what you've seen in your 606 00:44:26,150 --> 00:44:30,210 cards so far, you don't make your bet on the basis of your 607 00:44:30,210 --> 00:44:34,410 observation of what the other player has, and the other 608 00:44:34,410 --> 00:44:39,890 player will get very angry if you try to base your decision 609 00:44:39,890 --> 00:44:45,600 later on what the person you're playing with has. 610 00:44:45,600 --> 00:44:47,590 So you can't do that sort of thing. 611 00:44:47,590 --> 00:44:51,490 You can't peek ahead when you're doing this. 612 00:44:51,490 --> 00:44:54,890 You could design random variables where, in fact, you 613 00:44:54,890 --> 00:44:59,920 can look ahead, but the times where you use what people call 614 00:44:59,920 --> 00:45:05,510 stopping trials are situations in which you stop based on 615 00:45:05,510 --> 00:45:07,990 what has already happened. 616 00:45:07,990 --> 00:45:11,380 You can generalize this, and we will generalize it later, 617 00:45:11,380 --> 00:45:14,380 where you can stop on the basis of other random 618 00:45:14,380 --> 00:45:19,350 variables, which also evolve in time, but you stop on the 619 00:45:19,350 --> 00:45:25,730 basis of what has already happened up until time n. 620 00:45:25,730 --> 00:45:29,230 We're going to visualize conducting successive trials 621 00:45:29,230 --> 00:45:33,850 until sum n in which the event J equals n occurs. 622 00:45:33,850 --> 00:45:36,520 Further trials then cease. 623 00:45:36,520 --> 00:45:39,680 It is simpler conceptually to visualize stopping the 624 00:45:39,680 --> 00:45:43,720 observation of trials after the stopping trial, but 625 00:45:43,720 --> 00:45:45,530 continuing to conduct trials. 626 00:45:45,530 --> 00:45:49,310 In other words, if we start talking about a stopping 627 00:45:49,310 --> 00:45:55,100 process and we say we stop on the n-th trial, but we're 628 00:45:55,100 --> 00:45:58,020 forbidden to even talk about the n plus 629 00:45:58,020 --> 00:45:59,700 first random variable. 630 00:45:59,700 --> 00:46:05,890 Now the n plus first random variable occurs on some sample 631 00:46:05,890 --> 00:46:09,520 paths but doesn't occur on other sample paths. 632 00:46:09,520 --> 00:46:11,960 I don't know how to deal with that, and neither do you. 633 00:46:14,800 --> 00:46:17,800 And what that means is the way we're going to visualize these 634 00:46:17,800 --> 00:46:24,200 processes is physically they continue forever, but we just 635 00:46:24,200 --> 00:46:26,540 stop observing them at a certain point. 636 00:46:26,540 --> 00:46:30,840 So, this stopping rule is the time at which we stop 637 00:46:30,840 --> 00:46:35,090 observing as opposed to the time at which the random 638 00:46:35,090 --> 00:46:50,470 variable ceases existing, if we define the random variable, 639 00:46:50,470 --> 00:46:55,430 it has a value for all sample values, and sample values 640 00:46:55,430 --> 00:46:59,200 involve all of these paths. 641 00:46:59,200 --> 00:47:01,940 So you sort of have to define it that way. 642 00:47:01,940 --> 00:47:07,550 One thing you would like to do is in many of these situations 643 00:47:07,550 --> 00:47:11,040 you wind up never stopping. 644 00:47:11,040 --> 00:47:15,360 And the trouble is, when you're investigating stopping 645 00:47:15,360 --> 00:47:20,110 rules, stopping trials, you usually don't know ahead of 646 00:47:20,110 --> 00:47:24,530 time whether you're ever going to stop or not. 647 00:47:24,530 --> 00:47:27,660 And because of that if you don't know whether you're 648 00:47:27,660 --> 00:47:29,310 going to stop or not, you can't call 649 00:47:29,310 --> 00:47:30,810 it a stopping trial. 650 00:47:30,810 --> 00:47:35,990 So, what one normally does is to say that J is a possibly 651 00:47:35,990 --> 00:47:38,240 defective random variable. 652 00:47:38,240 --> 00:47:44,400 And if it's possibly defective you mean that all sample 653 00:47:44,400 --> 00:47:49,700 points are mapped into either finite J or perhaps J equals 654 00:47:49,700 --> 00:47:52,840 infinity, which means that you never stop. 655 00:47:52,840 --> 00:47:57,660 But you still have the same condition that we have here, 656 00:47:57,660 --> 00:48:02,290 that you stop on the basis of x1, x of n. 657 00:48:02,290 --> 00:48:05,400 And you also have the condition, which isn't quite 658 00:48:05,400 --> 00:48:08,310 clear here, but it's clear from the fact that J is a 659 00:48:08,310 --> 00:48:17,060 random variable, that the events J equals 1, J equals 2, 660 00:48:17,060 --> 00:48:20,840 J equals 3, are all disjoint. 661 00:48:20,840 --> 00:48:24,940 So you can't stop twice, you have to just stop once. 662 00:48:24,940 --> 00:48:30,140 And once you stop you can't start again, 663 00:48:30,140 --> 00:48:34,020 Now does everybody understand what is stopping trial or a 664 00:48:34,020 --> 00:48:35,740 stopping rule is at this point? 665 00:48:39,300 --> 00:48:44,770 If you don't understand it I think the trouble is you won't 666 00:48:44,770 --> 00:48:47,730 realize you don't understand it until you start seeing some 667 00:48:47,730 --> 00:48:52,540 of these examples where strange things are going on. 668 00:48:52,540 --> 00:48:55,790 I guess we have to just go ahead and see 669 00:48:55,790 --> 00:48:58,460 what happens then. 670 00:48:58,460 --> 00:49:00,300 So the examples, which I'm not going to 671 00:49:00,300 --> 00:49:02,730 develop in any detail. 672 00:49:02,730 --> 00:49:07,160 A gambler goes to a casino and he gambles until he's broke. 673 00:49:07,160 --> 00:49:10,480 So, he goes in with a finite amount of capital. 674 00:49:10,480 --> 00:49:14,480 He has some particular system that he's playing by. 675 00:49:14,480 --> 00:49:18,040 If he's lucky he gets to play for a long, long time, he gets 676 00:49:18,040 --> 00:49:22,030 a lot of amusement as he loses his money, and if he's unlucky 677 00:49:22,030 --> 00:49:24,600 he loses very quickly. 678 00:49:24,600 --> 00:49:28,490 If you look at the odds in casinos and you apply the 679 00:49:28,490 --> 00:49:31,000 strong law of large numbers, you realize that with 680 00:49:31,000 --> 00:49:34,480 probability 1 you get wiped out eventually. 681 00:49:34,480 --> 00:49:37,560 Because the odds are not in your favor. 682 00:49:37,560 --> 00:49:41,200 Casinos wouldn't be built if the odds were in your favor. 683 00:49:41,200 --> 00:49:44,410 Another example, just flip a coin until 10 684 00:49:44,410 --> 00:49:46,500 successive heads appear. 685 00:49:46,500 --> 00:49:48,000 10 heads in a row. 686 00:49:48,000 --> 00:49:50,260 That's a rather unlikely event. 687 00:49:50,260 --> 00:49:53,470 You're going to be able to figure out very easily from 688 00:49:53,470 --> 00:49:57,170 this theory what's the expected amount of time until 689 00:49:57,170 --> 00:49:59,730 10 successive heads appear. 690 00:49:59,730 --> 00:50:02,801 It's a very easy problem. 691 00:50:02,801 --> 00:50:08,070 A more important problem, this test and hypothesis with 692 00:50:08,070 --> 00:50:11,310 repeated trials. 693 00:50:11,310 --> 00:50:13,730 So, you have the hypothesis that a certain kind of 694 00:50:13,730 --> 00:50:18,710 treatment will make patients well, and you want to know 695 00:50:18,710 --> 00:50:22,580 whether to use this treatment. 696 00:50:22,580 --> 00:50:27,820 So you test it on mice, or people, or what have you. 697 00:50:27,820 --> 00:50:29,420 If you think it's pretty safe, you start 698 00:50:29,420 --> 00:50:32,070 testing it on people. 699 00:50:32,070 --> 00:50:35,015 And how many tests do you make? 700 00:50:37,950 --> 00:50:41,320 Well, you originally think that as a scientist you should 701 00:50:41,320 --> 00:50:45,240 plan an experiment ahead of time, and you should say, I'm 702 00:50:45,240 --> 00:50:47,810 going to do 1,000 tests. 703 00:50:47,810 --> 00:50:51,180 But if you're doing experiments on people and you 704 00:50:51,180 --> 00:50:55,700 find out that the first 10 patients die, you're going to 705 00:50:55,700 --> 00:50:59,380 stop the experiment at that point. 706 00:50:59,380 --> 00:51:03,990 And if you find out that the first 100 patients all live, 707 00:51:03,990 --> 00:51:05,505 well, you might continue the experiment. 708 00:51:05,505 --> 00:51:08,430 But you're going to publish the results at that point 709 00:51:08,430 --> 00:51:11,230 because you're going to try to get the FDA to approve this 710 00:51:11,230 --> 00:51:17,080 drug, or this operation, or this what have you. 711 00:51:17,080 --> 00:51:21,730 So if you view this stopping trial as the time at which 712 00:51:21,730 --> 00:51:26,410 you're going to try to publish something, then, again, if you 713 00:51:26,410 --> 00:51:30,510 have any sense, you are going to perform experiments, look 714 00:51:30,510 --> 00:51:34,420 at the experiments as you go, and decide what you're going 715 00:51:34,420 --> 00:51:37,700 to do as a function of what you've already seen. 716 00:51:37,700 --> 00:51:38,860 I mean, that's the way that all 717 00:51:38,860 --> 00:51:40,480 intelligent people operate. 718 00:51:43,200 --> 00:51:46,620 So if science says that's not a good way to operate, 719 00:51:46,620 --> 00:51:48,000 something's wrong with science. 720 00:51:48,000 --> 00:51:50,290 But fortunately science doesn't say that. 721 00:51:50,290 --> 00:51:52,170 Science allows you to do this. 722 00:51:52,170 --> 00:51:55,217 AUDIENCE: So not every J is potentially defective? 723 00:51:55,217 --> 00:51:56,675 You can come up with examples of 724 00:51:56,675 --> 00:51:57,656 where it would be defective. 725 00:51:57,656 --> 00:52:00,300 But not every one is necessarily [INAUDIBLE]. 726 00:52:03,470 --> 00:52:03,966 PROFESSOR: Yes. 727 00:52:03,966 --> 00:52:11,830 If a random variable is defective it means there's 728 00:52:11,830 --> 00:52:15,600 some probability that J is going to be infinite, but 729 00:52:15,600 --> 00:52:19,710 there's also presumably some probability that J is equal to 730 00:52:19,710 --> 00:52:23,070 1, a probability J is equal to 2, and so forth. 731 00:52:23,070 --> 00:52:27,050 So we have a distribution function for J, which instead 732 00:52:27,050 --> 00:52:31,870 of going up to 1 and stopping goes up to some smaller value 733 00:52:31,870 --> 00:52:35,202 and stops, and then it doesn't go any further. 734 00:52:35,202 --> 00:52:36,194 AUDIENCE: Not every [INAUDIBLE]. 735 00:52:36,194 --> 00:52:38,550 Some of these are definitely going to [INAUDIBLE]. 736 00:52:38,550 --> 00:52:39,800 PROFESSOR: Yes. 737 00:52:43,180 --> 00:52:50,200 So this testing hypotheses is really what triggered Abraham 738 00:52:50,200 --> 00:52:55,210 Wald to start trying to understand these situations, 739 00:52:55,210 --> 00:52:59,090 because he found out relatively quickly that it 740 00:52:59,090 --> 00:53:02,990 wasn't trivial to understand them, and all sorts of crazy 741 00:53:02,990 --> 00:53:04,350 things were happening. 742 00:53:04,350 --> 00:53:09,080 So he spent a lot of his time studying what happened in 743 00:53:09,080 --> 00:53:10,130 these cases. 744 00:53:10,130 --> 00:53:11,930 He called it sequential analysis. 745 00:53:11,930 --> 00:53:13,520 You might have heard the word. 746 00:53:13,520 --> 00:53:18,930 And sequential analysis is exactly the same idea. 747 00:53:18,930 --> 00:53:25,620 It's looking at analyzing experiments as you go in time, 748 00:53:25,620 --> 00:53:28,660 and either stopping or doing something else or changing the 749 00:53:28,660 --> 00:53:33,360 experiment or what have you, depending on what happens. 750 00:53:33,360 --> 00:53:36,700 Another thing is observe successive renewals in a 751 00:53:36,700 --> 00:53:40,750 renewal process until s sub n is greater 752 00:53:40,750 --> 00:53:43,760 than or equal to 100. 753 00:53:43,760 --> 00:53:47,820 Now, this particular thing is one of the things that we're 754 00:53:47,820 --> 00:53:51,230 going to use in studying renewal processes, it's why 755 00:53:51,230 --> 00:53:54,740 we're studying this topic right now. 756 00:53:54,740 --> 00:53:57,270 Well, we might study it now anyway, because we're going to 757 00:53:57,270 --> 00:54:01,460 use it in lots of other places, but it's also why you 758 00:54:01,460 --> 00:54:07,920 have to be very careful about talking about stopping time as 759 00:54:07,920 --> 00:54:10,310 opposed to stopping trials. 760 00:54:10,310 --> 00:54:16,910 Because if you stop an experiment at the arrival in a 761 00:54:16,910 --> 00:54:26,080 renewal process at which that renewal occurs after time 100, 762 00:54:26,080 --> 00:54:29,140 you don't know when you're going to stop. 763 00:54:29,140 --> 00:54:33,420 It might be a very long time after 100 before that first 764 00:54:33,420 --> 00:54:38,190 arrival after time 100 occurs, it might be a very short time. 765 00:54:38,190 --> 00:54:41,390 So it's not a stopping time that you're defining, it's a 766 00:54:41,390 --> 00:54:43,120 stopping trial. 767 00:54:43,120 --> 00:54:46,560 You are defining a rule which tells you at which trial 768 00:54:46,560 --> 00:54:50,090 you're going to stop as opposed to a rule that tells 769 00:54:50,090 --> 00:54:53,690 you in which time you're going to stop, and we'll see this as 770 00:54:53,690 --> 00:54:54,960 we go on in this. 771 00:54:58,250 --> 00:55:02,480 Suppose the random variables in this process we're looking 772 00:55:02,480 --> 00:55:07,970 at have a finite number of possible sample values. 773 00:55:07,970 --> 00:55:13,200 Then any possibly defective stopping trial, and a stopping 774 00:55:13,200 --> 00:55:15,720 trial is a set of rules for when you're going to stop 775 00:55:15,720 --> 00:55:22,350 observing things, any stopping trial can be represented as a 776 00:55:22,350 --> 00:55:24,280 rooted tree. 777 00:55:24,280 --> 00:55:26,970 If you don't know what a rooted tree is, a rooted tree 778 00:55:26,970 --> 00:55:28,080 is what I've drawn here. 779 00:55:28,080 --> 00:55:30,740 So, there's no confusion here. 780 00:55:30,740 --> 00:55:33,670 A rooted tree is something that starts on the left hand 781 00:55:33,670 --> 00:55:35,740 side and it grows for a while. 782 00:55:35,740 --> 00:55:36,910 It's like an ordinary tree. 783 00:55:36,910 --> 00:55:40,420 It has a root, and it has branches that go up. 784 00:55:43,580 --> 00:55:48,530 But any possibly defective stopping trial can be 785 00:55:48,530 --> 00:55:52,250 represented as a rooted tree where the trial at which each 786 00:55:52,250 --> 00:55:55,320 sample path stops is represented 787 00:55:55,320 --> 00:55:58,400 by a terminal node. 788 00:55:58,400 --> 00:56:00,780 So the example here I want to use is the same 789 00:56:00,780 --> 00:56:03,360 example as in the text. 790 00:56:03,360 --> 00:56:06,210 I was trying to get another example, which will be more 791 00:56:06,210 --> 00:56:07,440 interesting. 792 00:56:07,440 --> 00:56:11,490 The trouble is with these trees they get very big very 793 00:56:11,490 --> 00:56:17,840 quickly, and therefore this is not really a practical way of 794 00:56:17,840 --> 00:56:19,730 analyzing these problems. 795 00:56:19,730 --> 00:56:23,340 I think it's a nice conceptual way of analyzing. 796 00:56:23,340 --> 00:56:28,240 So, the experiment is x is a binary random variable, and 797 00:56:28,240 --> 00:56:33,090 stopping occurs when the pattern 1, 0 first occurs. 798 00:56:33,090 --> 00:56:36,030 You can certainly look at any other pattern you want to, any 799 00:56:36,030 --> 00:56:37,520 much longer pattern. 800 00:56:37,520 --> 00:56:42,010 But what I want to do here is to illustrate what the tree is 801 00:56:42,010 --> 00:56:46,220 that corresponds to this stopping rule. 802 00:56:46,220 --> 00:56:53,680 So if the first random variable has a value 1 and 803 00:56:53,680 --> 00:56:59,230 then the second one has a value 0, you observe 1, 0, and 804 00:56:59,230 --> 00:57:01,530 at that point the experiment is over. 805 00:57:01,530 --> 00:57:04,430 You've seen the pattern 1, 0, and you stop. 806 00:57:04,430 --> 00:57:06,440 So there's a big circle there. 807 00:57:06,440 --> 00:57:11,040 If you observe 1 followed by a 1 followed by 0, again you 808 00:57:11,040 --> 00:57:15,830 stop 1, 1, 1, followed by a 0 you stop, and so forth. 809 00:57:15,830 --> 00:57:20,380 If you observe a 0 and then you observe a 1 and then you 810 00:57:20,380 --> 00:57:24,020 observe a 0, that's the first time at which you've seen the 811 00:57:24,020 --> 00:57:27,200 pattern 1, 0 so you stop there. 812 00:57:27,200 --> 00:57:31,470 0, 1, 1, 0, you stop there, and so forth. 813 00:57:31,470 --> 00:57:34,280 So you get this kind of ladder structure. 814 00:57:34,280 --> 00:57:37,180 The point that I want to make is not that this particular 815 00:57:37,180 --> 00:57:39,150 structure is very interesting. 816 00:57:39,150 --> 00:57:42,800 It's that whatever kind of rule you decide to use for 817 00:57:42,800 --> 00:57:46,340 stopping, you can express it in this way. 818 00:57:46,340 --> 00:57:50,090 You can express the points at which you stop by big circles 819 00:57:50,090 --> 00:57:53,600 and the points at which you don't stop as intermediate 820 00:57:53,600 --> 00:57:55,350 nodes where you keep on going. 821 00:57:58,940 --> 00:58:03,010 I think in some sense this takes the picture which you 822 00:58:03,010 --> 00:58:07,860 have of sample values, of sample functions with the rule 823 00:58:07,860 --> 00:58:12,050 that each sample value, which is often the easiest way to 824 00:58:12,050 --> 00:58:18,280 express stopping rules, but in terms of random variables it 825 00:58:18,280 --> 00:58:21,550 doesn't always give you the right story. 826 00:58:21,550 --> 00:58:26,760 This gives you a story in terms of all possible choices 827 00:58:26,760 --> 00:58:28,790 of the random variables. 828 00:58:28,790 --> 00:58:32,420 The first toss has to be either a 1 or a 0. 829 00:58:32,420 --> 00:58:36,630 The second toss has to be a 1 or a 0, but if it's a 0 you 830 00:58:36,630 --> 00:58:38,580 stop and don't go any further. 831 00:58:38,580 --> 00:58:41,020 If it's a 1 you continue further. 832 00:58:41,020 --> 00:58:43,570 So you keep continuing along here, you can 833 00:58:43,570 --> 00:58:47,000 continue forever here. 834 00:58:47,000 --> 00:58:50,580 If you want to analyze this, it has a 835 00:58:50,580 --> 00:58:53,310 rather peculiar analysis. 836 00:58:53,310 --> 00:58:57,870 How long does it take until the first 1, 0 occurs? 837 00:58:57,870 --> 00:59:05,230 Well, if you start out with a 1 this experiment lasts until 838 00:59:05,230 --> 00:59:07,920 the first 0 appears. 839 00:59:07,920 --> 00:59:10,930 Because if the 0 doesn't appear right away another 1 840 00:59:10,930 --> 00:59:13,430 appears, so you keep going along this 841 00:59:13,430 --> 00:59:16,870 path until a 0 appears. 842 00:59:16,870 --> 00:59:21,180 If you see a 0 first and then you see a 1, you're back in 843 00:59:21,180 --> 00:59:24,160 the same situation again. 844 00:59:24,160 --> 00:59:26,370 You proceed until a 0 occurs. 845 00:59:26,370 --> 00:59:31,540 What this is really saying is the time until you stop here 846 00:59:31,540 --> 00:59:37,690 consists of an arbitrary number, possibly 0 of 0's, 847 00:59:37,690 --> 00:59:42,420 followed by an arbitrary number, possibly 0 of 1's, 848 00:59:42,420 --> 00:59:45,260 followed by a single 0. 849 00:59:45,260 --> 00:59:48,500 So the only patterns you can have here which would lead to 850 00:59:48,500 --> 00:59:53,670 these terminal nodes are some number of 0's followed by some 851 00:59:53,670 --> 00:59:56,320 number of 1's, followed by a final 0. 852 01:00:00,210 --> 01:00:04,280 But the example is not important, so that just tells 853 01:00:04,280 --> 01:00:06,630 you what it means. 854 01:00:06,630 --> 01:00:11,700 With all of this we finally have Wald's equality. 855 01:00:11,700 --> 01:00:18,140 And Wald's equality, it looks like a strange thing. 856 01:00:18,140 --> 01:00:23,880 I spent so much time talking about stopping rules because 857 01:00:23,880 --> 01:00:26,760 all the complexity lies in the stopping rules. 858 01:00:26,760 --> 01:00:30,790 Wald's equality itself is a very simple thing after you 859 01:00:30,790 --> 01:00:32,010 understand that. 860 01:00:32,010 --> 01:00:37,440 What Wald's equality says is, let x sub n be a sequence of 861 01:00:37,440 --> 01:00:42,420 IID random variables, each would mean x-bar. 862 01:00:42,420 --> 01:00:43,760 This is the kind of thing we've been 863 01:00:43,760 --> 01:00:45,320 talking about a lot. 864 01:00:45,320 --> 01:00:49,740 If J is a stopping trial for x sub n, n greater than or equal 865 01:00:49,740 --> 01:00:53,050 to 1, in other words, if it satisfies that definition 866 01:00:53,050 --> 01:00:57,230 where it never peeks ahead, and if the expected value of J 867 01:00:57,230 --> 01:01:02,695 is less than infinity, then the sum s sub J equals x1, x2, 868 01:01:02,695 --> 01:01:07,520 up to x sub J, the amount that the gambler has accumulated 869 01:01:07,520 --> 01:01:08,840 before the gambler stops. 870 01:01:11,730 --> 01:01:16,500 The sum of the stopping trial satisfied expected value of 871 01:01:16,500 --> 01:01:21,120 the gain as equal to the expected value of x times the 872 01:01:21,120 --> 01:01:24,610 expected number of times you play. 873 01:01:24,610 --> 01:01:29,820 It sort of says that even when you use these stopping rules 874 01:01:29,820 --> 01:01:33,080 to decide when to stop, there's no 875 01:01:33,080 --> 01:01:34,330 way to beat the house. 876 01:01:41,760 --> 01:01:46,830 If you're playing a fair game, your expected gain is equal to 877 01:01:46,830 --> 01:01:50,920 the expected gain for trial times the expected number of 878 01:01:50,920 --> 01:01:53,210 times you play. 879 01:01:53,210 --> 01:01:55,375 And there's no way you can get around that. 880 01:02:01,230 --> 01:02:03,430 Note that this is more than a [? poof ?] 881 01:02:03,430 --> 01:02:04,670 and less than a proof. 882 01:02:04,670 --> 01:02:07,530 I'll explain why it's more than a [? poof ?] 883 01:02:07,530 --> 01:02:11,630 and less than a proof as we go. 884 01:02:11,630 --> 01:02:19,060 s sub J, the random variable s sub J, it's equal to x sub 1 885 01:02:19,060 --> 01:02:21,630 times the indicator function of j greater 886 01:02:21,630 --> 01:02:24,070 than or equal to 1. 887 01:02:24,070 --> 01:02:27,860 In other words, well, the indicator function J greater 888 01:02:27,860 --> 01:02:31,900 than or equal to 1 is just universally 1. 889 01:02:31,900 --> 01:02:35,630 x sub 2 times the indicator function of J greater than or 890 01:02:35,630 --> 01:02:36,350 equal to 2. 891 01:02:36,350 --> 01:02:41,210 In other words, s sub J includes x sub 2 if the 892 01:02:41,210 --> 01:02:45,490 experiment proceeds long enough that you take the 893 01:02:45,490 --> 01:02:46,820 second trial. 894 01:02:46,820 --> 01:02:54,480 In other words, this quantity here, is 1 minus the 895 01:02:54,480 --> 01:02:58,980 probability that you stop at the end of the first trial. 896 01:02:58,980 --> 01:03:05,370 You continue here, each x sub n is included if the 897 01:03:05,370 --> 01:03:09,920 experiment has not been stopped before time n. 898 01:03:09,920 --> 01:03:15,360 So the experiments continue under the event that the 899 01:03:15,360 --> 01:03:19,770 stopping time was greater than or equal to the J. So the 900 01:03:19,770 --> 01:03:25,180 expected value of s of J is equal to the expected value of 901 01:03:25,180 --> 01:03:30,520 this sum here, of x sub n times the indicator function 902 01:03:30,520 --> 01:03:34,840 of J greater than or equal to n, which is equal to the sum 903 01:03:34,840 --> 01:03:38,730 over n of the expected value of x sub n times this 904 01:03:38,730 --> 01:03:39,980 indicator function. 905 01:03:43,100 --> 01:03:46,280 By this time I hope that most of you are at least a little 906 01:03:46,280 --> 01:03:52,610 bit sensitive to interchanging expectations and sums. 907 01:03:52,610 --> 01:03:56,730 And the notes and the problem sets deal with that. 908 01:03:56,730 --> 01:03:59,400 That's the part of the proof that I don't want to talk 909 01:03:59,400 --> 01:04:02,410 about here, because it isn't really very interesting. 910 01:04:02,410 --> 01:04:08,610 It's just one of these typical tedious analysis things. 911 01:04:08,610 --> 01:04:10,730 Most of the time this is valid. 912 01:04:10,730 --> 01:04:14,470 We will see an example where it isn't, so you'll see 913 01:04:14,470 --> 01:04:16,630 what's going on. 914 01:04:16,630 --> 01:04:20,060 The essence of the proof, however, the interesting part 915 01:04:20,060 --> 01:04:25,690 of the proof is to show that xn and this indicator function 916 01:04:25,690 --> 01:04:28,630 are independent random variables. 917 01:04:28,630 --> 01:04:34,140 So you can think of that as a separate limit, that the n-th 918 01:04:34,140 --> 01:04:40,020 trial and the indicator function of J greater than or 919 01:04:40,020 --> 01:04:43,930 equal to n are independent of each other. 920 01:04:43,930 --> 01:04:47,430 This seems a little bit weird, and it seems a little bit 921 01:04:47,430 --> 01:04:51,770 weird because this includes the event that 922 01:04:51,770 --> 01:04:54,030 J is equal to n. 923 01:04:54,030 --> 01:04:56,840 And the event that J is equal to n is something that's 924 01:04:56,840 --> 01:05:05,040 decided on the basis of the previous arrivals, or the 925 01:05:05,040 --> 01:05:07,440 previous what have you. 926 01:05:07,440 --> 01:05:11,130 It's highly influenced by x sub n. 927 01:05:11,130 --> 01:05:14,950 So, x sub n is not independent of the indicator 928 01:05:14,950 --> 01:05:19,240 function of J equals n. 929 01:05:19,240 --> 01:05:22,760 So this is strange. 930 01:05:22,760 --> 01:05:24,440 But we'll see how this works out. 931 01:05:27,750 --> 01:05:35,810 Now, if we want to show that x sub n and this indicator 932 01:05:35,810 --> 01:05:41,300 function are independent, the thing that you do is note that 933 01:05:41,300 --> 01:05:46,220 the indicator function of J greater than or equal to n, 934 01:05:46,220 --> 01:05:50,130 this is the set of events in which J is greater than or 935 01:05:50,130 --> 01:05:51,240 equal to n. 936 01:05:51,240 --> 01:05:53,910 What's the complement of that event? 937 01:05:53,910 --> 01:05:57,730 The complement of the event J greater than or equal to n is 938 01:05:57,730 --> 01:06:00,230 the event J less than n. 939 01:06:00,230 --> 01:06:04,280 So this indicator function is 1 minus 940 01:06:04,280 --> 01:06:06,110 this indicator function. 941 01:06:06,110 --> 01:06:09,890 J less than n is the complement of the event J 942 01:06:09,890 --> 01:06:11,500 greater than or equal to n. 943 01:06:11,500 --> 01:06:14,140 When you perform the experiment either this 944 01:06:14,140 --> 01:06:19,020 happens, in which case this is 1, or this happens, in which 945 01:06:19,020 --> 01:06:24,760 case this is 1, and if this is 1, this is 0. 946 01:06:24,760 --> 01:06:27,450 If this is 1, this is 0. 947 01:06:27,450 --> 01:06:32,020 Also, the indicator function and J less than n is a 948 01:06:32,020 --> 01:06:37,370 function of x1, x2, up to x sub n minus 1. 949 01:06:37,370 --> 01:06:42,190 Let me spell that out a little more because it looks a little 950 01:06:42,190 --> 01:06:45,020 strange the way it is. 951 01:06:45,020 --> 01:06:56,300 i of J less than n is equal to i if the indicator random 952 01:06:56,300 --> 01:07:01,090 variable for J equals 1. 953 01:07:03,950 --> 01:07:08,640 Union with the indicator random variable 954 01:07:08,640 --> 01:07:10,700 for J equals 2. 955 01:07:10,700 --> 01:07:19,910 Union indicator function for J equals n minus 1. 956 01:07:19,910 --> 01:07:22,190 These are all disjoint events. 957 01:07:22,190 --> 01:07:24,590 You can't stop at two different times. 958 01:07:24,590 --> 01:07:29,280 So if you stop before time n you either stop at time 1, or 959 01:07:29,280 --> 01:07:34,710 you stop at time 2, or you stop at time n minus 1. 960 01:07:37,540 --> 01:07:39,685 This event is independent. 961 01:07:43,740 --> 01:07:49,616 Well not independent, it depends on x sub 1. 962 01:07:53,160 --> 01:07:58,550 This event depends on x1 and x2. 963 01:07:58,550 --> 01:08:07,040 This event depends on x1 up to x sub n minus 1. 964 01:08:07,040 --> 01:08:12,320 OK, so this is what I mean when I say that the event J 965 01:08:12,320 --> 01:08:18,550 less than n is determined by x1 up to x of n minus 1, 966 01:08:18,550 --> 01:08:23,060 because each of these sub events is determined by a sub 967 01:08:23,060 --> 01:08:24,390 event up there. 968 01:08:29,310 --> 01:08:33,939 Since we're looking at the situation now where the x is 969 01:08:33,939 --> 01:08:44,510 our IID and this event is a function of x1 x of n minus 1, 970 01:08:44,510 --> 01:08:47,090 it just depends on those random variables. 971 01:08:47,090 --> 01:08:53,310 These random variables are independent of x sub n so i of 972 01:08:53,310 --> 01:09:00,840 J less than n is independent of x sub n and thus x sub n is 973 01:09:00,840 --> 01:09:05,359 independent of J greater than or equal to n. 974 01:09:05,359 --> 01:09:08,350 Now, as I said, this is very surprising, and it's very 975 01:09:08,350 --> 01:09:16,800 surprising because this event, J greater than or equal to n, 976 01:09:16,800 --> 01:09:20,540 typically depends very heavily on x sub n. 977 01:09:20,540 --> 01:09:24,930 It depends on J equals n plus 1, and so forth. 978 01:09:24,930 --> 01:09:28,319 So it's a little paradoxical and the resolution of the 979 01:09:28,319 --> 01:09:33,149 paradox is that given that J is greater than or equal to n, 980 01:09:33,149 --> 01:09:38,990 in other words, given that you haven't stopped before time n, 981 01:09:38,990 --> 01:09:43,779 the time at which you stop is very dependent on the 982 01:09:43,779 --> 01:09:46,170 observations that you make. 983 01:09:46,170 --> 01:09:51,260 But whether you stopped before time n or whether you haven't 984 01:09:51,260 --> 01:09:54,040 stopped before time n is independent of x sub n. 985 01:09:57,760 --> 01:10:01,680 So it really is this conditioning here that makes 986 01:10:01,680 --> 01:10:03,570 this a confusing issue. 987 01:10:03,570 --> 01:10:08,730 But whether or not J greater than n occurs depends only on 988 01:10:08,730 --> 01:10:12,330 x1 up to x sub n minus 1. 989 01:10:12,330 --> 01:10:17,060 And with that it's easy enough to finish the proof, it's just 990 01:10:17,060 --> 01:10:18,920 writing a few equations. 991 01:10:18,920 --> 01:10:23,320 The expected value of s sub J is equal to the sum over n and 992 01:10:23,320 --> 01:10:27,440 the expected value of x sub n times the indicator function 993 01:10:27,440 --> 01:10:30,290 of J greater than or equal to n. 994 01:10:30,290 --> 01:10:33,590 We've just shown that these are independent. 995 01:10:33,590 --> 01:10:36,870 This random variable is independent of this random 996 01:10:36,870 --> 01:10:40,150 variable, that's independent of the event J greater than or 997 01:10:40,150 --> 01:10:41,940 equal to n, so it's independent 998 01:10:41,940 --> 01:10:43,890 of the random variable. 999 01:10:43,890 --> 01:10:47,550 The indicator function is J greater than or equal to n, so 1000 01:10:47,550 --> 01:10:49,700 we can write this way. 1001 01:10:49,700 --> 01:10:53,140 Now, when we have this expression each of the x sub 1002 01:10:53,140 --> 01:10:58,650 n's are IID, so all of these are the same quantity x-bar. 1003 01:10:58,650 --> 01:11:02,160 So since they're all the same quantity x-bar we can just 1004 01:11:02,160 --> 01:11:04,550 take it outside of the summation. 1005 01:11:04,550 --> 01:11:09,805 So we have x-bar times the sum of the expected value of i of 1006 01:11:09,805 --> 01:11:13,100 J greater than or equal to n. 1007 01:11:13,100 --> 01:11:17,480 Now, the expected value of this indicator function, this 1008 01:11:17,480 --> 01:11:22,800 indicator function is 1 when J is greater than or equal to 1, 1009 01:11:22,800 --> 01:11:24,610 and it's 0 otherwise. 1010 01:11:24,610 --> 01:11:29,450 So the expected value of this is just a probability that J 1011 01:11:29,450 --> 01:11:31,940 is greater than or equal to n. 1012 01:11:31,940 --> 01:11:38,100 So, this is equal to the expected value of x times the 1013 01:11:38,100 --> 01:11:41,980 sum of the probabilities that J is greater 1014 01:11:41,980 --> 01:11:43,230 than or equal to n. 1015 01:11:48,040 --> 01:11:51,560 This is really adding the elements of the complimentary 1016 01:11:51,560 --> 01:11:55,770 distribution function for J, and that gives us the expected 1017 01:11:55,770 --> 01:12:02,290 value of J. If finite or not finite gives us this anyway. 1018 01:12:02,290 --> 01:12:06,515 So, this really is a proof except for that one step. 1019 01:12:15,050 --> 01:12:20,700 Except for this interchange of expectation and summation here 1020 01:12:20,700 --> 01:12:29,480 which is, well, we will get an idea of what that has to do 1021 01:12:29,480 --> 01:12:30,970 with anything. 1022 01:12:30,970 --> 01:12:33,570 So let's look at an example. 1023 01:12:33,570 --> 01:12:36,270 Stop when you're ahead. 1024 01:12:36,270 --> 01:12:41,360 And stop when you're ahead is a strategy for 1025 01:12:41,360 --> 01:12:45,180 gambling with coins. 1026 01:12:45,180 --> 01:12:47,440 You toss a coin with probability of 1027 01:12:47,440 --> 01:12:48,980 heads equal to p. 1028 01:12:52,280 --> 01:12:55,020 And we want to look at all the different cases for p, whether 1029 01:12:55,020 --> 01:12:59,860 p is fair or biased in your favor or biased in the other 1030 01:12:59,860 --> 01:13:02,250 person's favor. 1031 01:13:02,250 --> 01:13:05,090 And the rule is that you stop whenever you're 1032 01:13:05,090 --> 01:13:07,290 winnings reach one. 1033 01:13:07,290 --> 01:13:11,590 So you keep on dumbly gambling, and gambling, and 1034 01:13:11,590 --> 01:13:15,150 gambling, until finally you get to the point where you're 1035 01:13:15,150 --> 01:13:18,580 one ahead and when you're one ahead you sigh a sigh of 1036 01:13:18,580 --> 01:13:23,140 relief and say, now my wife won't be angry at me, or my 1037 01:13:23,140 --> 01:13:28,400 husband won't divorce me, or something of that sort. 1038 01:13:28,400 --> 01:13:33,400 And you walk away swearing never to gamble again. 1039 01:13:33,400 --> 01:13:37,990 Except, when you look at this you say, aha! 1040 01:13:37,990 --> 01:13:42,400 If the game is fair, I am sure to win, and after I win I 1041 01:13:42,400 --> 01:13:45,110 might as well play again, because I'm sure to win again, 1042 01:13:45,110 --> 01:13:48,270 and I'm sure to win again, and I'm sure to win again, and so 1043 01:13:48,270 --> 01:13:50,700 forth, which is all very strange. 1044 01:13:50,700 --> 01:13:53,870 And we'll see why that is in a little bit. 1045 01:13:53,870 --> 01:13:57,210 But first let's look at the case where p is 1046 01:13:57,210 --> 01:13:59,480 greater than 1/2. 1047 01:13:59,480 --> 01:14:03,340 In other words, you have a loaded coin and you've talked 1048 01:14:03,340 --> 01:14:08,270 somebody else, some poor sucker, into playing with you, 1049 01:14:08,270 --> 01:14:10,650 and you've decided you're going to stop the first time 1050 01:14:10,650 --> 01:14:12,420 you're ahead. 1051 01:14:12,420 --> 01:14:18,490 And because p is greater than 1/2, the expected value of 1052 01:14:18,490 --> 01:14:23,590 your gain each time you play the game is greater than 0. 1053 01:14:23,590 --> 01:14:27,820 You're applying IID random variables so that eventually 1054 01:14:27,820 --> 01:14:30,500 your winnings, if you kept on playing forever, 1055 01:14:30,500 --> 01:14:32,360 would become humongous. 1056 01:14:32,360 --> 01:14:37,030 So at some point you have to be 1 ahead in this process of 1057 01:14:37,030 --> 01:14:40,300 getting to the point where you're arbitrarily large 1058 01:14:40,300 --> 01:14:41,370 amounts ahead. 1059 01:14:41,370 --> 01:14:47,190 So with probability 1, you will eventually become ahead. 1060 01:14:47,190 --> 01:14:52,670 We would like to know since we know that you're going to be 1061 01:14:52,670 --> 01:14:56,030 ahead at some point, how long has it going to 1062 01:14:56,030 --> 01:14:59,410 take you to get ahead? 1063 01:14:59,410 --> 01:15:02,320 We know that j has to be a random variable in this case, 1064 01:15:02,320 --> 01:15:07,200 because we know you have to win with probability 1 s sub J 1065 01:15:07,200 --> 01:15:09,820 equals 1 with probability 1. 1066 01:15:09,820 --> 01:15:15,190 Namely, your winnings up to the time J. At the time J your 1067 01:15:15,190 --> 01:15:16,710 winnings are 1. 1068 01:15:16,710 --> 01:15:21,210 So s sub J is equal to 1, the expected value of s sub J is 1069 01:15:21,210 --> 01:15:28,330 equal to 1, and Wald says that the expected value of J, the 1070 01:15:28,330 --> 01:15:34,060 expected time you have to play, is just 1 over x-bar. 1071 01:15:37,590 --> 01:15:38,840 Isn't that neat? 1072 01:15:41,470 --> 01:15:44,800 Which says it's 1 over 2p minus 1. 1073 01:15:44,800 --> 01:15:52,520 The expected value of x is p plus minus 1 times 1 minus p 1074 01:15:52,520 --> 01:15:56,450 and if you look at that it's 2p minus 1, which is positive 1075 01:15:56,450 --> 01:15:57,790 when p is bigger than 1/2. 1076 01:16:00,650 --> 01:16:06,200 After all the abstraction of Wald's equality it's nice to 1077 01:16:06,200 --> 01:16:10,020 look at this and solve it in a different way to see if we get 1078 01:16:10,020 --> 01:16:10,940 the same answer. 1079 01:16:10,940 --> 01:16:12,190 So we'll do that. 1080 01:16:20,430 --> 01:16:26,350 If J is equal to 1, that means that on the first flip of the 1081 01:16:26,350 --> 01:16:31,780 coin you've got heads, you became 1 up and you stopped. 1082 01:16:31,780 --> 01:16:36,440 So the experiment was over at time 1 if the 1083 01:16:36,440 --> 01:16:39,070 first toss was a head. 1084 01:16:39,070 --> 01:16:43,350 So if the first toss was a tail, on the other hand, what 1085 01:16:43,350 --> 01:16:48,030 has to happen in order for you to win? 1086 01:16:48,030 --> 01:16:51,920 Well, you're sitting there at minus 1 and you're going to 1087 01:16:51,920 --> 01:16:55,900 stop when you get to plus 1. 1088 01:16:55,900 --> 01:16:59,670 Every time you move up one or down one, which means if 1089 01:16:59,670 --> 01:17:03,320 you're ever going to get from minus 1 up to plus 1, you've 1090 01:17:03,320 --> 01:17:07,410 got to go through 0 in order to get there. 1091 01:17:07,410 --> 01:17:11,410 So let's look at the expected time that it takes to get to 0 1092 01:17:11,410 --> 01:17:14,820 for the first time, and then we'll look at the expected 1093 01:17:14,820 --> 01:17:20,350 time that it takes to get from 0 to 1 for the first time. 1094 01:17:20,350 --> 01:17:24,960 Well, the expected time that it takes to get from minus 1 1095 01:17:24,960 --> 01:17:29,560 to 0 is exactly the same as the expected time that it 1096 01:17:29,560 --> 01:17:31,980 takes to get from 0 to 1. 1097 01:17:31,980 --> 01:17:39,700 So, the equation that you then have is the expected value of 1098 01:17:39,700 --> 01:17:42,690 J is going to be 1. 1099 01:17:42,690 --> 01:17:46,930 This is the first step you have to take anyway, and with 1100 01:17:46,930 --> 01:17:52,260 probability 1 minus p you went to minus 1, and if you went to 1101 01:17:52,260 --> 01:17:57,965 minus 1 you still have J-bar left to go to get to 0 again, 1102 01:17:57,965 --> 01:18:06,960 and another J-bar to get to 1, so J-bar is equal to 1 plus 1103 01:18:06,960 --> 01:18:10,940 failing the first time it takes 2J bar to get 1104 01:18:10,940 --> 01:18:12,720 back up to plus 1. 1105 01:18:12,720 --> 01:18:19,870 If you analyze this equation, J-bar is equal to 1 plus 1106 01:18:19,870 --> 01:18:26,490 2J-bar minus 2Jp-bar, you work that out and it's just 1 over 1107 01:18:26,490 --> 01:18:27,980 2p minus 1. 1108 01:18:27,980 --> 01:18:31,850 So fortunately we get the same answer as we got with the 1109 01:18:31,850 --> 01:18:33,100 Wald's equality. 1110 01:18:36,580 --> 01:18:39,010 Let's go one again. 1111 01:18:39,010 --> 01:18:43,880 Let's look at what happens if you go to the casino and you 1112 01:18:43,880 --> 01:18:50,130 play a game where you win $1 or lose $1. 1113 01:18:50,130 --> 01:18:55,350 But the probability in casinos is somehow always tilted to be 1114 01:18:55,350 --> 01:18:56,600 a little less. 1115 01:19:00,790 --> 01:19:06,080 If you play for red or black in roulette, what happens is 1116 01:19:06,080 --> 01:19:11,440 that red or black are equally likely, but there's the 0 and 1117 01:19:11,440 --> 01:19:15,090 double 0 where the house always wins. 1118 01:19:15,090 --> 01:19:18,470 So your probability of winning is always just a little less 1119 01:19:18,470 --> 01:19:23,990 than 1/2, so that's the situation we have here. 1120 01:19:23,990 --> 01:19:27,450 It's still possible to win and stop. 1121 01:19:27,450 --> 01:19:32,610 Like if you win on the first toss, then 1122 01:19:32,610 --> 01:19:34,880 you're going to stop. 1123 01:19:34,880 --> 01:19:38,340 So J equals 1 with probability p. 1124 01:19:38,340 --> 01:19:42,780 J equals 3 if you lose, and then you win, 1125 01:19:42,780 --> 01:19:43,930 and then you win. 1126 01:19:43,930 --> 01:19:48,660 So J equals 3 with probability p squared times 1 minus b. 1127 01:19:48,660 --> 01:19:52,680 And there are lots of other situations you can look at J 1128 01:19:52,680 --> 01:19:57,150 equals 5 and see when you win then, and you can count all 1129 01:19:57,150 --> 01:19:58,990 these things up. 1130 01:19:58,990 --> 01:20:04,730 Let's try to find a simple way of counting them up. 1131 01:20:04,730 --> 01:20:08,880 Let's let theta be the probability that J is less 1132 01:20:08,880 --> 01:20:10,960 than infinity. 1133 01:20:10,960 --> 01:20:13,480 In other words, theta is the probability that you're ever 1134 01:20:13,480 --> 01:20:15,010 going to stop. 1135 01:20:15,010 --> 01:20:18,610 Sometimes in this game you won't ever stop because you're 1136 01:20:18,610 --> 01:20:21,820 going to lose for while, and as soon as you lose a fair 1137 01:20:21,820 --> 01:20:26,090 amount you're very unlikely to ever recover because you're 1138 01:20:26,090 --> 01:20:27,640 drifting South all the time. 1139 01:20:40,970 --> 01:20:43,580 Since you're drifting South, theta is going to 1140 01:20:43,580 --> 01:20:46,750 be less than 1. 1141 01:20:46,750 --> 01:20:49,050 So how do we analyze this? 1142 01:20:49,050 --> 01:20:53,980 Same way as before, given that J is greater than 1, in other 1143 01:20:53,980 --> 01:20:57,970 words, given that you lose on the first trial, the event J 1144 01:20:57,970 --> 01:21:04,310 less than infinity requires that at some time you go from 1145 01:21:04,310 --> 01:21:06,580 minus 1 back up to 1. 1146 01:21:06,580 --> 01:21:12,170 In other words, there's some time m at which s sub m minus 1147 01:21:12,170 --> 01:21:14,330 s sub 1 is equal to 1. 1148 01:21:14,330 --> 01:21:17,760 s sub m is the time at which you first get from minus 1 1149 01:21:17,760 --> 01:21:21,860 back up to 0, and then there's some time that it takes to get 1150 01:21:21,860 --> 01:21:25,530 from 0 up to 1, and there's some 1151 01:21:25,530 --> 01:21:27,820 probability that ever happened. 1152 01:21:27,820 --> 01:21:33,400 The probability that you ever reach 0 is equal to theta, the 1153 01:21:33,400 --> 01:21:38,720 probability you ever get from 0 up to plus 1 is theta again. 1154 01:21:38,720 --> 01:21:42,740 So theta is equal to p, which is the probability that you 1155 01:21:42,740 --> 01:21:46,640 win right away, plus probability you lose on the 1156 01:21:46,640 --> 01:21:53,990 first toss, and you ever get from minus 1 to 0 and you ever 1157 01:21:53,990 --> 01:21:55,650 get from 0 to 1. 1158 01:21:55,650 --> 01:21:59,500 So theta equals p plus 1 minus p times theta squared. 1159 01:21:59,500 --> 01:22:01,010 There are two solutions to this. 1160 01:22:04,420 --> 01:22:09,120 One is that theta it is p over 1 minus p and the other is 1161 01:22:09,120 --> 01:22:13,230 theta equals 1, which we know is impossible. 1162 01:22:13,230 --> 01:22:16,610 And thus J is defective, and Wald's equation is 1163 01:22:16,610 --> 01:22:19,240 inapplicable. 1164 01:22:19,240 --> 01:22:23,600 But the solution that we have, which is useful to know, is 1165 01:22:23,600 --> 01:22:28,970 that your probability of ever winning is p over 1 minus p. 1166 01:22:28,970 --> 01:22:30,220 Nice to know. 1167 01:22:33,130 --> 01:22:37,280 That's what happens when you go to a casino. 1168 01:22:37,280 --> 01:22:41,570 Finally, let's consider the interesting case which is p 1169 01:22:41,570 --> 01:22:43,470 equals 1/2. 1170 01:22:43,470 --> 01:22:47,850 The limit as p approaches 1/2 from below, probability that J 1171 01:22:47,850 --> 01:23:02,840 less than infinity goes from p over 1 minus p goes to 1. 1172 01:23:02,840 --> 01:23:07,830 In other words, in a fair game, and this is not obvious, 1173 01:23:07,830 --> 01:23:11,350 there's probability 1 that you will eventually get to the 1174 01:23:11,350 --> 01:23:16,270 point where s sub n is equal to 1. 1175 01:23:16,270 --> 01:23:23,550 You can sort of see this if you view this in terms of the 1176 01:23:23,550 --> 01:23:24,840 central limit theorem. 1177 01:23:27,850 --> 01:23:30,910 And we looked pretty hard at the central limit theorem in 1178 01:23:30,910 --> 01:23:33,910 the case of these binary experiments. 1179 01:23:33,910 --> 01:23:42,170 As n gets bigger and bigger, the standard deviation of this 1180 01:23:42,170 --> 01:23:45,720 grid grows as the square root of n. 1181 01:23:45,720 --> 01:23:48,900 Now the standard deviation is growing as the square root of 1182 01:23:48,900 --> 01:23:53,675 n and over a period of n you're wobbling around. 1183 01:23:56,560 --> 01:24:00,050 You can see that it doesn't make any sense to have the 1184 01:24:00,050 --> 01:24:03,630 standard deviation growing with the square root of n if 1185 01:24:03,630 --> 01:24:06,820 you don't at some point go from the positive half down to 1186 01:24:06,820 --> 01:24:09,150 the negative half. 1187 01:24:09,150 --> 01:24:12,350 If you always stay in the positive half after a certain 1188 01:24:12,350 --> 01:24:16,920 point, then that much time later you're always going to 1189 01:24:16,920 --> 01:24:22,070 stay in the same half again, and that doesn't work in terms 1190 01:24:22,070 --> 01:24:24,600 of the square root of n. 1191 01:24:24,600 --> 01:24:28,850 So you can convince yourself on the basis of a very tedious 1192 01:24:28,850 --> 01:24:31,670 argument what this very simple argument shows you. 1193 01:24:35,010 --> 01:24:38,430 And as soon as we start studying Markov chains with 1194 01:24:38,430 --> 01:24:41,710 uncountably infinite number of states, we're going to analyze 1195 01:24:41,710 --> 01:24:43,470 this case again. 1196 01:24:43,470 --> 01:24:46,720 In other words, this is a very interesting situation because 1197 01:24:46,720 --> 01:24:48,750 something very interesting happens here and we're going 1198 01:24:48,750 --> 01:24:51,880 to analyze it at least three different ways. 1199 01:24:51,880 --> 01:24:54,560 So if you don't believe this way, you will 1200 01:24:54,560 --> 01:24:57,780 believe the other way. 1201 01:24:57,780 --> 01:25:04,351 And what happens as p approaches 1/2 from above you 1202 01:25:04,351 --> 01:25:08,040 know the expected value of J approaches infinity. 1203 01:25:08,040 --> 01:25:12,020 So what's going to happen at p equals 1/2 as the expected 1204 01:25:12,020 --> 01:25:17,500 time for you to win goes to infinity, the probability of 1205 01:25:17,500 --> 01:25:23,460 your winning goes to 1, and Wald's equality does not hold 1206 01:25:23,460 --> 01:25:28,800 because the expected value of x is equal to 0, and you look 1207 01:25:28,800 --> 01:25:33,050 at 0 times infinity and it doesn't tell you anything. 1208 01:25:33,050 --> 01:25:39,520 The expected value of s sub J s sub n at every value of n is 1209 01:25:39,520 --> 01:25:44,610 0, but s sub n when the experiment ends is equal to 1. 1210 01:25:44,610 --> 01:25:49,760 So there's something very funny going on here and if you 1211 01:25:49,760 --> 01:25:52,800 think in practical terms though, it takes an infinite 1212 01:25:52,800 --> 01:25:58,250 time to make your $1.00 and more important, it requires 1213 01:25:58,250 --> 01:26:00,610 access to an infinite capital. 1214 01:26:00,610 --> 01:26:04,570 We will see that if you limit your capital, if you limit how 1215 01:26:04,570 --> 01:26:09,050 far South you can go, then in fact you don't win with 1216 01:26:09,050 --> 01:26:13,720 probability 1, because you can go there also. 1217 01:26:13,720 --> 01:26:16,500 Next time we will apply this to renewal processes.