1
00:00:00,499 --> 00:00:02,930
In this segment we
develop some consequences

2
00:00:02,930 --> 00:00:04,890
of the independence
assumption that we

3
00:00:04,890 --> 00:00:09,140
have made on the trials that
constitute a Bernoulli process.

4
00:00:09,140 --> 00:00:12,440
These properties will
be pretty intuitive,

5
00:00:12,440 --> 00:00:14,260
but they play an important role.

6
00:00:14,260 --> 00:00:15,960
They're helpful in
solving problems,

7
00:00:15,960 --> 00:00:18,990
and they're also quite
helpful in understanding

8
00:00:18,990 --> 00:00:22,010
the continuous time version of
the Bernoulli process, namely

9
00:00:22,010 --> 00:00:25,640
the Poisson process that
we will be studying later.

10
00:00:25,640 --> 00:00:27,130
So here's the story.

11
00:00:27,130 --> 00:00:30,010
We start with a
Bernoulli processes

12
00:00:30,010 --> 00:00:31,460
with some parameter p.

13
00:00:34,210 --> 00:00:36,250
The process starts.

14
00:00:36,250 --> 00:00:39,480
A friend of yours
watches the processes,

15
00:00:39,480 --> 00:00:42,830
and they observe the results
of the different trials,

16
00:00:42,830 --> 00:00:46,000
let's say for five time steps.

17
00:00:46,000 --> 00:00:50,790
And at this time,
right after time five,

18
00:00:50,790 --> 00:00:54,400
they call you into
the room, and you

19
00:00:54,400 --> 00:00:58,680
start watching the
rest of the process.

20
00:00:58,680 --> 00:01:00,980
What will you see?

21
00:01:00,980 --> 00:01:03,340
The first random variable
that you will see

22
00:01:03,340 --> 00:01:06,090
is the result of whatever
happens in this time

23
00:01:06,090 --> 00:01:11,420
slot, which is the sixth
slot of the original process.

24
00:01:11,420 --> 00:01:14,030
The second random
variable that you will see

25
00:01:14,030 --> 00:01:17,590
is the result of the
seventh random variable

26
00:01:17,590 --> 00:01:20,570
in the original
process, and so on.

27
00:01:20,570 --> 00:01:23,570
So the process
that you get to see

28
00:01:23,570 --> 00:01:27,660
is the process
Yi, where i ranges

29
00:01:27,660 --> 00:01:30,580
over the non-negative integers.

30
00:01:30,580 --> 00:01:34,460
What properties does
this process have?

31
00:01:34,460 --> 00:01:37,090
Because of the assumption
that the different trials are

32
00:01:37,090 --> 00:01:40,020
independent, this means
that the first five trials

33
00:01:40,020 --> 00:01:44,130
are independent from the trials
that happen after time five.

34
00:01:44,130 --> 00:01:46,890
So one property is
that the process

35
00:01:46,890 --> 00:01:54,550
is Yi is independent
of whatever happens

36
00:01:54,550 --> 00:02:01,200
in the past, which
is X1 up to X5.

37
00:02:01,200 --> 00:02:06,910
Second, the random variable
that you see, X6, X7, and so on,

38
00:02:06,910 --> 00:02:10,699
are independent Bernoulli random
variables with parameter p.

39
00:02:10,699 --> 00:02:14,370
So the random variables
Yi constitute also

40
00:02:14,370 --> 00:02:20,110
a Bernoulli process
with parameter p.

41
00:02:20,110 --> 00:02:22,310
So the process that
you get to see,

42
00:02:22,310 --> 00:02:27,490
which is the sequence of
trials after time five,

43
00:02:27,490 --> 00:02:32,560
is identical, probabilistically,
to a Bernoulli process

44
00:02:32,560 --> 00:02:34,920
with parameter p
like the process Xi.

45
00:02:34,920 --> 00:02:37,910
So it's as if a Bernoulli
process was just

46
00:02:37,910 --> 00:02:40,960
starting fresh at
this particular time.

47
00:02:40,960 --> 00:02:44,530
And because of this,
we say that the process

48
00:02:44,530 --> 00:02:49,510
has a fresh-start property
after a certain time.

49
00:02:49,510 --> 00:02:52,850
In this example, we used
5 as the certain time,

50
00:02:52,850 --> 00:02:56,850
but instead of 5, we could
have any particular integer

51
00:02:56,850 --> 00:03:02,230
little n, in which case
our process Y1 starts

52
00:03:02,230 --> 00:03:09,060
with Xn plus 1, continues
with Xn plus 2 and so on.

53
00:03:09,060 --> 00:03:13,260
And here, instead of X5,
we would have written Xn.

54
00:03:13,260 --> 00:03:18,570
So after a deterministic
time n, what you see

55
00:03:18,570 --> 00:03:21,980
is the same as if we had
a Bernoulli process that

56
00:03:21,980 --> 00:03:24,930
was starting fresh at
this particular time,

57
00:03:24,930 --> 00:03:27,180
and which is also
independent from whatever

58
00:03:27,180 --> 00:03:29,890
happened in the past.

59
00:03:29,890 --> 00:03:33,010
Let us now complicate
the story a little bit.

60
00:03:33,010 --> 00:03:37,220
Suppose that your friend
watches the Bernoulli process,

61
00:03:37,220 --> 00:03:42,450
and they keep watching
it until a success is

62
00:03:42,450 --> 00:03:44,940
observed for the first time.

63
00:03:44,940 --> 00:03:48,220
Right at that time, they
call you into the room,

64
00:03:48,220 --> 00:03:52,880
and you started watching
the rest of the process.

65
00:03:52,880 --> 00:03:54,460
This is the length
of time that we

66
00:03:54,460 --> 00:03:58,350
have called T1, the number of
trials until the first success.

67
00:03:58,350 --> 00:04:01,060
So what is it that
you will be watching?

68
00:04:01,060 --> 00:04:03,200
The first random variable
that you will see

69
00:04:03,200 --> 00:04:09,010
is what happens
in slot T1 plus 1.

70
00:04:09,010 --> 00:04:11,160
The second random
variable that you will see

71
00:04:11,160 --> 00:04:18,350
is what happened in slot
T1 plus 2, and so on.

72
00:04:18,350 --> 00:04:21,589
And this defines,
again, a process,

73
00:04:21,589 --> 00:04:24,430
the sequence of the
Yi's This is what

74
00:04:24,430 --> 00:04:28,730
you will see starting
from this particular time.

75
00:04:28,730 --> 00:04:31,810
What kind of process is it?

76
00:04:31,810 --> 00:04:34,360
Well, these trials
happened in the past.

77
00:04:34,360 --> 00:04:36,370
We know what they were.

78
00:04:36,370 --> 00:04:39,680
But no matter what they
were, the future trials

79
00:04:39,680 --> 00:04:43,350
will still be
independent of the past,

80
00:04:43,350 --> 00:04:45,100
and each one of the
trials will have

81
00:04:45,100 --> 00:04:48,340
probability p of
being a success.

82
00:04:48,340 --> 00:04:50,550
So the properties
that we have, again,

83
00:04:50,550 --> 00:04:53,870
is that the trials
that you see are

84
00:04:53,870 --> 00:05:00,830
independent of the
past, which in this case

85
00:05:00,830 --> 00:05:05,940
is everything from
x1 up to time xT1.

86
00:05:05,940 --> 00:05:10,200
And what you see is
a Bernoulli process.

87
00:05:10,200 --> 00:05:12,460
We describe the
situation by saying

88
00:05:12,460 --> 00:05:17,390
that the process starts
fresh after time T1.

89
00:05:17,390 --> 00:05:20,950
And by this, again,
we mean that if you

90
00:05:20,950 --> 00:05:25,960
start watching the process right
after T1, what you will see

91
00:05:25,960 --> 00:05:28,470
will be a Bernoulli
process which

92
00:05:28,470 --> 00:05:33,540
is independent from whatever
happened in the past.

93
00:05:33,540 --> 00:05:35,840
Having just argued
that the process starts

94
00:05:35,840 --> 00:05:39,940
fresh at the time T1
of the first success,

95
00:05:39,940 --> 00:05:43,290
we can now ask why whether
such a property is also true

96
00:05:43,290 --> 00:05:44,690
more generally.

97
00:05:44,690 --> 00:05:49,780
That is, if we start watching
the process at some random time

98
00:05:49,780 --> 00:05:55,200
n, will the process
start fresh at that time?

99
00:05:55,200 --> 00:05:58,050
Let us look at some
additional examples.

100
00:05:58,050 --> 00:06:02,080
Suppose that capital N is the
time of the third success.

101
00:06:02,080 --> 00:06:05,670
So your friend watches
the Bernoulli process,

102
00:06:05,670 --> 00:06:10,360
and each time, they say,
did the third success occur?

103
00:06:10,360 --> 00:06:11,280
Not yet.

104
00:06:11,280 --> 00:06:12,200
Not yet.

105
00:06:12,200 --> 00:06:13,560
Not yet.

106
00:06:13,560 --> 00:06:14,840
Not yet.

107
00:06:14,840 --> 00:06:17,530
Yes, the third
success just occurred.

108
00:06:17,530 --> 00:06:20,350
And at that time, they
call you into the room

109
00:06:20,350 --> 00:06:24,920
and you start to watching what
happens from that time on.

110
00:06:24,920 --> 00:06:27,060
What will you be seeing?

111
00:06:27,060 --> 00:06:31,860
After that time, there will be
independent Bernoulli trials

112
00:06:31,860 --> 00:06:33,340
that take place.

113
00:06:33,340 --> 00:06:37,360
And these refer to the future
of the process, looking at [it]

114
00:06:37,360 --> 00:06:39,990
from this particular
point in time.

115
00:06:39,990 --> 00:06:42,750
And the future is
independent from whatever

116
00:06:42,750 --> 00:06:44,170
happened in the past.

117
00:06:44,170 --> 00:06:48,720
So what you actually see is,
indeed, a fresh Bernoulli

118
00:06:48,720 --> 00:06:51,040
process that starts
here and which

119
00:06:51,040 --> 00:06:53,190
is independent from the
random variables that

120
00:06:53,190 --> 00:06:55,430
occurred in the past.

121
00:06:55,430 --> 00:06:57,810
Let us look at another example.

122
00:06:57,810 --> 00:07:02,310
Let capital N be the first time
that three successes in a row

123
00:07:02,310 --> 00:07:05,290
have been recorded.

124
00:07:05,290 --> 00:07:07,890
So your friend, again,
watches the process.

125
00:07:07,890 --> 00:07:12,420
And they ask each time, did
we see three success in a row?

126
00:07:12,420 --> 00:07:13,660
Not yet.

127
00:07:13,660 --> 00:07:14,710
Not yet.

128
00:07:14,710 --> 00:07:15,780
Not yet.

129
00:07:15,780 --> 00:07:16,910
Not yet.

130
00:07:16,910 --> 00:07:18,470
Not yet.

131
00:07:18,470 --> 00:07:19,160
Yes.

132
00:07:19,160 --> 00:07:22,030
I just saw three
successes in a row.

133
00:07:22,030 --> 00:07:24,740
And now your friend
calls you in,

134
00:07:24,740 --> 00:07:26,810
and you start
watching the process

135
00:07:26,810 --> 00:07:28,410
from this point in time.

136
00:07:28,410 --> 00:07:30,840
By the same argument
as before, whatever

137
00:07:30,840 --> 00:07:34,020
happens in the future is going
to be Bernoulli trials that

138
00:07:34,020 --> 00:07:36,990
are independent from the
past, so you will, again,

139
00:07:36,990 --> 00:07:41,440
have a fresh-start property
starting from this time.

140
00:07:41,440 --> 00:07:44,820
So in both cases,
formally, what we have

141
00:07:44,820 --> 00:07:49,470
is that the process that you get
to observe starting after time

142
00:07:49,470 --> 00:07:53,620
capital N, after the time
that you're called and asked

143
00:07:53,620 --> 00:07:55,830
to start watching,
what you will see

144
00:07:55,830 --> 00:07:59,460
is going to be a sequence of
independent Bernoulli trials,

145
00:07:59,460 --> 00:08:01,490
that is, a Bernoulli process.

146
00:08:01,490 --> 00:08:03,420
And this sequence
of future trials

147
00:08:03,420 --> 00:08:06,110
is independent from
whatever happened

148
00:08:06,110 --> 00:08:08,470
in the past of the process.

149
00:08:08,470 --> 00:08:11,800
What both of these
examples have in common

150
00:08:11,800 --> 00:08:15,280
is that the random variable
N, the time at which you're

151
00:08:15,280 --> 00:08:19,270
called in, is
causally determined

152
00:08:19,270 --> 00:08:23,360
from the history of the process.

153
00:08:23,360 --> 00:08:25,050
What does that mean?

154
00:08:25,050 --> 00:08:28,430
It means that somebody
is watching the process,

155
00:08:28,430 --> 00:08:33,240
and at each point in time, based
on what they have seen so far,

156
00:08:33,240 --> 00:08:38,510
they decide whether
to call you in or not.

157
00:08:38,510 --> 00:08:43,250
What would be an example
of a non-causal time N?

158
00:08:43,250 --> 00:08:44,370
Here it is.

159
00:08:44,370 --> 00:08:49,090
N could be the time just before
the first occurrence of 1,

160
00:08:49,090 --> 00:08:50,120
1, 1.

161
00:08:50,120 --> 00:08:55,100
So in this example here, you
would be called into the room

162
00:08:55,100 --> 00:08:59,750
and start watching at this time.

163
00:08:59,750 --> 00:09:04,750
So your friend somehow knows
that a sequence of 1,1, 1

164
00:09:04,750 --> 00:09:09,750
is going to occur and calls
you just before it happens.

165
00:09:09,750 --> 00:09:10,960
How could that be?

166
00:09:10,960 --> 00:09:14,470
Well, imagine that the
Bernoulli process actually

167
00:09:14,470 --> 00:09:16,410
was run yesterday.

168
00:09:16,410 --> 00:09:18,090
It was recorded in a movie.

169
00:09:18,090 --> 00:09:20,404
Your friend has seen
the movie, so knows

170
00:09:20,404 --> 00:09:21,820
everything that's
going to happen.

171
00:09:21,820 --> 00:09:24,670
And so, when the movie
is replayed today,

172
00:09:24,670 --> 00:09:27,570
your friend can call you in
at this time and tell you,

173
00:09:27,570 --> 00:09:30,540
you know, something very
interesting is about happen.

174
00:09:30,540 --> 00:09:33,290
Come in and start watching.

175
00:09:33,290 --> 00:09:37,170
Now, what will you be watching?

176
00:09:37,170 --> 00:09:42,740
What you will watch will
be 1, 1, 1, with certainty.

177
00:09:42,740 --> 00:09:46,190
You're certain that the first
three trials that you will see

178
00:09:46,190 --> 00:09:47,580
will be 1's.

179
00:09:47,580 --> 00:09:50,830
And, well, the subsequent
one's will be random.

180
00:09:50,830 --> 00:09:54,690
But since you know that the
first three trials will be 1,

181
00:09:54,690 --> 00:09:56,880
this means that
statistically, they're

182
00:09:56,880 --> 00:10:01,510
not described by the statistics
of a Bernoulli process.

183
00:10:01,510 --> 00:10:03,910
In a Bernoulli
process, each trial

184
00:10:03,910 --> 00:10:08,220
has a probability of being 1
and the probability of being 0.

185
00:10:08,220 --> 00:10:11,360
But since, in your case,
you're certain to watch

186
00:10:11,360 --> 00:10:15,120
1's in the beginning, this
means that the random variables

187
00:10:15,120 --> 00:10:19,060
that you see do not conform to
the description of a Bernoulli

188
00:10:19,060 --> 00:10:20,290
process.

189
00:10:20,290 --> 00:10:26,100
So this is an example in which
N is not causally determined.

190
00:10:29,940 --> 00:10:32,540
And in this example,
you do not to get

191
00:10:32,540 --> 00:10:34,790
to see a Bernoulli process.

192
00:10:34,790 --> 00:10:39,220
We do not have the
fresh-start property.

193
00:10:39,220 --> 00:10:42,720
What happened here is
more generally true.

194
00:10:42,720 --> 00:10:46,830
We do have a fresh-start
property, but not always.

195
00:10:46,830 --> 00:10:50,210
We have it under the assumption
that the time at which you're

196
00:10:50,210 --> 00:10:53,040
asked to start
watching is determined

197
00:10:53,040 --> 00:10:57,500
from the past history of the
process in some causal manner.

198
00:10:57,500 --> 00:11:01,160
This is a general fact that
can be established rigorously.

199
00:11:01,160 --> 00:11:05,450
However we will not go through a
formal mathematical derivation.

200
00:11:05,450 --> 00:11:09,160
The formal argument for
the most general case

201
00:11:09,160 --> 00:11:12,610
involves somewhat tedious
symbol manipulations

202
00:11:12,610 --> 00:11:15,860
that do not provide
any additional insight.

203
00:11:15,860 --> 00:11:18,440
However, the intuition
behind this result

204
00:11:18,440 --> 00:11:22,150
should be fairly clear,
and we will use it freely

205
00:11:22,150 --> 00:11:24,533
whenever we need it.