1
00:00:00,810 --> 00:00:03,520
We will now go through two
examples of convergence in

2
00:00:03,520 --> 00:00:04,590
probability.

3
00:00:04,590 --> 00:00:07,370
Our first example is
quite trivial.

4
00:00:07,370 --> 00:00:11,090
We're dealing with a sequence
of random variables Yn that

5
00:00:11,090 --> 00:00:12,220
are discrete.

6
00:00:12,220 --> 00:00:15,680
Most of the probability
is concentrated at 0.

7
00:00:15,680 --> 00:00:19,690
But there is also a small
probability of a large value.

8
00:00:19,690 --> 00:00:22,040
Because the bulk of the
probability mass is

9
00:00:22,040 --> 00:00:25,540
concentrated at 0, it is a good
guess that this sequence

10
00:00:25,540 --> 00:00:27,070
converges to 0.

11
00:00:27,070 --> 00:00:29,380
Do we have, indeed,
convergence in

12
00:00:29,380 --> 00:00:30,760
probability to 0?

13
00:00:30,760 --> 00:00:32,710
We need to check
the definition.

14
00:00:32,710 --> 00:00:36,930
So we fix some epsilon, which
is a positive number.

15
00:00:36,930 --> 00:00:40,550
And we look at the probability
of the event that our random

16
00:00:40,550 --> 00:00:46,360
variable is epsilon or more away
than what we think is the

17
00:00:46,360 --> 00:00:48,460
limit of that sequence.

18
00:00:48,460 --> 00:00:50,370
We look at that probability.

19
00:00:50,370 --> 00:00:54,590
And in this example, it is equal
to 1 over n, which goes

20
00:00:54,590 --> 00:00:57,820
to 0 as n goes to infinity.

21
00:00:57,820 --> 00:01:01,990
And this verifies that, indeed,
in this example, Yn

22
00:01:01,990 --> 00:01:07,065
converges to 0, as n goes to
infinity in probability.

23
00:01:10,330 --> 00:01:12,900
Now, we make the following
observation.

24
00:01:12,900 --> 00:01:16,220
If we are to calculate the
expected value of this random

25
00:01:16,220 --> 00:01:19,390
variable, what we get
is the following.

26
00:01:19,390 --> 00:01:23,060
We get a value of 0 with this
probability, no contribution

27
00:01:23,060 --> 00:01:24,280
to the expectation.

28
00:01:24,280 --> 00:01:27,970
But we also get a value
of n squared with

29
00:01:27,970 --> 00:01:30,490
probability 1 over n.

30
00:01:30,490 --> 00:01:34,630
And so the expected value is
equal to n, which, actually,

31
00:01:34,630 --> 00:01:38,440
goes to infinity, as
n goes to infinity.

32
00:01:38,440 --> 00:01:43,120
So we have a situation where
the sequence of the random

33
00:01:43,120 --> 00:01:45,220
variables converges to 0.

34
00:01:45,220 --> 00:01:48,090
But the expectation does
not converge to 0.

35
00:01:48,090 --> 00:01:50,120
In fact, it goes to infinity.

36
00:01:50,120 --> 00:01:52,920
And this example serves
to make the point that

37
00:01:52,920 --> 00:01:56,509
convergence in probability does
not imply convergence of

38
00:01:56,509 --> 00:01:57,890
expectations.

39
00:01:57,890 --> 00:02:01,940
The reason is that convergence
in probability has to do with

40
00:02:01,940 --> 00:02:04,360
the bulk of the distribution.

41
00:02:04,360 --> 00:02:08,259
It only cares that the tail of
the distribution has small

42
00:02:08,259 --> 00:02:09,340
probability.

43
00:02:09,340 --> 00:02:11,980
On the other hand, the
expectation is highly

44
00:02:11,980 --> 00:02:15,410
sensitive to the tail
of the distribution.

45
00:02:15,410 --> 00:02:19,150
It might be that the tail only
has a small probability.

46
00:02:19,150 --> 00:02:21,900
But if that probability is
assigned to a very large

47
00:02:21,900 --> 00:02:26,140
value, then the expectation will
be strongly affected and

48
00:02:26,140 --> 00:02:29,020
can be quite different
from the limit

49
00:02:29,020 --> 00:02:32,270
of the random variable.

50
00:02:32,270 --> 00:02:35,570
Our second example is going to
be less trivial and more

51
00:02:35,570 --> 00:02:36,860
interesting.

52
00:02:36,860 --> 00:02:40,290
Consider random variables
that are independent and

53
00:02:40,290 --> 00:02:44,050
identically distributed and
whose common distribution is

54
00:02:44,050 --> 00:02:47,350
uniform on the unit interval,
so that the

55
00:02:47,350 --> 00:02:50,160
PDF takes this form.

56
00:02:50,160 --> 00:02:55,150
Are these random variables
convergent to something?

57
00:02:55,150 --> 00:02:56,480
The answer is no.

58
00:02:56,480 --> 00:03:00,980
And the reason is that as i
increases, the distribution

59
00:03:00,980 --> 00:03:02,020
does not change.

60
00:03:02,020 --> 00:03:04,380
And it does not to
get concentrated

61
00:03:04,380 --> 00:03:06,010
around a certain number.

62
00:03:06,010 --> 00:03:09,130
The distribution remains spread
out over the entire

63
00:03:09,130 --> 00:03:10,960
unit interval.

64
00:03:10,960 --> 00:03:14,330
But let us look now at some
related random variables.

65
00:03:14,330 --> 00:03:19,940
Let us define Yn to be the
minimum of the first n of the

66
00:03:19,940 --> 00:03:21,530
X's that we get.

67
00:03:21,530 --> 00:03:25,540
So if n is equal to 4, and we
obtain these four values, Yn

68
00:03:25,540 --> 00:03:28,100
would be equal to this value.

69
00:03:28,100 --> 00:03:33,560
Notice that if we draw more
values, then the new values

70
00:03:33,560 --> 00:03:36,810
might be above the minimum, in
which case the minimum does

71
00:03:36,810 --> 00:03:37,890
not change.

72
00:03:37,890 --> 00:03:41,610
But we might also get a value
that's below the minimum, in

73
00:03:41,610 --> 00:03:44,100
which case the minimum
moves down.

74
00:03:44,100 --> 00:03:48,400
The only thing that can happen
is that the minimum goes down.

75
00:03:48,400 --> 00:03:49,960
It cannot go up.

76
00:03:49,960 --> 00:03:52,680
And this gives us
this inequality.

77
00:03:52,680 --> 00:03:56,420
So the random variables
Yn tends to go down.

78
00:03:56,420 --> 00:03:58,720
How far down?

79
00:03:58,720 --> 00:04:03,640
If n is very large, we expect
that we will obtain some X's

80
00:04:03,640 --> 00:04:07,570
whose value happens to be very
close to 0, which means that

81
00:04:07,570 --> 00:04:12,580
Yn will go down to values that
are very close to 0.

82
00:04:12,580 --> 00:04:16,010
And this leads us to conjecture
that, perhaps, Yn

83
00:04:16,010 --> 00:04:18,390
does converge to 0.

84
00:04:18,390 --> 00:04:21,209
This is always the first step
when dealing with convergence

85
00:04:21,209 --> 00:04:22,029
in probability.

86
00:04:22,029 --> 00:04:25,940
The first step is to make an
educated guess about what the

87
00:04:25,940 --> 00:04:27,580
limit might be.

88
00:04:27,580 --> 00:04:30,610
And then we want to verify
that this is, indeed, the

89
00:04:30,610 --> 00:04:32,150
correct limit.

90
00:04:32,150 --> 00:04:37,380
To verify that, what we do is we
fix some positive epsilon.

91
00:04:37,380 --> 00:04:41,640
And we look for the probability
that the distance

92
00:04:41,640 --> 00:04:46,090
of the random variable Yn from
the conjectured limit has a

93
00:04:46,090 --> 00:04:49,610
magnitude that's larger than
or equal to epsilon.

94
00:04:49,610 --> 00:04:52,630
And what we need to show is that
this quantity converges

95
00:04:52,630 --> 00:04:58,320
to 0 as n goes to infinity,
no matter what epsilon is.

96
00:04:58,320 --> 00:05:01,940
Now, because Yn is a
non-negative random variable,

97
00:05:01,940 --> 00:05:05,270
this is the same as the
probability that Yn is larger

98
00:05:05,270 --> 00:05:07,330
than or equal to epsilon.

99
00:05:07,330 --> 00:05:10,210
Now, let us distinguish
between two cases.

100
00:05:10,210 --> 00:05:13,870
If epsilon is bigger than
1, we're asking for the

101
00:05:13,870 --> 00:05:18,190
probability that Yn is larger
than or equal to a certain

102
00:05:18,190 --> 00:05:20,460
number epsilon that's
out there.

103
00:05:20,460 --> 00:05:22,610
But this probability is 0.

104
00:05:22,610 --> 00:05:25,690
There's no way that the minimum
of these uniforms will

105
00:05:25,690 --> 00:05:29,000
take a value that's larger
than some epsilon that's

106
00:05:29,000 --> 00:05:30,660
larger than 1.

107
00:05:30,660 --> 00:05:34,150
So in that case, this quantity
is equal to zero.

108
00:05:34,150 --> 00:05:35,380
And we are done.

109
00:05:35,380 --> 00:05:38,530
But we need to check that this
quantity becomes small no

110
00:05:38,530 --> 00:05:40,290
matter what epsilon is.

111
00:05:40,290 --> 00:05:44,360
So now, let us consider taking
a small epsilon that is some

112
00:05:44,360 --> 00:05:47,260
number that's less than
or equal to 1.

113
00:05:47,260 --> 00:05:51,180
For that case, let us continue
with the calculation.

114
00:05:51,180 --> 00:05:56,300
The minimum is going to be at
least epsilon, if, and only

115
00:05:56,300 --> 00:05:59,490
if, all of the random variables

116
00:05:59,490 --> 00:06:01,160
are at least epsilon.

117
00:06:04,210 --> 00:06:07,950
So this is an equivalent way
of writing this particular

118
00:06:07,950 --> 00:06:09,300
event here.

119
00:06:09,300 --> 00:06:12,470
Now, because of independence,
this is the product of the

120
00:06:12,470 --> 00:06:17,080
probabilities that each one of
the random variables is larger

121
00:06:17,080 --> 00:06:20,370
than or equal to epsilon.

122
00:06:20,370 --> 00:06:23,470
The probability that X1 is
larger than or equal to

123
00:06:23,470 --> 00:06:26,150
epsilon can be found
as follows.

124
00:06:26,150 --> 00:06:29,420
If we have here epsilon, the
probability of being larger

125
00:06:29,420 --> 00:06:31,540
than or equal to epsilon
is the probability

126
00:06:31,540 --> 00:06:32,990
of this event here.

127
00:06:32,990 --> 00:06:35,020
So it's the area of
this rectangle.

128
00:06:35,020 --> 00:06:38,060
The base of that rectangle
is 1 minus epsilon.

129
00:06:38,060 --> 00:06:42,409
And so we obtain 1 minus epsilon
for this first term.

130
00:06:42,409 --> 00:06:45,180
But because the Xi's are
identically distributed, all

131
00:06:45,180 --> 00:06:48,280
the other terms that we multiply
are also the same.

132
00:06:48,280 --> 00:06:52,940
And so the answer is this
expression here.

133
00:06:52,940 --> 00:06:55,890
Now, epsilon is a
positive number.

134
00:06:55,890 --> 00:06:58,920
So 1 minus epsilon is strictly
less than 1.

135
00:06:58,920 --> 00:07:03,010
And when we take higher powers
of a number that's less than

136
00:07:03,010 --> 00:07:07,500
1, we obtain something
that converges to 0

137
00:07:07,500 --> 00:07:09,030
as n goes to infinity.

138
00:07:09,030 --> 00:07:11,690
And that's what we
needed to verify.

139
00:07:11,690 --> 00:07:16,090
Since this is the case for any
epsilon, we conclude that the

140
00:07:16,090 --> 00:07:22,370
random variables Yn converge to
zero in the sense that we

141
00:07:22,370 --> 00:07:24,135
have defined, in probability.

142
00:07:27,270 --> 00:07:31,170
Generalizing from this example,
when we want to show

143
00:07:31,170 --> 00:07:34,850
convergence in probability, the
first step is to make a

144
00:07:34,850 --> 00:07:38,540
guess as to what is
the value that the

145
00:07:38,540 --> 00:07:40,530
sequence converges to.

146
00:07:40,530 --> 00:07:44,260
In this example, that value
was equal to 0.

147
00:07:44,260 --> 00:07:48,240
Once we have made that
conjecture, then we write down

148
00:07:48,240 --> 00:07:51,540
an expression for the
probability of being epsilon

149
00:07:51,540 --> 00:07:54,690
away from the conjectured
limit.

150
00:07:54,690 --> 00:07:58,280
And then we calculate that
probability either exactly, as

151
00:07:58,280 --> 00:07:59,475
in this example.

152
00:07:59,475 --> 00:08:02,500
Or we try to bound it somehow
and still show

153
00:08:02,500 --> 00:08:03,750
that it goes to 0.