1
00:00:01,170 --> 00:00:05,680
This is a rather theoretical
exercise that has two purposes.

2
00:00:05,680 --> 00:00:09,970
One is to verify that the notion
of convergence in probability

3
00:00:09,970 --> 00:00:12,730
is quite natural and
that it has properties

4
00:00:12,730 --> 00:00:15,880
similar to the notion of
convergence of sequences.

5
00:00:15,880 --> 00:00:18,860
And the second purpose is to
get a little bit of practice

6
00:00:18,860 --> 00:00:22,390
with the formal definition of
convergence in probability.

7
00:00:22,390 --> 00:00:25,150
So what is the statement saying?

8
00:00:25,150 --> 00:00:29,840
It says that if we have a
sequence of random variables

9
00:00:29,840 --> 00:00:32,540
that converges to a
certain number, a,

10
00:00:32,540 --> 00:00:36,730
and this basically means
that when n is large,

11
00:00:36,730 --> 00:00:40,450
the distribution is highly
concentrated around a.

12
00:00:40,450 --> 00:00:42,712
And if we have another
sequence of random variables

13
00:00:42,712 --> 00:00:44,170
that converges to
a certain number,

14
00:00:44,170 --> 00:00:48,460
b, which means that the
probability distribution of Yn

15
00:00:48,460 --> 00:00:52,390
is heavily
concentrated around b.

16
00:00:52,390 --> 00:00:55,510
In that case, then the
probability distribution

17
00:00:55,510 --> 00:00:58,370
of the sum of the
two random variables

18
00:00:58,370 --> 00:01:02,880
is heavily concentrated in
the vicinity of a plus b.

19
00:01:02,880 --> 00:01:04,290
So what are we saying?

20
00:01:04,290 --> 00:01:07,810
If Xn is very close to
a with high probability

21
00:01:07,810 --> 00:01:11,370
and Yn is very close to
b with high probability,

22
00:01:11,370 --> 00:01:15,140
then the sum will also
be close to a plus b

23
00:01:15,140 --> 00:01:16,860
with high probability.

24
00:01:16,860 --> 00:01:20,070
This is the intuitive
content of the statement.

25
00:01:20,070 --> 00:01:23,510
Now we want to
establish this formally.

26
00:01:23,510 --> 00:01:25,860
Before establishing
this statement,

27
00:01:25,860 --> 00:01:28,340
however, it will
be a good practice

28
00:01:28,340 --> 00:01:31,100
to verify a property
of this type

29
00:01:31,100 --> 00:01:35,700
for the ordinary convergence
of sequences of numbers, not

30
00:01:35,700 --> 00:01:37,210
random variables.

31
00:01:37,210 --> 00:01:39,060
So let us do that.

32
00:01:39,060 --> 00:01:46,350
What we want to show is that
if a sequence of numbers, an,

33
00:01:46,350 --> 00:01:49,990
converges to some number a,
and another sequence converges

34
00:01:49,990 --> 00:01:53,300
to some number b,
we want to show that

35
00:01:53,300 --> 00:01:58,490
in that case, an plus bn
converges to the sum of a plus

36
00:01:58,490 --> 00:02:01,460
b, and we want to
do this formally.

37
00:02:04,620 --> 00:02:08,960
So let us start with the
definition of convergence.

38
00:02:08,960 --> 00:02:13,440
What does it mean that
an converges to a?

39
00:02:13,440 --> 00:02:24,590
It means that if I fix
some positive epsilon,

40
00:02:24,590 --> 00:02:36,329
then there exists some
number or some time

41
00:02:36,329 --> 00:02:45,670
such that if we consider
some n bigger than n0,

42
00:02:45,670 --> 00:02:51,850
then an is close
to a in the sense

43
00:02:51,850 --> 00:02:56,070
that this difference
is less than epsilon.

44
00:02:56,070 --> 00:02:59,800
Now this is true for
any positive epsilon,

45
00:02:59,800 --> 00:03:03,550
so if instead of epsilon,
I take epsilon over 2,

46
00:03:03,550 --> 00:03:07,940
this would also be true.

47
00:03:07,940 --> 00:03:13,090
Eventually, after some time,
we will have the property

48
00:03:13,090 --> 00:03:18,150
that an minus a is less
than epsilon over 2.

49
00:03:18,150 --> 00:03:24,460
Similarly, if bn
converges to b, then

50
00:03:24,460 --> 00:03:34,630
we will have the property that
there exists some time-- let's

51
00:03:34,630 --> 00:03:42,570
call it n0 prime-- such
that if n is bigger

52
00:03:42,570 --> 00:03:49,980
than that particular time,
then bn minus b is going

53
00:03:49,980 --> 00:03:51,880
to be less than epsilon over 2.

54
00:03:54,579 --> 00:03:59,090
So after time n0 and
after time n0 prime,

55
00:03:59,090 --> 00:04:02,790
these two inequalities
will be true.

56
00:04:02,790 --> 00:04:06,710
So if we wait long enough so
that both of these inequalities

57
00:04:06,710 --> 00:04:11,790
are true, that
is, if n is bigger

58
00:04:11,790 --> 00:04:19,339
than the maximum
of n0 and n0 prime,

59
00:04:19,339 --> 00:04:23,320
then we will have the following.

60
00:04:23,320 --> 00:04:32,170
We will have that an
plus bn minus a minus b

61
00:04:32,170 --> 00:04:34,870
which, by an
elementary inequality,

62
00:04:34,870 --> 00:04:42,750
is less than or equal to
an minus a plus bn minus b.

63
00:04:42,750 --> 00:04:45,420
Where is this
inequality coming from?

64
00:04:45,420 --> 00:04:48,190
This is a general inequality
about absolute values.

65
00:04:48,190 --> 00:04:52,630
If I give you two numbers,
the absolute value of x plus y

66
00:04:52,630 --> 00:04:57,560
is always less than or equal to
the sum of the absolute values.

67
00:04:57,560 --> 00:04:59,710
So we're using this
inequality where

68
00:04:59,710 --> 00:05:04,680
x is an minus a and
y is bn minus b.

69
00:05:04,680 --> 00:05:09,520
So we have this inequality,
but when time is big enough,

70
00:05:09,520 --> 00:05:13,520
an minus a is less
than epsilon over 2.

71
00:05:13,520 --> 00:05:19,420
bn minus b is also less
than epsilon over 2.

72
00:05:19,420 --> 00:05:22,850
And putting everything
together, this is epsilon.

73
00:05:22,850 --> 00:05:24,680
So what have we shown?

74
00:05:24,680 --> 00:05:28,160
That if an converges
to a and bn converges

75
00:05:28,160 --> 00:05:31,500
to b, so that all
these relations hold,

76
00:05:31,500 --> 00:05:36,460
then if time n is
large enough, then

77
00:05:36,460 --> 00:05:40,820
the difference between
this number and that number

78
00:05:40,820 --> 00:05:44,830
is going to be
less than epsilon.

79
00:05:44,830 --> 00:05:48,460
And this is true for
every positive epsilon,

80
00:05:48,460 --> 00:05:51,560
but that's just the
definition of convergence

81
00:05:51,560 --> 00:05:54,550
of this quantity
to that quantity.

82
00:05:54,550 --> 00:05:58,010
And this is the proof of
this elementary relation

83
00:05:58,010 --> 00:05:59,625
about convergence of numbers.

84
00:06:03,190 --> 00:06:08,210
Now let us turn to convergence
of random variables.

85
00:06:08,210 --> 00:06:14,320
We fix some epsilon
that's positive.

86
00:06:14,320 --> 00:06:17,410
In order to show
convergence in probability,

87
00:06:17,410 --> 00:06:27,270
we want to look
at this difference

88
00:06:27,270 --> 00:06:31,090
and look at the probability
that this difference is

89
00:06:31,090 --> 00:06:33,680
bigger than epsilon
in magnitude.

90
00:06:33,680 --> 00:06:37,590
And we want to show that
this quantity converges to 0.

91
00:06:42,350 --> 00:06:45,470
If it does, then we
will have established

92
00:06:45,470 --> 00:06:48,030
convergence in probability
because that's just

93
00:06:48,030 --> 00:06:50,770
the definition.

94
00:06:50,770 --> 00:06:53,640
Now, this is the event that
the sum of the random variables

95
00:06:53,640 --> 00:06:56,460
is close to a plus
b, and we want

96
00:06:56,460 --> 00:07:02,860
to use the fact that xn is
close to a and yn is close to b.

97
00:07:02,860 --> 00:07:07,320
So this is the event
that-- let's write it

98
00:07:07,320 --> 00:07:11,990
in a somewhat different way--
is the probability of the event

99
00:07:11,990 --> 00:07:20,460
that xn minus a
plus yn minus b is

100
00:07:20,460 --> 00:07:24,220
bigger than epsilon
in magnitude.

101
00:07:24,220 --> 00:07:28,950
Now, for a sum of two
numbers to be bigger

102
00:07:28,950 --> 00:07:31,570
than epsilon in
magnitude, it has

103
00:07:31,570 --> 00:07:37,409
to be the case that
either one of them

104
00:07:37,409 --> 00:07:44,430
is larger than epsilon
over 2 or the other number

105
00:07:44,430 --> 00:07:48,840
is bigger in magnitude
than epsilon over 2.

106
00:07:48,840 --> 00:07:54,670
So if this event happens,
this event must also happen.

107
00:07:54,670 --> 00:07:59,720
This means that this event
is a subset of this event.

108
00:07:59,720 --> 00:08:00,980
This is a smaller one.

109
00:08:00,980 --> 00:08:03,450
If this happens, then
this one happens.

110
00:08:03,450 --> 00:08:05,630
So since it's a
smaller event, it

111
00:08:05,630 --> 00:08:08,470
means that its
probability is less than

112
00:08:08,470 --> 00:08:12,560
or equal to the
probability of that event.

113
00:08:12,560 --> 00:08:14,740
Now we use the union bound.

114
00:08:14,740 --> 00:08:16,860
The probability that
something happens

115
00:08:16,860 --> 00:08:20,180
or something else is
happening is less than

116
00:08:20,180 --> 00:08:23,683
or equal to the sum of
their probabilities.

117
00:08:35,169 --> 00:08:41,110
And now, since Xn converges
to a in probability,

118
00:08:41,110 --> 00:08:47,560
then by definition, we know that
this quantity converges to 0

119
00:08:47,560 --> 00:08:49,070
as n goes to infinity.

120
00:08:51,900 --> 00:08:58,640
Similarly, since Yn converges
to b in probability,

121
00:08:58,640 --> 00:09:04,910
this quantity converges to
0 as n goes to infinity.

122
00:09:04,910 --> 00:09:07,900
This is a sequence of
numbers that converges to 0.

123
00:09:07,900 --> 00:09:12,470
This is another sequence of
numbers that converges to 0.

124
00:09:12,470 --> 00:09:18,880
Therefore, the sum of these two
sequences also converges to 0.

125
00:09:18,880 --> 00:09:22,290
In essence, here we're applying
what we established earlier

126
00:09:22,290 --> 00:09:24,260
about convergence of numbers.

127
00:09:24,260 --> 00:09:26,830
If a sequence converges to 0
and another sequence converges

128
00:09:26,830 --> 00:09:29,780
to 0, then the sum
of these sequences

129
00:09:29,780 --> 00:09:34,890
also converges to 0
as n goes to infinity.

130
00:09:34,890 --> 00:09:38,440
But this is exactly what
we need to show in order

131
00:09:38,440 --> 00:09:43,030
to establish convergence in
probability of Xn plus Yn.

132
00:09:43,030 --> 00:09:47,890
We have shown that if I
fix any epsilon, positive,

133
00:09:47,890 --> 00:09:50,920
no matter how small,
the probability

134
00:09:50,920 --> 00:09:54,860
that I am more
than epsilon away,

135
00:09:54,860 --> 00:09:59,080
the probability that Xn plus
Yn is more than epsilon away

136
00:09:59,080 --> 00:10:03,240
from the supposed
target or the limit,

137
00:10:03,240 --> 00:10:06,210
this probability must go to 0.

138
00:10:06,210 --> 00:10:09,060
And that's exactly what
we established here,

139
00:10:09,060 --> 00:10:11,680
and this completes
the derivation.