1
00:00:00,520 --> 00:00:02,960
We have seen that
under some conditions,

2
00:00:02,960 --> 00:00:08,270
the binomial PMF is well
approximated by a Poisson PMF.

3
00:00:08,270 --> 00:00:10,890
But we have also seen
the central limit theorem

4
00:00:10,890 --> 00:00:15,240
that tells us that the binomial
PMF can be approximated

5
00:00:15,240 --> 00:00:18,120
using a normal random variable.

6
00:00:18,120 --> 00:00:20,990
Can we reconcile
these two facts?

7
00:00:20,990 --> 00:00:24,110
Let's look into the situation
in some more detail.

8
00:00:24,110 --> 00:00:28,950
Consider a Poisson process
that has rate equal to 1.

9
00:00:28,950 --> 00:00:31,020
And consider that
Poisson process

10
00:00:31,020 --> 00:00:34,280
running over the
unit time interval.

11
00:00:34,280 --> 00:00:37,340
We take the unit
interval, and we split it

12
00:00:37,340 --> 00:00:40,720
into many small
sub-intervals, where

13
00:00:40,720 --> 00:00:46,120
each sub-interval
has a length of 1/n.

14
00:00:46,120 --> 00:00:50,320
And let Xi be the number of
arrivals that we get during

15
00:00:50,320 --> 00:00:52,600
the i'th interval.

16
00:00:52,600 --> 00:00:56,170
Xi is a Poisson random variable.

17
00:00:56,170 --> 00:00:59,010
And the mean of that
random variable,

18
00:00:59,010 --> 00:01:01,110
or the parameter of
that random variable,

19
00:01:01,110 --> 00:01:05,209
is just the duration of the time
interval, since the rate is 1.

20
00:01:05,209 --> 00:01:09,850
So it's a Poisson random
variable, with parameter 1/n.

21
00:01:09,850 --> 00:01:12,980
Now, let us look at the
total number of arrivals.

22
00:01:12,980 --> 00:01:17,350
The total number of arrivals
is the sum of how many arrivals

23
00:01:17,350 --> 00:01:21,130
we had during each one
of these intervals.

24
00:01:21,130 --> 00:01:24,070
And we know the
distribution of S. S

25
00:01:24,070 --> 00:01:30,430
is a Poisson random variable,
with parameter equal to 1.

26
00:01:30,430 --> 00:01:35,000
Now, here what we have is
a sum of random variables

27
00:01:35,000 --> 00:01:39,390
that are independent and
identically distributed.

28
00:01:39,390 --> 00:01:41,080
They are identically
distributed,

29
00:01:41,080 --> 00:01:43,800
because all of these intervals
have the same length.

30
00:01:43,800 --> 00:01:46,750
And they're independent,
because in the Poisson process,

31
00:01:46,750 --> 00:01:49,250
what happens in
different intervals are

32
00:01:49,250 --> 00:01:52,320
independent events.

33
00:01:52,320 --> 00:01:55,080
So we are in a
situation where we

34
00:01:55,080 --> 00:01:57,590
could apply the
central limit theorem.

35
00:01:57,590 --> 00:02:00,520
We have a sum of
many independent,

36
00:02:00,520 --> 00:02:02,980
identically distributed
random variables.

37
00:02:02,980 --> 00:02:06,850
And by letting n go to infinity,
the central limit theorem

38
00:02:06,850 --> 00:02:10,723
appears to tell us that
S is going to be normal.

39
00:02:13,800 --> 00:02:17,440
Now, how can we reconcile
these two facts?

40
00:02:17,440 --> 00:02:20,930
We know that the Poisson
distribution is not the same

41
00:02:20,930 --> 00:02:23,960
as a normal distribution.

42
00:02:23,960 --> 00:02:25,565
What is the catch?

43
00:02:25,565 --> 00:02:29,760
Well, the catch
is the following--

44
00:02:29,760 --> 00:02:33,640
the central limit theorem
applies to a situation where we

45
00:02:33,640 --> 00:02:37,010
fix a certain
probability distribution,

46
00:02:37,010 --> 00:02:39,530
the distribution of the Xi's.

47
00:02:39,530 --> 00:02:44,270
And it tells us that as we add
more and more of these Xi's,

48
00:02:44,270 --> 00:02:47,480
asymptotically, we obtain
a distribution that's

49
00:02:47,480 --> 00:02:50,630
well approximated by a normal.

50
00:02:50,630 --> 00:02:56,030
On the other hand, what we have
here is actually different.

51
00:02:56,030 --> 00:02:59,660
The Xi's do not have
a fixed distribution.

52
00:02:59,660 --> 00:03:06,260
But rather, the distribution
of Xi depends on n.

53
00:03:08,840 --> 00:03:13,030
That is, if we change
n so that we're

54
00:03:13,030 --> 00:03:15,500
adding more random
variables, we're

55
00:03:15,500 --> 00:03:17,490
adding more random
variables that are now

56
00:03:17,490 --> 00:03:20,200
coming from a
different distribution.

57
00:03:20,200 --> 00:03:23,260
And this is not a situation
to which the central limit

58
00:03:23,260 --> 00:03:24,880
theorem applies.

59
00:03:24,880 --> 00:03:28,980
And therefore, this conclusion
here is not justified.

60
00:03:28,980 --> 00:03:32,040
And so there's no
contradiction between the two

61
00:03:32,040 --> 00:03:34,790
types of approximations.

62
00:03:34,790 --> 00:03:37,460
To summarize, the
situation is as follows.

63
00:03:37,460 --> 00:03:40,500
Consider a binomial random
variable with some parameters

64
00:03:40,500 --> 00:03:41,800
n and p.

65
00:03:41,800 --> 00:03:44,200
Now, let p be fixed.

66
00:03:44,200 --> 00:03:46,410
And let n go to infinity.

67
00:03:46,410 --> 00:03:49,030
In that case, the
binomial random variable

68
00:03:49,030 --> 00:03:54,740
can be thought of as the sum of
n Bernoulli random variables.

69
00:03:54,740 --> 00:03:57,050
And those Bernoulli
random variables

70
00:03:57,050 --> 00:04:01,160
have a parameter
p, which is fixed.

71
00:04:01,160 --> 00:04:04,750
So we're dealing with the
sum of iid random variables

72
00:04:04,750 --> 00:04:06,050
from a fixed distribution.

73
00:04:06,050 --> 00:04:08,320
And this is the situation
where the central limit

74
00:04:08,320 --> 00:04:09,610
theorem applies.

75
00:04:09,610 --> 00:04:12,120
And we have a normal
approximation.

76
00:04:12,120 --> 00:04:18,750
On the other hand, if
we take the product

77
00:04:18,750 --> 00:04:22,250
n times p, which is the
expected value of this binomial,

78
00:04:22,250 --> 00:04:25,540
to stay constant,
but we let n go

79
00:04:25,540 --> 00:04:29,990
to infinity and at the same
time let p go to 0, then

80
00:04:29,990 --> 00:04:33,330
in this regime, in the
limit, this random variable

81
00:04:33,330 --> 00:04:37,370
will be well approximated by
a Poisson random variable.

82
00:04:37,370 --> 00:04:40,130
So we have two different
approximations.

83
00:04:40,130 --> 00:04:42,430
Both of them are
valid, but they're

84
00:04:42,430 --> 00:04:45,674
valid in different regimes.

85
00:04:45,674 --> 00:04:47,340
Now, although they're
different, there's

86
00:04:47,340 --> 00:04:52,770
actually an interesting case in
which the two will not really

87
00:04:52,770 --> 00:04:53,640
differ.

88
00:04:53,640 --> 00:04:55,400
And this is the following.

89
00:04:55,400 --> 00:04:59,090
Consider a Poisson random
variable with parameter n.

90
00:04:59,090 --> 00:05:03,550
And we're interested in the
limit as n goes to infinity.

91
00:05:03,550 --> 00:05:05,380
We can think of
this random variable

92
00:05:05,380 --> 00:05:08,000
as the number of arrivals
during an interval

93
00:05:08,000 --> 00:05:13,670
of length n in a Poisson process
with arrival rate equal to 1.

94
00:05:13,670 --> 00:05:17,540
Now, let's take this
interval and split it

95
00:05:17,540 --> 00:05:23,600
into n intervals, each of
which has a length of 1.

96
00:05:23,600 --> 00:05:28,470
And let us call Xi the number of
arrivals in the i'th interval.

97
00:05:28,470 --> 00:05:31,920
Our Poisson random variable
is going to be, of course,

98
00:05:31,920 --> 00:05:35,040
equal to the sum of
the number of arrivals

99
00:05:35,040 --> 00:05:37,900
during each one
of the intervals.

100
00:05:37,900 --> 00:05:40,430
Each one of these
random variables

101
00:05:40,430 --> 00:05:45,310
is Poisson with
parameter equal to 1.

102
00:05:45,310 --> 00:05:50,420
And these random variables
are actually iid.

103
00:05:50,420 --> 00:05:52,730
Now, what's happening
in this case

104
00:05:52,730 --> 00:05:56,250
is that even when we
increase n, because we're

105
00:05:56,250 --> 00:06:00,130
using constant length intervals,
the distribution of the Xi's

106
00:06:00,130 --> 00:06:01,400
doesn't change.

107
00:06:01,400 --> 00:06:04,470
So this is a situation
in which we're

108
00:06:04,470 --> 00:06:09,590
going to get approximately a
normal random variable as n

109
00:06:09,590 --> 00:06:12,650
goes to infinity.

110
00:06:12,650 --> 00:06:16,520
So what we see is that a
Poisson random variable, but

111
00:06:16,520 --> 00:06:19,010
with a very large
parameter, starts

112
00:06:19,010 --> 00:06:22,180
to approach the
normal distribution.

113
00:06:22,180 --> 00:06:24,010
And this in particular
will tell us

114
00:06:24,010 --> 00:06:27,630
that these two approximations
that we have, in some regime,

115
00:06:27,630 --> 00:06:30,810
they would start to agree.

116
00:06:30,810 --> 00:06:33,770
Now, all of this discussion
here has been asymptotic.

117
00:06:33,770 --> 00:06:37,940
We talk about p going to
0 or n going to infinity.

118
00:06:37,940 --> 00:06:41,920
But in any real situation, you
will be given actual numbers.

119
00:06:41,920 --> 00:06:44,909
And you cannot really tell,
is this number close to 0,

120
00:06:44,909 --> 00:06:46,900
or is it not?

121
00:06:46,900 --> 00:06:52,380
Here, we need some rules of
thumb or maybe some experience.

122
00:06:52,380 --> 00:06:55,300
Let's look at some examples.

123
00:06:55,300 --> 00:07:00,510
In this case, n times
p is equal to 1.

124
00:07:00,510 --> 00:07:03,200
So the number of
arrivals or the values

125
00:07:03,200 --> 00:07:07,380
of the binomial random variable
will take values 0, 1, 2,

126
00:07:07,380 --> 00:07:12,860
3, but with high probability,
not a lot more than that.

127
00:07:12,860 --> 00:07:15,930
So the binomial random
variable is really

128
00:07:15,930 --> 00:07:17,630
a discrete random variable.

129
00:07:17,630 --> 00:07:19,940
There's no way to
approximate it with a normal.

130
00:07:19,940 --> 00:07:22,770
On the other hand,
p is very small.

131
00:07:22,770 --> 00:07:25,630
So a Poisson
approximation would be

132
00:07:25,630 --> 00:07:29,300
very reasonable
in this situation.

133
00:07:29,300 --> 00:07:33,510
On the other hand, if
p is equal to 1/3, then

134
00:07:33,510 --> 00:07:36,690
definitely 1/3 is
not a small number.

135
00:07:36,690 --> 00:07:38,920
A Poisson approximation
would not apply.

136
00:07:38,920 --> 00:07:40,770
But n is pretty big.

137
00:07:40,770 --> 00:07:45,880
So that a normal approximation
would be appropriate.

138
00:07:45,880 --> 00:07:51,610
And finally, in this case,
we would obtain a Poisson

139
00:07:51,610 --> 00:07:58,159
approximation with parameter
100, because n times p is 100.

140
00:07:58,159 --> 00:08:01,810
But a Poisson random
variable with parameter 100

141
00:08:01,810 --> 00:08:04,570
is also well
approximated by a normal.

142
00:08:04,570 --> 00:08:07,190
Or to think about
it differently,

143
00:08:07,190 --> 00:08:10,130
we start with a Bernoulli
distribution that's

144
00:08:10,130 --> 00:08:17,190
very skewed, [the] probability
of success is just 100.

145
00:08:17,190 --> 00:08:20,720
And this makes it difficult
for the central limit theorem

146
00:08:20,720 --> 00:08:24,320
to apply when you start with a
very asymmetric distribution.

147
00:08:24,320 --> 00:08:28,840
On the other hand, because
we're adding so many of them,

148
00:08:28,840 --> 00:08:32,049
the central limit theorem
actually does take hold.

149
00:08:32,049 --> 00:08:36,299
And so this is an example
where both approximations

150
00:08:36,299 --> 00:08:38,490
will be valid.

151
00:08:38,490 --> 00:08:44,460
So finally, to conclude, we have
two different approximations.

152
00:08:44,460 --> 00:08:46,960
They're valid in
different regimes.

153
00:08:46,960 --> 00:08:49,660
And in practice, you
need to do some thinking

154
00:08:49,660 --> 00:08:54,640
before deciding to choose one
versus the other approximation.