1
00:00:00,710 --> 00:00:04,010
The subject of this segment is
the calculation of the PMF of

2
00:00:04,010 --> 00:00:09,040
the sum of two independent,
discrete random variables.

3
00:00:09,040 --> 00:00:11,750
This is the simplest example
of a function of two random

4
00:00:11,750 --> 00:00:16,050
variables, a function of the
form of g of X and Y, where

5
00:00:16,050 --> 00:00:18,480
the function g happens
to be just the

6
00:00:18,480 --> 00:00:21,080
sum of the two arguments.

7
00:00:21,080 --> 00:00:23,140
This is a very important
example, because there are

8
00:00:23,140 --> 00:00:25,340
many situations where
random variables get

9
00:00:25,340 --> 00:00:26,960
added to each other.

10
00:00:26,960 --> 00:00:29,890
We work with discrete random
variables as a warm up.

11
00:00:29,890 --> 00:00:32,020
And later, we will consider
the case of

12
00:00:32,020 --> 00:00:34,170
continuous random variables.

13
00:00:34,170 --> 00:00:38,020
So suppose that we know the PMFs
of X and Y and that we

14
00:00:38,020 --> 00:00:40,720
want to compute the probability
that the sum is

15
00:00:40,720 --> 00:00:42,250
equal to 3.

16
00:00:42,250 --> 00:00:44,840
It always helps to
have a picture.

17
00:00:44,840 --> 00:00:49,110
The sum of X and Y will
be equal to 3.

18
00:00:49,110 --> 00:00:51,720
This is an event that can
happen in many ways.

19
00:00:51,720 --> 00:00:56,010
For example, x could be 3 and Y
could be 0, or X could be 1

20
00:00:56,010 --> 00:00:58,910
and Y equal to 2.

21
00:00:58,910 --> 00:01:01,530
The probability of the event of
interest, that the sum is

22
00:01:01,530 --> 00:01:04,800
equal to 3, is going to be the
sum of the probabilities of

23
00:01:04,800 --> 00:01:08,390
all the different ways that
this event can happen.

24
00:01:08,390 --> 00:01:12,060
So it is going to be a
sum of various terms.

25
00:01:12,060 --> 00:01:15,800
And the typical term would be
the probability, let's say of

26
00:01:15,800 --> 00:01:20,210
this outcome, which is that
X is equal to 0 and

27
00:01:20,210 --> 00:01:22,370
Y is equal to 3.

28
00:01:22,370 --> 00:01:25,930
Another typical term in the sum
will be the probability of

29
00:01:25,930 --> 00:01:29,270
this outcome here, the
probability that X is equal to

30
00:01:29,270 --> 00:01:35,479
1, Y is equal to 2, and so on.

31
00:01:35,479 --> 00:01:38,350
Now, here comes an
important step.

32
00:01:38,350 --> 00:01:41,560
Because we have assumed that X
and Y are independent, the

33
00:01:41,560 --> 00:01:43,910
probability of these two
events happening is the

34
00:01:43,910 --> 00:01:48,180
product of the probabilities of
each one of these events.

35
00:01:48,180 --> 00:01:51,570
So it is the product of the
probability that X is equal to

36
00:01:51,570 --> 00:01:55,440
0, where now I'm using PMF
notation, times the

37
00:01:55,440 --> 00:01:58,600
probability that Y
is equal to 3.

38
00:01:58,600 --> 00:02:01,720
Similarly, the next term is
the probability that X is

39
00:02:01,720 --> 00:02:04,190
equal to 1 times the
probability that

40
00:02:04,190 --> 00:02:05,900
Y is equal to 2.

41
00:02:05,900 --> 00:02:09,220
Again, we can do this because
we are assuming that our two

42
00:02:09,220 --> 00:02:12,510
random variables are independent
of each other.

43
00:02:12,510 --> 00:02:15,846
Now, let us generalize.

44
00:02:15,846 --> 00:02:23,260
In the general case, the
probability that the sum takes

45
00:02:23,260 --> 00:02:28,610
on a particular value little z
can be calculated as follows.

46
00:02:28,610 --> 00:02:31,420
We look at all the different
ways that the sum of little z

47
00:02:31,420 --> 00:02:32,980
can be obtained.

48
00:02:32,980 --> 00:02:36,920
One way is that the random
variable X takes on a specific

49
00:02:36,920 --> 00:02:40,480
value little X. And at the same
time, the random variable

50
00:02:40,480 --> 00:02:44,890
Y takes the value that's needed
so that the sum of the

51
00:02:44,890 --> 00:02:52,200
two is equal to little Z. For a
given value of little X, we

52
00:02:52,200 --> 00:02:56,400
have a particular way that the
sum is equal to Z. And this

53
00:02:56,400 --> 00:02:59,579
particular way has a certain
probability.

54
00:02:59,579 --> 00:03:01,660
But little X could
be anything.

55
00:03:01,660 --> 00:03:04,730
And different choices of little
x give us different

56
00:03:04,730 --> 00:03:07,640
ways that the event of
interest can happen.

57
00:03:07,640 --> 00:03:12,740
So we add those probabilities
over all possible X's.

58
00:03:12,740 --> 00:03:15,800
And then we proceed
as follows.

59
00:03:15,800 --> 00:03:18,630
We invoke independence of
X and Y to derive this

60
00:03:18,630 --> 00:03:21,350
probability as a product
of two probabilities.

61
00:03:21,350 --> 00:03:25,560
And then we use PMF notation
instead of probability

62
00:03:25,560 --> 00:03:29,170
notation to obtain this
expression here.

63
00:03:33,010 --> 00:03:39,540
This formula is called the
convolution formula.

64
00:03:39,540 --> 00:03:42,320
It is the convolution
of two PMFs.

65
00:03:42,320 --> 00:03:47,420
What convolution means is that
somebody gives us the PMF of

66
00:03:47,420 --> 00:03:51,079
one random variable, gives
us also the PMF of

67
00:03:51,079 --> 00:03:52,640
another random variable.

68
00:03:52,640 --> 00:03:55,540
And when we say we're given the
PMF, it means we're given

69
00:03:55,540 --> 00:03:59,360
the values of the PMFs for all
the possible choices of little

70
00:03:59,360 --> 00:04:02,570
X and little y, the arguments
of the two PMFs.

71
00:04:02,570 --> 00:04:06,790
Then the convolution formula
does a certain calculation and

72
00:04:06,790 --> 00:04:11,690
spits out now a new PMF, which
is the PMF of the random

73
00:04:11,690 --> 00:04:17,459
variable Z. Let's now take a
closer look at what it takes

74
00:04:17,459 --> 00:04:20,350
to carry out of the calculations
involved in this

75
00:04:20,350 --> 00:04:22,450
convolution formula.

76
00:04:22,450 --> 00:04:25,210
Let's proceed by a simple
example that will illustrate

77
00:04:25,210 --> 00:04:26,140
the methodology.

78
00:04:26,140 --> 00:04:29,410
We're given two PMFs of
two random variables.

79
00:04:29,410 --> 00:04:32,159
And assuming that they are
independent, the PMF of their

80
00:04:32,159 --> 00:04:35,400
sum is determined by
this formula here.

81
00:04:35,400 --> 00:04:37,920
And we want to see what
those terms in this

82
00:04:37,920 --> 00:04:39,500
summation would be.

83
00:04:39,500 --> 00:04:41,640
Suppose that we're interested
in the probability that the

84
00:04:41,640 --> 00:04:43,220
sum is equal to 3.

85
00:04:43,220 --> 00:04:46,140
Now, the sum is going
to be equal to 3.

86
00:04:46,140 --> 00:04:48,330
This can happen in
several ways.

87
00:04:48,330 --> 00:04:54,580
We could have X equal to
1 and and Y equal to 2.

88
00:04:54,580 --> 00:04:56,750
This combination is
one way that the

89
00:04:56,750 --> 00:04:58,700
sum of 3 can be obtained.

90
00:04:58,700 --> 00:05:03,030
And that combination has a
probability of 1/3 times 3/6.

91
00:05:03,030 --> 00:05:06,010
And that would be one of the
terms in this summation.

92
00:05:06,010 --> 00:05:09,320
Another way that the sum of 3
can be obtained is by having X

93
00:05:09,320 --> 00:05:12,630
equal to 4 and y equal
to minus 1.

94
00:05:12,630 --> 00:05:17,750
And by multiplying this
probability 2/3 with 2/6, we

95
00:05:17,750 --> 00:05:21,900
obtain another contribution
to this summation.

96
00:05:21,900 --> 00:05:25,690
However, keeping track of these
correspondences here can

97
00:05:25,690 --> 00:05:29,970
become a little complicated if
we have richer our PMFs.

98
00:05:29,970 --> 00:05:33,460
So an alternative way of
arranging the calculation is

99
00:05:33,460 --> 00:05:34,940
the following.

100
00:05:34,940 --> 00:05:41,930
Let us take the PMF of Y, flip
it along this vertical axis.

101
00:05:41,930 --> 00:05:44,870
So these two terms would go to
the left side, and this term

102
00:05:44,870 --> 00:05:46,860
will go to the right
hand side.

103
00:05:46,860 --> 00:05:49,850
And then draw it underneath
the PMF of X.

104
00:05:49,850 --> 00:05:52,150
This is what we obtain.

105
00:05:52,150 --> 00:05:56,500
Then let us take this drawing
here and shift it to

106
00:05:56,500 --> 00:05:58,590
the right by 3.

107
00:05:58,590 --> 00:06:02,140
So the entry of minus 2 goes
to 1, minus 1 goes to 2,

108
00:06:02,140 --> 00:06:03,990
and 1 goes to 4.

109
00:06:03,990 --> 00:06:08,710
So what have we accomplished by
these two transformations?

110
00:06:08,710 --> 00:06:16,810
Well, the term that had
probability 3/6 and which were

111
00:06:16,810 --> 00:06:21,940
to be multiplied with the
probability 1/3 on that side,

112
00:06:21,940 --> 00:06:25,520
now this 3/6 sits here.

113
00:06:25,520 --> 00:06:28,260
So we have this correspondence.

114
00:06:28,260 --> 00:06:31,850
And we need to multiply
1/3 by 3/6.

115
00:06:31,850 --> 00:06:36,170
Similarly, the multiplication of
2/3 with 2/6 corresponds to

116
00:06:36,170 --> 00:06:39,980
the multiplication of this
probability here times the

117
00:06:39,980 --> 00:06:43,110
probability of this term here.

118
00:06:43,110 --> 00:06:47,100
So when the diagrams are
arranged this way, then we

119
00:06:47,100 --> 00:06:49,320
have a simpler job to do.

120
00:06:49,320 --> 00:06:53,380
We look at corresponding terms,
those that sit on top

121
00:06:53,380 --> 00:06:58,150
of each other, multiply them,
do that for all the possible

122
00:06:58,150 --> 00:07:02,450
choices, and then add those
products together.

123
00:07:02,450 --> 00:07:05,430
And this is what we do if
we're shifting by 3.

124
00:07:05,430 --> 00:07:09,470
Now, if we wanted to find the
probability that Z equal to 4,

125
00:07:09,470 --> 00:07:12,320
we would be doing the same
thing, except that this

126
00:07:12,320 --> 00:07:16,270
diagram would need to be shifted
by one more unit to

127
00:07:16,270 --> 00:07:20,100
the right so that we have
a total shift of 4.

128
00:07:20,100 --> 00:07:23,500
So we just repeat this procedure
for all possible

129
00:07:23,500 --> 00:07:28,540
values of Z which corresponds
to taking this diagram here

130
00:07:28,540 --> 00:07:32,790
and shifting it progressively
by different amounts.

131
00:07:32,790 --> 00:07:36,680
This turns out to be a fairly
simple and systematic way of

132
00:07:36,680 --> 00:07:38,830
arranging the calculations,
at least if you're

133
00:07:38,830 --> 00:07:40,430
doing them by hand.

134
00:07:40,430 --> 00:07:43,920
Of course, an alternative is to
carry out the calculations

135
00:07:43,920 --> 00:07:45,040
on a computer.

136
00:07:45,040 --> 00:07:49,000
This is a pretty simple formula
that is not hard to

137
00:07:49,000 --> 00:07:50,930
implement on a computer.