1
00:00:00,292 --> 00:00:01,750
PROFESSOR: In this
segment, we will

2
00:00:01,750 --> 00:00:04,360
discuss the multinomial
model and the multinomial

3
00:00:04,360 --> 00:00:06,970
probabilities, which are
a nice generalization

4
00:00:06,970 --> 00:00:09,190
of the binomial probabilities.

5
00:00:09,190 --> 00:00:11,050
The setting is as follows.

6
00:00:11,050 --> 00:00:13,030
We are dealing with
balls and the balls come

7
00:00:13,030 --> 00:00:14,250
into different colors.

8
00:00:14,250 --> 00:00:17,380
There are r possible
different colors.

9
00:00:17,380 --> 00:00:19,860
We pick a ball at random,
and when we do that,

10
00:00:19,860 --> 00:00:23,530
there is a certain probability
Pi that the ball that we picked

11
00:00:23,530 --> 00:00:25,580
has ith color.

12
00:00:25,580 --> 00:00:28,990
Now, we repeat this process
n times independently.

13
00:00:28,990 --> 00:00:32,409
Each time we get a ball
that has a random color.

14
00:00:32,409 --> 00:00:35,740
And we're interested in the
following kind of question.

15
00:00:35,740 --> 00:00:38,560
Somebody fixes for
us certain numbers--

16
00:00:38,560 --> 00:00:44,090
n1, n2, up to nr that
add up to n, and asks us,

17
00:00:44,090 --> 00:00:46,300
what is the probability
that when you carry out

18
00:00:46,300 --> 00:00:50,110
the experiment, you get exactly
n1 balls of the first color,

19
00:00:50,110 --> 00:00:54,040
exactly n2 balls of the
second color, and so on?

20
00:00:54,040 --> 00:00:59,740
So the numbers n1, n2, up to
nr are fixed given numbers.

21
00:00:59,740 --> 00:01:01,480
For a particular choice
of those numbers,

22
00:01:01,480 --> 00:01:04,750
we want to calculate
this probability.

23
00:01:04,750 --> 00:01:07,420
Now of course, this is
a more general model.

24
00:01:07,420 --> 00:01:10,940
It doesn't necessarily deal
with balls of different colors.

25
00:01:10,940 --> 00:01:15,520
For example, we might have
an experiment that gives us

26
00:01:15,520 --> 00:01:20,100
random numbers, where the
numbers range from 1 up to r,

27
00:01:20,100 --> 00:01:24,460
and at each time we get a
random number with probability

28
00:01:24,460 --> 00:01:27,940
Pi we get a number
which is equal to i.

29
00:01:27,940 --> 00:01:32,560
So we could use this to
model die rolls, for example.

30
00:01:32,560 --> 00:01:34,300
And there's actually
a special case

31
00:01:34,300 --> 00:01:36,820
of this problem, which
should be familiar.

32
00:01:36,820 --> 00:01:39,220
Suppose that we have
only two colors,

33
00:01:39,220 --> 00:01:41,530
and instead of
thinking of colors,

34
00:01:41,530 --> 00:01:43,660
let us think of the
two possibilities

35
00:01:43,660 --> 00:01:47,080
as being heads or tails.

36
00:01:47,080 --> 00:01:49,180
And we can make the
following analogy.

37
00:01:49,180 --> 00:01:55,746
Somebody gives us numbers
n1 and n2 that add up to n.

38
00:01:55,746 --> 00:01:57,370
And we're interested
in the probability

39
00:01:57,370 --> 00:01:59,920
that we get n1 of
the first color

40
00:01:59,920 --> 00:02:02,420
and n2 of the second color.

41
00:02:02,420 --> 00:02:05,850
Well, we could think of this
as a setting in which we

42
00:02:05,850 --> 00:02:08,380
are asking for the
probability that we obtain

43
00:02:08,380 --> 00:02:14,650
k heads and n minus k tails.

44
00:02:17,650 --> 00:02:20,770
So the question of what is
the probability that we obtain

45
00:02:20,770 --> 00:02:24,700
k heads and n minus k
tails is of the same kind

46
00:02:24,700 --> 00:02:28,320
as what is the probability that
we get n1 of the first color

47
00:02:28,320 --> 00:02:30,880
and n2 of the second color.

48
00:02:30,880 --> 00:02:34,690
Now, if heads have a
probability p of occurring,

49
00:02:34,690 --> 00:02:38,560
and tails has a probability
of 1 minus p of occurring,

50
00:02:38,560 --> 00:02:40,720
then we would have
the following analogy.

51
00:02:40,720 --> 00:02:43,270
The probability of
obtaining the first color,

52
00:02:43,270 --> 00:02:47,110
which correspond to heads,
that would be equal to p.

53
00:02:47,110 --> 00:02:49,630
The probability of obtaining
the second color, which

54
00:02:49,630 --> 00:02:54,130
correspond to tails,
this would be 1 minus p.

55
00:02:54,130 --> 00:02:56,620
Now, the probability
of obtaining k

56
00:02:56,620 --> 00:03:00,790
heads in those n
independent trials--

57
00:03:00,790 --> 00:03:02,890
we know what it is.

58
00:03:02,890 --> 00:03:04,930
By the binomial
probabilities, it

59
00:03:04,930 --> 00:03:09,650
is n choose k times
p to the k times one

60
00:03:09,650 --> 00:03:14,020
minus p to the power n minus k.

61
00:03:14,020 --> 00:03:19,150
Now we can translate this
answer to the multinomial case

62
00:03:19,150 --> 00:03:23,620
where we're dealing with colors,
and we do these substitutions.

63
00:03:23,620 --> 00:03:31,090
So n choose k is n factorial
divided by k factorial.

64
00:03:31,090 --> 00:03:37,690
In this case, k is the same
as n1, so we get n1 factorial.

65
00:03:37,690 --> 00:03:40,900
And then we are going to have
here n minus k factorial.

66
00:03:40,900 --> 00:03:44,020
But n minus k corresponds to n2.

67
00:03:44,020 --> 00:03:47,860
So here we get an n2 factorial.

68
00:03:47,860 --> 00:03:58,120
And then p corresponds to p1
and p2 corresponds to 1 minus p.

69
00:03:58,120 --> 00:04:01,230
So we get here p1 times p2.

70
00:04:01,230 --> 00:04:06,920
n to the power n minus k,
again, by analogy, is n2.

71
00:04:06,920 --> 00:04:10,750
So this is the form of the
multinomial probabilities

72
00:04:10,750 --> 00:04:13,530
for the special case where
we're dealing with two colors.

73
00:04:16,149 --> 00:04:19,250
Let us now look at
the general case.

74
00:04:19,250 --> 00:04:22,370
Let us start with an
example, to be concrete.

75
00:04:22,370 --> 00:04:26,560
Suppose that the number
of colors is equal to 3,

76
00:04:26,560 --> 00:04:33,910
and that we're going to
pick n equal to 7 balls.

77
00:04:33,910 --> 00:04:36,160
We carry out the
experiments, and we

78
00:04:36,160 --> 00:04:39,310
might obtain an
outcome which would

79
00:04:39,310 --> 00:04:41,390
be a sequence of this type.

80
00:04:41,390 --> 00:04:44,650
So the first ball
had the color 1,

81
00:04:44,650 --> 00:04:46,840
the second ball had
the first color,

82
00:04:46,840 --> 00:04:49,540
the third ball had
the third color,

83
00:04:49,540 --> 00:04:53,800
the fourth ball had the
first color, and so on.

84
00:04:53,800 --> 00:04:57,760
And suppose that
this was the outcome.

85
00:04:57,760 --> 00:05:01,060
One way of summarizing what
happened in this outcome

86
00:05:01,060 --> 00:05:15,040
would be to say that we
had four 1s, we had two 2s,

87
00:05:15,040 --> 00:05:18,380
and we had one 3.

88
00:05:18,380 --> 00:05:21,860
We could say that this is
the type of the outcome.

89
00:05:21,860 --> 00:05:23,830
It's of type 4, 2, 1--

90
00:05:23,830 --> 00:05:26,560
that is, we obtained
four of the first color,

91
00:05:26,560 --> 00:05:29,500
two of the second color,
and one of the third color.

92
00:05:32,530 --> 00:05:34,640
This is one possible outcome.

93
00:05:34,640 --> 00:05:36,760
What is the probability
of obtaining

94
00:05:36,760 --> 00:05:39,690
this particular outcome?

95
00:05:39,690 --> 00:05:42,630
The probability of obtaining
this particular outcome

96
00:05:42,630 --> 00:05:46,710
is, using independence, the
probability that we obtain

97
00:05:46,710 --> 00:05:51,960
color 1 in the first trial,
color 1 in the second trial,

98
00:05:51,960 --> 00:05:57,390
color 3 in the third trial,
color 1 in the fourth trial,

99
00:05:57,390 --> 00:06:01,980
color 2 in the next trial,
color 2 in the next trial,

100
00:06:01,980 --> 00:06:05,090
color 1 in the last trial.

101
00:06:05,090 --> 00:06:08,670
And we put all the
factors together,

102
00:06:08,670 --> 00:06:13,090
and we notice that this
is p1 to the fourth p2

103
00:06:13,090 --> 00:06:16,220
to the second times p3.

104
00:06:19,510 --> 00:06:23,590
It's not a coincidence that the
exponents that we have up here

105
00:06:23,590 --> 00:06:26,560
are exactly the
count that we had

106
00:06:26,560 --> 00:06:32,050
when we specified the type
of this particular outcome.

107
00:06:32,050 --> 00:06:35,200
Generalizing from
this example, we

108
00:06:35,200 --> 00:06:37,960
realize that the
probability of obtaining

109
00:06:37,960 --> 00:06:44,010
a particular sequence of a
certain type, that probability

110
00:06:44,010 --> 00:06:46,180
is of this form.

111
00:06:46,180 --> 00:06:49,210
For each color, we
have the probability

112
00:06:49,210 --> 00:06:52,660
of that color raised to
the power of how many times

113
00:06:52,660 --> 00:06:57,280
that particular color
appears in a sequence.

114
00:06:57,280 --> 00:07:02,110
So any particular sequence of
this type has this probability.

115
00:07:02,110 --> 00:07:04,510
What we're interested
in is to find

116
00:07:04,510 --> 00:07:07,150
the total probability
of obtaining

117
00:07:07,150 --> 00:07:10,270
some sequence of this type.

118
00:07:10,270 --> 00:07:12,250
How can we find
this probability?

119
00:07:12,250 --> 00:07:14,170
Well, we will take
the probability

120
00:07:14,170 --> 00:07:16,780
of each sequence of this type--

121
00:07:16,780 --> 00:07:19,900
which is this much,
and it's the same

122
00:07:19,900 --> 00:07:22,190
for any particular sequence--

123
00:07:22,190 --> 00:07:28,160
and multiply with the number
of sequences of this type.

124
00:07:28,160 --> 00:07:32,050
So how many sequences are
there of a certain type?

125
00:07:32,050 --> 00:07:34,400
Let us look back at our example.

126
00:07:34,400 --> 00:07:37,450
We had seven trials.

127
00:07:37,450 --> 00:07:42,700
So let us number here
the different trials.

128
00:07:42,700 --> 00:07:46,630
And when I tell you that
a particular sequence was

129
00:07:46,630 --> 00:07:50,800
obtained, that's the
same as telling you

130
00:07:50,800 --> 00:08:00,190
that in this set of trials,
we had the first color.

131
00:08:03,780 --> 00:08:09,000
In this set of trials,
the fifth and sixth trial,

132
00:08:09,000 --> 00:08:13,360
we had the second color.

133
00:08:13,360 --> 00:08:19,870
And in this trial, the third
trial, we had the third color.

134
00:08:19,870 --> 00:08:22,630
This is an alternative
way of telling you

135
00:08:22,630 --> 00:08:24,500
what sequence we obtained.

136
00:08:24,500 --> 00:08:26,530
I tell you at
which trials we had

137
00:08:26,530 --> 00:08:29,170
the first color, at which trials
we had the second, at which

138
00:08:29,170 --> 00:08:32,251
trials we had the third.

139
00:08:32,251 --> 00:08:34,070
But What do we have here?

140
00:08:34,070 --> 00:08:40,929
Here we have a partition of the
set of numbers from 1 up to 7

141
00:08:40,929 --> 00:08:43,299
into three subsets.

142
00:08:43,299 --> 00:08:46,300
And the cardinalities
of those subsets

143
00:08:46,300 --> 00:08:52,540
are the numbers that appear here
in the type of the sequence.

144
00:08:52,540 --> 00:08:58,650
The conclusion is that a
sequence of certain type

145
00:08:58,650 --> 00:09:03,330
is equivalent, or can be
alternatively specified,

146
00:09:03,330 --> 00:09:11,190
by giving you a partition
over this set of tosses, which

147
00:09:11,190 --> 00:09:16,440
is the set from 1 up to n,
how many trials we've had,

148
00:09:16,440 --> 00:09:20,620
a partition into subsets
of certain sizes.

149
00:09:20,620 --> 00:09:23,820
So this allows us now
to count the number

150
00:09:23,820 --> 00:09:26,430
of sequences of a certain type.

151
00:09:26,430 --> 00:09:29,730
It's exactly the same as
the number of partitions,

152
00:09:29,730 --> 00:09:31,650
and we know what this is.

153
00:09:31,650 --> 00:09:34,500
And putting everything
together, the probability

154
00:09:34,500 --> 00:09:37,710
of obtaining a sequence
of a certain type

155
00:09:37,710 --> 00:09:41,820
is equal to the count
of how many sequences

156
00:09:41,820 --> 00:09:44,130
do we have of the
certain type, which

157
00:09:44,130 --> 00:09:47,660
is the same as the number of
partitions of a certain type,

158
00:09:47,660 --> 00:09:51,600
times the probability of
any particular sequence

159
00:09:51,600 --> 00:09:56,050
of that type that
we're interested in.

160
00:09:56,050 --> 00:09:58,980
So this is a formula
that generalizes the one

161
00:09:58,980 --> 00:10:03,240
that we saw before for the case
where we have only two colors,

162
00:10:03,240 --> 00:10:08,140
and which corresponded to
the coin tossing setting.

163
00:10:08,140 --> 00:10:09,690
And it is a useful
model, because you

164
00:10:09,690 --> 00:10:13,380
can think of many situations in
which you have repeated trials,

165
00:10:13,380 --> 00:10:17,130
and at each trial,
you obtain one out

166
00:10:17,130 --> 00:10:20,640
of a finite set of
possible results.

167
00:10:20,640 --> 00:10:23,190
There are different
possible results.

168
00:10:23,190 --> 00:10:25,560
You repeat those
trials independently.

169
00:10:25,560 --> 00:10:30,010
And you may be interested in
the question of how many results

170
00:10:30,010 --> 00:10:34,657
of the first kind, of the second
kind, and so on there will be.