1
00:00:00,040 --> 00:00:02,460
The following content is
provided under a Creative

2
00:00:02,460 --> 00:00:03,870
Commons license.

3
00:00:03,870 --> 00:00:06,910
Your support will help MIT
OpenCourseWare continue to

4
00:00:06,910 --> 00:00:10,560
offer high-quality educational
resources for free.

5
00:00:10,560 --> 00:00:13,460
To make a donation or view
additional materials from

6
00:00:13,460 --> 00:00:19,290
hundreds of MIT courses, visit
MIT OpenCourseWare at

7
00:00:19,290 --> 00:00:20,540
ocw.mit.edu.

8
00:00:22,420 --> 00:00:25,310
PROFESSOR: OK, good morning.

9
00:00:25,310 --> 00:00:30,930
So today, we're going to have
a fairly packed lecture.

10
00:00:30,930 --> 00:00:34,060
We are going to conclude
with chapter two,

11
00:00:34,060 --> 00:00:35,560
discrete random variables.

12
00:00:35,560 --> 00:00:37,140
And we will be talking
mostly about

13
00:00:37,140 --> 00:00:39,322
multiple random variables.

14
00:00:39,322 --> 00:00:43,060
And this is also the
last lecture as far

15
00:00:43,060 --> 00:00:44,720
as quiz one is concerned.

16
00:00:44,720 --> 00:00:48,350
So it's going to cover the
material until today, and of

17
00:00:48,350 --> 00:00:52,550
course the next recitation
and tutorial as well.

18
00:00:52,550 --> 00:00:57,170
OK, so we're going to review
quickly what we introduced at

19
00:00:57,170 --> 00:01:01,040
the end of last lecture, where
we talked about the joint PMF

20
00:01:01,040 --> 00:01:02,300
of two random variables.

21
00:01:02,300 --> 00:01:05,040
We're going to talk about the
case of more than two random

22
00:01:05,040 --> 00:01:07,440
variables as well.

23
00:01:07,440 --> 00:01:09,910
We're going to talk about
the familiar concepts of

24
00:01:09,910 --> 00:01:14,300
conditioning and independence,
but applied to random

25
00:01:14,300 --> 00:01:16,460
variables instead of events.

26
00:01:16,460 --> 00:01:19,320
We're going to look at the
expectations once more, talk

27
00:01:19,320 --> 00:01:22,720
about a few properties that they
have, and then solve a

28
00:01:22,720 --> 00:01:25,900
couple of problems and calculate
a few things in

29
00:01:25,900 --> 00:01:28,180
somewhat clever ways.

30
00:01:28,180 --> 00:01:31,790
So the first point I want to
make is that, to a large

31
00:01:31,790 --> 00:01:34,870
extent, whatever is happening
in our chapter on discrete

32
00:01:34,870 --> 00:01:39,160
random variables is just an
exercise in notation.

33
00:01:39,160 --> 00:01:42,850
There is stuff and concepts that
you are already familiar

34
00:01:42,850 --> 00:01:45,230
with-- probabilities,
probabilities of two things

35
00:01:45,230 --> 00:01:47,490
happening, conditional
probabilities.

36
00:01:47,490 --> 00:01:51,760
And all that we're doing, to
some extent, is rewriting

37
00:01:51,760 --> 00:01:54,840
those familiar concepts
in new notation.

38
00:01:54,840 --> 00:01:57,810
So for example, this
is the joint PMF

39
00:01:57,810 --> 00:01:59,020
of two random variable.

40
00:01:59,020 --> 00:02:02,080
It gives us, for any pair or
possible values of those

41
00:02:02,080 --> 00:02:05,510
random variables, the
probability that that pair

42
00:02:05,510 --> 00:02:07,270
occurs simultaneously.

43
00:02:07,270 --> 00:02:10,020
So it's the probability that
simultaneously x takes that

44
00:02:10,020 --> 00:02:13,580
value, and y takes
that other value.

45
00:02:13,580 --> 00:02:17,210
And similarly, we have the
notion of the conditional PMF,

46
00:02:17,210 --> 00:02:21,060
which is just a list of the --
condition of -- the various

47
00:02:21,060 --> 00:02:23,750
conditional probabilities
of interest, conditional

48
00:02:23,750 --> 00:02:26,450
probability that one random
variable takes this value

49
00:02:26,450 --> 00:02:30,320
given that the other random
variable takes that value.

50
00:02:30,320 --> 00:02:33,640
Now, a remark about conditional
probabilities.

51
00:02:33,640 --> 00:02:36,640
Conditional probabilities
generally are like ordinary

52
00:02:36,640 --> 00:02:37,370
probabilities.

53
00:02:37,370 --> 00:02:40,170
You condition on something
particular.

54
00:02:40,170 --> 00:02:43,230
So here we condition
on a particular y.

55
00:02:43,230 --> 00:02:46,580
So think of little y as
a fixed quantity.

56
00:02:46,580 --> 00:02:49,800
And then look at this
as a function of x.

57
00:02:49,800 --> 00:02:54,430
So given that y, which we
condition on, given our new

58
00:02:54,430 --> 00:02:58,990
universe, we're considering the
various possibilities for

59
00:02:58,990 --> 00:03:01,290
x and the probabilities
that they have.

60
00:03:01,290 --> 00:03:04,000
Now, the probabilities over
all x's, of course,

61
00:03:04,000 --> 00:03:05,830
needs to add to 1.

62
00:03:05,830 --> 00:03:11,530
So we should have a relation
of this kind.

63
00:03:11,530 --> 00:03:14,420
So they're just like ordinary
probabilities over the

64
00:03:14,420 --> 00:03:18,230
different x's in a universe
where we are told the value of

65
00:03:18,230 --> 00:03:20,940
the random variable y.

66
00:03:20,940 --> 00:03:22,335
Now, how are these related?

67
00:03:25,200 --> 00:03:28,190
So we call these the marginal,
these the joint, these the

68
00:03:28,190 --> 00:03:29,150
conditional.

69
00:03:29,150 --> 00:03:31,510
And there are some relations
between these.

70
00:03:31,510 --> 00:03:35,430
For example, to find the
marginal from the joint, it's

71
00:03:35,430 --> 00:03:37,730
pretty straightforward.

72
00:03:37,730 --> 00:03:41,680
The probability that x takes a
particular value is the sum of

73
00:03:41,680 --> 00:03:45,030
the probabilities of all of the
different ways that this

74
00:03:45,030 --> 00:03:47,190
particular value may occur.

75
00:03:47,190 --> 00:03:48,380
What are the different ways?

76
00:03:48,380 --> 00:03:51,910
Well, it may occur together with
a certain y, or together

77
00:03:51,910 --> 00:03:55,110
with some other y, or together
with some other y.

78
00:03:55,110 --> 00:03:58,030
So you look at all the possible
y's that can go

79
00:03:58,030 --> 00:04:01,750
together with this x, and add
the probabilities of all of

80
00:04:01,750 --> 00:04:07,220
those pairs for which we get
this particular value of x.

81
00:04:07,220 --> 00:04:13,120
And then there's a relation
between that connects these

82
00:04:13,120 --> 00:04:16,230
two probabilities with the
conditional probability.

83
00:04:16,230 --> 00:04:18,630
And it's this relation.

84
00:04:18,630 --> 00:04:20,279
It's nothing new.

85
00:04:20,279 --> 00:04:25,160
It's just new notation for
writing what we already know,

86
00:04:25,160 --> 00:04:28,130
that the probability of two
things happening is the

87
00:04:28,130 --> 00:04:31,460
probability that the first thing
happens, and then given

88
00:04:31,460 --> 00:04:34,210
that the first thing happens,
the probability that the

89
00:04:34,210 --> 00:04:36,140
second one happened.

90
00:04:36,140 --> 00:04:39,050
So how do we go from
one to the other?

91
00:04:39,050 --> 00:04:42,960
Think of A as being the event
that X takes the value, little

92
00:04:42,960 --> 00:04:49,120
x, and B being the event that
Y takes the value, little y.

93
00:04:49,120 --> 00:04:52,230
So the joint probability is the
probability that these two

94
00:04:52,230 --> 00:04:54,220
things happen simultaneously.

95
00:04:54,220 --> 00:04:58,140
It's the probability that X
takes this value times the

96
00:04:58,140 --> 00:05:03,280
conditional probability that Y
takes this value, given that X

97
00:05:03,280 --> 00:05:04,670
took that first value.

98
00:05:04,670 --> 00:05:08,470
So it's the familiar
multiplication rule, but just

99
00:05:08,470 --> 00:05:11,030
transcribed in our
new notation.

100
00:05:11,030 --> 00:05:13,690
So nothing new so far.

101
00:05:13,690 --> 00:05:17,480
OK, why did we go through this
exercise and this notation?

102
00:05:17,480 --> 00:05:19,980
It's because in the experiments
where we're

103
00:05:19,980 --> 00:05:23,160
interested in the real world,
typically there's going to be

104
00:05:23,160 --> 00:05:24,630
lots of uncertain quantities.

105
00:05:24,630 --> 00:05:27,150
There's going to be multiple
random variables.

106
00:05:27,150 --> 00:05:31,520
And we want to be able to talk
about them simultaneously.

107
00:05:31,520 --> 00:05:31,665
Okay.

108
00:05:31,665 --> 00:05:35,110
Why two and not more than two?

109
00:05:35,110 --> 00:05:37,620
How about three random
variables?

110
00:05:37,620 --> 00:05:41,290
Well, if you understand what's
going on in this slide, you

111
00:05:41,290 --> 00:05:45,720
should be able to kind of
automatically generalize this

112
00:05:45,720 --> 00:05:48,260
to the case of multiple
random variables.

113
00:05:48,260 --> 00:05:51,590
So for example, if we have three
random variables, X, Y,

114
00:05:51,590 --> 00:05:56,720
and Z, and you see an expression
like this, it

115
00:05:56,720 --> 00:05:58,670
should be clear what it means.

116
00:05:58,670 --> 00:06:02,070
It's the probability that
X takes this value and

117
00:06:02,070 --> 00:06:06,240
simultaneously Y takes that
value and simultaneously Z

118
00:06:06,240 --> 00:06:07,765
takes that value.

119
00:06:07,765 --> 00:06:13,280
I guess that's an uppercase Z
here, that's a lowercase z.

120
00:06:13,280 --> 00:06:20,500
And if I ask you to find the
marginal of X, if I tell you

121
00:06:20,500 --> 00:06:24,340
the joint PMF of the three
random variables and I ask you

122
00:06:24,340 --> 00:06:27,320
for this value, how
would you find it?

123
00:06:27,320 --> 00:06:31,350
Well, you will try to generalize
this relation here.

124
00:06:31,350 --> 00:06:35,250
The probability that x occurs
is the sum of the

125
00:06:35,250 --> 00:06:44,450
probabilities of all events
that make X to take that

126
00:06:44,450 --> 00:06:45,870
particular value.

127
00:06:45,870 --> 00:06:47,400
So what are all the events?

128
00:06:47,400 --> 00:06:51,530
Well, this particular x can
happen together with some y

129
00:06:51,530 --> 00:06:52,790
and some z.

130
00:06:52,790 --> 00:06:55,150
We don't care which y and z.

131
00:06:55,150 --> 00:06:57,890
Any y and z will do.

132
00:06:57,890 --> 00:07:01,220
So when we consider all
possibilities, we need to add

133
00:07:01,220 --> 00:07:04,760
here over all possible values
of y's and z's.

134
00:07:04,760 --> 00:07:08,020
So consider all triples,
x, y, z.

135
00:07:08,020 --> 00:07:12,380
Fix x and consider all the
possibilities for the

136
00:07:12,380 --> 00:07:16,600
remaining variables, y and z,
add these up, and that gives

137
00:07:16,600 --> 00:07:24,740
you the marginal PMF of X. And
then there's other things that

138
00:07:24,740 --> 00:07:26,140
you can do.

139
00:07:26,140 --> 00:07:29,340
This is the multiplication
rule for two events.

140
00:07:29,340 --> 00:07:32,510
We saw back in chapter one that
there's a multiplication

141
00:07:32,510 --> 00:07:35,130
rule when you talk about
more than two events.

142
00:07:35,130 --> 00:07:38,860
And you can write a chain of
conditional probabilities.

143
00:07:38,860 --> 00:07:43,860
We can certainly do the same
in our new notation.

144
00:07:43,860 --> 00:07:45,810
So let's look at this
rule up here.

145
00:07:48,700 --> 00:07:51,220
Multiplication rule for three
random variables,

146
00:07:51,220 --> 00:07:53,000
what does it say?

147
00:07:53,000 --> 00:07:55,280
The probability of three
things happening

148
00:07:55,280 --> 00:07:59,770
simultaneously, X, Y, Z taking
specific values, little x,

149
00:07:59,770 --> 00:08:03,110
little y, little z, that
probability is the probability

150
00:08:03,110 --> 00:08:07,210
that the first thing happens,
that X takes that value.

151
00:08:07,210 --> 00:08:09,880
Given that X takes that value,
we multiply it with the

152
00:08:09,880 --> 00:08:14,650
probability that Y takes
also a certain value.

153
00:08:14,650 --> 00:08:18,560
And now, given that X and Y have
taken those particular

154
00:08:18,560 --> 00:08:21,730
values, we multiply with a
conditional probability that

155
00:08:21,730 --> 00:08:24,380
the third thing happens,
given that the

156
00:08:24,380 --> 00:08:26,960
first two things happen.

157
00:08:26,960 --> 00:08:30,080
So this is just the
multiplication rule for three

158
00:08:30,080 --> 00:08:33,530
events, which would be
probability of A intersection

159
00:08:33,530 --> 00:08:35,669
B intersection C equals--

160
00:08:35,669 --> 00:08:37,909
you know the rest
of the formula.

161
00:08:37,909 --> 00:08:42,330
You just rewrite this formula
in PMF notation.

162
00:08:42,330 --> 00:08:45,310
Probability of A intersection
B intersection C is the

163
00:08:45,310 --> 00:08:49,450
probability of A, which
corresponds to this term,

164
00:08:49,450 --> 00:08:54,010
times the probability of B given
A, times the probability

165
00:08:54,010 --> 00:09:00,700
of C given A and B.

166
00:09:00,700 --> 00:09:04,920
So what else is there that's
left from chapter one that we

167
00:09:04,920 --> 00:09:10,190
can or should generalize
to random variables?

168
00:09:10,190 --> 00:09:12,560
Well, there's the notion
of independence.

169
00:09:12,560 --> 00:09:16,720
So let's define what
independence means.

170
00:09:16,720 --> 00:09:19,970
Instead of talking about just
two random variables, let's go

171
00:09:19,970 --> 00:09:22,470
directly to the case of multiple
random variables.

172
00:09:22,470 --> 00:09:24,400
When we talked about events,
things were a little

173
00:09:24,400 --> 00:09:25,100
complicated.

174
00:09:25,100 --> 00:09:28,480
We had a simple definition for
independence of two events.

175
00:09:28,480 --> 00:09:31,950
Two events are independent if
the probability of both is

176
00:09:31,950 --> 00:09:33,740
equal to the product of
the probabilities.

177
00:09:33,740 --> 00:09:35,830
But for three events, it
was kind of messy.

178
00:09:35,830 --> 00:09:38,460
We needed to write down
lots of conditions.

179
00:09:38,460 --> 00:09:41,140
For random variables,
things in some sense

180
00:09:41,140 --> 00:09:42,060
are a little simpler.

181
00:09:42,060 --> 00:09:46,360
We only need to write down one
formula and take this as the

182
00:09:46,360 --> 00:09:49,020
definition of independence.

183
00:09:49,020 --> 00:09:53,630
Three random variables are
independent if and only if, by

184
00:09:53,630 --> 00:09:58,390
definition, their joint
probability mass function

185
00:09:58,390 --> 00:10:02,560
factors out into individual
probability mass functions.

186
00:10:02,560 --> 00:10:08,190
So the probability that all
three things happen is the

187
00:10:08,190 --> 00:10:11,840
product of the individual
probabilities that each one of

188
00:10:11,840 --> 00:10:14,170
these three things
is happening.

189
00:10:14,170 --> 00:10:17,580
So independence means
mathematically that you can

190
00:10:17,580 --> 00:10:21,030
just multiply probabilities to
get to the probability of

191
00:10:21,030 --> 00:10:22,706
several things happening
simultaneously.

192
00:10:25,680 --> 00:10:31,040
So with three events, we have
to write a huge number of

193
00:10:31,040 --> 00:10:34,500
equations, of equalities
that have to hold.

194
00:10:34,500 --> 00:10:37,500
How can it be that with random
variables we can only manage

195
00:10:37,500 --> 00:10:39,370
with one equality?

196
00:10:39,370 --> 00:10:41,230
Well, the catch is
that this is not

197
00:10:41,230 --> 00:10:43,260
really just one equality.

198
00:10:43,260 --> 00:10:48,390
We require this to be true for
every little x, y, and z.

199
00:10:48,390 --> 00:10:52,600
So in some sense, this is a
bunch of conditions that are

200
00:10:52,600 --> 00:10:56,300
being put on the joint PMF, a
bunch of conditions that we

201
00:10:56,300 --> 00:10:58,130
need to check.

202
00:10:58,130 --> 00:11:01,040
So this is the mathematical
definition.

203
00:11:01,040 --> 00:11:05,400
What is the intuitive content
of this definition?

204
00:11:05,400 --> 00:11:11,130
The intuitive content is
the same as for events.

205
00:11:11,130 --> 00:11:15,020
Random variables are independent
if knowing

206
00:11:15,020 --> 00:11:19,490
something about the realized
values of some of these random

207
00:11:19,490 --> 00:11:25,510
variables does not change our
beliefs about the likelihood

208
00:11:25,510 --> 00:11:29,510
of various values for the
remaining random variables.

209
00:11:29,510 --> 00:11:34,250
So independence would translate,
for example, to a

210
00:11:34,250 --> 00:11:39,690
condition such as the
conditional PMF of X , given

211
00:11:39,690 --> 00:11:46,420
y, should be equal to the
marginal PMF of X. What is

212
00:11:46,420 --> 00:11:47,490
this saying?

213
00:11:47,490 --> 00:11:53,070
That you have some original
beliefs about how likely it is

214
00:11:53,070 --> 00:11:55,210
for X to take this value.

215
00:11:55,210 --> 00:11:58,350
Now, someone comes and
tells you that Y took

216
00:11:58,350 --> 00:12:00,140
on a certain value.

217
00:12:00,140 --> 00:12:03,470
This causes you, in principle,
to revise your beliefs.

218
00:12:03,470 --> 00:12:06,430
And your new beliefs will be
captured by the conditional

219
00:12:06,430 --> 00:12:08,750
PMF, or the conditional
probabilities.

220
00:12:08,750 --> 00:12:12,820
Independence means that your
revised beliefs actually will

221
00:12:12,820 --> 00:12:15,420
be the same as your
original beliefs.

222
00:12:15,420 --> 00:12:19,960
Telling you information about
the value of Y doesn't change

223
00:12:19,960 --> 00:12:24,400
what you expect for the
random variable X.

224
00:12:24,400 --> 00:12:28,750
Why didn't we use this
definition for independence?

225
00:12:28,750 --> 00:12:31,900
Well, because this definition
only makes sense when this

226
00:12:31,900 --> 00:12:34,330
conditional is well-defined.

227
00:12:34,330 --> 00:12:43,290
And this conditional is only
well-defined if the events

228
00:12:43,290 --> 00:12:46,130
that Y takes on that particular
value has positive

229
00:12:46,130 --> 00:12:47,220
probability.

230
00:12:47,220 --> 00:12:51,730
We cannot condition on events
that have zero probability, so

231
00:12:51,730 --> 00:12:55,460
conditional probabilities are
only defined for y's that are

232
00:12:55,460 --> 00:12:59,500
likely to occur, that have
a positive probability.

233
00:12:59,500 --> 00:13:03,640
Now, similarly, with multiple
random variables, if they're

234
00:13:03,640 --> 00:13:07,970
independent, you would have
relations such as the

235
00:13:07,970 --> 00:13:14,290
conditional of X, given y and
z, should be the same as the

236
00:13:14,290 --> 00:13:17,340
marginal of X. What
is this saying?

237
00:13:17,340 --> 00:13:21,220
Again, that if I tell you the
values, the realized values of

238
00:13:21,220 --> 00:13:25,900
random variables Y and Z, this
is not going to change your

239
00:13:25,900 --> 00:13:28,900
beliefs about how likely
x is to occur.

240
00:13:28,900 --> 00:13:30,900
Whatever you believed in the
beginning, you're going to

241
00:13:30,900 --> 00:13:33,000
believe the same thing
afterwards.

242
00:13:33,000 --> 00:13:36,130
So it's important to keep that
intuition in mind, because

243
00:13:36,130 --> 00:13:39,200
sometimes this way you can tell
whether random variables

244
00:13:39,200 --> 00:13:42,820
are independent without having
to do calculations and to

245
00:13:42,820 --> 00:13:44,930
check this formula.

246
00:13:44,930 --> 00:13:47,300
OK, so let's check our concepts

247
00:13:47,300 --> 00:13:49,250
with a simple example.

248
00:13:49,250 --> 00:13:52,220
Let's look at two random
variables that are discrete,

249
00:13:52,220 --> 00:13:55,100
take values between
one and for each.

250
00:13:55,100 --> 00:13:57,890
And this is a table that
gives us the joint PMF.

251
00:13:57,890 --> 00:14:05,720
So it tells us the probability
that X equals to 2 and Y

252
00:14:05,720 --> 00:14:08,040
equals to 1 happening
simultaneously.

253
00:14:08,040 --> 00:14:10,810
It's an event that has
probability 1/20.

254
00:14:10,810 --> 00:14:14,510
Are these two random variables
independent?

255
00:14:14,510 --> 00:14:17,610
You can try to check a
condition like this.

256
00:14:17,610 --> 00:14:21,940
But can we tell directly
from the table?

257
00:14:21,940 --> 00:14:28,470
If I tell you a value of Y,
could that give you useful

258
00:14:28,470 --> 00:14:29,720
information about X?

259
00:14:32,180 --> 00:14:32,860
Certainly.

260
00:14:32,860 --> 00:14:38,680
If I tell you that Y is equal
to 1, this tells you that X

261
00:14:38,680 --> 00:14:40,990
must be equal to 2.

262
00:14:40,990 --> 00:14:44,870
But if I tell you that Y was
equal to 3, this tells you

263
00:14:44,870 --> 00:14:47,540
that, still, X could
be anything.

264
00:14:47,540 --> 00:14:52,220
So telling you the value of
Y kind of changes what you

265
00:14:52,220 --> 00:14:57,240
expect or what you consider
possible for the values of the

266
00:14:57,240 --> 00:14:59,020
other random variable.

267
00:14:59,020 --> 00:15:03,070
So by just inspecting here, we
can tell that the random

268
00:15:03,070 --> 00:15:04,860
variables are not independent.

269
00:15:08,290 --> 00:15:08,470
Okay.

270
00:15:08,470 --> 00:15:10,990
What's the other concept we
introduced in chapter one?

271
00:15:10,990 --> 00:15:14,060
We introduced the concept of
conditional independence.

272
00:15:14,060 --> 00:15:17,120
And conditional independence is
like ordinary independence

273
00:15:17,120 --> 00:15:20,420
but applied to a conditional
universe where we're given

274
00:15:20,420 --> 00:15:21,780
some information.

275
00:15:21,780 --> 00:15:24,610
So suppose someone tells you
that the outcome of the

276
00:15:24,610 --> 00:15:30,420
experiment is such that X is
less than or equal to 2 and Y

277
00:15:30,420 --> 00:15:33,920
is larger than or equal to 3.

278
00:15:33,920 --> 00:15:37,670
So we are given the information
that we now live

279
00:15:37,670 --> 00:15:40,010
inside this universe.

280
00:15:40,010 --> 00:15:42,080
So what happens inside
this universe?

281
00:15:42,080 --> 00:15:47,200
Inside this universe, our random
variables are going to

282
00:15:47,200 --> 00:15:55,140
have a new joint PMF which is
conditioned on the event that

283
00:15:55,140 --> 00:15:58,650
we were told that
it has occurred.

284
00:15:58,650 --> 00:16:04,780
So let A correspond to this
sort of event here.

285
00:16:04,780 --> 00:16:06,900
And now we're dealing with
conditional probabilities.

286
00:16:06,900 --> 00:16:09,490
What are those conditional
probabilities?

287
00:16:09,490 --> 00:16:11,490
We can put them in a table.

288
00:16:11,490 --> 00:16:14,220
So it's a two by two table,
since we only have two

289
00:16:14,220 --> 00:16:15,540
possible values.

290
00:16:15,540 --> 00:16:18,080
What are they going to be?

291
00:16:18,080 --> 00:16:20,740
Well, these probabilities
show up in the ratios

292
00:16:20,740 --> 00:16:22,910
1, 2, 2, and 4.

293
00:16:22,910 --> 00:16:25,480
Those ratios have to
stay the same.

294
00:16:25,480 --> 00:16:29,700
The probabilities need
to add up to one.

295
00:16:29,700 --> 00:16:34,030
So what should the denominators
be since these

296
00:16:34,030 --> 00:16:35,380
numbers add up to nine?

297
00:16:35,380 --> 00:16:37,820
These are the conditional
probabilities.

298
00:16:37,820 --> 00:16:40,575
So this is the conditional
PMF in this example.

299
00:16:43,870 --> 00:16:46,990
Now, in this conditional
universe, is x

300
00:16:46,990 --> 00:16:48,255
independent from y?

301
00:16:51,230 --> 00:17:01,450
If I tell you that y takes this
value, so we live in this

302
00:17:01,450 --> 00:17:04,980
universe, what do you
know about x?

303
00:17:04,980 --> 00:17:08,109
What you know about x is at this
value is twice as likely

304
00:17:08,109 --> 00:17:09,930
as that value.

305
00:17:09,930 --> 00:17:13,859
If I condition on y taking this
value, so we're living

306
00:17:13,859 --> 00:17:16,450
here, what do you
know about x?

307
00:17:16,450 --> 00:17:21,660
What you know about x is that
this value is twice as likely

308
00:17:21,660 --> 00:17:23,240
as that value.

309
00:17:23,240 --> 00:17:24,500
So it's the same.

310
00:17:24,500 --> 00:17:30,250
Whether we live here or we live
there, this x is twice as

311
00:17:30,250 --> 00:17:33,670
likely as that x.

312
00:17:33,670 --> 00:17:41,560
So the conditional PMF in this
new universe, the conditional

313
00:17:41,560 --> 00:17:55,970
PMF of X given y, in the new
universe is the same as the

314
00:17:55,970 --> 00:18:01,250
marginal PMF of X, but of course
in the new universe.

315
00:18:01,250 --> 00:18:04,370
So no matter what y is,
the conditional

316
00:18:04,370 --> 00:18:06,860
PMF of X is the same.

317
00:18:06,860 --> 00:18:12,150
And that conditional
PMF is 1/3 and 2/3.

318
00:18:12,150 --> 00:18:15,150
This is the conditional PMF of
X in the new universe no

319
00:18:15,150 --> 00:18:17,000
matter what y occurs.

320
00:18:17,000 --> 00:18:20,330
So Y does not give us any
information about X, doesn't

321
00:18:20,330 --> 00:18:25,620
cause us to change our beliefs
inside this little universe.

322
00:18:25,620 --> 00:18:28,440
And therefore the two random
variables are independent.

323
00:18:28,440 --> 00:18:31,180
Now, the other way that you
can verify that we have

324
00:18:31,180 --> 00:18:34,960
independence is to find the
marginal PMFs of the two

325
00:18:34,960 --> 00:18:36,250
random variables.

326
00:18:36,250 --> 00:18:39,650
The marginal PMF of
X, you find it by

327
00:18:39,650 --> 00:18:41,100
adding those two terms.

328
00:18:41,100 --> 00:18:42,720
You get 1/3.

329
00:18:42,720 --> 00:18:44,620
Adding those two terms,
you get 2/3.

330
00:18:44,620 --> 00:18:48,530
Marginal PMF of Y, you find it,
you add these two terms,

331
00:18:48,530 --> 00:18:51,410
and you get 1/3.

332
00:18:51,410 --> 00:18:56,470
And the marginal PMF of Y
here is going to be 2/3.

333
00:18:56,470 --> 00:18:59,700
And then you ask the question,
is the joint the product of

334
00:18:59,700 --> 00:19:00,860
the marginals?

335
00:19:00,860 --> 00:19:02,630
And indeed it is.

336
00:19:02,630 --> 00:19:05,330
This times this gives you 1/9.

337
00:19:05,330 --> 00:19:08,050
This times this gives you 2/9.

338
00:19:08,050 --> 00:19:12,180
So the values in the table with
the joint PMFs is the

339
00:19:12,180 --> 00:19:17,220
product of the marginal PMFs of
X and Y in this universe,

340
00:19:17,220 --> 00:19:19,090
so the two random variables are

341
00:19:19,090 --> 00:19:21,850
independent inside this universe.

342
00:19:21,850 --> 00:19:26,704
So we say that they're
conditionally independent.

343
00:19:26,704 --> 00:19:28,500
All right.

344
00:19:28,500 --> 00:19:32,720
Now let's move to the new topic,
to the new concept that

345
00:19:32,720 --> 00:19:35,170
we introduce in this chapter,
which is the concept of

346
00:19:35,170 --> 00:19:36,440
expectations.

347
00:19:36,440 --> 00:19:38,200
So what are the things
to know here?

348
00:19:38,200 --> 00:19:40,150
One is the general idea.

349
00:19:40,150 --> 00:19:43,140
The way to think about
expectations is that it's

350
00:19:43,140 --> 00:19:46,080
something like the average value
for random variable if

351
00:19:46,080 --> 00:19:49,590
you do an experiment over and
over, and if you interpret

352
00:19:49,590 --> 00:19:51,550
probabilities as frequencies.

353
00:19:51,550 --> 00:19:57,030
So you get x's over and over
with a certain frequency --

354
00:19:57,030 --> 00:19:58,670
P(x) --

355
00:19:58,670 --> 00:20:01,160
a particular value, little
x, gets realized.

356
00:20:01,160 --> 00:20:03,960
And each time that this happens,
you get x dollars.

357
00:20:03,960 --> 00:20:06,040
How many dollars do you
get on the average?

358
00:20:06,040 --> 00:20:09,330
Well, this formula gives you
that particular average.

359
00:20:09,330 --> 00:20:13,190
So first thing we do is to write
down a definition for

360
00:20:13,190 --> 00:20:15,420
this sort of concept.

361
00:20:15,420 --> 00:20:19,810
But then the other things you
need to know is how to

362
00:20:19,810 --> 00:20:23,990
calculate expectations using
shortcuts sometimes, and what

363
00:20:23,990 --> 00:20:25,440
properties they have.

364
00:20:25,440 --> 00:20:28,500
The most important shortcut
there is is that, if you want

365
00:20:28,500 --> 00:20:31,250
to calculate the expected value,
the average value for a

366
00:20:31,250 --> 00:20:36,380
random variable, you do not need
to find the PMF of that

367
00:20:36,380 --> 00:20:37,530
random variable.

368
00:20:37,530 --> 00:20:41,180
But you can work directly with
the x's and the y's.

369
00:20:41,180 --> 00:20:44,210
So you do the experiment
over and over.

370
00:20:44,210 --> 00:20:46,670
The outcome of the experiment
is a pair (x,y).

371
00:20:46,670 --> 00:20:49,400
And each time that a certain
(x,y) happens,

372
00:20:49,400 --> 00:20:51,280
you get so many dollars.

373
00:20:51,280 --> 00:20:54,990
So this fraction of the time,
a certain (x,y) happens.

374
00:20:54,990 --> 00:20:58,050
And that fraction of the time,
you get so many dollars, so

375
00:20:58,050 --> 00:21:00,860
this is the average number
of dollars that you get.

376
00:21:00,860 --> 00:21:05,230
So what you end up, since it
is the average, then that

377
00:21:05,230 --> 00:21:07,830
means that it corresponds
to the expected value.

378
00:21:07,830 --> 00:21:09,820
Now, this is something that, of
course, needs a little bit

379
00:21:09,820 --> 00:21:10,850
of mathematical proof.

380
00:21:10,850 --> 00:21:13,880
But this is just a different
way of accounting.

381
00:21:13,880 --> 00:21:16,510
And it turns out we give
you the right answer.

382
00:21:16,510 --> 00:21:19,420
And it's a very useful
shortcut.

383
00:21:19,420 --> 00:21:22,070
Now, when we're talking about
functions of random variables,

384
00:21:22,070 --> 00:21:26,620
in general, we cannot speak
just about averages.

385
00:21:26,620 --> 00:21:29,690
That is, the expected value
of a function of a random

386
00:21:29,690 --> 00:21:31,860
variable is not the same
as the function of

387
00:21:31,860 --> 00:21:33,320
the expected values.

388
00:21:33,320 --> 00:21:36,120
A function of averages is
not the same as the

389
00:21:36,120 --> 00:21:38,380
average of a function.

390
00:21:38,380 --> 00:21:40,510
So in general, this
is not true.

391
00:21:40,510 --> 00:21:43,960
But what it's important to know
is to know the exceptions

392
00:21:43,960 --> 00:21:45,370
to this rule.

393
00:21:45,370 --> 00:21:48,620
And the important exceptions
are mainly two.

394
00:21:48,620 --> 00:21:51,560
One is the case of linear

395
00:21:51,560 --> 00:21:53,040
functions of a random variable.

396
00:21:53,040 --> 00:21:54,800
We discussed this last time.

397
00:21:54,800 --> 00:21:59,810
So the expected value of
temperature in Celsius is, you

398
00:21:59,810 --> 00:22:03,340
first find the expected value of
temperature in Fahrenheit,

399
00:22:03,340 --> 00:22:05,810
and then you do the conversion
to Celsius.

400
00:22:05,810 --> 00:22:08,600
So whether you first average and
then do the conversion to

401
00:22:08,600 --> 00:22:11,730
the new units or not, it
shouldn't matter when you get

402
00:22:11,730 --> 00:22:13,740
the result.

403
00:22:13,740 --> 00:22:16,740
The other property that turns
out to be true when you talk

404
00:22:16,740 --> 00:22:19,280
about multiple random variables
is that expectation

405
00:22:19,280 --> 00:22:21,070
still behaves linearly.

406
00:22:21,070 --> 00:22:26,600
So let X, Y, and Z be the score
of a random student at

407
00:22:26,600 --> 00:22:29,940
each one of the three
sections of the SAT.

408
00:22:29,940 --> 00:22:36,310
So the overall SAT score is X
plus Y plus Z. This is the

409
00:22:36,310 --> 00:22:40,940
average score, the average
total SAT score.

410
00:22:40,940 --> 00:22:43,790
Another way to calculate that
average is to look at the

411
00:22:43,790 --> 00:22:47,480
first section of the SAT and
see what was the average.

412
00:22:47,480 --> 00:22:50,710
Look at the second section, look
at what was the average,

413
00:22:50,710 --> 00:22:53,470
and so the third, and
add the averages.

414
00:22:53,470 --> 00:22:56,910
So you can do the averages for
each section separately, add

415
00:22:56,910 --> 00:23:00,500
the averages, or you can find
total scores for each student

416
00:23:00,500 --> 00:23:01,710
and average them.

417
00:23:01,710 --> 00:23:05,690
So I guess you probably believe
that this is correct

418
00:23:05,690 --> 00:23:09,030
if you talk just about
averaging scores.

419
00:23:09,030 --> 00:23:12,580
Since expectations are just the
variation of averages, it

420
00:23:12,580 --> 00:23:16,010
turns out that this is
also true in general.

421
00:23:16,010 --> 00:23:19,760
And the derivation of this is
very simple, based on the

422
00:23:19,760 --> 00:23:21,320
expected value rule.

423
00:23:21,320 --> 00:23:24,450
And you can look at
it in the notes.

424
00:23:24,450 --> 00:23:27,740
So this is one exception,
which is linearity.

425
00:23:27,740 --> 00:23:31,540
The second important exception
is the case of independent

426
00:23:31,540 --> 00:23:34,520
random variables, that the
product of two random

427
00:23:34,520 --> 00:23:37,830
variables has an expectation
which is the product of the

428
00:23:37,830 --> 00:23:38,980
expectations.

429
00:23:38,980 --> 00:23:41,400
In general, this is not true.

430
00:23:41,400 --> 00:23:47,010
But for the case where we have
independence, the expectation

431
00:23:47,010 --> 00:23:48,080
works out as follows.

432
00:23:48,080 --> 00:23:55,130
Using the expected value rule,
this is how you calculate the

433
00:23:55,130 --> 00:23:59,170
expected value of a function
of a random variable.

434
00:23:59,170 --> 00:24:04,810
So think of this as being your
g(X, Y) and this being your

435
00:24:04,810 --> 00:24:06,160
g(little x, y).

436
00:24:06,160 --> 00:24:08,760
So this is something that's
generally true.

437
00:24:08,760 --> 00:24:20,350
Now, if we have independence,
then the PMFs factor out, and

438
00:24:20,350 --> 00:24:25,660
then you can separate this sum
by bringing together the x

439
00:24:25,660 --> 00:24:30,130
terms, bring them outside
the y summation.

440
00:24:30,130 --> 00:24:34,370
And you find that this is the
same as expected value of X

441
00:24:34,370 --> 00:24:38,890
times the expected value of Y.
So independence is used in

442
00:24:38,890 --> 00:24:40,140
this step here.

443
00:24:44,020 --> 00:24:48,640
OK, now what if X and Y are
independent, but instead of

444
00:24:48,640 --> 00:24:51,020
taking the expectation of
X times Y, we take the

445
00:24:51,020 --> 00:24:56,600
expectation of the product of
two functions of X and Y?

446
00:24:56,600 --> 00:24:59,560
I claim that the expected value
of the product is still

447
00:24:59,560 --> 00:25:02,630
going to be the product of
the expected values.

448
00:25:02,630 --> 00:25:04,180
How do we show that?

449
00:25:04,180 --> 00:25:09,230
We could show it by just redoing
this derivation here.

450
00:25:09,230 --> 00:25:13,500
Instead of X and Y, we would
have g(X) and h(Y), so the

451
00:25:13,500 --> 00:25:14,850
algebra goes through.

452
00:25:14,850 --> 00:25:17,720
But there's a better way to
think about it which is more

453
00:25:17,720 --> 00:25:18,960
conceptual.

454
00:25:18,960 --> 00:25:20,886
And here's the idea.

455
00:25:20,886 --> 00:25:25,750
If X and Y are independent,
what does it mean?

456
00:25:25,750 --> 00:25:31,180
X does not convey any
information about Y. If X

457
00:25:31,180 --> 00:25:36,350
conveys no information about Y,
does X convey information

458
00:25:36,350 --> 00:25:40,500
about h(Y)?

459
00:25:40,500 --> 00:25:41,940
No.

460
00:25:41,940 --> 00:25:46,160
If X tells me nothing about Y,
nothing new, it shouldn't tell

461
00:25:46,160 --> 00:25:50,580
me anything about h(Y).

462
00:25:50,580 --> 00:25:59,270
Now, if X tells me nothing about
h of h(Y), could g(X)

463
00:25:59,270 --> 00:26:01,470
tell me something about h(Y)?

464
00:26:01,470 --> 00:26:02,250
No.

465
00:26:02,250 --> 00:26:06,780
So the idea is that, if X is
unrelated to Y, doesn't have

466
00:26:06,780 --> 00:26:11,080
any useful information, then
g(X) could not have any useful

467
00:26:11,080 --> 00:26:13,250
information for h(Y).

468
00:26:13,250 --> 00:26:21,030
So if X and Y are independent,
then g(X) and h(Y) are also

469
00:26:21,030 --> 00:26:22,280
independent.

470
00:26:27,150 --> 00:26:29,430
So this is something that
one can try to prove

471
00:26:29,430 --> 00:26:31,500
mathematically, but it's more
important to understand

472
00:26:31,500 --> 00:26:34,530
conceptually why this is so.

473
00:26:34,530 --> 00:26:38,220
It's in terms of conveying
information.

474
00:26:38,220 --> 00:26:44,950
So if X tells me nothing about
Y, X cannot tell me anything

475
00:26:44,950 --> 00:26:48,490
about Y cubed, or X cannot
tell me anything by Y

476
00:26:48,490 --> 00:26:51,030
squared, and so on.

477
00:26:51,030 --> 00:26:52,260
That's the idea.

478
00:26:52,260 --> 00:26:57,180
And once we are convinced that
g(X) and h(Y) are independent,

479
00:26:57,180 --> 00:27:00,550
then we can apply our previous
rule, that for independent

480
00:27:00,550 --> 00:27:04,390
random variables, expectations
multiply the right way.

481
00:27:04,390 --> 00:27:08,660
Apply the previous rule, but
apply it now to these two

482
00:27:08,660 --> 00:27:10,490
independent random variables.

483
00:27:10,490 --> 00:27:12,785
And we get the conclusion
that we wanted.

484
00:27:15,500 --> 00:27:19,050
Now, besides expectations, we
also introduced the concept of

485
00:27:19,050 --> 00:27:20,300
the variance.

486
00:27:23,560 --> 00:27:27,450
And if you remember the
definition of the variance,

487
00:27:27,450 --> 00:27:31,100
let me write down the formula
for the variance of aX.

488
00:27:31,100 --> 00:27:34,920
It's the expected value of the
random variable that we're

489
00:27:34,920 --> 00:27:39,630
looking at minus the expected
value of the random variable

490
00:27:39,630 --> 00:27:42,050
that we're looking at.

491
00:27:42,050 --> 00:27:44,780
So this is the difference
of the random

492
00:27:44,780 --> 00:27:47,850
variable from its mean.

493
00:27:47,850 --> 00:27:50,880
And we take that difference
and square it, so it's the

494
00:27:50,880 --> 00:27:53,070
squared distance from the
mean, and then take

495
00:27:53,070 --> 00:27:55,250
expectations of the
whole thing.

496
00:27:55,250 --> 00:27:59,570
So when you look at that
expression, you realize that a

497
00:27:59,570 --> 00:28:01,780
can be pulled out of
those expressions.

498
00:28:04,540 --> 00:28:10,340
And because there is a squared,
when you pull out the

499
00:28:10,340 --> 00:28:12,980
a, it's going to come
out as an a-squared.

500
00:28:12,980 --> 00:28:16,050
So that gives us the rule for
finding the variance of a

501
00:28:16,050 --> 00:28:18,990
scale or product of
a random variable.

502
00:28:18,990 --> 00:28:22,370
The variance captures the idea
of how wide, how spread out a

503
00:28:22,370 --> 00:28:24,210
certain distribution is.

504
00:28:24,210 --> 00:28:26,600
Bigger variance means it's
more spread out.

505
00:28:26,600 --> 00:28:29,360
Now, if you take a random
variable and the constants to

506
00:28:29,360 --> 00:28:31,960
it, what does it do to
its distribution?

507
00:28:31,960 --> 00:28:35,480
It just shifts it, but it
doesn't change its width.

508
00:28:35,480 --> 00:28:37,140
So intuitively it
means that the

509
00:28:37,140 --> 00:28:39,030
variance should not change.

510
00:28:39,030 --> 00:28:42,360
You can check that
mathematically, but it should

511
00:28:42,360 --> 00:28:44,290
also make sense intuitively.

512
00:28:44,290 --> 00:28:47,710
So the variance, when you add
the constant, does not change.

513
00:28:47,710 --> 00:28:51,680
Now, can you add variances is
the way we added expectations?

514
00:28:51,680 --> 00:28:54,760
Does variance behave linearly?

515
00:28:54,760 --> 00:28:57,810
It turns out that not always.

516
00:28:57,810 --> 00:28:59,270
Here, we need a condition.

517
00:28:59,270 --> 00:29:03,880
It's only in special cases--

518
00:29:03,880 --> 00:29:06,210
for example, when the two
random variables are

519
00:29:06,210 --> 00:29:07,190
independent--

520
00:29:07,190 --> 00:29:09,300
that you can add variances.

521
00:29:09,300 --> 00:29:13,300
The variance of the sum is the
sum of the variances if X and

522
00:29:13,300 --> 00:29:15,370
Y are independent.

523
00:29:15,370 --> 00:29:18,880
The derivation of this is,
again, very short and simple.

524
00:29:18,880 --> 00:29:22,590
We'll skip it, but it's an
important fact to remember.

525
00:29:22,590 --> 00:29:26,140
Now, to appreciate why this
equality is not true always,

526
00:29:26,140 --> 00:29:28,980
we can think of some
extreme examples.

527
00:29:28,980 --> 00:29:32,250
Suppose that X is the same as
Y. What's going to be the

528
00:29:32,250 --> 00:29:34,520
variance of X plus Y?

529
00:29:34,520 --> 00:29:39,810
Well, X plus Y, in this case,
is the same as 2X, so we're

530
00:29:39,810 --> 00:29:44,620
going to get 4 times the
variance of X, which is

531
00:29:44,620 --> 00:29:49,770
different than the variance of
X plus the variance of X.

532
00:29:49,770 --> 00:29:52,920
So that expression would give
us twice the variance of X.

533
00:29:52,920 --> 00:29:56,460
But actually now it's 4 times
the variance of X. The other

534
00:29:56,460 --> 00:30:01,990
extreme would be if X is equal
to -Y. Then the variance is

535
00:30:01,990 --> 00:30:05,390
the variance of the random
variable, which is always

536
00:30:05,390 --> 00:30:07,020
equal to 0.

537
00:30:07,020 --> 00:30:09,980
Now, a random variable which
is always equal to 0 has no

538
00:30:09,980 --> 00:30:10,700
uncertainty.

539
00:30:10,700 --> 00:30:14,570
It is always equal to its mean
value, so the variance, in

540
00:30:14,570 --> 00:30:17,090
this case, turns out to be 0.

541
00:30:17,090 --> 00:30:19,940
So in both of these cases,
of course we have random

542
00:30:19,940 --> 00:30:23,020
variables that are extremely
dependent.

543
00:30:23,020 --> 00:30:24,740
Why are they dependent?

544
00:30:24,740 --> 00:30:27,940
Because if I tell you something
about Y, it tells

545
00:30:27,940 --> 00:30:32,020
you an awful lot about the value
of X. There's a lot of

546
00:30:32,020 --> 00:30:34,910
information about X if
I tell you Y, in this

547
00:30:34,910 --> 00:30:37,050
case or in that case.

548
00:30:37,050 --> 00:30:39,940
And finally, a short drill.

549
00:30:39,940 --> 00:30:42,570
If I tell you that the random
variables are independent and

550
00:30:42,570 --> 00:30:44,840
you want to calculate the
variance of a linear

551
00:30:44,840 --> 00:30:48,330
combination of this kind,
then how do you argue?

552
00:30:48,330 --> 00:30:51,940
You argue that, since X and Y
are independent, this means

553
00:30:51,940 --> 00:30:55,660
that X and 3Y are also
independent.

554
00:30:55,660 --> 00:30:59,610
X has no information about Y, so
X has no information about

555
00:30:59,610 --> 00:31:05,000
-Y. X has no information about
-Y, so X should not have any

556
00:31:05,000 --> 00:31:10,270
information about -3Y.

557
00:31:10,270 --> 00:31:14,400
So X and -3Y are independent.

558
00:31:14,400 --> 00:31:18,480
So the variance of Z should be
the variance of X plus the

559
00:31:18,480 --> 00:31:26,910
variance of -3Y, which is the
variance of X plus 9 times the

560
00:31:26,910 --> 00:31:31,760
variance of Y. The important
thing to note here is that no

561
00:31:31,760 --> 00:31:34,080
matter what happens, you
end up getting a

562
00:31:34,080 --> 00:31:37,000
plus here, not a minus.

563
00:31:37,000 --> 00:31:41,160
So that's the sort of important
thing to remember in

564
00:31:41,160 --> 00:31:42,410
this type of calculation.

565
00:31:44,820 --> 00:31:48,890
So this has been all concepts,
reviews, new

566
00:31:48,890 --> 00:31:50,390
concepts and all that.

567
00:31:50,390 --> 00:31:52,720
It's the usual fire hose.

568
00:31:52,720 --> 00:31:56,680
Now let's use them to do
something useful finally.

569
00:31:56,680 --> 00:31:59,220
So let's revisit our old
example, the binomial

570
00:31:59,220 --> 00:32:03,350
distribution, which counts the
number of successes in

571
00:32:03,350 --> 00:32:06,230
independent trials of a coin.

572
00:32:06,230 --> 00:32:09,030
It's a biased coin that has
a probability of heads, or

573
00:32:09,030 --> 00:32:13,000
probability of success, equal
to p at each trial.

574
00:32:13,000 --> 00:32:16,160
Finally, we can go through the
exercise of calculating the

575
00:32:16,160 --> 00:32:18,820
expected value of this
random variable.

576
00:32:18,820 --> 00:32:21,790
And there's the way of
calculating that expectation

577
00:32:21,790 --> 00:32:24,260
that would be the favorite
of those people who enjoy

578
00:32:24,260 --> 00:32:27,500
algebra, which is to write down
the definition of the

579
00:32:27,500 --> 00:32:28,740
expected value.

580
00:32:28,740 --> 00:32:31,980
We add over all possible values
of the random variable,

581
00:32:31,980 --> 00:32:35,580
over all the possible k's, and
weigh them according to the

582
00:32:35,580 --> 00:32:38,440
probabilities that this
particular k occurs.

583
00:32:38,440 --> 00:32:42,250
The probability that X takes on
a particular value k is, of

584
00:32:42,250 --> 00:32:44,820
course, the binomial
PMF, which is

585
00:32:44,820 --> 00:32:47,560
this familiar formula.

586
00:32:47,560 --> 00:32:50,480
Clearly, that would be a messy
and challenging calculation.

587
00:32:50,480 --> 00:32:52,490
Can we find a shortcut?

588
00:32:52,490 --> 00:32:54,010
There's a very clever trick.

589
00:32:54,010 --> 00:32:56,690
There's lots of problems in
probability that you can

590
00:32:56,690 --> 00:33:00,000
approach really nicely by
breaking up the random

591
00:33:00,000 --> 00:33:03,830
variable of interest into a
sum of simpler and more

592
00:33:03,830 --> 00:33:06,010
manageable random variables.

593
00:33:06,010 --> 00:33:09,700
And if you can make it to be a
sum of random variables that

594
00:33:09,700 --> 00:33:12,590
are just 0's or 1's,
so much the better.

595
00:33:12,590 --> 00:33:13,990
Life is easier.

596
00:33:13,990 --> 00:33:16,850
Random variables that take
values 0 or 1, we call them

597
00:33:16,850 --> 00:33:18,380
indicator variables.

598
00:33:18,380 --> 00:33:21,700
They indicate whether an event
has occurred or not.

599
00:33:21,700 --> 00:33:25,600
In this case, we look at each
coin flip one at a time.

600
00:33:25,600 --> 00:33:29,710
For the i-th flip, if it
resulted in heads or a

601
00:33:29,710 --> 00:33:32,110
success, we record it 1.

602
00:33:32,110 --> 00:33:34,220
If not, we record it 0.

603
00:33:34,220 --> 00:33:37,540
And then we look at the
random variable.

604
00:33:37,540 --> 00:33:42,580
If we take the sum of the Xi's,
what is it going to be?

605
00:33:42,580 --> 00:33:48,030
We add one each time that we get
a success, so the sum is

606
00:33:48,030 --> 00:33:50,820
going to be the total
number of successes.

607
00:33:50,820 --> 00:33:53,900
So we break up the random
variable of interest as a sum

608
00:33:53,900 --> 00:33:57,610
of really nice and simple
random variables.

609
00:33:57,610 --> 00:34:00,380
And now we can use the linearity
of expectations.

610
00:34:00,380 --> 00:34:02,800
We're going to find the
expectation of X by finding

611
00:34:02,800 --> 00:34:05,700
the expectation of the Xi's
and then adding the

612
00:34:05,700 --> 00:34:06,770
expectations.

613
00:34:06,770 --> 00:34:09,520
What's the expected
value of Xi?

614
00:34:09,520 --> 00:34:13,050
Well, Xi takes the value 1 with
probability p, and takes

615
00:34:13,050 --> 00:34:15,610
the value 0 with probability
1-p.

616
00:34:15,610 --> 00:34:19,070
So the expected value
of Xi is just p.

617
00:34:19,070 --> 00:34:24,889
So the expected value of X is
going to be just n times p.

618
00:34:24,889 --> 00:34:29,560
Because X is the sum of n terms,
each one of which has

619
00:34:29,560 --> 00:34:33,050
expectation p, the expected
value of the sum is the sum of

620
00:34:33,050 --> 00:34:34,600
the expected values.

621
00:34:34,600 --> 00:34:38,440
So I guess that's a pretty good
shortcut for doing this

622
00:34:38,440 --> 00:34:40,790
horrendous calculation
up there.

623
00:34:40,790 --> 00:34:47,210
So in case you didn't realize
it, that's what we just

624
00:34:47,210 --> 00:34:51,940
established without
doing any algebra.

625
00:34:51,940 --> 00:34:52,219
Good.

626
00:34:52,219 --> 00:34:56,150
How about the variance
of X, of Xi?

627
00:34:56,150 --> 00:34:57,570
Two ways to calculate it.

628
00:34:57,570 --> 00:35:01,160
One is by using directly the
formula for the variance,

629
00:35:01,160 --> 00:35:02,370
which would be --

630
00:35:02,370 --> 00:35:03,900
let's see what it would be.

631
00:35:03,900 --> 00:35:06,800
With probability
p, you get a 1.

632
00:35:06,800 --> 00:35:11,270
And in this case, you are
so far from the mean.

633
00:35:11,270 --> 00:35:13,950
That's your squared distance
from the mean.

634
00:35:13,950 --> 00:35:18,750
With probability 1-p, you
get a 0, which is so far

635
00:35:18,750 --> 00:35:20,380
away from the mean.

636
00:35:20,380 --> 00:35:24,380
And then you can simplify that
formula and get an answer.

637
00:35:24,380 --> 00:35:28,660
How about a slightly easier
way of doing it.

638
00:35:28,660 --> 00:35:31,360
Instead of doing the algebra
here, let me indicate the

639
00:35:31,360 --> 00:35:33,420
slightly easier way.

640
00:35:33,420 --> 00:35:36,070
We have a formula for the
variance that tells us that we

641
00:35:36,070 --> 00:35:42,290
can find the variance by
proceeding this way.

642
00:35:42,290 --> 00:35:45,980
That's a formula that's
generally true for variances.

643
00:35:45,980 --> 00:35:47,380
Why is this easier?

644
00:35:47,380 --> 00:35:49,560
What's the expected value
of Xi squared?

645
00:35:52,240 --> 00:35:53,290
Backtrack.

646
00:35:53,290 --> 00:35:57,140
What is Xi squared, after all?

647
00:35:57,140 --> 00:35:59,510
It's the same thing as Xi.

648
00:35:59,510 --> 00:36:04,200
Since Xi takes value 0 and 1, Xi
squared also takes the same

649
00:36:04,200 --> 00:36:05,780
values, 0 and 1.

650
00:36:05,780 --> 00:36:09,050
So the expected value of Xi
squared is the same as the

651
00:36:09,050 --> 00:36:11,990
expected value of Xi,
which is equal to p.

652
00:36:15,120 --> 00:36:20,530
And the expected value of Xi
squared is p squared, so we

653
00:36:20,530 --> 00:36:24,680
get the final answer,
p times (1-p).

654
00:36:24,680 --> 00:36:28,630
If you were to work through and
do the cancellations in

655
00:36:28,630 --> 00:36:32,400
this messy expression here,
after one line you would also

656
00:36:32,400 --> 00:36:34,050
get to the same formula.

657
00:36:34,050 --> 00:36:38,240
But this sort of illustrates
that working with this formula

658
00:36:38,240 --> 00:36:40,550
for the variance, sometimes
things work

659
00:36:40,550 --> 00:36:43,090
out a little faster.

660
00:36:43,090 --> 00:36:45,420
Finally, are we in business?

661
00:36:45,420 --> 00:36:47,820
Can we calculate the variance
of the random

662
00:36:47,820 --> 00:36:50,100
variable X as well?

663
00:36:50,100 --> 00:36:52,650
Well, we have the rule that
for independent random

664
00:36:52,650 --> 00:36:55,680
variables, the variance
of the sum is

665
00:36:55,680 --> 00:36:57,870
the sum of the variances.

666
00:36:57,870 --> 00:37:00,930
So to find the variance of X,
we just need to add the

667
00:37:00,930 --> 00:37:02,960
variances of the Xi's.

668
00:37:02,960 --> 00:37:07,140
We have n Xi's, and each
one of them has

669
00:37:07,140 --> 00:37:10,110
variance p_n times (1-p).

670
00:37:10,110 --> 00:37:12,290
And we are done.

671
00:37:12,290 --> 00:37:17,780
So this way, we have calculated
both the mean and

672
00:37:17,780 --> 00:37:21,550
the variance of the binomial
random variable.

673
00:37:21,550 --> 00:37:27,280
It's interesting to look at this
particular formula and

674
00:37:27,280 --> 00:37:29,180
see what it tells us.

675
00:37:29,180 --> 00:37:33,470
If you are to plot the variance
of X as a function of

676
00:37:33,470 --> 00:37:36,050
p, it has this shape.

677
00:37:45,900 --> 00:37:51,310
And the maximum is
here at 1/2.

678
00:37:51,310 --> 00:37:55,150
p times (1-p) is 0 when
p is equal to 0.

679
00:37:55,150 --> 00:37:58,570
And when p equals to 1, it's a
quadratic, so it must have

680
00:37:58,570 --> 00:38:00,250
this particular shape.

681
00:38:00,250 --> 00:38:02,080
So what does it tell us?

682
00:38:02,080 --> 00:38:05,880
If you think about variance as
a measure of uncertainty, it

683
00:38:05,880 --> 00:38:10,290
tells you that coin flips
are most uncertain when

684
00:38:10,290 --> 00:38:12,620
your coin is fair.

685
00:38:12,620 --> 00:38:16,190
When p is equal to 1/2, that's
when you have the most

686
00:38:16,190 --> 00:38:17,050
randomness.

687
00:38:17,050 --> 00:38:18,790
And this is kind of intuitive.

688
00:38:18,790 --> 00:38:21,460
if on the other hand I tell you
that the coin is extremely

689
00:38:21,460 --> 00:38:26,490
biased, p very close to 1, which
means it almost always

690
00:38:26,490 --> 00:38:29,460
gives you heads, then
that would be

691
00:38:29,460 --> 00:38:30,630
a case of low variance.

692
00:38:30,630 --> 00:38:32,870
There's low variability
in the results.

693
00:38:32,870 --> 00:38:35,270
There's little uncertainty about
what's going to happen.

694
00:38:35,270 --> 00:38:39,570
It's going to be mostly heads
with some occasional tails.

695
00:38:39,570 --> 00:38:42,010
So p equals 1/2.

696
00:38:42,010 --> 00:38:45,350
Fair coin, that's the coin which
is the most uncertain of

697
00:38:45,350 --> 00:38:47,240
all coins, in some sense.

698
00:38:47,240 --> 00:38:49,240
And it corresponds to the
biggest variance.

699
00:38:49,240 --> 00:38:53,760
It corresponds to an X that has
the widest distribution.

700
00:38:53,760 --> 00:38:57,680
Now that we're on a roll and we
can calculate such hugely

701
00:38:57,680 --> 00:39:01,400
complicated sums in simple ways,
let us try to push our

702
00:39:01,400 --> 00:39:05,100
luck and do a problem with
this flavor, but a little

703
00:39:05,100 --> 00:39:06,590
harder than that.

704
00:39:06,590 --> 00:39:07,960
So you go to one of those

705
00:39:07,960 --> 00:39:09,910
old-fashioned cocktail parties.

706
00:39:09,910 --> 00:39:16,010
All males at least will have
those standard big hats which

707
00:39:16,010 --> 00:39:16,990
look identical.

708
00:39:16,990 --> 00:39:19,700
They check them in when
they walk in.

709
00:39:19,700 --> 00:39:23,390
And when they walk out, since
they look pretty identical,

710
00:39:23,390 --> 00:39:26,830
they just pick a random
hat and go home.

711
00:39:26,830 --> 00:39:31,080
So n people, they pick their
hats completely at random,

712
00:39:31,080 --> 00:39:33,950
quote, unquote, and
then leave.

713
00:39:33,950 --> 00:39:36,970
And the question is, to say
something about the number of

714
00:39:36,970 --> 00:39:42,070
people who end up, by accident
or by luck, to get back their

715
00:39:42,070 --> 00:39:45,170
own hat, the exact same hat
that they checked in.

716
00:39:45,170 --> 00:39:48,490
OK, first what do we mean
completely at random?

717
00:39:48,490 --> 00:39:51,060
Completely at random, we
basically mean that any

718
00:39:51,060 --> 00:39:54,180
permutation of the hats
is equally likely.

719
00:39:54,180 --> 00:39:58,520
Any way of distributing those
n hats to the n people, any

720
00:39:58,520 --> 00:40:01,350
particular way is as likely
as any other way.

721
00:40:01,350 --> 00:40:05,230
So there's complete symmetry
between hats and people.

722
00:40:05,230 --> 00:40:08,490
So what we want to do is to
calculate the expected value

723
00:40:08,490 --> 00:40:11,460
and the variance of this random
variable X. Let's start

724
00:40:11,460 --> 00:40:13,240
with the expected value.

725
00:40:13,240 --> 00:40:17,840
Let's reuse the trick from
the binomial case.

726
00:40:17,840 --> 00:40:21,110
So total number of hats picked,
we're going to think

727
00:40:21,110 --> 00:40:24,140
of total number of hats
picked as a sum of

728
00:40:24,140 --> 00:40:26,900
(0, 1) random variables.

729
00:40:26,900 --> 00:40:30,470
X1 tells us whether person
1 got their own hat back.

730
00:40:30,470 --> 00:40:32,920
If they did, we record a 1.

731
00:40:32,920 --> 00:40:34,960
X2, the same thing.

732
00:40:34,960 --> 00:40:40,910
By adding all X's is how many
1's did we get, which counts

733
00:40:40,910 --> 00:40:45,510
how many people selected
their own hats.

734
00:40:45,510 --> 00:40:48,100
So we broke down the random
variable of interest, the

735
00:40:48,100 --> 00:40:51,500
number of people who get their
own hats back, as a sum of

736
00:40:51,500 --> 00:40:53,570
random variables.

737
00:40:53,570 --> 00:40:56,200
And these random variables,
again, are easy to handle,

738
00:40:56,200 --> 00:40:58,010
because they're binary.

739
00:40:58,010 --> 00:40:59,250
The only take two values.

740
00:40:59,250 --> 00:41:03,500
What's the probability that Xi
is equal to 1, the i-th person

741
00:41:03,500 --> 00:41:06,730
has a probability that they
get their own hat?

742
00:41:06,730 --> 00:41:09,430
There's n hats by symmetry.

743
00:41:09,430 --> 00:41:11,890
The chance is that they end up
getting their own hat, as

744
00:41:11,890 --> 00:41:14,930
opposed to any one of the
other n - 1 hats,

745
00:41:14,930 --> 00:41:18,020
is going to be 1/n.

746
00:41:18,020 --> 00:41:20,710
So what's the expected
value of Xi?

747
00:41:20,710 --> 00:41:23,130
It's one times 1/n.

748
00:41:23,130 --> 00:41:26,510
With probability 1/n, you get
your own hat, or you get a

749
00:41:26,510 --> 00:41:30,960
value of 0 with probability
1-1/n, which is 1/n.

750
00:41:34,660 --> 00:41:38,360
All right, so we got the
expected value of the Xi's.

751
00:41:38,360 --> 00:41:41,510
And remember, we want to do is
to calculate the expected

752
00:41:41,510 --> 00:41:46,900
value of X by using this
decomposition?

753
00:41:46,900 --> 00:41:52,230
Are the random variables Xi
independent of each other?

754
00:41:52,230 --> 00:41:55,470
You can try to answer that
question by writing down a

755
00:41:55,470 --> 00:41:58,510
joint PMF for the X's,
but I'm sure that

756
00:41:58,510 --> 00:42:00,000
you will not succeed.

757
00:42:00,000 --> 00:42:02,740
But can you think intuitively?

758
00:42:02,740 --> 00:42:05,940
If I tell you information about
some of the Xi's, does

759
00:42:05,940 --> 00:42:08,920
it give you information about
the remaining ones?

760
00:42:08,920 --> 00:42:09,300
Yeah.

761
00:42:09,300 --> 00:42:13,950
If I tell you that out of 10
people, 9 of them got their

762
00:42:13,950 --> 00:42:16,710
own hat back, does that
tell you something

763
00:42:16,710 --> 00:42:18,330
about the 10th person?

764
00:42:18,330 --> 00:42:18,690
Yes.

765
00:42:18,690 --> 00:42:22,510
If 9 got their own hat, then the
10th must also have gotten

766
00:42:22,510 --> 00:42:24,170
their own hat back.

767
00:42:24,170 --> 00:42:27,170
So the first 9 random variables
tell you something

768
00:42:27,170 --> 00:42:28,790
about the 10th one.

769
00:42:28,790 --> 00:42:33,000
And conveying information of
this sort, that's the case of

770
00:42:33,000 --> 00:42:34,410
dependence.

771
00:42:34,410 --> 00:42:38,100
All right, so the random
variables are not independent.

772
00:42:38,100 --> 00:42:39,030
Are we stuck?

773
00:42:39,030 --> 00:42:43,240
Can we still calculate the
expected value of X?

774
00:42:43,240 --> 00:42:45,210
Yes, we can.

775
00:42:45,210 --> 00:42:50,710
And the reason we can is that
expectations are linear.

776
00:42:50,710 --> 00:42:53,940
Expectation of a sum of random
variables is the sum of the

777
00:42:53,940 --> 00:42:55,140
expectations.

778
00:42:55,140 --> 00:42:57,490
And that's always true.

779
00:42:57,490 --> 00:43:00,710
There's no independence
assumption that's being used

780
00:43:00,710 --> 00:43:02,540
to apply that rule.

781
00:43:02,540 --> 00:43:06,980
So we have that the expected
value of X is the sum of the

782
00:43:06,980 --> 00:43:09,580
expected value of the Xi's.

783
00:43:09,580 --> 00:43:12,970
And this is a property
that's always true.

784
00:43:12,970 --> 00:43:14,350
You don't need independence.

785
00:43:14,350 --> 00:43:15,590
You don't care.

786
00:43:15,590 --> 00:43:18,660
So we're adding n terms,
each one of which has

787
00:43:18,660 --> 00:43:20,430
expected value 1/n.

788
00:43:20,430 --> 00:43:22,670
And the final answer is 1.

789
00:43:22,670 --> 00:43:27,430
So out of the 100 people who
selected hats at random, on

790
00:43:27,430 --> 00:43:32,590
the average, you expect only one
of them to end up getting

791
00:43:32,590 --> 00:43:35,830
their own hat back.

792
00:43:35,830 --> 00:43:36,640
Very good.

793
00:43:36,640 --> 00:43:41,620
So since we are succeeding so
far, let's try to see if we

794
00:43:41,620 --> 00:43:44,620
can succeed in calculating
the variance as well.

795
00:43:44,620 --> 00:43:46,580
And of course, we will.

796
00:43:46,580 --> 00:43:50,160
But it's going to be a little
more complicated.

797
00:43:50,160 --> 00:43:52,760
The reason it's going to be a
little more complicated is

798
00:43:52,760 --> 00:43:56,500
because the Xi's are not
independent, so the variance

799
00:43:56,500 --> 00:44:00,280
of the sum is not the same as
the sum of the variances.

800
00:44:00,280 --> 00:44:04,320
So it's not enough to find the
variances of the Xi's.

801
00:44:04,320 --> 00:44:06,930
We'll have to do more work.

802
00:44:06,930 --> 00:44:08,550
And here's what's involved.

803
00:44:08,550 --> 00:44:12,320
Let's start with the general
formula for the variance,

804
00:44:12,320 --> 00:44:15,950
which, as I mentioned before,
it's usually the simpler way

805
00:44:15,950 --> 00:44:18,430
to go about calculating
variances.

806
00:44:18,430 --> 00:44:21,800
So we need to calculate the
expected value for X-squared,

807
00:44:21,800 --> 00:44:27,110
and subtract from it the
expectation squared.

808
00:44:27,110 --> 00:44:31,010
Well, we already found the
expected value of X. It's

809
00:44:31,010 --> 00:44:31,870
equal to 1.

810
00:44:31,870 --> 00:44:34,580
So 1-squared gives us just 1.

811
00:44:34,580 --> 00:44:37,980
So we're left with the task of
calculating the expected value

812
00:44:37,980 --> 00:44:43,440
of X-squared, the random
variable X-squared.

813
00:44:43,440 --> 00:44:45,610
Let's try to follow
the same idea.

814
00:44:45,610 --> 00:44:49,770
Write this messy random
variable, X-squared, as a sum

815
00:44:49,770 --> 00:44:54,440
of hopefully simpler
random variables.

816
00:44:54,440 --> 00:44:59,350
So X is the sum of the
Xi's, so you square

817
00:44:59,350 --> 00:45:01,560
both sides of this.

818
00:45:01,560 --> 00:45:05,150
And then you expand the
right-hand side.

819
00:45:05,150 --> 00:45:09,390
When you expand the right-hand
side, you get the squares of

820
00:45:09,390 --> 00:45:11,420
the terms that appear here.

821
00:45:11,420 --> 00:45:14,230
And then you get all
the cross-terms.

822
00:45:14,230 --> 00:45:19,100
For every pair of (i,j) that
are different, i different

823
00:45:19,100 --> 00:45:24,030
than j, you're going to have
a cross-term in the sum.

824
00:45:24,030 --> 00:45:29,230
So now, in order to calculate
the expected value of

825
00:45:29,230 --> 00:45:32,480
X-squared, what does
our task reduce to?

826
00:45:32,480 --> 00:45:36,230
It reduces to calculating the
expected value of this term

827
00:45:36,230 --> 00:45:38,690
and calculating the expected
value of that term.

828
00:45:38,690 --> 00:45:41,060
So let's do them
one at a time.

829
00:45:41,060 --> 00:45:47,040
Expected value of Xi squared,
what is it going to be?

830
00:45:47,040 --> 00:45:48,660
Same trick as before.

831
00:45:48,660 --> 00:45:53,350
Xi takes value 0 or 1, so Xi
squared takes just the same

832
00:45:53,350 --> 00:45:55,290
values, 0 or 1.

833
00:45:55,290 --> 00:45:57,010
So that's the easy one.

834
00:45:57,010 --> 00:46:00,680
That's the same as expected
value of Xi, which we already

835
00:46:00,680 --> 00:46:04,410
know to be 1/n.

836
00:46:04,410 --> 00:46:07,830
So this gives us a first
contribution down here.

837
00:46:10,840 --> 00:46:14,220
The expected value of this
term is going to be what?

838
00:46:14,220 --> 00:46:17,210
We have n terms in
the summation.

839
00:46:17,210 --> 00:46:21,800
And each one of these terms
has an expectation of 1/n.

840
00:46:21,800 --> 00:46:24,710
So we did a piece
of the puzzle.

841
00:46:24,710 --> 00:46:28,480
So now let's deal with the
second piece of the puzzle.

842
00:46:28,480 --> 00:46:32,020
Let's find the expected
value of Xi times Xj.

843
00:46:32,020 --> 00:46:35,540
Now by symmetry, the expected
value of Xi times Xj is going

844
00:46:35,540 --> 00:46:39,900
to be the same no matter
what i and j you see.

845
00:46:39,900 --> 00:46:44,930
So let's just think about X1
and X2 and try to find the

846
00:46:44,930 --> 00:46:48,260
expected value of X1 and X2.

847
00:46:48,260 --> 00:46:51,710
X1 times X2 is a random
variable.

848
00:46:51,710 --> 00:46:53,960
What values does it take?

849
00:46:53,960 --> 00:46:56,570
Only 0 or 1?

850
00:46:56,570 --> 00:47:00,000
Since X1 and X2 are 0 or 1,
their product can only take

851
00:47:00,000 --> 00:47:02,010
the values of 0 or 1.

852
00:47:02,010 --> 00:47:04,990
So to find the probability
distribution of this random

853
00:47:04,990 --> 00:47:07,320
variable, it's just sufficient
to find the probability that

854
00:47:07,320 --> 00:47:09,530
it takes the value of 1.

855
00:47:09,530 --> 00:47:14,500
Now, what does X1 times
X2 equal to 1 mean?

856
00:47:14,500 --> 00:47:19,500
It means that X1 was
1 and X2 was 1.

857
00:47:19,500 --> 00:47:22,390
The only way that you can get
a product of 1 is if both of

858
00:47:22,390 --> 00:47:24,350
them turned out to be 1's.

859
00:47:24,350 --> 00:47:29,570
So that's the same as saying,
persons 1 and 2 both picked

860
00:47:29,570 --> 00:47:31,980
their own hats.

861
00:47:31,980 --> 00:47:35,510
The probability that person 1
and person 2 both pick their

862
00:47:35,510 --> 00:47:39,600
own hats is the probability of
two things happening, which is

863
00:47:39,600 --> 00:47:42,320
the product of the first thing
happening times the

864
00:47:42,320 --> 00:47:44,310
conditional probability
of the second, given

865
00:47:44,310 --> 00:47:46,160
that the first happened.

866
00:47:46,160 --> 00:47:48,690
And in words, this is the
probability that the first

867
00:47:48,690 --> 00:47:51,840
person picked their own hat
times the probability that the

868
00:47:51,840 --> 00:47:54,920
second person picks their own
hat, given that the first

869
00:47:54,920 --> 00:47:56,990
person already picked
their own.

870
00:47:56,990 --> 00:47:58,820
So what's the probability
that the first person

871
00:47:58,820 --> 00:48:00,760
picks their own hat?

872
00:48:00,760 --> 00:48:03,040
We know that it's 1/n.

873
00:48:03,040 --> 00:48:05,030
Now, how about the
second person?

874
00:48:05,030 --> 00:48:09,540
If I tell you that one person
has their own hat, and that

875
00:48:09,540 --> 00:48:13,240
person takes their hat and goes
away, from the point of

876
00:48:13,240 --> 00:48:17,250
view of the second person,
there's n - 1 people left

877
00:48:17,250 --> 00:48:19,770
looking at n - 1 hats.

878
00:48:19,770 --> 00:48:22,330
And they're getting just
hats at random.

879
00:48:22,330 --> 00:48:24,930
What's the chance that
I will get my own?

880
00:48:24,930 --> 00:48:26,180
It's 1/n - 1.

881
00:48:29,210 --> 00:48:33,700
So think of them as person 1
goes, picks a hat at random,

882
00:48:33,700 --> 00:48:36,850
it happens to be their
own, and it leaves.

883
00:48:36,850 --> 00:48:40,120
You're left with n - 1 people,
and there are n

884
00:48:40,120 --> 00:48:41,250
- 1 hats out there.

885
00:48:41,250 --> 00:48:44,490
Person 2 goes and picks a hat
at random, with probability

886
00:48:44,490 --> 00:48:48,820
1/n - 1, is going to
pick his own hat.

887
00:48:48,820 --> 00:48:52,400
So the expected value now of
this random variable is,

888
00:48:52,400 --> 00:48:54,520
again, that same number,
because this is

889
00:48:54,520 --> 00:48:57,500
a 0, 1 random variable.

890
00:48:57,500 --> 00:49:02,370
So this is the same as expected
value of Xi times Xj

891
00:49:02,370 --> 00:49:04,810
when i different than j.

892
00:49:04,810 --> 00:49:09,830
So here, all that's left to do
is to add the expectations of

893
00:49:09,830 --> 00:49:10,540
these terms.

894
00:49:10,540 --> 00:49:14,480
Each one of these terms has an
expected value that's 1/n

895
00:49:14,480 --> 00:49:16,910
times (1/n - 1).

896
00:49:16,910 --> 00:49:19,170
And how many terms do we have?

897
00:49:19,170 --> 00:49:21,410
How many of these are
we adding up?

898
00:49:24,840 --> 00:49:28,950
It's n-squared - n.

899
00:49:28,950 --> 00:49:31,830
When you expand the quadratic,
there's a total

900
00:49:31,830 --> 00:49:33,890
of n-squared terms.

901
00:49:33,890 --> 00:49:37,860
Some are self-terms,
n of them.

902
00:49:37,860 --> 00:49:42,170
And the remaining number of
terms is n-squared - n.

903
00:49:42,170 --> 00:49:48,310
So here we got n-squared
- n terms.

904
00:49:48,310 --> 00:49:51,200
And so we need to multiply
here with n-squared - n.

905
00:49:53,810 --> 00:49:59,980
And after you realize that this
number here is 1, and you

906
00:49:59,980 --> 00:50:03,490
realize that this is the same
as the denominator, you get

907
00:50:03,490 --> 00:50:06,750
the answer that the expected
value of X squared equals 2.

908
00:50:06,750 --> 00:50:10,120
And then, finally going up to
the top formula, we get the

909
00:50:10,120 --> 00:50:14,720
expected value of X squared,
which is 2 - 1, and the

910
00:50:14,720 --> 00:50:17,610
variance is just equal to 1.

911
00:50:17,610 --> 00:50:21,680
So the variance of this random
variable, number of people who

912
00:50:21,680 --> 00:50:25,130
get their own hats back,
is also equal to 1,

913
00:50:25,130 --> 00:50:26,540
equal to the mean.

914
00:50:26,540 --> 00:50:27,690
Looks like magic.

915
00:50:27,690 --> 00:50:29,220
Why is this the case?

916
00:50:29,220 --> 00:50:31,550
Well, there's a deeper
explanation why these two

917
00:50:31,550 --> 00:50:33,630
numbers should come out
to be the same.

918
00:50:33,630 --> 00:50:35,980
But this is something that would
probably have to wait a

919
00:50:35,980 --> 00:50:39,420
couple of chapters before we
could actually explain it.

920
00:50:39,420 --> 00:50:40,730
And so I'll stop here.