1
00:00:00,000 --> 00:00:00,460

2
00:00:00,460 --> 00:00:00,660
Hi.

3
00:00:00,660 --> 00:00:03,240
In this session, we're going to
cover a nice review problem

4
00:00:03,240 --> 00:00:05,210
that will look at how
to infer one random

5
00:00:05,210 --> 00:00:06,880
variable based on another.

6
00:00:06,880 --> 00:00:09,090
And in this problem, we're given
two random variables--

7
00:00:09,090 --> 00:00:10,090
X and Y--

8
00:00:10,090 --> 00:00:13,950
and we're also given their joint
pdf, which we're told is

9
00:00:13,950 --> 00:00:16,840
a constant 2/3 within the
region bounded by

10
00:00:16,840 --> 00:00:17,930
these orange lines.

11
00:00:17,930 --> 00:00:21,280
And outside of the region,
the joint pdf is 0.

12
00:00:21,280 --> 00:00:24,150
So the first thing we're going
to look at, or the first thing

13
00:00:24,150 --> 00:00:29,140
that we're asked to do, is find
the LMS estimator of Y

14
00:00:29,140 --> 00:00:36,230
based on X. Now, remember that
LMS estimator is really just a

15
00:00:36,230 --> 00:00:37,990
conditional expectation.

16
00:00:37,990 --> 00:00:42,860
So the LMS estimator of Y based
on X is the conditional

17
00:00:42,860 --> 00:00:45,950
expectation of Y, given X.

18
00:00:45,950 --> 00:00:48,960
Now, when we have a plot of
the joint pdf and we're

19
00:00:48,960 --> 00:00:53,200
dealing with these two random
variables, and especially when

20
00:00:53,200 --> 00:00:56,620
the joint pdf is constant like
this, it's often easy to

21
00:00:56,620 --> 00:00:59,680
calculate this conditional
expectation visually.

22
00:00:59,680 --> 00:01:02,810
So what we really need to do
is just say, given any

23
00:01:02,810 --> 00:01:07,520
particular value of X, what
is the conditional

24
00:01:07,520 --> 00:01:08,670
expectation of Y?

25
00:01:08,670 --> 00:01:12,390
So what we can do is we can just
pick some values of X and

26
00:01:12,390 --> 00:01:15,520
see visually what that initial
expectation is.

27
00:01:15,520 --> 00:01:24,900
So for example, if X is 1/2,
given that X is 1/2, and since

28
00:01:24,900 --> 00:01:29,510
this whole joint pdf is uniform,
then the conditional

29
00:01:29,510 --> 00:01:32,780
slice of Y will be from
here to here.

30
00:01:32,780 --> 00:01:36,260
And that slice, the conditional
distribution of Y,

31
00:01:36,260 --> 00:01:38,910
given that X is 1/2, will
also be uniform.

32
00:01:38,910 --> 00:01:41,450
So it'll be uniform
from here to here.

33
00:01:41,450 --> 00:01:42,990
And if it's uniform, we know
that the conditional

34
00:01:42,990 --> 00:01:46,150
expectation will just be
the midpoint here.

35
00:01:46,150 --> 00:01:48,990

36
00:01:48,990 --> 00:01:51,670
And so that would be what the
conditional expectation of Y

37
00:01:51,670 --> 00:01:53,830
would be, given that X is 1/2.

38
00:01:53,830 --> 00:01:57,640
And we could do the same
thing for X equals 1.

39
00:01:57,640 --> 00:02:01,020
And we'll see that again,
because everything is uniform,

40
00:02:01,020 --> 00:02:03,030
this slice is also going
to be uniform.

41
00:02:03,030 --> 00:02:06,030
And so the conditional
expectation will again be the

42
00:02:06,030 --> 00:02:07,780
midpoint, which is there.

43
00:02:07,780 --> 00:02:12,220
And then if we just look at it
within this region, it's

44
00:02:12,220 --> 00:02:15,850
always going to be
the midpoint.

45
00:02:15,850 --> 00:02:18,820
And so we get that the initial
expectation of Y, given X,

46
00:02:18,820 --> 00:02:23,070
will just look like that line,
which you can think of it as

47
00:02:23,070 --> 00:02:29,420
just bisecting this angle formed
by these two parts of

48
00:02:29,420 --> 00:02:31,000
the region.

49
00:02:31,000 --> 00:02:33,230
But things are a little bit
different, though, when we

50
00:02:33,230 --> 00:02:37,160
move to the region where
X is between 1 and 2.

51
00:02:37,160 --> 00:02:44,480
Between 1 and 2, say
at 1 and 1/2, this

52
00:02:44,480 --> 00:02:45,370
line doesn't continue.

53
00:02:45,370 --> 00:02:51,690
Because now, the slice of Y goes
from here to here, and

54
00:02:51,690 --> 00:02:52,800
again, it's still uniform.

55
00:02:52,800 --> 00:02:58,400
So the midpoint would
be there.

56
00:02:58,400 --> 00:03:02,580
And similarly for X equals
2, it would be here.

57
00:03:02,580 --> 00:03:08,330
And so for X between 1 and 2,
the conditional expectation

58
00:03:08,330 --> 00:03:09,950
actually looks like this.

59
00:03:09,950 --> 00:03:12,720
So you see that there's actually
two linear parts of

60
00:03:12,720 --> 00:03:16,210
it, but there's a kink
at X equals 1.

61
00:03:16,210 --> 00:03:19,780
And so by looking at this
visually and taking advantage

62
00:03:19,780 --> 00:03:22,630
of the fact that everything is
uniform, we can pretty easily

63
00:03:22,630 --> 00:03:24,650
figure out what this conditional
expectation is.

64
00:03:24,650 --> 00:03:29,420
So now, let's actually just
write it out algebraically.

65
00:03:29,420 --> 00:03:34,620
So for X between 0 and 1, we
said that it's this line,

66
00:03:34,620 --> 00:03:41,490
which if we look at it, that's
just 1/2 of X. Now, this is

67
00:03:41,490 --> 00:03:44,800
for X between 0 and 1.

68
00:03:44,800 --> 00:03:49,000
And if X is between 1 and 2,
it's going to be this line,

69
00:03:49,000 --> 00:03:50,620
which is a slope of 1.

70
00:03:50,620 --> 00:03:53,600
And if we extend this
down, it hits the

71
00:03:53,600 --> 00:03:56,170
y-axis at negative 1/2.

72
00:03:56,170 --> 00:04:02,840
So it's X minus 1/2, if
X is between 1 and 2.

73
00:04:02,840 --> 00:04:06,110

74
00:04:06,110 --> 00:04:08,840
And otherwise, it's undefined.

75
00:04:08,840 --> 00:04:11,100
So we'll focus on these
two cases here.

76
00:04:11,100 --> 00:04:13,720

77
00:04:13,720 --> 00:04:21,209
Now, the second part of the
question, now that we know

78
00:04:21,209 --> 00:04:24,330
what the LMS estimator is, we're
asked to find what is

79
00:04:24,330 --> 00:04:27,860
the traditional mean squared
error of this estimator?

80
00:04:27,860 --> 00:04:29,590
So we want to know
how good is it.

81
00:04:29,590 --> 00:04:31,520
And one way of capturing
that is to look at the

82
00:04:31,520 --> 00:04:32,750
mean squared error.

83
00:04:32,750 --> 00:04:37,130
And so recall that the
conditional mean squared error

84
00:04:37,130 --> 00:04:41,755
is given by this expression.

85
00:04:41,755 --> 00:04:48,930

86
00:04:48,930 --> 00:04:52,510
So what we're saying is this is
what we estimate Y to be.

87
00:04:52,510 --> 00:04:56,930
This is what y really is, so
this difference is how wrong,

88
00:04:56,930 --> 00:04:59,370
or the error in our estimate.

89
00:04:59,370 --> 00:05:04,090
We square it, because otherwise,
positive and

90
00:05:04,090 --> 00:05:05,580
negative errors might cancel
each other out,

91
00:05:05,580 --> 00:05:06,610
so we square it.

92
00:05:06,610 --> 00:05:10,880
And then this just looking
at each individual

93
00:05:10,880 --> 00:05:12,750
value of x for now.

94
00:05:12,750 --> 00:05:13,805
So this is why it's
the conditional

95
00:05:13,805 --> 00:05:15,540
mean squared error.

96
00:05:15,540 --> 00:05:17,290
So how do we calculate this?

97
00:05:17,290 --> 00:05:24,390
Well, remember that this g of
X, we said the LMS estimator

98
00:05:24,390 --> 00:05:26,900
is just a conditional
expectation.

99
00:05:26,900 --> 00:05:30,220
So it's just expectation
of Y, given X.

100
00:05:30,220 --> 00:05:32,945
Well, then if you look at this,
what this reminds you

101
00:05:32,945 --> 00:05:35,900
of, it reminds you of the
definition of what a

102
00:05:35,900 --> 00:05:37,190
conditional variance is.

103
00:05:37,190 --> 00:05:42,420
A variance is just, you take the
random variable, subtract

104
00:05:42,420 --> 00:05:45,230
its mean, square it, and take
the expectation of that.

105
00:05:45,230 --> 00:05:47,690
This is no different, except
that everything is now the

106
00:05:47,690 --> 00:05:50,060
conditional world of X.

107
00:05:50,060 --> 00:05:55,410
So this is actually the
conditional variance of Y,

108
00:05:55,410 --> 00:05:59,800
given X is little x.

109
00:05:59,800 --> 00:06:01,720
What is the conditional
variance of Y, given

110
00:06:01,720 --> 00:06:02,980
that X is little x?

111
00:06:02,980 --> 00:06:05,300
Now, we can again go back
to this plot to

112
00:06:05,300 --> 00:06:06,560
try to help us out.

113
00:06:06,560 --> 00:06:10,710
We can split this up
into regions again.

114
00:06:10,710 --> 00:06:15,130
So just take some little x as
an example and see what the

115
00:06:15,130 --> 00:06:16,120
variance is.

116
00:06:16,120 --> 00:06:22,590
So if little x is 1/2, then we
know that the conditional

117
00:06:22,590 --> 00:06:24,760
distribution of Y would
be uniform, we said,

118
00:06:24,760 --> 00:06:26,540
from 0 up to here.

119
00:06:26,540 --> 00:06:29,670
Well, that point is this
from 0 to 1/2.

120
00:06:29,670 --> 00:06:32,860
And remember, the variance of a
uniform distribution is just

121
00:06:32,860 --> 00:06:34,180
the width of the uniform
distribution

122
00:06:34,180 --> 00:06:36,150
squared, divided by 12.

123
00:06:36,150 --> 00:06:42,060
And so in this case, the width
would be 1/2 squared over 12.

124
00:06:42,060 --> 00:06:48,240
And in general, for the region
of X between 0 and 1, the

125
00:06:48,240 --> 00:06:51,960
width of the conditional
distribution of Y will always

126
00:06:51,960 --> 00:06:58,080
be X, because the width will
go from 0 to wherever X is.

127
00:06:58,080 --> 00:07:02,980
So because of that, the
conditional variance will just

128
00:07:02,980 --> 00:07:09,800
be X squared, the width squared,
over 12, when X is

129
00:07:09,800 --> 00:07:12,340
between 0 and 1.

130
00:07:12,340 --> 00:07:14,640
Now, let's think about the
other case, where X is

131
00:07:14,640 --> 00:07:16,130
between 1 and 2.

132
00:07:16,130 --> 00:07:18,900
Well, if X is between 1 and
2, we're over here.

133
00:07:18,900 --> 00:07:22,440
And now, if we take the
conditional distribution of Y,

134
00:07:22,440 --> 00:07:23,740
it's again uniform.

135
00:07:23,740 --> 00:07:26,760
But the width now, instead of
varying with Y, it's always

136
00:07:26,760 --> 00:07:30,090
going to be the same width.

137
00:07:30,090 --> 00:07:33,360
Each of these slices have the
same width, and the width goes

138
00:07:33,360 --> 00:07:39,610
from here-- this is X minus
1, and that's X.

139
00:07:39,610 --> 00:07:42,550
So if the width is always going
to be a constant of 1.

140
00:07:42,550 --> 00:07:45,310
And so this variance is
going to be 1/12.

141
00:07:45,310 --> 00:07:48,360

142
00:07:48,360 --> 00:07:51,160
And from that, we get our answer
for the conditional

143
00:07:51,160 --> 00:07:52,110
mean squared error.

144
00:07:52,110 --> 00:07:55,820
Now, part c asks us to find the
mean squared error, which

145
00:07:55,820 --> 00:07:58,105
is given by this expression.

146
00:07:58,105 --> 00:08:02,220

147
00:08:02,220 --> 00:08:04,490
And we'll see that it looks very
similar to this, which

148
00:08:04,490 --> 00:08:06,070
was the conditional mean
squared error.

149
00:08:06,070 --> 00:08:08,520
And now, given what we
know from part b,

150
00:08:08,520 --> 00:08:09,710
this is easy to calculate.

151
00:08:09,710 --> 00:08:13,330
We can just apply total
expectation, because this is

152
00:08:13,330 --> 00:08:22,130
just equal to the integral
of the conditional

153
00:08:22,130 --> 00:08:23,380
mean squared error.

154
00:08:23,380 --> 00:08:27,430

155
00:08:27,430 --> 00:08:32,880
And then we need to also
multiply this by the pf of x,

156
00:08:32,880 --> 00:08:37,260
and then integrate over X. And
that integral will should be

157
00:08:37,260 --> 00:08:43,289
from X equals 0 to 2, because
that's the only range that

158
00:08:43,289 --> 00:08:46,240
applies for x, given
this joint pdf.

159
00:08:46,240 --> 00:08:48,310
Now, in order to do this first,
though, we need to

160
00:08:48,310 --> 00:08:54,610
figure out what the
pdf of X is.

161
00:08:54,610 --> 00:08:57,190

162
00:08:57,190 --> 00:09:00,620
In order to do that, we can go
back to our original joint pdf

163
00:09:00,620 --> 00:09:02,600
of X and Y and marginalize it.

164
00:09:02,600 --> 00:09:05,810
So marginalizing, you could
think of it as taking this

165
00:09:05,810 --> 00:09:10,510
joint pdf and collapsing it onto
the x-axis so that you

166
00:09:10,510 --> 00:09:13,410
take everything and
integrate out Y.

167
00:09:13,410 --> 00:09:15,900
Now to do that, let's
do that up here.

168
00:09:15,900 --> 00:09:20,310

169
00:09:20,310 --> 00:09:25,450
We can split it up into
two sections.

170
00:09:25,450 --> 00:09:32,540
So the section of X between 0
and 1, we integrate the joint

171
00:09:32,540 --> 00:09:38,620
pdf from Y equals 0 to Y
equals X, which is this

172
00:09:38,620 --> 00:09:40,320
portion of this line.

173
00:09:40,320 --> 00:09:50,790
So we integrate Y. The joint pdf
is 2/3, and we integrate Y

174
00:09:50,790 --> 00:10:05,180
out from Y equals 0 to X. And
then for the portion of X from

175
00:10:05,180 --> 00:10:07,890
1 to 2, we again integrate
Y out.

176
00:10:07,890 --> 00:10:20,620
Now we integrate Y from
X minus 1 up to X.

177
00:10:20,620 --> 00:10:30,810
So this is X between 0 and 1,
and this is X between 1 and 2.

178
00:10:30,810 --> 00:10:33,580

179
00:10:33,580 --> 00:10:39,280
So we just do some little bit
of calculus, and we get that

180
00:10:39,280 --> 00:10:46,030
this is going to be 2/3 X when
X is between 0 and 1.

181
00:10:46,030 --> 00:10:55,270
And it's going to be 2/3 when
X is between 1 and 2.

182
00:10:55,270 --> 00:10:59,350
So now that we have what the
marginal pdf of X is, we can

183
00:10:59,350 --> 00:11:03,170
plug that into this, and plug in
what we had for b, and then

184
00:11:03,170 --> 00:11:04,910
calculate what this
actually is.

185
00:11:04,910 --> 00:11:08,810
So remember, we need to take
care of these two cases, these

186
00:11:08,810 --> 00:11:09,390
two regions--

187
00:11:09,390 --> 00:11:11,720
X between 0 and 1, and
X between 1 and 2.

188
00:11:11,720 --> 00:11:17,160
So the conditional mean squared
error for X between 0

189
00:11:17,160 --> 00:11:19,780
and 1 is X squared over 2.

190
00:11:19,780 --> 00:11:26,390
So between 0 and 1, this first
part is X squared over 12.

191
00:11:26,390 --> 00:11:32,745
The pdf of X in that same
region is 2/3 x.

192
00:11:32,745 --> 00:11:37,830
And we integrate that in the
region from x equals 0 to 1.

193
00:11:37,830 --> 00:11:41,340
And then, we also have the
second region which

194
00:11:41,340 --> 00:11:43,280
is X from 1 to 2.

195
00:11:43,280 --> 00:11:45,690
In that region, the traditional
mean squared error

196
00:11:45,690 --> 00:11:47,990
from part b is 1/12.

197
00:11:47,990 --> 00:11:55,560
The marginal pdf of X is 2/3,
and we do this integral.

198
00:11:55,560 --> 00:11:59,530
And if you just carry out some
calculus here, you'll get that

199
00:11:59,530 --> 00:12:02,525
the final answer is
equal to 5/72.

200
00:12:02,525 --> 00:12:08,660

201
00:12:08,660 --> 00:12:14,420
Now, the last part of this
question asks us, is this mean

202
00:12:14,420 --> 00:12:17,380
squared error the same thing--

203
00:12:17,380 --> 00:12:24,475
does it equal the expectation
of the conditional variance?

204
00:12:24,475 --> 00:12:27,000

205
00:12:27,000 --> 00:12:29,480
And it turns out that
yes, it does.

206
00:12:29,480 --> 00:12:36,980
And to see that, we can just
take this, and use the law of

207
00:12:36,980 --> 00:12:41,100
iterated expectations, because
iterated expectations tells us

208
00:12:41,100 --> 00:12:50,220
this is in fact equal to the
expectation of Y minus g of X

209
00:12:50,220 --> 00:12:56,580
squared, given X. That's just
applying law of iterated

210
00:12:56,580 --> 00:12:59,320
expectations.

211
00:12:59,320 --> 00:13:04,350
And then, if we look at this,
this part that's inside is

212
00:13:04,350 --> 00:13:07,200
exactly equal to the conditional
variance of Y,

213
00:13:07,200 --> 00:13:11,250
given X. And so these two
are, in fact, the same.

214
00:13:11,250 --> 00:13:15,980
In part c, we calculated what
the marginal pdf of X is, and

215
00:13:15,980 --> 00:13:18,530
it'll actually be used later
on in this problem.

216
00:13:18,530 --> 00:13:20,800
So for future reference, let's
just write it down here in

217
00:13:20,800 --> 00:13:22,140
this corner.

218
00:13:22,140 --> 00:13:24,490
Now, so far in this
problem, we've

219
00:13:24,490 --> 00:13:27,920
looked at the LMS estimator.

220
00:13:27,920 --> 00:13:29,590
And of course, there are
other estimators that

221
00:13:29,590 --> 00:13:30,550
you can use as well.

222
00:13:30,550 --> 00:13:35,240
And now in part d, let's look
at the linear LMS estimator.

223
00:13:35,240 --> 00:13:42,810
Now remember, the linear LMS
estimator is special, because

224
00:13:42,810 --> 00:13:48,040
it forces the estimator to have
a linear relationship.

225
00:13:48,040 --> 00:13:51,310
So the estimator is going to
be a linear function of X.

226
00:13:51,310 --> 00:13:53,820
Now, compare that
to what the LMS

227
00:13:53,820 --> 00:13:55,590
estimator was in this case.

228
00:13:55,590 --> 00:13:58,790
It was two linear pieces,
but there was a kink.

229
00:13:58,790 --> 00:14:03,150
And so the entire estimator
wasn't actually linear in X.

230
00:14:03,150 --> 00:14:07,360
Now, the LLMS estimator, or the
linear LMS estimator, will

231
00:14:07,360 --> 00:14:10,180
give us the linear estimator.

232
00:14:10,180 --> 00:14:11,610
It's going to be a linear
function of X.

233
00:14:11,610 --> 00:14:15,910
And we know that we have
a formula for this.

234
00:14:15,910 --> 00:14:23,930
Is the expectation of Y plus the
covariance of X and Y over

235
00:14:23,930 --> 00:14:32,770
the variance of X times X minus
expectation of X. All

236
00:14:32,770 --> 00:14:35,350
right, so in order to calculate
what this is, we

237
00:14:35,350 --> 00:14:38,810
just need to calculate
what four things are.

238
00:14:38,810 --> 00:14:43,080
Now, let's start with this last
one, the expected value

239
00:14:43,080 --> 00:14:47,900
of X. To calculate the expected
value of X, we just

240
00:14:47,900 --> 00:14:48,590
use a formula.

241
00:14:48,590 --> 00:14:52,010
And from before, we know
what the pdf of X is.

242
00:14:52,010 --> 00:14:56,910
And so we know that this
is just going to be X

243
00:14:56,910 --> 00:15:01,160
times fx of x dx.

244
00:15:01,160 --> 00:15:09,150
And in particular, this will
give us that from 0 to 1, it's

245
00:15:09,150 --> 00:15:13,200
going to be X times the
pdf of X is 2/3 X,

246
00:15:13,200 --> 00:15:16,486
so it's 2/3 X squared.

247
00:15:16,486 --> 00:15:19,720

248
00:15:19,720 --> 00:15:24,950
And from 1 to 2, it's going to
be X times the pdf of X, which

249
00:15:24,950 --> 00:15:31,200
is just 2/3, so it's 2/3 X dx.

250
00:15:31,200 --> 00:15:34,000
And when you calculate
this out, you'll get

251
00:15:34,000 --> 00:15:38,060
that is equal to 11/9.

252
00:15:38,060 --> 00:15:41,250
Now, let's calculate the
variance of X next.

253
00:15:41,250 --> 00:15:43,760
In order to calculate that,
let's use the formula that

254
00:15:43,760 --> 00:15:46,920
variance is equal to the
expectation of X squared minus

255
00:15:46,920 --> 00:15:49,220
the expectation of X
quantity squared.

256
00:15:49,220 --> 00:15:52,440
We had the expectation of X,
so let's calculate what the

257
00:15:52,440 --> 00:15:56,940
expectation of X squared is.

258
00:15:56,940 --> 00:15:59,060
Now, it's the same idea.

259
00:15:59,060 --> 00:16:06,060
Instead, we have X squared
times f of X dx.

260
00:16:06,060 --> 00:16:08,860
And we'll get the same
sort of formula.

261
00:16:08,860 --> 00:16:11,880
We'll split it up again into
two different parts from X

262
00:16:11,880 --> 00:16:14,490
equals 0 to X equals 1.

263
00:16:14,490 --> 00:16:16,260
It's going to be X squared
times pdf, so

264
00:16:16,260 --> 00:16:20,430
it's 2/3 X cubed dx.

265
00:16:20,430 --> 00:16:23,490
And then from X equals 1 to
2, it's going to be X

266
00:16:23,490 --> 00:16:25,060
squared times 2/3.

267
00:16:25,060 --> 00:16:28,770
So it's 2/3 X squared dx.

268
00:16:28,770 --> 00:16:30,890
And when we calculate this
out, we'll get that

269
00:16:30,890 --> 00:16:32,750
it's equal to 31/18.

270
00:16:32,750 --> 00:16:35,900

271
00:16:35,900 --> 00:16:38,730
From that, we know that the
variance is going to be equal

272
00:16:38,730 --> 00:16:44,680
to expectation of X squared
minus expectation of X

273
00:16:44,680 --> 00:16:46,860
quantity squared.

274
00:16:46,860 --> 00:16:52,940
Now, expectation of X
squared is 31/18.

275
00:16:52,940 --> 00:16:55,010
Expectation of X is 11/9.

276
00:16:55,010 --> 00:16:58,010
And when we calculate this,
we get that the

277
00:16:58,010 --> 00:17:06,400
variance is equal to 37/162.

278
00:17:06,400 --> 00:17:08,349
So now we have this,
and we have that.

279
00:17:08,349 --> 00:17:11,170
Let's calculate what expectation
of Y is.

280
00:17:11,170 --> 00:17:16,190
Expectation of Y, let's
calculate it using the law of

281
00:17:16,190 --> 00:17:19,310
iterated expectations.

282
00:17:19,310 --> 00:17:20,944
The law of iterated expectations
tells us that

283
00:17:20,944 --> 00:17:25,849
this is equal to the expectation
of Y conditioned

284
00:17:25,849 --> 00:17:27,700
on X.

285
00:17:27,700 --> 00:17:29,410
Now, we already know what
expectation of Y

286
00:17:29,410 --> 00:17:30,430
conditioned on X is.

287
00:17:30,430 --> 00:17:33,400
That was the LMS estimator that
we calculated earlier.

288
00:17:33,400 --> 00:17:35,770
It's this.

289
00:17:35,770 --> 00:17:39,760
So now we just need to
calculate this out,

290
00:17:39,760 --> 00:17:41,650
and we can do that.

291
00:17:41,650 --> 00:17:48,210
So we know that in the range
from X between 0 and 1, it's

292
00:17:48,210 --> 00:17:55,720
equal to 1/2 X. So in the range
from 0 to 1, it's equal

293
00:17:55,720 --> 00:18:04,130
to 1/2 X. But then, we have to
use total expectation, so we

294
00:18:04,130 --> 00:18:07,940
have to multiply by the pdf
of X in that region

295
00:18:07,940 --> 00:18:11,790
which is 2/3 X dx.

296
00:18:11,790 --> 00:18:16,900
And then in the range from
X equals 1 to 2, this

297
00:18:16,900 --> 00:18:18,840
conditional expectation
is X minus 1/2.

298
00:18:18,840 --> 00:18:22,100

299
00:18:22,100 --> 00:18:25,440
And the pdf of X in that
region is 2/3.

300
00:18:25,440 --> 00:18:30,200

301
00:18:30,200 --> 00:18:35,500
Now, when we calculate out this
value, we'll get that

302
00:18:35,500 --> 00:18:38,060
it's equal to 7/9.

303
00:18:38,060 --> 00:18:40,620

304
00:18:40,620 --> 00:18:44,770
And now, the last piece is the
covariance of X and Y.

305
00:18:44,770 --> 00:18:47,760
Remember, the covariance, we
can calculate that as the

306
00:18:47,760 --> 00:18:52,810
expectation of X times Y minus
the expectation of X times the

307
00:18:52,810 --> 00:18:56,030
expectation of Y. We already
know the expectation of X and

308
00:18:56,030 --> 00:18:58,590
the expectation of Y, so we
just need to calculate the

309
00:18:58,590 --> 00:19:03,212
expectation of X times Y,
the product of the two.

310
00:19:03,212 --> 00:19:08,740
And for that, we'll use the
definition, and we'll use the

311
00:19:08,740 --> 00:19:11,990
joint pdf that we have.

312
00:19:11,990 --> 00:19:16,610
So this is going to be a double
integral of X times Y

313
00:19:16,610 --> 00:19:17,860
times the joint pdf.

314
00:19:17,860 --> 00:19:22,360

315
00:19:22,360 --> 00:19:25,920
And the tricky part here is just
figuring out what these

316
00:19:25,920 --> 00:19:26,430
limits are.

317
00:19:26,430 --> 00:19:30,312
So we'll integrate
in this order--

318
00:19:30,312 --> 00:19:32,590
X and Y.

319
00:19:32,590 --> 00:19:35,880
Now, let's split this up.

320
00:19:35,880 --> 00:19:38,450
So let's focus on
splitting X up.

321
00:19:38,450 --> 00:19:46,990
So for X between 0 and 1, we
just need to figure out what's

322
00:19:46,990 --> 00:19:51,240
the rate right range of Y to
integrate over such that this

323
00:19:51,240 --> 00:19:52,120
is actually non-zero.

324
00:19:52,120 --> 00:19:55,160
Because remember, the
joint pdf is easy.

325
00:19:55,160 --> 00:19:56,230
It's just a constant 2/3.

326
00:19:56,230 --> 00:19:58,660
But it's only a constant
2/3 within this region.

327
00:19:58,660 --> 00:20:01,110
So the difficult part is just
figuring out what the limits

328
00:20:01,110 --> 00:20:03,100
are in order to specify
that region.

329
00:20:03,100 --> 00:20:11,410
So for X between 0 and 1, Y
has to be between 0 and X,

330
00:20:11,410 --> 00:20:18,100
because this line is Y equals
X. So we need to integrate

331
00:20:18,100 --> 00:20:19,926
from 0 to X--

332
00:20:19,926 --> 00:20:23,595
X times Y times the joint
pdf, which is 2/3.

333
00:20:23,595 --> 00:20:28,260

334
00:20:28,260 --> 00:20:34,380
And now, let's do the other
part, which is X from 1 to 2.

335
00:20:34,380 --> 00:20:38,170
Well, if X is from 1 to 2, in
order to fall into this

336
00:20:38,170 --> 00:20:43,210
region, Y has to be between X
minus 1 and X. So we integrate

337
00:20:43,210 --> 00:20:48,110
Y from X minus 1 to X. Against,
it's X times Y times

338
00:20:48,110 --> 00:20:49,640
the joint pdf, which is 2/3.

339
00:20:49,640 --> 00:20:54,070

340
00:20:54,070 --> 00:20:57,360
And now, once we have this set
up, the rest of it we can just

341
00:20:57,360 --> 00:20:58,400
do some calculus.

342
00:20:58,400 --> 00:21:00,680
And what we find is
that the final

343
00:21:00,680 --> 00:21:03,940
answer is equal to 41/36.

344
00:21:03,940 --> 00:21:07,940

345
00:21:07,940 --> 00:21:13,640
Now, what that tells us is that
the covariance of X and

346
00:21:13,640 --> 00:21:19,430
Y, which is just expectation
of X times Y, the product,

347
00:21:19,430 --> 00:21:25,080
minus expectation of X times
expectation of Y.

348
00:21:25,080 --> 00:21:27,640
We know expectation
of X times Y now.

349
00:21:27,640 --> 00:21:29,440
It's 41/36.

350
00:21:29,440 --> 00:21:31,920
Expectation of X is 11/9.

351
00:21:31,920 --> 00:21:34,640
Expectation of Y is 7/9.

352
00:21:34,640 --> 00:21:37,050
So when we substitute all of
that in, we get that this

353
00:21:37,050 --> 00:21:43,830
covariance is 61/324.

354
00:21:43,830 --> 00:21:45,910
All right, so now we have
everything we need.

355
00:21:45,910 --> 00:21:48,760
Expectation of Y is here.

356
00:21:48,760 --> 00:21:51,620
Covariance is here.

357
00:21:51,620 --> 00:21:54,360
Variance of X is here.

358
00:21:54,360 --> 00:21:57,210
And expectation of X is here.

359
00:21:57,210 --> 00:22:01,970
So let's substitute that in, and
we can figure out what the

360
00:22:01,970 --> 00:22:05,410
actual LLMS estimator is.

361
00:22:05,410 --> 00:22:09,950
So expectation of Y
we know is 7/9.

362
00:22:09,950 --> 00:22:13,570
Expectation of X is 11/9.

363
00:22:13,570 --> 00:22:16,800

364
00:22:16,800 --> 00:22:20,980
And when you divide the
covariance, which is 61/324,

365
00:22:20,980 --> 00:22:32,040
by the variance, which is
37/162, we'll get 61/74.

366
00:22:32,040 --> 00:22:36,290
And so that is the LLMS
estimator that we calculated.

367
00:22:36,290 --> 00:22:40,760
And notice that it is,
in fact, linear in X.

368
00:22:40,760 --> 00:22:44,720
So let's plot that and see
what it looks like.

369
00:22:44,720 --> 00:22:52,300
So it's going to be a
line, and it's going

370
00:22:52,300 --> 00:22:54,610
to look like this.

371
00:22:54,610 --> 00:23:00,700
So at X equals 2, it's actually
a little bit below 1

372
00:23:00,700 --> 00:23:05,360
and 1/2, which is what the
LMS estimator would be.

373
00:23:05,360 --> 00:23:13,010
At X equals 1, it's actually a
little bit above 1/2, which is

374
00:23:13,010 --> 00:23:15,150
what the LMS estimator
would be.

375
00:23:15,150 --> 00:23:21,120
And then it crosses 0 around
roughly 1/4, and it drops

376
00:23:21,120 --> 00:23:22,160
actually below 0.

377
00:23:22,160 --> 00:23:26,040
So if we connect the dots,
it's going to look

378
00:23:26,040 --> 00:23:28,820
something like this.

379
00:23:28,820 --> 00:23:32,230
So notice that it's actually not
too far away from the LMS

380
00:23:32,230 --> 00:23:33,850
estimator here.

381
00:23:33,850 --> 00:23:38,340
But it doesn't have the kink
because it is a line.

382
00:23:38,340 --> 00:23:41,700
And note also that it actually
drops below.

383
00:23:41,700 --> 00:23:45,070
So when X is very small, you
actually estimate negative

384
00:23:45,070 --> 00:23:51,880
values of Y, which is actually
impossible, given the joint

385
00:23:51,880 --> 00:23:53,410
pdf distribution that
we're given.

386
00:23:53,410 --> 00:23:57,950
And that is sometimes a feature
or artifact of the

387
00:23:57,950 --> 00:24:00,530
linear LMS estimator, that
you'll get values that don't

388
00:24:00,530 --> 00:24:03,820
necessarily seem
to make sense.

389
00:24:03,820 --> 00:24:06,720
So now that we've calculated
the linear LMS estimator in

390
00:24:06,720 --> 00:24:10,740
part d, which is this, and the
LMS estimator in part a, which

391
00:24:10,740 --> 00:24:14,420
is this, we've also compared
them visually.

392
00:24:14,420 --> 00:24:17,750
The linear LMS estimator
is the one in pink,

393
00:24:17,750 --> 00:24:18,680
the straight line.

394
00:24:18,680 --> 00:24:22,080
And the LMS estimator is the
one in black with the kink.

395
00:24:22,080 --> 00:24:24,170
It's an interesting question
to now ask, which

396
00:24:24,170 --> 00:24:25,480
one of these is better?

397
00:24:25,480 --> 00:24:27,390
And in order to judge that, we
need to come up with some sort

398
00:24:27,390 --> 00:24:30,060
of criterion to compare
the two with.

399
00:24:30,060 --> 00:24:34,280
And the one that we're going
to look at in part e is the

400
00:24:34,280 --> 00:24:35,040
mean squared error.

401
00:24:35,040 --> 00:24:37,420
Which one gives the lower
mean squared error.

402
00:24:37,420 --> 00:24:44,530
And so specifically, we're going
to ask ourselves which

403
00:24:44,530 --> 00:24:49,075
of these two estimators
gives us the smaller

404
00:24:49,075 --> 00:24:50,325
mean squared error?

405
00:24:50,325 --> 00:24:54,000

406
00:24:54,000 --> 00:24:57,500
Is it the linear LMS estimator
given by l of X?

407
00:24:57,500 --> 00:25:02,180
Or is it the LMS estimator,
given by g of X?

408
00:25:02,180 --> 00:25:08,400
Now, we know that the LMS
estimator is the one that

409
00:25:08,400 --> 00:25:09,560
actually minimizes this.

410
00:25:09,560 --> 00:25:13,600
The LMS estimator is designed
to minimize the

411
00:25:13,600 --> 00:25:14,140
mean squared error.

412
00:25:14,140 --> 00:25:18,030
And so we know that given any
estimator of X, this one will

413
00:25:18,030 --> 00:25:20,470
have the smallest mean
squared error.

414
00:25:20,470 --> 00:25:24,430
And so the linear LMS estimator,
its mean squared

415
00:25:24,430 --> 00:25:29,480
error has to be at least as
large as the LMS estimators.

416
00:25:29,480 --> 00:25:32,220
And the last part of the
question now asks us to look

417
00:25:32,220 --> 00:25:37,680
at a third type of estimator,
which is the MEP estimator.

418
00:25:37,680 --> 00:25:41,390
Now, we want to ask, why is it
that we haven't been using the

419
00:25:41,390 --> 00:25:43,260
MEP estimator in this problem?

420
00:25:43,260 --> 00:25:45,360
Well, remember what the
MEP estimator does.

421
00:25:45,360 --> 00:25:50,730
In this case, what we would
do is it would take the

422
00:25:50,730 --> 00:25:54,030
conditional distribution ratio
of Y given any value of X. And

423
00:25:54,030 --> 00:25:59,200
then it would pick the value
of Y that gives the highest

424
00:25:59,200 --> 00:26:01,780
value in the conditional
distribution.

425
00:26:01,780 --> 00:26:04,190
And that would be the
MEP estimate of Y.

426
00:26:04,190 --> 00:26:08,060
But the problem in this case is
that if you take any slice

427
00:26:08,060 --> 00:26:11,700
here, so a condition on any
value of X, any of these

428
00:26:11,700 --> 00:26:18,240
slices, if you just take this
out and look at it, it's going

429
00:26:18,240 --> 00:26:20,040
to be uniform.

430
00:26:20,040 --> 00:26:28,910
This is what the conditional
distribution of Y given X is.

431
00:26:28,910 --> 00:26:33,662
It's going to be uniform between
0 and X. Now, what the

432
00:26:33,662 --> 00:26:37,610
MEP rule tells us is we're going
to pick the value of Y

433
00:26:37,610 --> 00:26:43,310
that gives us the highest point
in this conditional

434
00:26:43,310 --> 00:26:44,190
distribution.

435
00:26:44,190 --> 00:26:47,380
You can think of it as a
posterior distribution.

436
00:26:47,380 --> 00:26:49,370
Now, what's the problem here?

437
00:26:49,370 --> 00:26:52,860
Well, every single point gives
us exactly the same value for

438
00:26:52,860 --> 00:26:54,140
this conditional distribution.

439
00:26:54,140 --> 00:26:56,780
And so there's no
unique MEP rule.

440
00:26:56,780 --> 00:27:02,790
Every single value of Y has
just the same conditional

441
00:27:02,790 --> 00:27:03,510
distribution.

442
00:27:03,510 --> 00:27:08,260
So there's no sensible way of
choosing a value based on the

443
00:27:08,260 --> 00:27:10,490
MEP rule in this case.

444
00:27:10,490 --> 00:27:16,090
But compare that with the LMS
estimator, which is just get

445
00:27:16,090 --> 00:27:17,560
conditional expectation.

446
00:27:17,560 --> 00:27:19,600
In that case, we can always
find a conditional

447
00:27:19,600 --> 00:27:20,580
expectation.

448
00:27:20,580 --> 00:27:23,640
In this case, the conditional
expectation is the midpoint,

449
00:27:23,640 --> 00:27:27,690
which is X/2, just as
had found in part a.

450
00:27:27,690 --> 00:27:31,640
OK, so in this problem, we
reviewed a bunch of different

451
00:27:31,640 --> 00:27:35,823
ideas in terms of inference, and
we took a joint pdf of X

452
00:27:35,823 --> 00:27:39,600
and Y, and we used that to
calculate the LMS estimator,

453
00:27:39,600 --> 00:27:41,200
the linear LMS estimator.

454
00:27:41,200 --> 00:27:44,280
We compared the two, and then we
also looked at why in this

455
00:27:44,280 --> 00:27:47,670
case, the MEP estimator doesn't
really make sense.

456
00:27:47,670 --> 00:27:49,330
All right, so I hope that
was helpful, and we'll

457
00:27:49,330 --> 00:27:50,580
see you next time.

458
00:27:50,580 --> 00:27:51,430