1
00:00:00,000 --> 00:00:02,830

2
00:00:02,830 --> 00:00:04,030
Welcome back.

3
00:00:04,030 --> 00:00:07,370
So now we're going to finish
the rest of this problem.

4
00:00:07,370 --> 00:00:12,730
For part e, we've calculated
what the map and LMS

5
00:00:12,730 --> 00:00:13,840
estimators are.

6
00:00:13,840 --> 00:00:16,600
And now we're going to calculate
what the conditional

7
00:00:16,600 --> 00:00:17,750
mean squared error is.

8
00:00:17,750 --> 00:00:23,130
So it's a way to measure how
good these estimators are.

9
00:00:23,130 --> 00:00:25,740
So let's start out
generically.

10
00:00:25,740 --> 00:00:32,530
For any estimator theta hat,
the conditional MSE is--

11
00:00:32,530 --> 00:00:33,940
conditional mean
squared error--

12
00:00:33,940 --> 00:00:37,810

13
00:00:37,810 --> 00:00:39,060
is equal to this.

14
00:00:39,060 --> 00:00:42,610

15
00:00:42,610 --> 00:00:50,750
It's the estimator minus the
actual value squared

16
00:00:50,750 --> 00:00:55,450
conditioned on X being equal
to some little x.

17
00:00:55,450 --> 00:00:56,870
So the mean squared error.

18
00:00:56,870 --> 00:01:00,470
So you take the error, which is
the difference between your

19
00:01:00,470 --> 00:01:02,530
estimator and the true
value, square it, and

20
00:01:02,530 --> 00:01:03,570
then take the mean.

21
00:01:03,570 --> 00:01:07,190
And it's conditioned on the
actual value of what x is.

22
00:01:07,190 --> 00:01:11,260
Or, conditioned on the
data that you get is.

23
00:01:11,260 --> 00:01:15,200
So to calculate this, we use
our standard definition of

24
00:01:15,200 --> 00:01:16,730
what conditional expectation
would be.

25
00:01:16,730 --> 00:01:23,850
So it's theta hat minus
theta squared.

26
00:01:23,850 --> 00:01:33,780
And we weight that by the
appropriate conditional PDF,

27
00:01:33,780 --> 00:01:36,460
which in this case would
be the posterior.

28
00:01:36,460 --> 00:01:41,120
And we integrate this from x--

29
00:01:41,120 --> 00:01:44,450
from theta equals x
to theta equals 1.

30
00:01:44,450 --> 00:01:50,160
Now, we can go through some
algebra and this will tell us

31
00:01:50,160 --> 00:01:55,460
that this is theta hat squared
minus 2 theta hat theta minus

32
00:01:55,460 --> 00:01:57,410
plus theta squared.

33
00:01:57,410 --> 00:02:02,210
And this posterior we know from
before is 1 over theta

34
00:02:02,210 --> 00:02:08,669
times absolute value
of log x d theta.

35
00:02:08,669 --> 00:02:15,540
And when we do out this
integral, it's going to be--

36
00:02:15,540 --> 00:02:18,070
we can split up in to three
different terms.

37
00:02:18,070 --> 00:02:21,585
So there's theta hat squared
times this and

38
00:02:21,585 --> 00:02:22,280
you integrate it.

39
00:02:22,280 --> 00:02:26,210
But in fact, this is just
a conditional density.

40
00:02:26,210 --> 00:02:30,990
When you integrate it from x to
1, this will just integrate

41
00:02:30,990 --> 00:02:34,940
up to 1 because it is
a valid density.

42
00:02:34,940 --> 00:02:39,410
So the first term is just
theta hat squared.

43
00:02:39,410 --> 00:02:43,360
Now, the second term is you can
pull out of 2 theta hat

44
00:02:43,360 --> 00:02:51,320
and integrate theta times 1
over theta times absolute

45
00:02:51,320 --> 00:02:59,500
value of log of x d
theta from x to 1.

46
00:02:59,500 --> 00:03:04,430
And then the last one is
integral of theta squared 1

47
00:03:04,430 --> 00:03:13,130
over theta absolute value of
log x d theta from x to 1.

48
00:03:13,130 --> 00:03:16,660
OK, so we can do some more--

49
00:03:16,660 --> 00:03:24,040
with some more calculus, we get
a final answer is this.

50
00:03:24,040 --> 00:03:28,530
So this will integrate to
1 minus x over absolute

51
00:03:28,530 --> 00:03:31,600
value of log x.

52
00:03:31,600 --> 00:03:38,900
And this will integrate to 1
minus x squared over 2 times

53
00:03:38,900 --> 00:03:43,280
absolute value of log x.

54
00:03:43,280 --> 00:03:48,460
So this tells us for any generic
estimate theta hat,

55
00:03:48,460 --> 00:03:51,100
this would be what the
conditional mean

56
00:03:51,100 --> 00:03:52,590
squared error would be.

57
00:03:52,590 --> 00:03:58,150
Now, let's calculate what it
actually is for the specific

58
00:03:58,150 --> 00:04:00,030
estimates that we actually
came up with.

59
00:04:00,030 --> 00:04:04,120
So for the MAP rule, the
estimate of theta hat is just

60
00:04:04,120 --> 00:04:06,220
equal to x.

61
00:04:06,220 --> 00:04:09,180
So when we plug that into
this, we get that the

62
00:04:09,180 --> 00:04:19,430
conditional MSE is just equal to
x squared minus 2x 1 minus

63
00:04:19,430 --> 00:04:30,920
x absolute value of log x plus 1
minus x squared over 2 times

64
00:04:30,920 --> 00:04:35,290
absolute value of log of x.

65
00:04:35,290 --> 00:04:41,880
And for the LMS estimate,
remember this was equal to--

66
00:04:41,880 --> 00:04:48,500
theta hat was 1 minus x over
absolute value of log x.

67
00:04:48,500 --> 00:04:52,080
And so when you plug this
particular theta hat into this

68
00:04:52,080 --> 00:04:54,570
formula, what you get is that
the conditional mean squared

69
00:04:54,570 --> 00:05:01,580
error is equal to 1 minus x
squared over 2 times absolute

70
00:05:01,580 --> 00:05:07,990
value of log of x minus
1 minus x over log

71
00:05:07,990 --> 00:05:12,480
of x quantity squared.

72
00:05:12,480 --> 00:05:17,830
So these two expressions tells
us what the mean squared error

73
00:05:17,830 --> 00:05:21,290
is for the MAP rule
and the LMS rule.

74
00:05:21,290 --> 00:05:23,740
And it's kind of hard to
actually interpret exactly

75
00:05:23,740 --> 00:05:27,130
which one is better based on
just these expressions.

76
00:05:27,130 --> 00:05:33,980
So it's helpful to plot out
what the conditional mean

77
00:05:33,980 --> 00:05:36,440
squared error is.

78
00:05:36,440 --> 00:05:38,130
So we're plotting for x.

79
00:05:38,130 --> 00:05:41,030
For each possible actual
data that we observe--

80
00:05:41,030 --> 00:05:42,845
data point that we observe,
what is the

81
00:05:42,845 --> 00:05:44,100
mean squared error?

82
00:05:44,100 --> 00:05:48,980
So let's do the MAP
rule first.

83
00:05:48,980 --> 00:05:51,000
The MAP rule would look
something like this.

84
00:05:51,000 --> 00:05:57,070

85
00:05:57,070 --> 00:06:02,810
And it turns out that the LMS
rule is better, and it will

86
00:06:02,810 --> 00:06:11,980
look like this dotted line
here on the bottom.

87
00:06:11,980 --> 00:06:16,850
And so it turns out that if your
metric for how good your

88
00:06:16,850 --> 00:06:21,220
estimate is is the conditional
mean squared error, then LMS

89
00:06:21,220 --> 00:06:22,540
is better than MAP.

90
00:06:22,540 --> 00:06:28,650
And this is true because LMS
is actually designed to

91
00:06:28,650 --> 00:06:31,790
actually minimize what this
mean squared error is.

92
00:06:31,790 --> 00:06:35,865
And so in this case, the LMS
estimator should have a better

93
00:06:35,865 --> 00:06:40,680
mean squared error than
the map estimator.

94
00:06:40,680 --> 00:06:47,440
OK, now the last part of the
problem, we calculate one more

95
00:06:47,440 --> 00:06:53,360
type of estimator, which is
the linear LMS estimator.

96
00:06:53,360 --> 00:06:59,410
So notice that the LMS estimator
was this one.

97
00:06:59,410 --> 00:07:02,820
It was 1 minus x over absolute
value of log of x.

98
00:07:02,820 --> 00:07:08,160
And this is not linear in x,
which means sometimes it's

99
00:07:08,160 --> 00:07:10,560
difficult to calculate.

100
00:07:10,560 --> 00:07:16,160
And so what we do is we tried to
come up with a linear form

101
00:07:16,160 --> 00:07:19,800
of this, something that is like
ax plus b, where a and b

102
00:07:19,800 --> 00:07:23,070
are some constant numbers.

103
00:07:23,070 --> 00:07:27,480
But that also does well in terms
of having a small mean

104
00:07:27,480 --> 00:07:28,780
squared error.

105
00:07:28,780 --> 00:07:33,860
And so we know from the class
that in order to calculate the

106
00:07:33,860 --> 00:07:42,290
linear LMS, the linear LMS we
know we just need to calculate

107
00:07:42,290 --> 00:07:43,930
a few different parts.

108
00:07:43,930 --> 00:07:48,870
So it's equal to the expectation
of the parameter

109
00:07:48,870 --> 00:07:58,910
plus the covariance of theta and
x over the variance of x

110
00:07:58,910 --> 00:08:02,850
times x minus expectation
of x.

111
00:08:02,850 --> 00:08:06,980

112
00:08:06,980 --> 00:08:08,560
Now, in order to do this,
we just need to

113
00:08:08,560 --> 00:08:09,730
calculate four things.

114
00:08:09,730 --> 00:08:12,530
We need the expectation of
theta, the covariance, the

115
00:08:12,530 --> 00:08:15,725
variance, and the expectation
of x.

116
00:08:15,725 --> 00:08:18,840
OK, so let's calculate what
these things are.

117
00:08:18,840 --> 00:08:22,120
Expectation of theta.

118
00:08:22,120 --> 00:08:24,350
We know that theta is uniformly
distributed

119
00:08:24,350 --> 00:08:25,490
between 0 and 1.

120
00:08:25,490 --> 00:08:28,710
And so the expectation
of theta is the

121
00:08:28,710 --> 00:08:29,560
easiest one to calculate.

122
00:08:29,560 --> 00:08:31,610
It's just 1/2.

123
00:08:31,610 --> 00:08:35,590
What about the expectation
of x?

124
00:08:35,590 --> 00:08:38,280
Well, expectation of x is a
little bit more complicated.

125
00:08:38,280 --> 00:08:41,909
But remember, like in previous
problems, it's helpful when

126
00:08:41,909 --> 00:08:47,090
you have a hierarchy of
randomness to try to use the

127
00:08:47,090 --> 00:08:49,720
law of iterated expectations.

128
00:08:49,720 --> 00:08:54,960
So the delay, which
is x, is random.

129
00:08:54,960 --> 00:08:57,270
But it's randomness depends
on the actual

130
00:08:57,270 --> 00:08:59,790
distribution, which is theta.

131
00:08:59,790 --> 00:09:01,100
Which itself is random.

132
00:09:01,100 --> 00:09:05,420
And so let's try to condition
on theta and see

133
00:09:05,420 --> 00:09:06,940
if that helps us.

134
00:09:06,940 --> 00:09:10,590
OK, so if we knew what theta
was, then what is the

135
00:09:10,590 --> 00:09:12,040
expectation of x?

136
00:09:12,040 --> 00:09:14,540
Well, we know that given
theta, x is uniformly

137
00:09:14,540 --> 00:09:16,300
distributed between
0 and theta.

138
00:09:16,300 --> 00:09:21,230
And so the mean would be
just theta over 2.

139
00:09:21,230 --> 00:09:26,580
And so this would just be
expectation of theta over 2.

140
00:09:26,580 --> 00:09:30,490
And we know this is just 1/2
times the expectation of

141
00:09:30,490 --> 00:09:32,150
theta, which is 1/2.

142
00:09:32,150 --> 00:09:33,930
So this is just 1/4.

143
00:09:33,930 --> 00:09:36,510

144
00:09:36,510 --> 00:09:41,690
Now, let's calculate
the variance of x.

145
00:09:41,690 --> 00:09:47,180
The variance of x takes some
more work because we need to

146
00:09:47,180 --> 00:09:51,000
use the law of total variance,
which is this.

147
00:09:51,000 --> 00:09:56,140
That the variance of theta is
equal to the expectation of

148
00:09:56,140 --> 00:09:59,630
the conditional variance plus
the variance of the

149
00:09:59,630 --> 00:10:01,460
conditional expectation.

150
00:10:01,460 --> 00:10:05,510

151
00:10:05,510 --> 00:10:07,540
Let's see if we can figure
out what these

152
00:10:07,540 --> 00:10:08,270
different parts are.

153
00:10:08,270 --> 00:10:12,015
What is the conditional variance
of x given theta?

154
00:10:12,015 --> 00:10:16,070
Well, given theta, x we know
is uniformly distributed

155
00:10:16,070 --> 00:10:17,740
between 0 and theta.

156
00:10:17,740 --> 00:10:24,820
And remember for uniform
distribution of width c, the

157
00:10:24,820 --> 00:10:28,680
variance of that uniform
distribution is just c

158
00:10:28,680 --> 00:10:31,330
squared over 12.

159
00:10:31,330 --> 00:10:33,960
And so in this case, what is
the width of this uniform

160
00:10:33,960 --> 00:10:34,550
distribution?

161
00:10:34,550 --> 00:10:36,730
Well, it's uniformly distributed
between 0 and

162
00:10:36,730 --> 00:10:38,820
theta, so the width is theta.

163
00:10:38,820 --> 00:10:42,535
So this variance should be
theta squared over 12.

164
00:10:42,535 --> 00:10:45,190

165
00:10:45,190 --> 00:10:48,540
OK, what about the expectation
of x given theta?

166
00:10:48,540 --> 00:10:50,830
Well, we already argued earlier
that the expectation

167
00:10:50,830 --> 00:10:56,530
of x given theta is
just theta over 2.

168
00:10:56,530 --> 00:10:58,150
So now let's fill in the rest.

169
00:10:58,150 --> 00:11:02,390
What's the expectation of
theta squared over 12?

170
00:11:02,390 --> 00:11:07,760
Well, that takes a little bit
more work because this is

171
00:11:07,760 --> 00:11:09,810
just-- you can think
of it as 1/12.

172
00:11:09,810 --> 00:11:11,160
You could pull the 1/12
out times the

173
00:11:11,160 --> 00:11:13,140
expectation of theta squared.

174
00:11:13,140 --> 00:11:16,340
Well, the expectation of theta
squared we can calculate from

175
00:11:16,340 --> 00:11:21,300
the variance of theta plus
the expectation of

176
00:11:21,300 --> 00:11:24,730
theta quantity squared.

177
00:11:24,730 --> 00:11:27,690
Because that is just the
definition of variance.

178
00:11:27,690 --> 00:11:31,920
Variance is equal to expectation
of theta squared

179
00:11:31,920 --> 00:11:35,440
minus expectation of theta
quantity squared.

180
00:11:35,440 --> 00:11:37,560
So we've just reversed
the formula.

181
00:11:37,560 --> 00:11:41,180
Now, the second half is the
variance of theta over 2.

182
00:11:41,180 --> 00:11:43,000
Well, remember when you pull
out a constant from a

183
00:11:43,000 --> 00:11:44,920
variance, you have
to square it.

184
00:11:44,920 --> 00:11:49,370
So this is just equal to 1/4
times the variance of theta.

185
00:11:49,370 --> 00:11:53,040

186
00:11:53,040 --> 00:11:54,740
Well, what is the variance
of theta?

187
00:11:54,740 --> 00:11:58,460
The variance of theta is
the variance of uniform

188
00:11:58,460 --> 00:11:59,600
between 0 and 1.

189
00:11:59,600 --> 00:12:00,810
So the width is 1.

190
00:12:00,810 --> 00:12:03,100
So you get 1 squared over 12.

191
00:12:03,100 --> 00:12:05,760
And the variance is 1/12.

192
00:12:05,760 --> 00:12:06,860
What is the mean of theta?

193
00:12:06,860 --> 00:12:08,990
It's 1/2 when you square
that, you get 1/4.

194
00:12:08,990 --> 00:12:12,350

195
00:12:12,350 --> 00:12:13,900
Finally for here, the
variance of theta

196
00:12:13,900 --> 00:12:15,240
like we said, is 1/12.

197
00:12:15,240 --> 00:12:18,060
So you get 1/12.

198
00:12:18,060 --> 00:12:21,050
And now, when you combine all
these, you get that the

199
00:12:21,050 --> 00:12:29,780
variance ends up being 7/144.

200
00:12:29,780 --> 00:12:31,340
Now we have almost everything.

201
00:12:31,340 --> 00:12:32,520
The last thing we need
to calculate is

202
00:12:32,520 --> 00:12:34,030
this covariance term.

203
00:12:34,030 --> 00:12:39,640
What is the covariance
of theta and x?

204
00:12:39,640 --> 00:12:43,480
Well, the covariance we know is
just the expectation of the

205
00:12:43,480 --> 00:12:48,570
product of theta and x minus
the product of the

206
00:12:48,570 --> 00:12:50,420
expectations.

207
00:12:50,420 --> 00:12:55,980
So the expectation of x times
the expectation of theta.

208
00:12:55,980 --> 00:12:58,490
All right, so we already know
what expectation of theta is.

209
00:12:58,490 --> 00:12:59,180
That's 1/2.

210
00:12:59,180 --> 00:13:01,030
And expectation of x was 1/4.

211
00:13:01,030 --> 00:13:03,020
So the only thing that we don't
know is expectation of

212
00:13:03,020 --> 00:13:05,230
the product of the two.

213
00:13:05,230 --> 00:13:11,230
So once again, let's try to
use iterated expectations.

214
00:13:11,230 --> 00:13:20,010
So let's calculate this as
the expectation of this

215
00:13:20,010 --> 00:13:22,730
conditional expectation.

216
00:13:22,730 --> 00:13:25,180
So we, again, condition
on theta.

217
00:13:25,180 --> 00:13:29,590
And minus the expectation
of theta is 1/2.

218
00:13:29,590 --> 00:13:34,100
Times 1/4, which is the
expectation of x.

219
00:13:34,100 --> 00:13:36,485
Now, what is this conditional
expectation?

220
00:13:36,485 --> 00:13:39,730

221
00:13:39,730 --> 00:13:42,490
Well, the expectation
of theta--

222
00:13:42,490 --> 00:13:46,280
if you know what theta is, then
the expectation of theta

223
00:13:46,280 --> 00:13:47,450
is just theta.

224
00:13:47,450 --> 00:13:49,770
You already know what it is, so
you know for sure that the

225
00:13:49,770 --> 00:13:52,000
expectation is just
equal to theta.

226
00:13:52,000 --> 00:13:55,430
And what is the expectation
of x given theta?

227
00:13:55,430 --> 00:13:57,700
Well, the expectation of x given
theta we already said

228
00:13:57,700 --> 00:13:59,210
was theta over 2.

229
00:13:59,210 --> 00:14:03,220
So what you get is this entire
expression is just going to be

230
00:14:03,220 --> 00:14:10,070
equal to theta times theta
over 2, or expectation of

231
00:14:10,070 --> 00:14:15,120
theta squared over
2 minus 1/8.

232
00:14:15,120 --> 00:14:18,700
Now, what is the expectation
of theta squared over 2?

233
00:14:18,700 --> 00:14:20,650
Well, we know that--

234
00:14:20,650 --> 00:14:23,170
we already calculated out
what expectation of

235
00:14:23,170 --> 00:14:25,440
theta squared is.

236
00:14:25,440 --> 00:14:30,360
So we know that expectation
of theta squared

237
00:14:30,360 --> 00:14:33,070
is 1/12 plus 1/4.

238
00:14:33,070 --> 00:14:38,140
So what we get is we need a 1/2
times 1/12 plus 1/4, which

239
00:14:38,140 --> 00:14:42,950
is 1/3 minus 1/8.

240
00:14:42,950 --> 00:14:48,110
So the answer is 1/6 minus
1/8, which is 1/24.

241
00:14:48,110 --> 00:14:54,410

242
00:14:54,410 --> 00:14:57,630
Now, let's actually plug
this in and figure out

243
00:14:57,630 --> 00:14:59,380
what this value is.

244
00:14:59,380 --> 00:15:01,860
So when you get everything--

245
00:15:01,860 --> 00:15:04,480

246
00:15:04,480 --> 00:15:11,750
when you combine everything,
you get that the

247
00:15:11,750 --> 00:15:13,370
LMS estimator is--

248
00:15:13,370 --> 00:15:18,230
the linear LMS estimator
is going to be--

249
00:15:18,230 --> 00:15:22,600

250
00:15:22,600 --> 00:15:23,860
expectation of theta is 1/2.

251
00:15:23,860 --> 00:15:27,360

252
00:15:27,360 --> 00:15:31,090
The covariance is 1/24.

253
00:15:31,090 --> 00:15:33,790
The variance is 7/144.

254
00:15:33,790 --> 00:15:43,830
And when you divide that, it's
equal to 6/7 times x minus 1/4

255
00:15:43,830 --> 00:15:46,940
because expectation
of x is 1/4.

256
00:15:46,940 --> 00:15:50,610
And you can simplify this a
little bit and get that this

257
00:15:50,610 --> 00:15:55,160
is equal to 6/7 times
x plus 2/7.

258
00:15:55,160 --> 00:15:58,580

259
00:15:58,580 --> 00:16:02,760
So now we have three different
types of estimators.

260
00:16:02,760 --> 00:16:05,670
The map estimator,
which is this.

261
00:16:05,670 --> 00:16:06,900
Notice that it's kind
of complicated.

262
00:16:06,900 --> 00:16:07,950
You have x squared terms.

263
00:16:07,950 --> 00:16:10,150
You have more x squared terms.

264
00:16:10,150 --> 00:16:13,980
And you have absolute
value of log of x.

265
00:16:13,980 --> 00:16:17,590
And then you have the LMS, which
is, again, nonlinear.

266
00:16:17,590 --> 00:16:20,010
And now you have something
that looks very simple--

267
00:16:20,010 --> 00:16:20,490
much simpler.

268
00:16:20,490 --> 00:16:24,420
It's just 6/7 x plus 2/7.

269
00:16:24,420 --> 00:16:27,560
And that is the linear
LMS estimator.

270
00:16:27,560 --> 00:16:34,200
And it turns out that you can,
again, plot these to see what

271
00:16:34,200 --> 00:16:35,270
this one looks like.

272
00:16:35,270 --> 00:16:44,220
So here is our original plot
of x and theta hat.

273
00:16:44,220 --> 00:16:45,470
So the map estimator--

274
00:16:45,470 --> 00:16:50,230

275
00:16:50,230 --> 00:16:53,500
sorry, the map estimator was
just theta hat equals x.

276
00:16:53,500 --> 00:16:57,600
This was the mean squared error
of the map estimator.

277
00:16:57,600 --> 00:17:02,490
So the map estimator is just
this diagonal straight line.

278
00:17:02,490 --> 00:17:05,950
The LMS estimator looked
like this.

279
00:17:05,950 --> 00:17:10,079
And it turns out that the linear
LMS estimator will look

280
00:17:10,079 --> 00:17:17,520
something like this.

281
00:17:17,520 --> 00:17:20,800
So it is fairly close to
the LMS estimator, but

282
00:17:20,800 --> 00:17:22,180
not quite the same.

283
00:17:22,180 --> 00:17:25,740
And note, especially that
depending on what x is, if x

284
00:17:25,740 --> 00:17:28,380
is fairly close to the 1, you
might actually get an estimate

285
00:17:28,380 --> 00:17:31,010
of theta that's greater
than 1.

286
00:17:31,010 --> 00:17:34,040
So for example, if you observe
that Julian is actually an

287
00:17:34,040 --> 00:17:37,610
hour late, then x is 1 and your
estimate of theta from

288
00:17:37,610 --> 00:17:40,310
the linear LMS estimator
would be 8/7, which is

289
00:17:40,310 --> 00:17:43,400
greater than 1.

290
00:17:43,400 --> 00:17:49,240
That doesn't quite make sense
because we know that theta is

291
00:17:49,240 --> 00:17:51,230
bounded to be only
between 0 and 1.

292
00:17:51,230 --> 00:17:54,350
So you shouldn't get an estimate
of theta that's

293
00:17:54,350 --> 00:17:55,420
greater than 1.

294
00:17:55,420 --> 00:17:58,670
And that's one of the side
effects of having the linear

295
00:17:58,670 --> 00:17:59,480
LMS estimator.

296
00:17:59,480 --> 00:18:02,942
So that sometimes you will have
an estimator that doesn't

297
00:18:02,942 --> 00:18:05,430
quite make sense.

298
00:18:05,430 --> 00:18:09,590
But what you get instead when
sacrificing that is you get a

299
00:18:09,590 --> 00:18:13,540
simple form of the estimator
that's linear.

300
00:18:13,540 --> 00:18:16,070
And now, let's actually
consider what

301
00:18:16,070 --> 00:18:18,530
the performance is.

302
00:18:18,530 --> 00:18:22,790
And it turns out that the
performance in terms of the

303
00:18:22,790 --> 00:18:26,880
conditional mean squared error
is actually fairly close to

304
00:18:26,880 --> 00:18:28,570
the LMS estimator.

305
00:18:28,570 --> 00:18:30,620
So it looks like this.

306
00:18:30,620 --> 00:18:33,810
Pretty close, pretty close,
until you get close to 1.

307
00:18:33,810 --> 00:18:36,080
In which case, it does worse.

308
00:18:36,080 --> 00:18:39,810
And it does worse precisely
because it will come up with

309
00:18:39,810 --> 00:18:42,150
estimates of theta which
are greater than 1,

310
00:18:42,150 --> 00:18:44,390
which are too large.

311
00:18:44,390 --> 00:18:48,410
But otherwise, it does pretty
well with a estimator that is

312
00:18:48,410 --> 00:18:52,800
much simpler in form than
the LMS estimator.

313
00:18:52,800 --> 00:18:56,940
So in this problem, which had
several parts, we actually

314
00:18:56,940 --> 00:18:59,740
went through, basically, all
the different concepts and

315
00:18:59,740 --> 00:19:03,340
tools within Chapter Eight
for Bayesian inference.

316
00:19:03,340 --> 00:19:08,750
We talked about the prior, the
posterior, calculating the

317
00:19:08,750 --> 00:19:10,060
posterior using the
Bayes' rule.

318
00:19:10,060 --> 00:19:12,170
We calculated the
MAP estimator.

319
00:19:12,170 --> 00:19:14,630
We calculated the
LMS estimator.

320
00:19:14,630 --> 00:19:17,430
From those, we calculated what
the mean squared error for

321
00:19:17,430 --> 00:19:19,790
each one of those and
compared the two.

322
00:19:19,790 --> 00:19:23,310
And then, we looked at the
linear LMS estimator as

323
00:19:23,310 --> 00:19:26,930
another example and calculated
what that estimator is, along

324
00:19:26,930 --> 00:19:30,560
with the mean squared error
for that and compared all

325
00:19:30,560 --> 00:19:32,140
three of these.

326
00:19:32,140 --> 00:19:34,800
So I hope that was a good review
problem for Chapter

327
00:19:34,800 --> 00:19:36,320
Eight, and we'll see
you next time.

328
00:19:36,320 --> 00:19:37,570