1
00:00:01,910 --> 00:00:04,430
We now continue with
our example and turn

2
00:00:04,430 --> 00:00:07,270
to the performance
evaluation question.

3
00:00:07,270 --> 00:00:09,320
As you recall, we
have a Theta that

4
00:00:09,320 --> 00:00:11,590
has a certain
prior distribution.

5
00:00:11,590 --> 00:00:14,420
We're given a model
for the observations.

6
00:00:14,420 --> 00:00:18,140
We came up with the joint
distribution for X and Theta,

7
00:00:18,140 --> 00:00:21,260
which was uniform on
this particular shape,

8
00:00:21,260 --> 00:00:23,710
and we found that the
least mean squares

9
00:00:23,710 --> 00:00:26,100
estimator, namely the
conditional expectation

10
00:00:26,100 --> 00:00:29,020
of Theta given any
particular value of X,

11
00:00:29,020 --> 00:00:33,360
is given by this particular
piecewise linear function.

12
00:00:33,360 --> 00:00:36,780
Now, let us look at the
performance of this estimator.

13
00:00:36,780 --> 00:00:40,970
We judge the performance given
any particular value of X

14
00:00:40,970 --> 00:00:45,300
by looking at the corresponding
mean squared error, which

15
00:00:45,300 --> 00:00:50,850
is the square of the distance
between the unknown parameter

16
00:00:50,850 --> 00:00:54,070
and the estimate with
which we came up.

17
00:00:54,070 --> 00:00:56,640
And as we have discussed,
this is the same

18
00:00:56,640 --> 00:00:59,860
as the variance of Theta but
in the conditional universe

19
00:00:59,860 --> 00:01:02,123
where X has been observed.

20
00:01:02,123 --> 00:01:04,164
It's the variance of the
conditional distribution

21
00:01:04,164 --> 00:01:05,820
of Theta.

22
00:01:05,820 --> 00:01:08,180
As we have discussed,
if I tell you

23
00:01:08,180 --> 00:01:11,789
that X takes on this
particular value,

24
00:01:11,789 --> 00:01:15,590
Theta is uniform
on this interval.

25
00:01:15,590 --> 00:01:18,510
Therefore, the conditional
variance of Theta

26
00:01:18,510 --> 00:01:22,020
is the variance of a
uniform on an interval

27
00:01:22,020 --> 00:01:23,990
of this particular length.

28
00:01:23,990 --> 00:01:27,789
Now, we know that the
variance of a uniform

29
00:01:27,789 --> 00:01:36,150
on an interval from a to b
is equal to b minus a squared

30
00:01:36,150 --> 00:01:38,330
divided by 12.

31
00:01:38,330 --> 00:01:42,030
In this particular instance,
the interval has length 2.

32
00:01:42,030 --> 00:01:45,479
Therefore, we have 2
squared divided by 12.

33
00:01:45,479 --> 00:01:50,190
So the variance is equal to 1/3.

34
00:01:50,190 --> 00:01:53,490
This is what we get when
the picture is of this type.

35
00:01:53,490 --> 00:01:57,750
On the other hand, if X
falls in this range, then

36
00:01:57,750 --> 00:02:03,350
this interval on which Theta
is now constrained to live,

37
00:02:03,350 --> 00:02:05,040
has a smaller length
and we're going

38
00:02:05,040 --> 00:02:07,220
to get a different variance.

39
00:02:07,220 --> 00:02:11,400
So in order to keep track,
let us come up with a plot.

40
00:02:11,400 --> 00:02:16,440
When X is between 5 and 9, Theta
has a conditional distribution

41
00:02:16,440 --> 00:02:19,140
which is uniform on an
interval of length 2

42
00:02:19,140 --> 00:02:21,460
and a variance of 1/3.

43
00:02:21,460 --> 00:02:27,480
And therefore, the variance
is constant, takes this value.

44
00:02:27,480 --> 00:02:31,600
In the extreme case,
when X is equal to 3,

45
00:02:31,600 --> 00:02:34,250
then this interval has 0 length.

46
00:02:34,250 --> 00:02:36,410
In fact, we have
perfect certainty

47
00:02:36,410 --> 00:02:37,900
about the value of Theta.

48
00:02:37,900 --> 00:02:41,740
If X is equal to 3, then we
know that Theta is equal to 4.

49
00:02:41,740 --> 00:02:43,000
There's no uncertainty.

50
00:02:43,000 --> 00:02:45,510
There's zero variance.

51
00:02:45,510 --> 00:02:47,190
What happens in between?

52
00:02:47,190 --> 00:02:52,030
As we increase x
moving away from 3,

53
00:02:52,030 --> 00:02:56,500
the length of this interval
increases linearly with x.

54
00:02:56,500 --> 00:03:00,050
And this means that the
variance increases quadratically

55
00:03:00,050 --> 00:03:04,050
with x, so we have a
quadratic that starts at 0

56
00:03:04,050 --> 00:03:06,160
and rises to this value.

57
00:03:06,160 --> 00:03:09,310
And by a symmetric
argument, on the other side,

58
00:03:09,310 --> 00:03:13,580
we also get function,
which is 0 at 11,

59
00:03:13,580 --> 00:03:19,570
and which rises quadratically
as x gets reduced from 11 to 9.

60
00:03:19,570 --> 00:03:22,770
So this is a complete plot
of the conditional variance

61
00:03:22,770 --> 00:03:26,565
of Theta as a function of
the particular observation

62
00:03:26,565 --> 00:03:29,079
that we have obtained.

63
00:03:29,079 --> 00:03:33,160
We notice that some x's are
more favorable than others.

64
00:03:33,160 --> 00:03:36,280
An observation equal to
3 is extremely favorable

65
00:03:36,280 --> 00:03:38,070
because it tells
us unambiguously

66
00:03:38,070 --> 00:03:39,420
the value of Theta.

67
00:03:39,420 --> 00:03:43,760
But other choices of x,
other possible observations,

68
00:03:43,760 --> 00:03:46,190
will lead to more
uncertainty in Theta,

69
00:03:46,190 --> 00:03:49,970
and this is reflected
in this diagram.

70
00:03:49,970 --> 00:03:56,360
In case we are now interested in
the overall mean squared error,

71
00:03:56,360 --> 00:04:00,490
then we have to
calculate the average

72
00:04:00,490 --> 00:04:06,470
of this conditional variance
where the average is taken over

73
00:04:06,470 --> 00:04:10,630
all values of X.
This is going to be

74
00:04:10,630 --> 00:04:19,910
an integral of the conditional
variance of Theta integrated

75
00:04:19,910 --> 00:04:22,590
over all possibilities for x.

76
00:04:22,590 --> 00:04:25,410
But, of course, each
possibility of x

77
00:04:25,410 --> 00:04:31,160
has to be weighted according to
the corresponding probability,

78
00:04:31,160 --> 00:04:35,970
or in this case, probability
density function of X.

79
00:04:35,970 --> 00:04:38,760
What is the PDF of X?

80
00:04:38,760 --> 00:04:40,900
It is not a given
to us, but it is

81
00:04:40,900 --> 00:04:43,510
something that can
be easily determined

82
00:04:43,510 --> 00:04:45,860
from what we have already done.

83
00:04:45,860 --> 00:04:49,409
We know the joint
distribution of Theta and X,

84
00:04:49,409 --> 00:04:53,830
and whenever we know the joint
we can also find the marginal.

85
00:04:53,830 --> 00:04:57,090
So once we find the
marginal PDF of X,

86
00:04:57,090 --> 00:05:00,640
then we can plug it in,
multiply by this function

87
00:05:00,640 --> 00:05:04,350
that we have already obtained,
carry out the integration,

88
00:05:04,350 --> 00:05:07,550
and we will end up
with a numerical value.

89
00:05:07,550 --> 00:05:10,530
Since it is an average
of what we have here,

90
00:05:10,530 --> 00:05:15,250
it's going to end up being
some number between 0 and 1/3.

91
00:05:15,250 --> 00:05:18,950
It's the average of these
values, and closer to 1/3

92
00:05:18,950 --> 00:05:21,600
rather than 0.

93
00:05:21,600 --> 00:05:24,930
And this would complete
the way that a performance

94
00:05:24,930 --> 00:05:28,432
of a particular
estimator gets evaluated.