1
00:00:00,890 --> 00:00:04,710
As we have already discussed,
we can estimate an unknown

2
00:00:04,710 --> 00:00:08,710
mean of a certain random
variable by generating several

3
00:00:08,710 --> 00:00:11,970
independent samples of that
random variable and taking

4
00:00:11,970 --> 00:00:13,210
their average.

5
00:00:13,210 --> 00:00:16,030
And this procedure is well
justified, because of the weak

6
00:00:16,030 --> 00:00:20,320
law of large numbers, which
tells us that this estimator

7
00:00:20,320 --> 00:00:24,730
converges when n goes to
infinity, in probability, to

8
00:00:24,730 --> 00:00:26,120
the true mean.

9
00:00:26,120 --> 00:00:28,610
Now we can apply this
idea more generally.

10
00:00:28,610 --> 00:00:31,550
Suppose we want to estimate
the expected value of a

11
00:00:31,550 --> 00:00:36,830
function of a random variable
X. Now g of X is itself a

12
00:00:36,830 --> 00:00:37,770
random variable.

13
00:00:37,770 --> 00:00:40,340
So if we have samples
of g of X, we can

14
00:00:40,340 --> 00:00:41,880
use the same procedure.

15
00:00:41,880 --> 00:00:43,130
How do we do that?

16
00:00:43,130 --> 00:00:47,000
We generate independent samples
of X, and these give

17
00:00:47,000 --> 00:00:52,250
us independent samples of g of
X. We use those independent

18
00:00:52,250 --> 00:00:56,260
samples, we average them, and
by the weak law of large

19
00:00:56,260 --> 00:00:59,730
numbers, this quantity, as
n goes to infinity, will

20
00:00:59,730 --> 00:01:05,200
converge in probability to the
expected value of g of X.

21
00:01:05,200 --> 00:01:08,960
We already used an idea of this
form when we tried to

22
00:01:08,960 --> 00:01:11,810
estimate an unknown variance.

23
00:01:11,810 --> 00:01:14,820
A variance is defined
as an expectation.

24
00:01:14,820 --> 00:01:19,550
And now we can generate samples
of X, many independent

25
00:01:19,550 --> 00:01:21,990
samples, calculate
this quantity,

26
00:01:21,990 --> 00:01:23,520
and take the average.

27
00:01:23,520 --> 00:01:27,150
However, we might not know the
mean of the distribution.

28
00:01:27,150 --> 00:01:31,539
So instead of the true mean,
we use an estimated mean,

29
00:01:31,539 --> 00:01:35,630
which is estimated the usual
way using a sample average.

30
00:01:35,630 --> 00:01:39,360
So when n is large, this
estimated mean is

31
00:01:39,360 --> 00:01:41,380
close to the true mean.

32
00:01:41,380 --> 00:01:44,160
So using the estimated mean
here will not make a

33
00:01:44,160 --> 00:01:45,940
substantial difference.

34
00:01:45,940 --> 00:01:49,840
And then we have essentially
independent

35
00:01:49,840 --> 00:01:53,350
samples of this quantity.

36
00:01:53,350 --> 00:01:58,400
And by averaging them, we obtain
an estimate of the

37
00:01:58,400 --> 00:02:01,350
variance, which asymptotically,
as n goes to

38
00:02:01,350 --> 00:02:05,080
infinity, will be equal
to the true variance.

39
00:02:05,080 --> 00:02:07,780
Now we can push this
idea even further.

40
00:02:07,780 --> 00:02:10,410
Suppose we wish to estimate
a covariance.

41
00:02:10,410 --> 00:02:13,310
What's a natural way
of doing this?

42
00:02:13,310 --> 00:02:17,870
We can generate independent
samples of the pair of the

43
00:02:17,870 --> 00:02:21,680
random variables X and Y, so
this will be a typical

44
00:02:21,680 --> 00:02:25,900
independent sample, and replace
the expected value by

45
00:02:25,900 --> 00:02:28,540
a sample average.

46
00:02:28,540 --> 00:02:33,290
That is, we take our i-th
sample, i-th pair, and

47
00:02:33,290 --> 00:02:36,660
calculate this quantity, which
looks very much like the

48
00:02:36,660 --> 00:02:40,190
quantity in here except that
we're using the estimated

49
00:02:40,190 --> 00:02:43,050
means in place of
the true means.

50
00:02:43,050 --> 00:02:47,610
We obtain these quantities and
average n of them, again using

51
00:02:47,610 --> 00:02:49,320
the weak law of large numbers.

52
00:02:49,320 --> 00:02:52,620
One can argue that this estimate
will converge to the

53
00:02:52,620 --> 00:02:56,220
true value of the covariance
as n goes to infinity.

54
00:02:56,220 --> 00:03:01,050
And once we have estimates of a
covariance and of variance,

55
00:03:01,050 --> 00:03:03,740
then we can use that to
estimate correlation

56
00:03:03,740 --> 00:03:04,990
coefficients.

57
00:03:04,990 --> 00:03:07,960
Look at this formula, which
is the definition of the

58
00:03:07,960 --> 00:03:09,820
correlation coefficient.

59
00:03:09,820 --> 00:03:13,340
If we just replace all
quantities involved here by

60
00:03:13,340 --> 00:03:16,700
corresponding estimates, this
gives us an estimate of the

61
00:03:16,700 --> 00:03:18,579
correlation coefficient.

62
00:03:18,579 --> 00:03:22,550
All of these ways of forming
estimates can be shown.

63
00:03:22,550 --> 00:03:25,750
We are omitting the details of
the argument, but hopefully

64
00:03:25,750 --> 00:03:28,230
you get the idea by now.

65
00:03:28,230 --> 00:03:31,510
All of these quantities are
consistent estimators.

66
00:03:31,510 --> 00:03:35,570
That is, when the sample size
goes to infinity, they

67
00:03:35,570 --> 00:03:40,230
approach the correct values of
what we're trying to estimate.

68
00:03:40,230 --> 00:03:44,440
So this is just an opening of
what else a statistician might

69
00:03:44,440 --> 00:03:45,850
be interested in.

70
00:03:45,850 --> 00:03:49,350
And if you're wondering what's
the further agenda after this

71
00:03:49,350 --> 00:03:52,390
point, it would be something
like the following.

72
00:03:52,390 --> 00:03:56,020
Typically, a statistician might
to want to say as much

73
00:03:56,020 --> 00:03:57,940
as possible about
the probability

74
00:03:57,940 --> 00:04:00,150
distribution of an estimator.

75
00:04:00,150 --> 00:04:03,540
For example, we have here an
estimate of a covariance.

76
00:04:03,540 --> 00:04:07,660
This estimate is going to be a
random variable because it is

77
00:04:07,660 --> 00:04:09,840
determined by random
quantities.

78
00:04:09,840 --> 00:04:12,580
What is the probability
distribution of this quantity?

79
00:04:12,580 --> 00:04:15,050
Can we describe it
approximately?

80
00:04:15,050 --> 00:04:16,760
What is the mean squared error

81
00:04:16,760 --> 00:04:19,089
associated with this estimator?

82
00:04:19,089 --> 00:04:22,079
And if you wish to construct
confidence intervals how

83
00:04:22,079 --> 00:04:23,120
would you do it?

84
00:04:23,120 --> 00:04:25,910
These are all topics that
statisticians have studied in

85
00:04:25,910 --> 00:04:29,510
depth, and you could see more
about these topics if you were

86
00:04:29,510 --> 00:04:32,860
to take a further class on
statistics and inference.

87
00:04:32,860 --> 00:04:35,450
But we will not go any deeper
in this course.