1
00:00:00,690 --> 00:00:03,360
Before we dive into the
heart of the subject,

2
00:00:03,360 --> 00:00:06,170
I want to make a few comments
on the different problem

3
00:00:06,170 --> 00:00:09,750
types that show up in
the field of inference.

4
00:00:09,750 --> 00:00:11,940
You can think of a
general distinction

5
00:00:11,940 --> 00:00:15,390
between model building
versus making inferences

6
00:00:15,390 --> 00:00:18,140
about unobserved variables.

7
00:00:18,140 --> 00:00:20,560
We said a little
earlier that one

8
00:00:20,560 --> 00:00:23,070
of the main uses of
the field of inference

9
00:00:23,070 --> 00:00:26,820
is to construct models
of certain situations.

10
00:00:26,820 --> 00:00:30,370
But in many cases, we
already have a model.

11
00:00:30,370 --> 00:00:35,150
On the other hand, there may
be variables that are unknown,

12
00:00:35,150 --> 00:00:38,550
that are unobserved-- variables
that are part of the model,

13
00:00:38,550 --> 00:00:40,700
but whose values are not known.

14
00:00:40,700 --> 00:00:43,070
In such cases, we
still want to use

15
00:00:43,070 --> 00:00:46,890
data to make some
predictions or decisions

16
00:00:46,890 --> 00:00:49,310
about those
unobserved variables.

17
00:00:49,310 --> 00:00:54,200
So model building might or
might not be part of the problem

18
00:00:54,200 --> 00:00:56,010
that we're dealing with.

19
00:00:56,010 --> 00:00:59,830
To illustrate the difference
between these two versions

20
00:00:59,830 --> 00:01:03,690
of the problem, let us
think of a concrete setting.

21
00:01:03,690 --> 00:01:07,060
You have a transmitter
who is sending a signal.

22
00:01:07,060 --> 00:01:12,180
Call it S. And that signal
goes through some medium.

23
00:01:12,180 --> 00:01:15,000
It could be just the atmosphere.

24
00:01:15,000 --> 00:01:18,460
And what that medium does
is that it attenuates

25
00:01:18,460 --> 00:01:21,340
the signal by a
certain factor, a.

26
00:01:21,340 --> 00:01:27,240
And then as the signal travels,
it also gets hit by some noise,

27
00:01:27,240 --> 00:01:32,670
call it W, and what the
receiver sees is an observation,

28
00:01:32,670 --> 00:01:40,780
X. So the situation is described
by this simple equation here.

29
00:01:40,780 --> 00:01:45,120
This situation often brings
up the following inference

30
00:01:45,120 --> 00:01:46,710
problem.

31
00:01:46,710 --> 00:01:50,720
We want to find out
what the medium is.

32
00:01:50,720 --> 00:01:52,560
How do we do this?

33
00:01:52,560 --> 00:01:56,190
We send a pilot
signal, S, that is

34
00:01:56,190 --> 00:01:58,759
a signal that we
know what it is.

35
00:01:58,759 --> 00:02:04,220
We observe X, and then
using this equation,

36
00:02:04,220 --> 00:02:07,050
and, knowing that W
is random noise coming

37
00:02:07,050 --> 00:02:09,509
from some distribution,
we try to make

38
00:02:09,509 --> 00:02:12,860
an inference about
the variable a.

39
00:02:12,860 --> 00:02:16,260
So this is an instance
of model building.

40
00:02:16,260 --> 00:02:20,010
We're trying to make a model
of the medium that's involved.

41
00:02:20,010 --> 00:02:23,370
But we can also think
of a different problem.

42
00:02:23,370 --> 00:02:26,270
Suppose that we know
what the medium is.

43
00:02:26,270 --> 00:02:30,320
Perhaps we already went through
this particular phase here.

44
00:02:30,320 --> 00:02:33,220
But we're sitting
at the receiver,

45
00:02:33,220 --> 00:02:36,079
and we do not know
what has been sent.

46
00:02:36,079 --> 00:02:39,230
And we want to
find out what S is.

47
00:02:39,230 --> 00:02:41,700
So we are looking
again at this equation.

48
00:02:41,700 --> 00:02:43,840
This time we know
a, and we're trying

49
00:02:43,840 --> 00:02:47,220
to make inferences about S.

50
00:02:47,220 --> 00:02:50,140
You notice that these two
versions of the problem

51
00:02:50,140 --> 00:02:54,390
are essentially of the same
mathematical structure.

52
00:02:54,390 --> 00:02:56,840
We have a linear equation.

53
00:02:56,840 --> 00:03:00,930
In one case, we know S.
We want to find out a.

54
00:03:00,930 --> 00:03:03,070
In the other case, we know a.

55
00:03:03,070 --> 00:03:05,980
We want to find out what S is.

56
00:03:05,980 --> 00:03:08,030
So even though
the interpretation

57
00:03:08,030 --> 00:03:11,510
of these two problems
[is] quite different,

58
00:03:11,510 --> 00:03:14,560
the mathematical structure
is exactly the same.

59
00:03:14,560 --> 00:03:15,690
This is fortunate.

60
00:03:15,690 --> 00:03:18,820
It means that one and
the same methodology

61
00:03:18,820 --> 00:03:23,780
would be applicable to
both types of problems.

62
00:03:23,780 --> 00:03:26,730
There is another distinction
between problem types

63
00:03:26,730 --> 00:03:30,100
which turns out to be a
little more substantial.

64
00:03:30,100 --> 00:03:33,829
There are problems that we call
hypothesis testing problems.

65
00:03:33,829 --> 00:03:36,460
In those problems the
unknown takes one out

66
00:03:36,460 --> 00:03:38,710
of a few possible values.

67
00:03:38,710 --> 00:03:42,040
That is, we may have a
few different alternative

68
00:03:42,040 --> 00:03:43,190
models of the world.

69
00:03:43,190 --> 00:03:46,120
And we're trying to figure
out which one of those models

70
00:03:46,120 --> 00:03:47,550
is the correct one.

71
00:03:47,550 --> 00:03:51,620
We're going to decide in favor
of one of the candidate models,

72
00:03:51,620 --> 00:03:54,329
and what we want to
achieve is that we

73
00:03:54,329 --> 00:03:55,900
make a correct decision.

74
00:03:55,900 --> 00:03:59,170
Or if not, we want to
have a small probability

75
00:03:59,170 --> 00:04:02,000
of making an incorrect decision.

76
00:04:02,000 --> 00:04:05,450
An example of this kind is
the radar detection problem

77
00:04:05,450 --> 00:04:09,170
that we had discussed in the
very beginning of this course,

78
00:04:09,170 --> 00:04:11,620
in which we were
getting a signal.

79
00:04:11,620 --> 00:04:14,200
We were getting a radar reading.

80
00:04:14,200 --> 00:04:16,880
And the question was
to make an inference

81
00:04:16,880 --> 00:04:19,470
whether the radar is
seeing an airplane

82
00:04:19,470 --> 00:04:22,830
or whether an airplane
is not present.

83
00:04:22,830 --> 00:04:25,070
So in hypothesis
testing problems,

84
00:04:25,070 --> 00:04:27,600
we're essentially
making a choice

85
00:04:27,600 --> 00:04:31,480
out of a small number of
discrete possible choices.

86
00:04:31,480 --> 00:04:35,740
Instead, in estimation
problems, the unknown quantities

87
00:04:35,740 --> 00:04:38,480
are more of a numerical type.

88
00:04:38,480 --> 00:04:41,390
They could even take
continuous values.

89
00:04:41,390 --> 00:04:44,000
And what we want
to do is to come up

90
00:04:44,000 --> 00:04:47,400
with an estimate of an
unknown quantity that

91
00:04:47,400 --> 00:04:52,510
is close to the true but
unknown value of the quantity

92
00:04:52,510 --> 00:04:54,540
that we're trying to estimate.

93
00:04:54,540 --> 00:04:56,870
So here, our
performance objective

94
00:04:56,870 --> 00:05:00,200
is in terms of some kind
of distance function.

95
00:05:00,200 --> 00:05:02,710
We want to be
close to the truth.

96
00:05:02,710 --> 00:05:06,310
And typically, we have a
continuum of possible choices

97
00:05:06,310 --> 00:05:11,210
that is, our estimates can
be general real numbers.

98
00:05:11,210 --> 00:05:14,080
Generally speaking, these two
types of problems, hypothesis

99
00:05:14,080 --> 00:05:18,270
testing and estimation, have
some significant differences

100
00:05:18,270 --> 00:05:20,360
in the way that
they are treated,

101
00:05:20,360 --> 00:05:22,999
as we will be seeing next.