1
00:00:00,000 --> 00:00:02,690
SPEAKER: The following content
is provided under a Creative

2
00:00:02,690 --> 00:00:03,630
Commons license.

3
00:00:03,630 --> 00:00:06,600
Your support will help MIT
OpeCourseWare continue to

4
00:00:06,600 --> 00:00:09,970
offer high quality educational
resources for free.

5
00:00:09,970 --> 00:00:12,815
To make a donation or to view
additional material from

6
00:00:12,815 --> 00:00:16,860
hundreds of MIT courses, visit
MIT OpenCourseWare at

7
00:00:16,860 --> 00:00:18,110
ocw.mit.edu.

8
00:00:21,490 --> 00:00:26,730
PROFESSOR: OK, we talked about
this a little bit last time.

9
00:00:26,730 --> 00:00:31,570
We were talking about the
detection of vectors in white

10
00:00:31,570 --> 00:00:33,210
Gaussian noise.

11
00:00:33,210 --> 00:00:35,670
When we talk about vectors we'll
often refer to white

12
00:00:35,670 --> 00:00:41,310
Gaussian noise as noise where
each component of the vector

13
00:00:41,310 --> 00:00:44,700
is independent of each
other when they're

14
00:00:44,700 --> 00:00:47,140
all the same variance.

15
00:00:47,140 --> 00:00:51,240
Usually we take the variance
to be n0 over 2,

16
00:00:51,240 --> 00:00:54,030
capital N0 over 2.

17
00:00:54,030 --> 00:00:59,530
And we'll say more about that as
we go on, but that's just--

18
00:00:59,530 --> 00:01:02,800
I guess one thing I ought
to say about it now--

19
00:01:02,800 --> 00:01:06,980
people keep wondering why we
call this N0 over 2 instead of

20
00:01:06,980 --> 00:01:08,440
something else.

21
00:01:08,440 --> 00:01:10,580
As I said before, there's
really no reason

22
00:01:10,580 --> 00:01:12,860
for it except custom.

23
00:01:12,860 --> 00:01:16,770
The one important thing that
you can always remember and

24
00:01:16,770 --> 00:01:22,380
which is always true is that any
time you're talking about

25
00:01:22,380 --> 00:01:28,650
a sequence of real noise
variables they all have

26
00:01:28,650 --> 00:01:34,320
variants N0 over 2, and the same
coordinate system that

27
00:01:34,320 --> 00:01:37,370
you're using to measure
the signals.

28
00:01:37,370 --> 00:01:39,650
OK, the only thing that ever
appears in any of these

29
00:01:39,650 --> 00:01:46,880
formulas is a ratio of signal
power or signal energy to

30
00:01:46,880 --> 00:01:49,430
noise signal power.

31
00:01:49,430 --> 00:01:58,530
And if we're up it passband,
we're dealing with the power,

32
00:01:58,530 --> 00:02:03,160
which is two times larger
than that at baseband.

33
00:02:03,160 --> 00:02:05,220
And because of that-- and
this is what really gets

34
00:02:05,220 --> 00:02:09,560
confusing-- is that when you
talk about N0 over 2 at

35
00:02:09,560 --> 00:02:13,070
passband, you are talking about
something which is twice

36
00:02:13,070 --> 00:02:17,240
as big as the N0 over 2, the
same N0 over 2 you were

37
00:02:17,240 --> 00:02:19,320
talking about at baseband.

38
00:02:19,320 --> 00:02:24,380
And the reason is since the
signal is twice as big there,

39
00:02:24,380 --> 00:02:28,120
the noise is also said
to be twice as big.

40
00:02:28,120 --> 00:02:29,710
I can't do anything
about that.

41
00:02:29,710 --> 00:02:31,950
It's just the way that everybody
does things.

42
00:02:31,950 --> 00:02:34,730
The other thing that everybody
does-- since everyone gets

43
00:02:34,730 --> 00:02:36,350
confused about that--

44
00:02:36,350 --> 00:02:40,230
is after they get all done
dealing with anything in a

45
00:02:40,230 --> 00:02:43,000
paper they're writing or
something, they always look at

46
00:02:43,000 --> 00:02:46,110
the signal to noise ratio that
they have and they remove all

47
00:02:46,110 --> 00:02:49,350
the factors of two that they
know shouldn't be there.

48
00:02:49,350 --> 00:02:53,190
So that you shouldn't trust
anything in the literature too

49
00:02:53,190 --> 00:02:56,870
much as far as factors
of two are concerned.

50
00:02:56,870 --> 00:02:59,630
And I try to be careful in the
notes about that, but you

51
00:02:59,630 --> 00:03:03,350
shouldn't trust the notes too
far along those lines, either.

52
00:03:03,350 --> 00:03:08,440
So well eventually I'll get the
notes straightened out on

53
00:03:08,440 --> 00:03:12,590
all of that but I think they're
pretty close now.

54
00:03:12,590 --> 00:03:16,240
But anyway, we were looking at
this question of how do you

55
00:03:16,240 --> 00:03:21,980
detect antipodal vectors in
white Gaussian noise?

56
00:03:21,980 --> 00:03:27,150
And the picture that we
can draw is this.

57
00:03:27,150 --> 00:03:31,820
Namely we have two signals.

58
00:03:31,820 --> 00:03:37,830
One is the vector, a, one
is the vector, minus a.

59
00:03:37,830 --> 00:03:41,910
It's in some finite dimensional
system, but we're

60
00:03:41,910 --> 00:03:45,310
viewing it as far as drawing
a picture as a

61
00:03:45,310 --> 00:03:46,940
two dimensional system.

62
00:03:46,940 --> 00:03:50,120
So a has some arbitrary
component

63
00:03:50,120 --> 00:03:51,440
in the first direction.

64
00:03:51,440 --> 00:03:54,330
Some arbitrary component in
the second direction.

65
00:03:54,330 --> 00:03:57,650
Minus a is, of course,
the reverse of that.

66
00:03:57,650 --> 00:04:00,160
This point right in here is
the zero point, which is

67
00:04:00,160 --> 00:04:01,530
halfway in between them.

68
00:04:06,290 --> 00:04:10,810
And the output that we observe
is either plus or minus a,

69
00:04:10,810 --> 00:04:16,260
plus this independent zero mean
noise, Z, which has the

70
00:04:16,260 --> 00:04:19,700
kind of circular symmetry
indicated here with these

71
00:04:19,700 --> 00:04:22,160
little circles.

72
00:04:22,160 --> 00:04:25,170
Each of the Z sub i have the
same variance, they're

73
00:04:25,170 --> 00:04:27,740
independent of each other.

74
00:04:27,740 --> 00:04:32,280
And when you write down the
likelihood of the probability

75
00:04:32,280 --> 00:04:37,110
density of this output, given
that the hypothesis was zero.

76
00:04:37,110 --> 00:04:41,050
Namely that plus a was the
signal which was chosen.

77
00:04:41,050 --> 00:04:44,290
OK, remember in all of these
things there's this process

78
00:04:44,290 --> 00:04:48,040
now going on that we usually
don't talk about anymore.

79
00:04:48,040 --> 00:04:51,730
But there's an input coming into
the communication channel

80
00:04:51,730 --> 00:04:53,810
which we're now calling
capital H. It's the

81
00:04:53,810 --> 00:04:56,620
hypothesis-- which is the thing
you're trying to detect

82
00:04:56,620 --> 00:04:57,970
when you're all done--

83
00:04:57,970 --> 00:05:03,110
that input which is one up to
capital N, or sometimes zero

84
00:05:03,110 --> 00:05:05,720
up to capital N minus 1.

85
00:05:05,720 --> 00:05:10,720
Is then mapped into a signal
from a single set of capital N

86
00:05:10,720 --> 00:05:11,980
different signals.

87
00:05:11,980 --> 00:05:15,560
So they're impossible signals
in this signal alphabet.

88
00:05:15,560 --> 00:05:20,600
You map the hypothesis
into one of those.

89
00:05:20,600 --> 00:05:24,400
From those, you generally
form a waveform.

90
00:05:24,400 --> 00:05:26,740
This waveform might be
modulated up to high

91
00:05:26,740 --> 00:05:30,410
frequency, back to low
frequency again.

92
00:05:30,410 --> 00:05:32,050
Detected or whatever.

93
00:05:32,050 --> 00:05:36,070
You got some vector,
v, at that point.

94
00:05:36,070 --> 00:05:42,570
Which is a sequence of samples
that you're going to be taking

95
00:05:42,570 --> 00:05:44,320
as far as most cases
are concerned.

96
00:05:44,320 --> 00:05:46,350
We'll talk more about
that later today.

97
00:05:46,350 --> 00:05:49,980
But anyway, v is a vector which
is plus or minus a at

98
00:05:49,980 --> 00:05:52,180
this point, plus this
Gaussian noise.

99
00:05:52,180 --> 00:05:55,110
We can write down the
probability density of that

100
00:05:55,110 --> 00:06:01,160
vector, v, which is if
hypothesis zero occurs.

101
00:06:01,160 --> 00:06:04,770
Namely if a zero enters the
communication channel, plus a

102
00:06:04,770 --> 00:06:06,860
is the signal which is chosen.

103
00:06:06,860 --> 00:06:13,090
Then what happens is that the
output is a plus Z. Which

104
00:06:13,090 --> 00:06:16,630
means that the probability
density of the

105
00:06:16,630 --> 00:06:19,800
noise is v minus a.

106
00:06:19,800 --> 00:06:23,480
So we have this probability
density here.

107
00:06:23,480 --> 00:06:27,110
When we look at the log
likelihood ratio, we're taking

108
00:06:27,110 --> 00:06:31,630
the logarithm of this
probability density divided by

109
00:06:31,630 --> 00:06:35,180
the probability density of the
alternative hypothesis.

110
00:06:35,180 --> 00:06:37,850
Namely v given one.

111
00:06:37,850 --> 00:06:40,230
Which is the same as this
formula except there's the

112
00:06:40,230 --> 00:06:42,830
plus a there instead
of a minus a.

113
00:06:42,830 --> 00:06:45,400
So you get this thing here.

114
00:06:45,400 --> 00:06:47,650
Now, why I wanted to talk about
this today is we're

115
00:06:47,650 --> 00:06:50,420
going to talk about the
complex case also and

116
00:06:50,420 --> 00:06:54,040
something very, very peculiar
and funny happens there.

117
00:06:54,040 --> 00:06:59,460
OK so the log likelihood ratio
is the scaled difference of

118
00:06:59,460 --> 00:07:05,120
the energy of the distance
between v and a.

119
00:07:05,120 --> 00:07:07,390
Which is this term here.

120
00:07:07,390 --> 00:07:09,820
This is just a squared distance
between the vector,

121
00:07:09,820 --> 00:07:11,970
v, and the vector, a.

122
00:07:11,970 --> 00:07:16,180
It's v minus a is that distance
there, squared.

123
00:07:16,180 --> 00:07:20,170
And the other term here is the
term that comes from the

124
00:07:20,170 --> 00:07:24,170
probability density
of v given one.

125
00:07:24,170 --> 00:07:27,330
Which turns out to be that
distance there squared.

126
00:07:27,330 --> 00:07:30,540
So you have the difference
between these two things.

127
00:07:30,540 --> 00:07:35,930
This is just the inner product
of v minus a with v minus a.

128
00:07:35,930 --> 00:07:38,770
And if you multiply that all out
it's the inner product v

129
00:07:38,770 --> 00:07:44,740
with itself, plus the inner
product of a with itself,

130
00:07:44,740 --> 00:07:50,660
minus the inner product of v and
a, minus the inner product

131
00:07:50,660 --> 00:07:52,990
of a with v.

132
00:07:52,990 --> 00:07:55,760
The only things that don't
cancel out between these two

133
00:07:55,760 --> 00:07:58,990
things is the inner product of v
with a, the inner product of

134
00:07:58,990 --> 00:08:02,650
a with v, the inner product of v
with a, the inner product of

135
00:08:02,650 --> 00:08:07,350
a with v. So those things last
because there's a minus sign

136
00:08:07,350 --> 00:08:09,270
here and a plus sign here.

137
00:08:09,270 --> 00:08:11,720
There's a minus sign here
and a plus sign here.

138
00:08:11,720 --> 00:08:15,280
So the plus and minus signs
cancel out, so you just get

139
00:08:15,280 --> 00:08:18,130
four of these terms here, which
is four times the inner

140
00:08:18,130 --> 00:08:20,230
product over n0.

141
00:08:20,230 --> 00:08:22,960
What happens to that
geometrically?

142
00:08:22,960 --> 00:08:27,020
What is the inner product
of v and a?

143
00:08:27,020 --> 00:08:32,210
Well it's the projection
of v on the vector a.

144
00:08:32,210 --> 00:08:37,110
Which happens to be the line
between minus a and plus a.

145
00:08:37,110 --> 00:08:39,890
Mainly the fact that it's the
line between minus a and plus

146
00:08:39,890 --> 00:08:42,230
a is the thing which is valuable
whether you're

147
00:08:42,230 --> 00:08:45,010
dealing with antipodal
communication or any other

148
00:08:45,010 --> 00:08:46,190
kind of communication.

149
00:08:46,190 --> 00:08:49,810
You're always looking at this
line between two points.

150
00:08:49,810 --> 00:08:53,940
So what this thing says is you
form the inner product, which

151
00:08:53,940 --> 00:08:58,950
says drop a perpendicular from
Z down to here, and in terms

152
00:08:58,950 --> 00:09:03,760
of where that perpendicular
lands here,

153
00:09:03,760 --> 00:09:04,890
you make your decision.

154
00:09:04,890 --> 00:09:07,770
Namely you compare that
with the threshold.

155
00:09:07,770 --> 00:09:08,200
OK?

156
00:09:08,200 --> 00:09:10,160
So we have two different
ways of doing this.

157
00:09:10,160 --> 00:09:14,700
One of them is compare this
distance with this distance.

158
00:09:14,700 --> 00:09:17,640
Or actually here you compare the
square distance here with

159
00:09:17,640 --> 00:09:19,310
the square distance there.

160
00:09:19,310 --> 00:09:23,380
Which says, if you're using
maximum likelihood then the

161
00:09:23,380 --> 00:09:26,440
threshold that you're dealing
with here is zero and you

162
00:09:26,440 --> 00:09:29,050
simply make your decision on
whether the log likelihood

163
00:09:29,050 --> 00:09:32,210
ratio is positive or minus.

164
00:09:32,210 --> 00:09:37,530
Which means in terms of the
projection theorem here, what

165
00:09:37,530 --> 00:09:40,940
you're doing is taking a
perpendicular bisector of the

166
00:09:40,940 --> 00:09:45,770
line between minus a and plus
a, and putting a plane there

167
00:09:45,770 --> 00:09:51,490
and that's the plane that
separates what goes into one

168
00:09:51,490 --> 00:09:53,290
and what goes into zero.

169
00:09:53,290 --> 00:09:54,930
This goes into zero.

170
00:09:54,930 --> 00:09:56,590
This goes into one.

171
00:09:56,590 --> 00:10:01,150
OK, so is it clear to all of you
that this is really saying

172
00:10:01,150 --> 00:10:02,100
the same thing as this?

173
00:10:02,100 --> 00:10:04,620
This inner product just
comes from here.

174
00:10:04,620 --> 00:10:08,310
You can either look at this as
minimum distance decoding.

175
00:10:08,310 --> 00:10:10,250
You just find the
point which is

176
00:10:10,250 --> 00:10:13,400
closest to what you receive.

177
00:10:15,950 --> 00:10:19,755
You find the hypothesized
signal, which is closest to

178
00:10:19,755 --> 00:10:21,830
the actual observation
that you make.

179
00:10:21,830 --> 00:10:24,600
You make your decision
in terms of that.

180
00:10:24,600 --> 00:10:27,690
Or you do this projection
and make your

181
00:10:27,690 --> 00:10:29,570
decision in terms of that.

182
00:10:29,570 --> 00:10:36,570
And if you use a triangle thing
which says that this

183
00:10:36,570 --> 00:10:39,400
squared distance plus this
squared distance is equal to

184
00:10:39,400 --> 00:10:40,500
that squared distance.

185
00:10:40,500 --> 00:10:44,090
We all remember that from third
grade or something.

186
00:10:44,090 --> 00:10:46,620
I don't know when.

187
00:10:46,620 --> 00:10:50,380
But that simply says the same
thing that this says.

188
00:10:50,380 --> 00:10:52,880
This just says it
more generally.

189
00:10:52,880 --> 00:10:58,690
In terms of an arbitrary finite
conventional vector,

190
00:10:58,690 --> 00:11:05,450
rather than just the case of
where you're looking at two

191
00:11:05,450 --> 00:11:07,110
dimensions.

192
00:11:07,110 --> 00:11:09,380
OK that's--

193
00:11:09,380 --> 00:11:13,670
we will probably come back to
look at that in a little bit,

194
00:11:13,670 --> 00:11:17,770
but now I want to look at
complex antipodal vectors in

195
00:11:17,770 --> 00:11:19,740
white Gaussian noise.

196
00:11:19,740 --> 00:11:25,780
So the set up there is that
the input is some vector.

197
00:11:25,780 --> 00:11:29,710
I usually use u's to mean
complex numbers.

198
00:11:29,710 --> 00:11:35,420
So the vector there is u1 up to
u sub j where u sub j is a

199
00:11:35,420 --> 00:11:36,440
complex number.

200
00:11:36,440 --> 00:11:39,110
So we're dealing with complex
vectors at this point.

201
00:11:39,110 --> 00:11:45,750
So we have two points, minus
a -- minus u and plus u.

202
00:11:45,750 --> 00:11:48,460
Where instead of being
a real vector

203
00:11:48,460 --> 00:11:51,060
they're complex vectors.

204
00:11:51,060 --> 00:11:55,560
And if you can't visualize
things geometrically, in terms

205
00:11:55,560 --> 00:11:58,160
of complex vectors,
join the crew.

206
00:11:58,160 --> 00:11:59,950
I can't either.

207
00:11:59,950 --> 00:12:04,330
The only thing I can never do
is talk about real vectors,

208
00:12:04,330 --> 00:12:07,740
try to get some idea of what's
going on from that, and use

209
00:12:07,740 --> 00:12:10,280
mathematics for the
complex vectors.

210
00:12:10,280 --> 00:12:13,540
Because the reason we use
complex vectors is that,

211
00:12:13,540 --> 00:12:16,570
analytically they're just as
simple as real vectors.

212
00:12:16,570 --> 00:12:19,160
The reason we use real vectors
is because we can draw

213
00:12:19,160 --> 00:12:20,370
pictures of them.

214
00:12:20,370 --> 00:12:24,230
I defy anybody to draw a picture
in four dimensions.

215
00:12:24,230 --> 00:12:27,960
Some books will do it, but
I can't understand them.

216
00:12:27,960 --> 00:12:30,140
And anyway.

217
00:12:30,140 --> 00:12:36,040
OK, so under hypothesis zero
what gets sent as u?

218
00:12:36,040 --> 00:12:39,530
And under hypothesis one, what
gets sent as minus u?

219
00:12:39,530 --> 00:12:46,080
So we're still talking about
binary communication and if

220
00:12:46,080 --> 00:12:48,790
you're talking about antipodal
vectors, you can't do much

221
00:12:48,790 --> 00:12:52,750
other than talk about binary
communication.

222
00:12:52,750 --> 00:12:56,430
Because if you're sending a, the
only vector antipodal to a

223
00:12:56,430 --> 00:12:57,750
is minus a.

224
00:12:57,750 --> 00:12:59,780
And then you're stuck
and you're done.

225
00:12:59,780 --> 00:13:03,750
So we're still talking
about binary vectors.

226
00:13:03,750 --> 00:13:07,280
Remember the reason why
we're doing this--

227
00:13:07,280 --> 00:13:09,690
because one of the things we
did last time, one of the

228
00:13:09,690 --> 00:13:13,170
things that's in the notes and
stressed again and again is

229
00:13:13,170 --> 00:13:16,250
that once you understand the
antipodal case, you can

230
00:13:16,250 --> 00:13:22,220
translate those two points
anywhere you want to and the

231
00:13:22,220 --> 00:13:28,470
maximum likelihood decision and
the MAP decision are still

232
00:13:28,470 --> 00:13:29,430
the same thing.

233
00:13:29,430 --> 00:13:33,960
You take those two points and
you just translate them in

234
00:13:33,960 --> 00:13:39,170
space until the mean between
them sits on the zero point.

235
00:13:39,170 --> 00:13:42,860
And then you're back to the
antipodal case again.

236
00:13:42,860 --> 00:13:49,170
OK, so the reason why we're
doing this is really so we can

237
00:13:49,170 --> 00:13:50,600
look at the more general case.

238
00:13:50,600 --> 00:13:52,870
But we don't have to have
that mean sitting

239
00:13:52,870 --> 00:13:54,630
around all the time.

240
00:13:54,630 --> 00:13:59,170
OK, Z then is going to be a
vector of j complex IID

241
00:13:59,170 --> 00:14:00,990
Gaussian random variables.

242
00:14:00,990 --> 00:14:04,900
IID real and imaginary parts.

243
00:14:04,900 --> 00:14:08,610
Namely the real part of each
Gaussian, complex Gaussian

244
00:14:08,610 --> 00:14:14,510
random variable has variance
n0 over 2 and the imaginary

245
00:14:14,510 --> 00:14:17,510
part also has variance
n0 over 2.

246
00:14:17,510 --> 00:14:21,650
These complex vectors, if you
look at the probability

247
00:14:21,650 --> 00:14:25,670
density for them and you draw it
in two dimensions, one for

248
00:14:25,670 --> 00:14:32,710
the real part one for the
imaginary part, you get this

249
00:14:32,710 --> 00:14:36,720
circular symmetry that we've
always associated.

250
00:14:36,720 --> 00:14:39,230
Those are supposed to be circles
and not ellipses.

251
00:14:42,260 --> 00:14:44,910
And those are sometimes
called proper complex

252
00:14:44,910 --> 00:14:46,860
Gaussian random variables.

253
00:14:46,860 --> 00:14:51,260
Because almost everywhere where
you see complex Gaussian

254
00:14:51,260 --> 00:14:55,020
random variables, the real
and imaginary parts are

255
00:14:55,020 --> 00:14:58,190
independent of each other and
both have the same variance.

256
00:15:01,460 --> 00:15:04,610
Again, when you look at formulas
in papers, formulas

257
00:15:04,610 --> 00:15:09,870
everywhere else, they are almost
always assuming this

258
00:15:09,870 --> 00:15:12,910
kind of circular symmetry.

259
00:15:12,910 --> 00:15:16,590
Or what's often called proper
complex random variables.

260
00:15:16,590 --> 00:15:19,380
Sort of accepting the fact that
anything else is very,

261
00:15:19,380 --> 00:15:22,230
very improper.

262
00:15:22,230 --> 00:15:26,450
It's improper because formulas
don't work in that case.

263
00:15:26,450 --> 00:15:31,150
OK, so we have a vector of these
complex IID Gaussian

264
00:15:31,150 --> 00:15:33,120
random variables.

265
00:15:33,120 --> 00:15:38,120
Under H equals zero, the
observation, v, is given by v

266
00:15:38,120 --> 00:15:42,680
equals u plus Z. And under
hypothesis one, the

267
00:15:42,680 --> 00:15:46,040
observation is given by minus
u plus Z. So I have exactly

268
00:15:46,040 --> 00:15:48,270
the same cases as
we had before.

269
00:15:48,270 --> 00:15:51,800
OK in other words almost all
formulas stay the same when

270
00:15:51,800 --> 00:15:54,680
you go from real to complex.

271
00:15:54,680 --> 00:15:58,370
But I don't trust the complex
case and you shouldn't either

272
00:15:58,370 --> 00:16:00,520
until you at least go
through it once.

273
00:16:00,520 --> 00:16:05,410
So what I'm going to do now is
translate this complex case to

274
00:16:05,410 --> 00:16:06,650
the real case.

275
00:16:06,650 --> 00:16:11,110
In other words, for each complex
variable I'm going to

276
00:16:11,110 --> 00:16:12,590
make two real variables.

277
00:16:12,590 --> 00:16:15,760
One the real part and one
the imaginary part.

278
00:16:15,760 --> 00:16:18,040
We know that the real part and
the imaginary part are

279
00:16:18,040 --> 00:16:19,700
independent of each other.

280
00:16:19,700 --> 00:16:22,820
And if these Gaussian random
variables are independent of

281
00:16:22,820 --> 00:16:25,750
each other, then the real and
imaginary parts of each of

282
00:16:25,750 --> 00:16:29,330
them are independent of the real
and imaginary parts of

283
00:16:29,330 --> 00:16:31,050
each of the other side.

284
00:16:31,050 --> 00:16:32,110
OK?

285
00:16:32,110 --> 00:16:37,610
So we can go from j Gaussian
random variables, independant

286
00:16:37,610 --> 00:16:40,300
Gaussian random variables,
which are complex.

287
00:16:40,300 --> 00:16:44,320
To 2j Gaussian random variables,
which at this point

288
00:16:44,320 --> 00:16:45,920
are going to be real.

289
00:16:45,920 --> 00:16:51,830
OK so it's just a translation
from j complex variables to 2j

290
00:16:51,830 --> 00:16:53,120
real variables.

291
00:16:53,120 --> 00:16:59,750
Again we can't draw pictures of
things in this j dimension,

292
00:16:59,750 --> 00:17:03,360
we can start to draw pictures
in 2j dimensions.

293
00:17:03,360 --> 00:17:06,220
If you talk about a probability
density for a

294
00:17:06,220 --> 00:17:10,310
complex random variable, what
are you talking about?

295
00:17:10,310 --> 00:17:14,030
How do you write the probability
density for just a

296
00:17:14,030 --> 00:17:18,030
plane Gaussian complex
random variable?

297
00:17:18,030 --> 00:17:19,280
What is it?

298
00:17:22,600 --> 00:17:23,650
Anybody know what it is?

299
00:17:23,650 --> 00:17:28,260
Is it one dimensional or
is it two dimensional?

300
00:17:28,260 --> 00:17:30,900
What does probability
density mean?

301
00:17:30,900 --> 00:17:34,060
It means probability
per unit area.

302
00:17:34,060 --> 00:17:35,970
What does area mean when
you're talking

303
00:17:35,970 --> 00:17:38,630
about complex numbers?

304
00:17:38,630 --> 00:17:43,000
Well you sort of mean what
you've drawn there, yes.

305
00:17:43,000 --> 00:17:48,800
And you're looking at areas in
this complex plane here.

306
00:17:48,800 --> 00:17:51,720
So that in fact when you write
the probability density for a

307
00:17:51,720 --> 00:17:55,460
complex random variable, what
you have already done whether

308
00:17:55,460 --> 00:17:59,360
you want to do it or not is
you've converted the problem

309
00:17:59,360 --> 00:18:01,660
to real and imaginary part.

310
00:18:01,660 --> 00:18:05,490
That's what the probability
densities are.

311
00:18:05,490 --> 00:18:08,640
Excuse me for a belaboring this
but, if I don't belabor

312
00:18:08,640 --> 00:18:12,370
it I mean there's a catch
here that comes

313
00:18:12,370 --> 00:18:13,870
along in a little bit.

314
00:18:13,870 --> 00:18:18,130
And you won't understand to
catch if you don't understand

315
00:18:18,130 --> 00:18:22,380
why these things are almost the
same as real variables up

316
00:18:22,380 --> 00:18:25,220
until the catch comes.

317
00:18:25,220 --> 00:18:30,020
OK, so we're going to
deal with these 2j

318
00:18:30,020 --> 00:18:32,530
dimensional real vectors.

319
00:18:32,530 --> 00:18:35,990
The components will be real part
of u j, imaginary part of

320
00:18:35,990 --> 00:18:39,660
u j for what goes into
the channel.

321
00:18:39,660 --> 00:18:44,730
And we'll let capital Y capital
Z prime be the two j

322
00:18:44,730 --> 00:18:48,780
dimensional real versions
of V and Z.

323
00:18:48,780 --> 00:18:53,650
OK so that we'll call Y the real
part, an imaginary part

324
00:18:53,650 --> 00:18:56,580
of Z. And you notice that what's
going on here is the

325
00:18:56,580 --> 00:19:00,910
same thing that's going on
when you modulate QAM.

326
00:19:00,910 --> 00:19:04,555
You take a complex signal, you
multiply it by either the 2pi

327
00:19:04,555 --> 00:19:11,920
j carrier frequency times t, and
then we started to look at

328
00:19:11,920 --> 00:19:14,810
orthonormal expansions, you
remember that what we looked

329
00:19:14,810 --> 00:19:17,560
at-- as far as the real signals
that were actually

330
00:19:17,560 --> 00:19:20,840
being transmitted on the
channel-- was the real part of

331
00:19:20,840 --> 00:19:24,890
that u of t times z to the
blah, blah, blah, and the

332
00:19:24,890 --> 00:19:28,880
imaginary part of u of t
times blah, blah, blah.

333
00:19:28,880 --> 00:19:32,310
So you've got two orthonormal
functions in place of one

334
00:19:32,310 --> 00:19:34,640
complex orthonormal function.

335
00:19:34,640 --> 00:19:36,620
And that's the same thing
that's going on here.

336
00:19:36,620 --> 00:19:41,490
It's just not with immodulation
put in, it's just

337
00:19:41,490 --> 00:19:45,480
dealing with the real parts and
imaginary parts directly.

338
00:19:45,480 --> 00:19:54,870
OK, so if we do that we get
a bunch of equations.

339
00:19:54,870 --> 00:19:56,040
They're sort of familiar
looking

340
00:19:56,040 --> 00:19:58,670
equations by now I hope.

341
00:19:58,670 --> 00:20:02,560
This is just the probability
density of this real 2j

342
00:20:02,560 --> 00:20:05,910
dimensional random variable.

343
00:20:05,910 --> 00:20:09,050
Which is all this junk that
we're used to seeing.

344
00:20:09,050 --> 00:20:15,910
We can collapse that into
e to the minus the norm

345
00:20:15,910 --> 00:20:18,190
squared of y minus a.

346
00:20:18,190 --> 00:20:20,790
This is the norm squared
in this 2j

347
00:20:20,790 --> 00:20:22,230
dimensional real space.

348
00:20:22,230 --> 00:20:26,590
It's not the norm squared
in this complex space.

349
00:20:26,590 --> 00:20:29,160
What gets confusing is
that those two norms

350
00:20:29,160 --> 00:20:31,150
are exactly the same.

351
00:20:31,150 --> 00:20:33,530
As we'll see in just
a few minutes.

352
00:20:33,530 --> 00:20:36,310
But anyway, what we're dealing
with now is this

353
00:20:36,310 --> 00:20:40,390
norm in real space.

354
00:20:40,390 --> 00:20:43,500
OK, note that's y--

355
00:20:43,500 --> 00:20:46,390
oh let me translate
this one for you.

356
00:20:49,540 --> 00:20:54,690
If we think of this v that we
received j complex random

357
00:20:54,690 --> 00:21:02,100
variables as being: real part
of v1, imaginary part of v1;

358
00:21:02,100 --> 00:21:07,510
real part of v2, imaginary part
of v2; and so forth, then

359
00:21:07,510 --> 00:21:13,140
y2j minus 1 minus
a2j minus one.

360
00:21:13,140 --> 00:21:15,120
I can't ever get these
formulas right.

361
00:21:18,110 --> 00:21:21,990
That should be a2j minus 1.

362
00:21:21,990 --> 00:21:23,240
There.

363
00:21:29,290 --> 00:21:32,200
This squared, plus
this squared--

364
00:21:32,200 --> 00:21:35,880
OK, in other words the real part
squared of the difference

365
00:21:35,880 --> 00:21:38,900
plus the imaginary part squared
of the difference-- is

366
00:21:38,900 --> 00:21:43,190
really just the same is
vj minus uj squared.

367
00:21:43,190 --> 00:21:47,080
OK, in other words you take the
complex number v sub j,

368
00:21:47,080 --> 00:21:50,970
you subtract off the real
number u sub j.

369
00:21:50,970 --> 00:21:55,380
And the way to do that, you
visualize this one complex

370
00:21:55,380 --> 00:21:59,870
variable in the complex plane,
and what you're doing is

371
00:21:59,870 --> 00:22:03,810
you're taking the square of the
real part of the distance,

372
00:22:03,810 --> 00:22:05,760
you're adding it to
the square of the

373
00:22:05,760 --> 00:22:07,960
imaginary part of the distance.

374
00:22:07,960 --> 00:22:11,160
OK, all of this is stuff you
learned in high school.

375
00:22:11,160 --> 00:22:14,390
Just viewed in a slightly
different way.

376
00:22:14,390 --> 00:22:18,790
OK, now when you take the
probability density with

377
00:22:18,790 --> 00:22:21,570
respect to these complex
vectors--

378
00:22:21,570 --> 00:22:24,310
which is what I want
to get at--

379
00:22:24,310 --> 00:22:28,000
probability density of these
complex variables really means

380
00:22:28,000 --> 00:22:30,890
the same thing as that with
the real variables.

381
00:22:30,890 --> 00:22:38,740
But then you wind up
with the magnitude

382
00:22:38,740 --> 00:22:41,750
of vj minus uj squared.

383
00:22:41,750 --> 00:22:44,800
And this term is really exactly
the same as these two

384
00:22:44,800 --> 00:22:45,940
terms there.

385
00:22:45,940 --> 00:22:49,150
So when you take this
probability density, you wind

386
00:22:49,150 --> 00:22:53,820
up with these terms the same and
with these terms the same.

387
00:22:53,820 --> 00:22:57,880
OK, in other words the complex
norm squared is the same as

388
00:22:57,880 --> 00:23:02,820
the real norm squared, when you
go from complex to real

389
00:23:02,820 --> 00:23:07,870
and imaginary parts of,
well-- as I said, this

390
00:23:07,870 --> 00:23:10,830
is the same as that.

391
00:23:10,830 --> 00:23:15,410
OK so when we look at the log
likelihood ratio, then, let's

392
00:23:15,410 --> 00:23:20,150
do the log likelihood ratio in
terms of the real parts first.

393
00:23:20,150 --> 00:23:24,820
We get the difference between
this norm squared of y minus

394
00:23:24,820 --> 00:23:28,030
a, and the norm squared
of y plus a.

395
00:23:28,030 --> 00:23:31,670
This is the part that comes from
the hypothesis zero, this

396
00:23:31,670 --> 00:23:35,120
is the part that comes from
the hypothesis one.

397
00:23:35,120 --> 00:23:37,350
OK, so we get these
two terms here.

398
00:23:37,350 --> 00:23:40,630
When we take the inner products
here, we get the same

399
00:23:40,630 --> 00:23:41,880
thing that we got before.

400
00:23:41,880 --> 00:23:45,440
Four times the inner
product of yna.

401
00:23:45,440 --> 00:23:47,560
Now, the whole reason for going
through all of this is

402
00:23:47,560 --> 00:23:49,340
this next formula.

403
00:23:49,340 --> 00:23:53,950
When you do this, you wind up
with this very bizarre four

404
00:23:53,950 --> 00:23:57,515
times the real part of the
inner product of v and u,

405
00:23:57,515 --> 00:23:59,400
divided by N0.

406
00:23:59,400 --> 00:24:01,700
And you get it in exactly
the same way

407
00:24:01,700 --> 00:24:03,050
that we got it before.

408
00:24:03,050 --> 00:24:06,010
Namely you take this norm
here, which is an inner

409
00:24:06,010 --> 00:24:09,080
product squared of v minus u.

410
00:24:09,080 --> 00:24:10,800
Let me write it out.

411
00:24:10,800 --> 00:24:12,050
I'll write it out here.

412
00:24:16,130 --> 00:24:26,730
Norm squared of y minus a, is
the norm squared of y plus the

413
00:24:26,730 --> 00:24:43,010
norm squared of a, plus the
inner product of minus ya,

414
00:24:43,010 --> 00:24:50,170
plus the inner product
of minus ay.

415
00:24:50,170 --> 00:24:52,320
What's the sum of these
two inner products?

416
00:24:52,320 --> 00:24:56,570
This inner product, by
definition in terms of

417
00:24:56,570 --> 00:25:02,630
integrals, is the integral
of minus y times a--

418
00:25:02,630 --> 00:25:04,770
complex conjugate.

419
00:25:04,770 --> 00:25:09,330
This is minus a times y--
complex conjugate.

420
00:25:09,330 --> 00:25:11,840
So in one case the complex
conjugate is here.

421
00:25:11,840 --> 00:25:13,860
In the other case it's
on the other term.

422
00:25:13,860 --> 00:25:16,320
In other words this and
this are complex

423
00:25:16,320 --> 00:25:18,920
conjugates of each other.

424
00:25:18,920 --> 00:25:21,970
What happens when you add
two complex conjugates?

425
00:25:21,970 --> 00:25:24,360
You get the real part
of the two of them.

426
00:25:24,360 --> 00:25:26,790
Okay so when you add these two
things you just get that real

427
00:25:26,790 --> 00:25:29,540
part there.

428
00:25:29,540 --> 00:25:31,300
OK, and then when you
do the other term,

429
00:25:31,300 --> 00:25:32,460
the same thing happens.

430
00:25:32,460 --> 00:25:36,590
The same cancellation that
we had before occurs.

431
00:25:36,590 --> 00:25:39,930
And these two inner products add
up so you wind up with the

432
00:25:39,930 --> 00:25:48,900
real part of vu over N0

433
00:25:48,900 --> 00:25:52,610
When we look at the picture
here, what does it mean?

434
00:25:52,610 --> 00:25:54,450
Well I suggest you first
look at the one

435
00:25:54,450 --> 00:25:56,910
dimensional case here.

436
00:25:56,910 --> 00:26:00,590
Namely on this one dimensional
case think of V as being a one

437
00:26:00,590 --> 00:26:03,620
dimensional complex
random variable.

438
00:26:03,620 --> 00:26:04,880
Then we can draw a picture.

439
00:26:04,880 --> 00:26:06,490
The picture make sense.

440
00:26:06,490 --> 00:26:10,070
And what we're dealing with is
the real and imaginary parts,

441
00:26:10,070 --> 00:26:17,170
and these distances here, when
you talk about the norm of V

442
00:26:17,170 --> 00:26:22,110
minus a-- namely what
corresponds to this line here,

443
00:26:22,110 --> 00:26:24,150
the length of this line--

444
00:26:24,150 --> 00:26:27,210
what do you really mean by it?

445
00:26:27,210 --> 00:26:32,940
If you took the inner product,
if you took the norm of v,

446
00:26:32,940 --> 00:26:38,990
with i times a-- namely that
the square root of minus 1

447
00:26:38,990 --> 00:26:41,800
times a-- would you get the same
thing or wouldn't you?

448
00:26:45,180 --> 00:26:48,970
If I take a complex number, and
I take the inner product

449
00:26:48,970 --> 00:26:53,090
of that complex number--
namely the product--

450
00:26:53,090 --> 00:27:06,050
of that, when I take the inner
product of ya, is this the

451
00:27:06,050 --> 00:27:12,710
same as the inner product
of y and i times a?

452
00:27:15,410 --> 00:27:16,190
Not at all.

453
00:27:16,190 --> 00:27:18,590
The two things are totally
different.

454
00:27:18,590 --> 00:27:22,240
Namely, inner products
are complex things.

455
00:27:22,240 --> 00:27:24,930
Norms are real things.

456
00:27:24,930 --> 00:27:28,080
And these norms, when you're
dealing with complex numbers,

457
00:27:28,080 --> 00:27:29,670
have real parts in them.

458
00:27:29,670 --> 00:27:33,900
In other words, this distance
that we're talking about here

459
00:27:33,900 --> 00:27:37,980
is not just the norm squared--

460
00:27:37,980 --> 00:27:40,840
well it is the norm squared--
because the norm has this

461
00:27:40,840 --> 00:27:43,580
complex feature built into it.

462
00:27:43,580 --> 00:27:46,580
Because people kept making that
mistake all the time.

463
00:27:46,580 --> 00:27:50,160
So they fudged the mathematics
to make it come out right.

464
00:27:50,160 --> 00:27:53,440
But after doing that, you have
to fudge the mathematics to

465
00:27:53,440 --> 00:27:56,210
come back to something that
makes sense here.

466
00:27:56,210 --> 00:27:59,350
So you have the real part of
this inner product, here.

467
00:27:59,350 --> 00:28:01,890
So in fact, what you're doing
when you're taking the inner

468
00:28:01,890 --> 00:28:05,590
product of two vectors and
you're trying to relate it to

469
00:28:05,590 --> 00:28:09,400
this plane here, this separation
plane, is you have

470
00:28:09,400 --> 00:28:11,300
to look at that separation
plane.

471
00:28:11,300 --> 00:28:13,620
You have to look at
that projection in

472
00:28:13,620 --> 00:28:15,050
terms of real numbers.

473
00:28:15,050 --> 00:28:17,880
Namely, you have to look
at the projection.

474
00:28:17,880 --> 00:28:23,110
First as being a complex
projection of v onto a, which

475
00:28:23,110 --> 00:28:25,520
gives you a complex number.

476
00:28:25,520 --> 00:28:29,470
And then after you do that you
have to you visualize yourself

477
00:28:29,470 --> 00:28:31,830
in a two dimensional
real space.

478
00:28:31,830 --> 00:28:34,870
And you have to project once
more from the two dimensional

479
00:28:34,870 --> 00:28:38,180
thing to the one dimensional
thing.

480
00:28:38,180 --> 00:28:40,860
And here where we're just
looking at one dimension to

481
00:28:40,860 --> 00:28:44,470
start with, we have to draw it
as a two dimensional space.

482
00:28:44,470 --> 00:28:48,490
And suddenly we are dealing
with this real part there

483
00:28:48,490 --> 00:28:50,440
while we're not dealing
with that here.

484
00:28:50,440 --> 00:28:53,740
Which is why when people say
minimum distance detection

485
00:28:53,740 --> 00:28:57,050
when they're dealing with
complex numbers it sounds

486
00:28:57,050 --> 00:28:59,100
very, very simple.

487
00:28:59,100 --> 00:29:00,960
But in fact, it's
not so simple.

488
00:29:03,500 --> 00:29:06,570
If you view this in the complex
plane, is this a

489
00:29:06,570 --> 00:29:11,410
linear operation or isn't it?

490
00:29:11,410 --> 00:29:15,000
When you're looking at things
as complex vectors.

491
00:29:15,000 --> 00:29:21,220
Is this thing a sub space of
the complex vector space?

492
00:29:21,220 --> 00:29:23,170
No, it's not.

493
00:29:23,170 --> 00:29:24,970
It's not a sub space.

494
00:29:24,970 --> 00:29:29,490
Because, to be a sub space you
have to be able to multiply by

495
00:29:29,490 --> 00:29:31,390
arbitrary scalors--

496
00:29:31,390 --> 00:29:33,960
which includes complex
numbers-- and

497
00:29:33,960 --> 00:29:35,840
stay in the same space.

498
00:29:35,840 --> 00:29:39,410
And here the complex numbers
are important.

499
00:29:39,410 --> 00:29:40,650
OK?

500
00:29:40,650 --> 00:29:43,110
You should go back and
think about that.

501
00:29:43,110 --> 00:29:46,650
You will be confused about it
for the first ten times you

502
00:29:46,650 --> 00:29:48,370
think about it.

503
00:29:48,370 --> 00:29:51,900
For those of you who stick with
it and carry through on

504
00:29:51,900 --> 00:29:53,990
it, you'll be happy because
you'll never be

505
00:29:53,990 --> 00:29:56,800
confused about it again.

506
00:29:56,800 --> 00:30:01,080
OK, anyway thats real
numbers there.

507
00:30:01,080 --> 00:30:06,860
And the most straightforward way
to deal with complex noise

508
00:30:06,860 --> 00:30:09,650
is to first turn it
into real noise.

509
00:30:09,650 --> 00:30:12,410
If you do that you never
get confused.

510
00:30:12,410 --> 00:30:16,690
And otherwise you only have
half the analytical work.

511
00:30:16,690 --> 00:30:19,300
You only have half the
writing to do.

512
00:30:19,300 --> 00:30:22,040
But you never know whether
you've done the right thing

513
00:30:22,040 --> 00:30:24,470
until you go back and check.

514
00:30:24,470 --> 00:30:28,830
OK the probability of error
for maximum likelihood

515
00:30:28,830 --> 00:30:33,170
detection, in other words where
the threshold for the

516
00:30:33,170 --> 00:30:37,820
log likelihood ratio is zero, is
simply the same thing that

517
00:30:37,820 --> 00:30:39,070
it was before.

518
00:30:39,070 --> 00:30:42,150
Namely it's the q function.

519
00:30:42,150 --> 00:30:47,530
This tale of the Gaussian
normal function.

520
00:30:47,530 --> 00:30:51,120
Of the norm of a divided by the
square root of N0 over 2.

521
00:30:51,120 --> 00:30:57,220
In other words it's the length
of a divided by the by the

522
00:30:57,220 --> 00:31:05,020
length of a one standard
deviation of the noise.

523
00:31:05,020 --> 00:31:12,260
When you put that in terms of,
well, if you write this as the

524
00:31:12,260 --> 00:31:16,220
square root of the norm squared
then you get this

525
00:31:16,220 --> 00:31:17,330
formula here.

526
00:31:17,330 --> 00:31:20,780
Because e sub b is just
the energy of

527
00:31:20,780 --> 00:31:22,690
these antipodal signals.

528
00:31:22,690 --> 00:31:25,580
When you look at this in terms
of the complex random

529
00:31:25,580 --> 00:31:28,860
variables, you get
the same thing.

530
00:31:28,860 --> 00:31:29,230
OK?

531
00:31:29,230 --> 00:31:33,740
You get u instead of a because,
in fact, in the

532
00:31:33,740 --> 00:31:37,110
complex plane and the real
plane, distances turn out to

533
00:31:37,110 --> 00:31:38,890
be the same.

534
00:31:38,890 --> 00:31:45,380
But again, in all cases it's
square root of 2eb over n0

535
00:31:45,380 --> 00:31:51,490
Now, that is true for any vector
at all where these

536
00:31:51,490 --> 00:31:53,450
norms are appropriate.

537
00:31:53,450 --> 00:31:58,910
When we start dealing with
functions what happens?

538
00:31:58,910 --> 00:32:01,910
When we start dealing with
functions, what we're going to

539
00:32:01,910 --> 00:32:04,680
do is we're going to take this
vector, we're going to turn it

540
00:32:04,680 --> 00:32:06,420
into a wave form.

541
00:32:06,420 --> 00:32:08,480
We're going to transmit
the wave form.

542
00:32:08,480 --> 00:32:10,650
The wave form is going
to come back to us.

543
00:32:10,650 --> 00:32:15,400
We're going to demodulate it,
get back to a number again.

544
00:32:15,400 --> 00:32:19,410
And that's the next thing that
I want to talk about.

545
00:32:19,410 --> 00:32:21,550
Because what I want to convince
you of is the

546
00:32:21,550 --> 00:32:26,150
property of white Gaussian noise
which is so important.

547
00:32:26,150 --> 00:32:31,990
Is that it doesn't make any
difference how you modulate.

548
00:32:31,990 --> 00:32:34,630
All modulation systems
are the same.

549
00:32:34,630 --> 00:32:36,990
All modulation schemes
are the same.

550
00:32:36,990 --> 00:32:39,110
All frequencies are the same.

551
00:32:39,110 --> 00:32:42,220
There is no way you can avoid
white Gaussian noise.

552
00:32:42,220 --> 00:32:44,550
There is no way you can
get screwed by it.

553
00:32:44,550 --> 00:32:48,610
No matter what you do, the
same thing happens.

554
00:32:48,610 --> 00:32:52,540
You can always take all these
problems where you're dealing

555
00:32:52,540 --> 00:32:53,540
with wave forms.

556
00:32:53,540 --> 00:32:58,070
You can convert them to finite
dimensional vector problems.

557
00:32:58,070 --> 00:33:00,410
And when you convert it into
a finite dimensional vector

558
00:33:00,410 --> 00:33:03,940
problem all of the orthonormal
functions that you're using

559
00:33:03,940 --> 00:33:05,590
all pass away.

560
00:33:05,590 --> 00:33:07,930
Because none of them are
relevant anymore.

561
00:33:07,930 --> 00:33:08,350
OK?

562
00:33:08,350 --> 00:33:11,580
That's the bottom line
of all of that.

563
00:33:14,100 --> 00:33:15,520
OK.

564
00:33:15,520 --> 00:33:20,370
We haven't really talked about
M-ARY hypothesis testing.

565
00:33:20,370 --> 00:33:22,590
So I want to talk about
it a little bit now.

566
00:33:22,590 --> 00:33:28,110
I talked about it just a
shade, but not much.

567
00:33:28,110 --> 00:33:32,010
When we want to detect between
m different hypothesis--

568
00:33:32,010 --> 00:33:34,560
namely in the vector case
we're going to now be

569
00:33:34,560 --> 00:33:37,860
detecting not between antipodal
signals but m

570
00:33:37,860 --> 00:33:41,910
signals which are placed
any place at all.

571
00:33:41,910 --> 00:33:47,610
We already said what the MAP,
optimal MAP test was.

572
00:33:47,610 --> 00:33:52,260
You see an observation, you're
trying to guess what the input

573
00:33:52,260 --> 00:33:55,380
was, or what the
hypothesis was.

574
00:33:55,380 --> 00:34:02,550
And in general, the MAP test
says try to find that j,

575
00:34:02,550 --> 00:34:06,970
namely that hypothesis, for
which the a priori probability

576
00:34:06,970 --> 00:34:11,570
of hypothesis j, times the
likelihood-- namely the

577
00:34:11,570 --> 00:34:14,910
probability that you
see v given h of

578
00:34:14,910 --> 00:34:19,130
v, given j is maximum.

579
00:34:19,130 --> 00:34:23,595
OK, this is just standard
formula for finding a

580
00:34:23,595 --> 00:34:25,330
posteriori probabilities.

581
00:34:25,330 --> 00:34:32,110
Where you factor out the
marginal on, when you cancel

582
00:34:32,110 --> 00:34:33,270
out the marginal on v.

583
00:34:33,270 --> 00:34:37,850
In other words, what this rule
says is to do MAP testing,

584
00:34:37,850 --> 00:34:41,930
what you do is you find the a
posteriori probabilities of

585
00:34:41,930 --> 00:34:45,260
each of the hypotheses and you
choose the a posteriori

586
00:34:45,260 --> 00:34:48,420
probability which is largest.

587
00:34:48,420 --> 00:34:50,540
Perfect common sense.

588
00:34:50,540 --> 00:34:54,130
The way we're going to do
that, the way which is

589
00:34:54,130 --> 00:34:58,330
particularly convenient, is you
do it the same way that

590
00:34:58,330 --> 00:35:01,810
we've been doing all along
for binary hypotheses.

591
00:35:01,810 --> 00:35:05,170
The way to do a MAP test, at
least one way to do a MAP

592
00:35:05,170 --> 00:35:09,980
test, is you compare every pair
of hypothesis and you

593
00:35:09,980 --> 00:35:12,180
choose the most likely
of each pair.

594
00:35:12,180 --> 00:35:15,290
And when you've got it all
done, you have a winner.

595
00:35:15,290 --> 00:35:16,310
OK?

596
00:35:16,310 --> 00:35:19,230
Mainly you can always compare
any objects if they're

597
00:35:19,230 --> 00:35:20,750
comparable.

598
00:35:20,750 --> 00:35:25,330
And after you compare each pair,
you take the one which

599
00:35:25,330 --> 00:35:28,240
beats every other one and
that's the winner.

600
00:35:28,240 --> 00:35:28,630
OK?

601
00:35:28,630 --> 00:35:33,900
So what you do is you do a
pairwise test between each

602
00:35:33,900 --> 00:35:34,790
hypothesis.

603
00:35:34,790 --> 00:35:38,920
Namely, the likelihood ratio
of m relative to m prime.

604
00:35:38,920 --> 00:35:43,670
Is the likelihood of the output
conditional on m,

605
00:35:43,670 --> 00:35:45,790
divided by the likelihood
of the output

606
00:35:45,790 --> 00:35:47,350
conditional on m prime.

607
00:35:47,350 --> 00:35:53,020
You compare it with the a priori
probabilities, and the

608
00:35:53,020 --> 00:35:56,820
point here is that nothing
really has been added.

609
00:35:56,820 --> 00:35:59,710
You have the same problem
you had before, OK?

610
00:35:59,710 --> 00:36:01,350
Nothing new.

611
00:36:01,350 --> 00:36:04,780
It's just gotten n square
times as complicated.

612
00:36:04,780 --> 00:36:09,870
The computation is free now, so
you have exactly the same

613
00:36:09,870 --> 00:36:12,840
problem that you had before.

614
00:36:12,840 --> 00:36:16,000
If you have to write it out on
paper, yeah it's much more

615
00:36:16,000 --> 00:36:17,050
complicated.

616
00:36:17,050 --> 00:36:20,600
But conceptually, it's not.

617
00:36:20,600 --> 00:36:23,070
You have to remember that
the signals are

618
00:36:23,070 --> 00:36:24,810
not antipodal here.

619
00:36:24,810 --> 00:36:28,740
But what we're dealing with
mostly at this point is this

620
00:36:28,740 --> 00:36:30,840
Gaussian noise case.

621
00:36:30,840 --> 00:36:35,490
And here, what you observe,
is signal plus noise.

622
00:36:35,490 --> 00:36:41,800
And Z is zero-mean jointly Gauss
and s is discrete with n

623
00:36:41,800 --> 00:36:43,530
possible values.

624
00:36:43,530 --> 00:36:47,540
OK, so let's see what
that means.

625
00:36:47,540 --> 00:36:48,790
Here's a picture of it.

626
00:36:51,230 --> 00:36:54,890
If you have three singles which
are each two dimensional

627
00:36:54,890 --> 00:36:59,170
vectors, suppose one of them is
s0, suppose one of them is

628
00:36:59,170 --> 00:37:02,880
s1, suppose one of them s2.

629
00:37:02,880 --> 00:37:04,320
OK?

630
00:37:04,320 --> 00:37:09,570
And now you want to pairwise,
see which one is most likely.

631
00:37:09,570 --> 00:37:12,850
And let's think of doing this
first for the maximum

632
00:37:12,850 --> 00:37:14,090
likelihood case.

633
00:37:14,090 --> 00:37:15,100
What do you do?

634
00:37:15,100 --> 00:37:21,200
You set up a perpendicular
bisector between s0 and s1.

635
00:37:21,200 --> 00:37:23,940
That's this line here.

636
00:37:23,940 --> 00:37:27,550
And if you weren't to worry
about s2, everything on this

637
00:37:27,550 --> 00:37:31,190
side would go into
H equals zero.

638
00:37:31,190 --> 00:37:34,620
And everything on this side
would go into H equals one.

639
00:37:34,620 --> 00:37:37,150
Namely, whatever's closest
to this point

640
00:37:37,150 --> 00:37:38,350
gets mapped into it.

641
00:37:38,350 --> 00:37:41,770
Whatever's closest to this point
gets mapped into it.

642
00:37:41,770 --> 00:37:46,160
If you're doing MAP testing,
what happens?

643
00:37:46,160 --> 00:37:52,820
In the test between this and
this you had the same

644
00:37:52,820 --> 00:37:56,320
orientation for this line, but
it just gets shifted a little

645
00:37:56,320 --> 00:38:00,140
bit this way or a little
bit this way.

646
00:38:00,140 --> 00:38:00,630
OK?

647
00:38:00,630 --> 00:38:02,720
Then you compare this
with this and you

648
00:38:02,720 --> 00:38:05,210
get this line here.

649
00:38:05,210 --> 00:38:07,110
Same argument as before.

650
00:38:07,110 --> 00:38:11,000
It's just comparing two things
are not antipodal they've just

651
00:38:11,000 --> 00:38:13,730
been shifted off from the
origin a little bit.

652
00:38:13,730 --> 00:38:16,470
But for the maximum likelihood
you still take the

653
00:38:16,470 --> 00:38:19,430
perpendicular bisector
between them.

654
00:38:19,430 --> 00:38:22,770
And then you compare these two
and you got a perpendicular

655
00:38:22,770 --> 00:38:24,410
bisector between those.

656
00:38:26,970 --> 00:38:30,050
And these perpendicular
bisectors, in two dimensions,

657
00:38:30,050 --> 00:38:32,660
always come together
at one point.

658
00:38:32,660 --> 00:38:33,920
And I don't know why.

659
00:38:38,010 --> 00:38:40,430
And if you looked at it often
enough, you probably know why.

660
00:38:40,430 --> 00:38:43,230
And you could probably prove
it in about ten minutes.

661
00:38:43,230 --> 00:38:46,740
But in fact these things always
come together somehow.

662
00:38:46,740 --> 00:38:49,830
If you do the MAP test, they
always come together also.

663
00:38:49,830 --> 00:38:54,670
You can shift each of them in
arbitrary ways and somehow

664
00:38:54,670 --> 00:38:58,230
they always come together
in this point.

665
00:38:58,230 --> 00:39:02,410
OK, the separators between
decision regions here are the

666
00:39:02,410 --> 00:39:08,290
set of points where the real
part of the inner product, vu,

667
00:39:08,290 --> 00:39:09,060
is constant.

668
00:39:09,060 --> 00:39:09,430
OK?

669
00:39:09,430 --> 00:39:13,670
Again, for dealing with complex
vectors, you got to

670
00:39:13,670 --> 00:39:17,020
both do the projection and then
do the projection again

671
00:39:17,020 --> 00:39:19,550
onto the real part of
this projection.

672
00:39:19,550 --> 00:39:21,580
So it's sort of a two
way projection.

673
00:39:21,580 --> 00:39:27,220
Because probability densities in
j dimensional complex space

674
00:39:27,220 --> 00:39:30,380
are really 2j dimensional
quantities.

675
00:39:30,380 --> 00:39:32,690
And when you're comparing
them, you really have to

676
00:39:32,690 --> 00:39:35,610
compare things in terms
of that 2j dimensional

677
00:39:35,610 --> 00:39:38,060
probability density.

678
00:39:38,060 --> 00:39:40,890
OK so that's why
that comes out.

679
00:39:40,890 --> 00:39:43,620
These are best visualized in
separate, real, and imaginary

680
00:39:43,620 --> 00:39:44,110
coordinates.

681
00:39:44,110 --> 00:39:48,850
And for maximum likelihood
detection, the regions are

682
00:39:48,850 --> 00:39:50,360
Voronoi regions.

683
00:39:50,360 --> 00:39:50,600
OK?

684
00:39:50,600 --> 00:39:58,520
We talked about Voronoi regions
in terms of doing

685
00:39:58,520 --> 00:40:00,010
quantization.

686
00:40:00,010 --> 00:40:02,840
And we found out if you wanted
to minimize the mean square

687
00:40:02,840 --> 00:40:06,620
error, what you did was you
set up regions, which are

688
00:40:06,620 --> 00:40:10,020
perpendicular bisectors between
all the points.

689
00:40:10,020 --> 00:40:12,810
And here you get the same
perpendicular bisectors

690
00:40:12,810 --> 00:40:14,880
between the points.

691
00:40:14,880 --> 00:40:16,800
And everybody-- because
of that-- thinks that

692
00:40:16,800 --> 00:40:21,740
quantization has a great deal
to do with error probability

693
00:40:21,740 --> 00:40:24,430
when you have large
sets of signals.

694
00:40:24,430 --> 00:40:28,230
And it probably has something
to do with it, but I don't

695
00:40:28,230 --> 00:40:30,530
know what other than the fact
that you've got Voronoi

696
00:40:30,530 --> 00:40:33,900
regions in each case.

697
00:40:33,900 --> 00:40:35,150
Which is what you get.

698
00:40:45,060 --> 00:40:55,440
OK, so that's where we are with
both complex vectors and

699
00:40:55,440 --> 00:40:56,640
real vectors.

700
00:40:56,640 --> 00:40:59,480
I want to now just restrict
attention to real wave forms

701
00:40:59,480 --> 00:41:02,770
so I don't have to keep going
back and forth between the

702
00:41:02,770 --> 00:41:04,640
real and imaginary case.

703
00:41:04,640 --> 00:41:07,800
If you're thinking in terms of
QAM, we're now thinking in

704
00:41:07,800 --> 00:41:10,920
terms of what goes
on at passband.

705
00:41:10,920 --> 00:41:13,300
Why do we want to think of
what goes on at passband?

706
00:41:13,300 --> 00:41:16,890
Because that's where
the noise hits us.

707
00:41:16,890 --> 00:41:21,620
And in a fundamental sense,
all of the stuff about QAM

708
00:41:21,620 --> 00:41:24,250
really isn't fundamental.

709
00:41:24,250 --> 00:41:27,520
I mean, it's all done-- all this
stuff down at passband--

710
00:41:27,520 --> 00:41:30,170
is all done because people
thought it was easier to

711
00:41:30,170 --> 00:41:32,020
implement things there.

712
00:41:32,020 --> 00:41:36,830
It's the only reason for all of
that mess with dealing with

713
00:41:36,830 --> 00:41:39,540
all of these complex signals.

714
00:41:39,540 --> 00:41:42,240
If we really want to deal with
the problem in a fundamental

715
00:41:42,240 --> 00:41:44,600
way, what we want to
do is to choose a

716
00:41:44,600 --> 00:41:47,220
signal set up at passband.

717
00:41:47,220 --> 00:41:49,890
Do detection up at passband.

718
00:41:49,890 --> 00:41:52,620
And then after we find out what
the optimal detection is

719
00:41:52,620 --> 00:41:56,430
up at passband, see if we can
actually implement that down

720
00:41:56,430 --> 00:41:58,200
at baseband.

721
00:41:58,200 --> 00:42:01,330
So the fundamental problem is
looking at single sets up at

722
00:42:01,330 --> 00:42:06,080
passband and analyze
what they all mean.

723
00:42:06,080 --> 00:42:11,290
OK, so we're going to generalize
both PAM and QAM.

724
00:42:11,290 --> 00:42:14,590
And now we're going to look at
the general problem where what

725
00:42:14,590 --> 00:42:18,580
we're dealing with
is a single set.

726
00:42:18,580 --> 00:42:21,020
Which is m signals.

727
00:42:21,020 --> 00:42:24,760
Each of them we're going to
visualize as a vector in j

728
00:42:24,760 --> 00:42:28,340
dimensional space. m different
signals, j dimensional space.

729
00:42:28,340 --> 00:42:31,590
Don't confuse the dimension
of space with

730
00:42:31,590 --> 00:42:33,420
the number of signals.

731
00:42:33,420 --> 00:42:33,770
OK?

732
00:42:33,770 --> 00:42:36,760
You can have an arbitrarily
large dimensional space and

733
00:42:36,760 --> 00:42:38,770
just binary signals.

734
00:42:38,770 --> 00:42:42,670
Or you can have an arbitrarily
large set of signals and you

735
00:42:42,670 --> 00:42:43,970
can be dealing with it.

736
00:42:43,970 --> 00:42:48,050
In PAM, for example, it's just
all done in one dimension.

737
00:42:48,050 --> 00:42:50,110
So j there is equal to 1.

738
00:42:50,110 --> 00:42:52,600
QAM j is equal to 2.

739
00:42:52,600 --> 00:42:54,540
We now want to look at
more general things.

740
00:42:54,540 --> 00:42:58,110
Partly because we want to look
at orthoginal wave forms.

741
00:42:58,110 --> 00:43:02,080
We want to look at orthoginal
wave forms for two reasons.

742
00:43:02,080 --> 00:43:05,890
One is that we would like to
show that by using orthoginal

743
00:43:05,890 --> 00:43:10,280
wave forms, you can reach what
we've called the capacity of a

744
00:43:10,280 --> 00:43:12,970
white Gaussian noise channel.

745
00:43:12,970 --> 00:43:17,480
And two, when we get to studying
wireless it's very,

746
00:43:17,480 --> 00:43:21,700
very useful to base
signal sets on

747
00:43:21,700 --> 00:43:23,510
orthonormal sets of functions.

748
00:43:23,510 --> 00:43:25,820
And we'll see why each
of those things

749
00:43:25,820 --> 00:43:28,710
happen as we move on.

750
00:43:28,710 --> 00:43:31,890
OK so we're going to denote the
single set as the set of

751
00:43:31,890 --> 00:43:35,170
vectors, a1 up to a sub m.

752
00:43:35,170 --> 00:43:38,480
And in that signal set, we will
denote the vector, a sub

753
00:43:38,480 --> 00:43:44,116
m, as a j dimensional vector,
a sub m1, a sub m2, up to a

754
00:43:44,116 --> 00:43:45,960
sub m capital j.

755
00:43:45,960 --> 00:43:47,950
So j is at the dimension.

756
00:43:47,950 --> 00:43:52,810
m is just a component
of these vectors.

757
00:43:52,810 --> 00:43:55,540
I'm going to create
a set of capital J

758
00:43:55,540 --> 00:43:57,930
orthonormal wave forms.

759
00:43:57,930 --> 00:43:59,150
They can be anything at all.

760
00:43:59,150 --> 00:44:02,220
I don't care what they are.

761
00:44:02,220 --> 00:44:05,080
I'm going to use those
orthonormal wave forms in

762
00:44:05,080 --> 00:44:07,290
order to modulate the signal--

763
00:44:07,290 --> 00:44:09,310
which is now a vector--

764
00:44:09,310 --> 00:44:13,240
up to some waveform.

765
00:44:13,240 --> 00:44:17,910
This is really the standard way
we've been turning signals

766
00:44:17,910 --> 00:44:20,820
into waveforms all along.

767
00:44:20,820 --> 00:44:23,910
It's just that now we're looking
at the general case

768
00:44:23,910 --> 00:44:26,820
instead of all of these specific
cases that we've been

769
00:44:26,820 --> 00:44:27,370
looking at.

770
00:44:27,370 --> 00:44:34,370
All of the special cases all fit
into this same category.

771
00:44:34,370 --> 00:44:38,620
So we have these capital
M different waveforms.

772
00:44:38,620 --> 00:44:45,110
And we're going to transmit one
of them and then at the

773
00:44:45,110 --> 00:44:49,370
receiver we're going to try to
decide which one was sent.

774
00:44:49,370 --> 00:44:52,240
OK, well one of the reasons why
I'm going through all of

775
00:44:52,240 --> 00:44:57,580
this generality is that there's
an issue we haven't

776
00:44:57,580 --> 00:45:00,210
talked about yet.

777
00:45:00,210 --> 00:45:05,020
All of the stuff we've done on
detection so far we have had

778
00:45:05,020 --> 00:45:07,030
one hypothesis.

779
00:45:07,030 --> 00:45:09,870
Could be M-ary could
be binary.

780
00:45:09,870 --> 00:45:11,500
We have sent something.

781
00:45:11,500 --> 00:45:13,190
We have received something.

782
00:45:13,190 --> 00:45:14,890
We have made a detection.

783
00:45:14,890 --> 00:45:15,270
OK?

784
00:45:15,270 --> 00:45:18,780
In other words, for all of the
antipodal stuff we've done, we

785
00:45:18,780 --> 00:45:20,640
built a communication system.

786
00:45:20,640 --> 00:45:22,230
We set it all up.

787
00:45:22,230 --> 00:45:24,400
We transmitted one bit.

788
00:45:24,400 --> 00:45:25,980
We've received the one bit.

789
00:45:25,980 --> 00:45:27,570
We've made a decision on it.

790
00:45:27,570 --> 00:45:29,890
Then we've torn down the
communication system

791
00:45:29,890 --> 00:45:31,930
and we've gone home.

792
00:45:31,930 --> 00:45:35,130
You really want to transmit a
whole sequence of symbols or

793
00:45:35,130 --> 00:45:37,870
signals or waveforms.

794
00:45:37,870 --> 00:45:42,600
So we want to deal
with that now.

795
00:45:42,600 --> 00:45:45,040
So we need some way
to transmit a

796
00:45:45,040 --> 00:45:48,770
succession of M-ary signals.

797
00:45:48,770 --> 00:45:52,930
And we'll call this succession
of signals-- mainly the

798
00:45:52,930 --> 00:45:55,780
signals are the things that
get chosen from the signal

799
00:45:55,780 --> 00:45:59,210
set-- we'll call them
x of k, k of z.

800
00:45:59,210 --> 00:46:01,030
Which is what we've been
calling them all along.

801
00:46:01,030 --> 00:46:04,540
We've been transmitting a
sequence of things when we're

802
00:46:04,540 --> 00:46:05,430
dealing with PAM.

803
00:46:05,430 --> 00:46:07,880
And aa in am.

804
00:46:07,880 --> 00:46:12,400
Why do I call them
xk instead of ak?

805
00:46:15,740 --> 00:46:18,940
Well I can't call them ak
because when I talk about ak

806
00:46:18,940 --> 00:46:23,000
I'm talking about the k'th
signal in the signal set.

807
00:46:23,000 --> 00:46:26,940
And here what I'm talking about
now is transmitting a

808
00:46:26,940 --> 00:46:28,940
sequence of choices.

809
00:46:28,940 --> 00:46:32,200
Each one of these choices, the
first choice is a choice from

810
00:46:32,200 --> 00:46:33,340
this set here.

811
00:46:33,340 --> 00:46:36,920
The second thing that I send
is a choice from this set.

812
00:46:36,920 --> 00:46:40,030
The third thing that I send
is a choice from this set.

813
00:46:40,030 --> 00:46:46,610
So x1, x2, x3, and x4 and so
forth are different choices

814
00:46:46,610 --> 00:46:48,870
among these M-ary signals.

815
00:46:48,870 --> 00:46:52,680
If m is 2 to the 6--

816
00:46:52,680 --> 00:46:55,770
OK in other words, every time
I transmit a signal I'm

817
00:46:55,770 --> 00:46:58,140
transmitting six bits.

818
00:46:58,140 --> 00:46:58,560
OK.

819
00:46:58,560 --> 00:47:01,980
In a communication system
we transmit six bits.

820
00:47:01,980 --> 00:47:04,060
Then we transmit another
six bits.

821
00:47:04,060 --> 00:47:09,200
Then we transmit another six
bits, and so on forever.

822
00:47:09,200 --> 00:47:14,030
OK, so I need to talk about
these things as ways of a

823
00:47:14,030 --> 00:47:15,610
succession of signals.

824
00:47:15,610 --> 00:47:19,670
The thing that I'm trying to
get at is, how do you know

825
00:47:19,670 --> 00:47:23,940
when you send one of these
signals that you don't have to

826
00:47:23,940 --> 00:47:27,180
worry about the other signals?

827
00:47:27,180 --> 00:47:28,450
How do you know that they don't

828
00:47:28,450 --> 00:47:29,660
interfere with each other?

829
00:47:29,660 --> 00:47:33,870
Well, we sort of solved the
problem of them interfering

830
00:47:33,870 --> 00:47:37,370
with each other in dealing with
Nyquist, but we haven't

831
00:47:37,370 --> 00:47:40,580
dealt with that problem at all
since we started to talk about

832
00:47:40,580 --> 00:47:42,180
random processes.

833
00:47:42,180 --> 00:47:45,350
So we don't know whether we've
really solved it or not.

834
00:47:45,350 --> 00:47:48,950
So at this point we have to
solve that problem, and that's

835
00:47:48,950 --> 00:47:50,200
what we're aiming at.

836
00:47:52,530 --> 00:47:57,190
So, the one way to be able to
transmit a whole sequence of

837
00:47:57,190 --> 00:48:02,510
signals is to have these choices
of vectors here and to

838
00:48:02,510 --> 00:48:07,210
develop a set of orthonormal
waveforms, v1 up to v sub j,

839
00:48:07,210 --> 00:48:13,040
which all have the property that
if you time shift them

840
00:48:13,040 --> 00:48:18,180
each by capital T,
they're stilll--

841
00:48:18,180 --> 00:48:20,950
if you time shift them by
capital T, they have to be

842
00:48:20,950 --> 00:48:22,900
orthonormal to each other.

843
00:48:22,900 --> 00:48:25,760
The question you're facing is
whether these things that

844
00:48:25,760 --> 00:48:28,400
you're transmitting are
orthoganol to all of these

845
00:48:28,400 --> 00:48:30,700
things that you're
transmitting.

846
00:48:30,700 --> 00:48:33,890
Now, in the Nyquist problem, we
dealt with the problem of

847
00:48:33,890 --> 00:48:38,470
how do you take one waveform
here and make it orthonormal

848
00:48:38,470 --> 00:48:40,040
to all of its time shifts.

849
00:48:40,040 --> 00:48:42,270
And we solved that problem.

850
00:48:42,270 --> 00:48:44,520
In the quiz you solved the
problem-- although you

851
00:48:44,520 --> 00:48:46,860
probably didn't recognize it--

852
00:48:46,860 --> 00:48:49,510
of dealing with orthonormal
functions both

853
00:48:49,510 --> 00:48:51,320
in time and in frequency.

854
00:48:51,320 --> 00:48:54,200
And that's the kind of thing
we would like to use here.

855
00:48:54,200 --> 00:48:57,990
If I take a set of orthonormal
pulses and then I modulate

856
00:48:57,990 --> 00:49:01,590
those orthonormal pulses up
to a higher frequency--

857
00:49:01,590 --> 00:49:04,560
which is out of the range of
this first frequency-- then I

858
00:49:04,560 --> 00:49:08,140
can send one sequence of
orthonormal functions down

859
00:49:08,140 --> 00:49:11,070
here, another set of orthonormal
functions up here

860
00:49:11,070 --> 00:49:12,810
in a different frequency
range.

861
00:49:12,810 --> 00:49:14,930
Another one up here in
a different frequency

862
00:49:14,930 --> 00:49:16,340
range and so forth.

863
00:49:16,340 --> 00:49:19,370
So then all of these orthonormal
functions are

864
00:49:19,370 --> 00:49:21,170
going to be orthonormal
to each other.

865
00:49:21,170 --> 00:49:21,580
Yeah?

866
00:49:21,580 --> 00:49:27,355
AUDIENCE: Are you saying
that each x of

867
00:49:27,355 --> 00:49:30,260
k is its own frequency?

868
00:49:30,260 --> 00:49:30,560
Because each x of k is
infinitely long.

869
00:49:30,560 --> 00:49:35,120
PROFESSOR: Each x of
k is going to--

870
00:49:35,120 --> 00:49:41,880
each x of k is just a vector
of j components.

871
00:49:41,880 --> 00:49:45,500
I'm going to modulate that
vector, x of k, into a

872
00:49:45,500 --> 00:49:51,230
waveform, x of t, which might be
finite duration or it might

873
00:49:51,230 --> 00:49:53,020
be infinite duration.

874
00:49:53,020 --> 00:49:57,670
I mean, it's going to go to zero
very, very fast, anyway.

875
00:49:57,670 --> 00:50:01,200
And whether it is absolutely
time limited or not is

876
00:50:01,200 --> 00:50:04,550
something I don't really care
about at this point.

877
00:50:04,550 --> 00:50:09,770
But the point is I can create
functions where, in fact, I

878
00:50:09,770 --> 00:50:13,240
have a whole sequence of
functions here and they're

879
00:50:13,240 --> 00:50:15,420
orthonormal to all
of the shifts.

880
00:50:15,420 --> 00:50:22,390
One way of doing this, for
example, is to make capital J

881
00:50:22,390 --> 00:50:25,540
a bunch of little time
shifts on functions.

882
00:50:25,540 --> 00:50:29,610
I can pick a function p of t,
which is orthonormal to all of

883
00:50:29,610 --> 00:50:33,240
its shifts, in terms of t1.

884
00:50:33,240 --> 00:50:38,530
I can send J of those pulses
to take care of x of k.

885
00:50:38,530 --> 00:50:43,000
And then I can use a capital T
in here, which is j times this

886
00:50:43,000 --> 00:50:44,730
little t that I was using.

887
00:50:44,730 --> 00:50:47,700
And I can send another
set of functions.

888
00:50:47,700 --> 00:50:51,720
So I can do that, I can move
up and down in frequency.

889
00:50:51,720 --> 00:50:53,710
I can choose any old
set of orthonormal

890
00:50:53,710 --> 00:50:55,460
functions that I want to.

891
00:50:55,460 --> 00:50:59,910
But the thing that I want to do
is I want to make sure that

892
00:50:59,910 --> 00:51:08,910
for each vector, x of k, that
I'm sending in time when I

893
00:51:08,910 --> 00:51:14,140
modulate it to a waveform, that
waveform is orthonormal

894
00:51:14,140 --> 00:51:16,860
to the waveforms for
every other k.

895
00:51:16,860 --> 00:51:18,970
And there are lots of
ways of doing that.

896
00:51:18,970 --> 00:51:21,190
OK, mainly there are
lots of choices

897
00:51:21,190 --> 00:51:23,900
of orthonormal functions.

898
00:51:23,900 --> 00:51:28,670
OK so anyway what I'm going to
be doing is making all of

899
00:51:28,670 --> 00:51:31,480
these signals orthogonal
to each other.

900
00:51:36,350 --> 00:51:39,880
OK, so the transmitting waveform
for this sequence of

901
00:51:39,880 --> 00:51:45,980
modulated signals is x of t,
which is the sum of x of k

902
00:51:45,980 --> 00:51:47,410
times t minus kt.

903
00:51:47,410 --> 00:51:50,860
Mainly the same thing we
were doing before.

904
00:51:50,860 --> 00:51:54,920
Except now I have also the
problem that each of these

905
00:51:54,920 --> 00:51:59,560
waveforms, x of k of t,
has to be some sum

906
00:51:59,560 --> 00:52:01,340
of orthonormal functions.

907
00:52:01,340 --> 00:52:03,380
So the problem becomes a little
more difficult than

908
00:52:03,380 --> 00:52:05,380
what it was before.

909
00:52:05,380 --> 00:52:06,570
But in fact it's--

910
00:52:06,570 --> 00:52:09,140
I mean this is just standard
communication.

911
00:52:09,140 --> 00:52:11,290
Every wireless system
in the world uses

912
00:52:11,290 --> 00:52:12,610
this kind of scheme.

913
00:52:12,610 --> 00:52:14,700
They don't use QAM or PAM.

914
00:52:14,700 --> 00:52:19,320
They use something much
more like this.

915
00:52:19,320 --> 00:52:23,780
OK, so now our problem is you
want to detect a generic x

916
00:52:23,780 --> 00:52:25,680
from this sequence.

917
00:52:25,680 --> 00:52:30,020
OK, in other words, one of these
x sub k in sequence, we

918
00:52:30,020 --> 00:52:33,670
want to be able to detect
what signal was sent.

919
00:52:33,670 --> 00:52:38,920
We want to detect which
hypothesis chose a signal

920
00:52:38,920 --> 00:52:42,940
which was then formed into
a waveform, x of k of t.

921
00:52:42,940 --> 00:52:45,180
And if I can do this
for one k, I can do

922
00:52:45,180 --> 00:52:46,720
it for all of them.

923
00:52:46,720 --> 00:52:51,630
So I want to solve the problem
for one generic value of k.

924
00:52:51,630 --> 00:52:53,690
OK, how is this problem
different from what I was

925
00:52:53,690 --> 00:52:55,120
looking at before?

926
00:52:55,120 --> 00:52:57,830
Before I was looking at the
problem where we built a

927
00:52:57,830 --> 00:53:02,450
communication system, we tuned
it all up, we sent one bit.

928
00:53:02,450 --> 00:53:06,300
We detected it, we tore it all
down and we went home.

929
00:53:06,300 --> 00:53:08,460
Now what we're doing is
we're building the

930
00:53:08,460 --> 00:53:09,680
communication system.

931
00:53:09,680 --> 00:53:10,870
We're tuning at all up.

932
00:53:10,870 --> 00:53:14,470
We're sending a sequence
of bits.

933
00:53:14,470 --> 00:53:18,860
And then all I'm interested in
at the moment is detecting the

934
00:53:18,860 --> 00:53:20,500
k'th of them.

935
00:53:20,500 --> 00:53:23,870
But if I find a way to detect
the k'th of them, I can then

936
00:53:23,870 --> 00:53:25,740
use it for every k.

937
00:53:25,740 --> 00:53:26,060
OK?

938
00:53:26,060 --> 00:53:29,450
So I'm going to build a detector
which is going to

939
00:53:29,450 --> 00:53:33,100
detect, in some optimal
way, each one of these

940
00:53:33,100 --> 00:53:35,030
signals that gets sent.

941
00:53:35,030 --> 00:53:35,300
OK?

942
00:53:35,300 --> 00:53:37,400
Is it clear how the problem
is different?

943
00:53:37,400 --> 00:53:40,000
Mainly I have to deal with the
fact that these other signals

944
00:53:40,000 --> 00:53:42,240
are floating around there.

945
00:53:42,240 --> 00:53:44,780
And that's my problem.

946
00:53:44,780 --> 00:53:49,380
Ok, so the input to the channel
is hypothesis H. That

947
00:53:49,380 --> 00:53:52,220
takes values one up the m.

948
00:53:52,220 --> 00:53:56,950
The symbol, m, is mapped into
the signal, vector a sub m,

949
00:53:56,950 --> 00:54:02,716
it's modulated into x of t
equals summation over j, a sub

950
00:54:02,716 --> 00:54:05,110
mj, phi j of t.

951
00:54:05,110 --> 00:54:09,980
OK, this waveform, now, is a
function of which particular

952
00:54:09,980 --> 00:54:11,380
signal I'm sending.

953
00:54:11,380 --> 00:54:15,040
Which is a function of which
particular hypothesis entered

954
00:54:15,040 --> 00:54:16,290
the encoder.

955
00:54:19,360 --> 00:54:25,860
The trouble with this material
is all the complication comes

956
00:54:25,860 --> 00:54:28,810
in this awful notation,
which you can't avoid.

957
00:54:28,810 --> 00:54:30,790
Because you're dealing with
sequences, you're dealing with

958
00:54:30,790 --> 00:54:33,860
vectors, and you're dealing
with wave forms

959
00:54:33,860 --> 00:54:35,820
all at the same time.

960
00:54:35,820 --> 00:54:40,070
What's going on, after you
understand it, you'll say why

961
00:54:40,070 --> 00:54:43,000
was it so difficult to
understand this?

962
00:54:43,000 --> 00:54:45,550
Because eventually when you see
it, it becomes very, very

963
00:54:45,550 --> 00:54:51,630
simple And I understand why
there's just too much stuff

964
00:54:51,630 --> 00:54:54,970
all going on at the same time.

965
00:54:54,970 --> 00:54:57,500
OK, so what I'm going to do now
is I'm going to take these

966
00:54:57,500 --> 00:55:02,420
J, capital J, orthonormal
waveforms.

967
00:55:02,420 --> 00:55:06,030
And we've already seen that you
can start out with any old

968
00:55:06,030 --> 00:55:09,990
orthonormal waveforms and if
you want to you can extend

969
00:55:09,990 --> 00:55:13,790
that set of waveforms into
an orthonormal set that

970
00:55:13,790 --> 00:55:16,530
spans all of L2.

971
00:55:16,530 --> 00:55:16,930
OK?

972
00:55:16,930 --> 00:55:20,030
So I'm going to imagine
that we've done that.

973
00:55:20,030 --> 00:55:23,050
It's taken us a long time,
but we've done it.

974
00:55:23,050 --> 00:55:24,320
We're all through with it.

975
00:55:24,320 --> 00:55:27,070
We have this orthonormal
set now.

976
00:55:27,070 --> 00:55:33,080
If I'm smart, that orthonormal
set, which I generated, will

977
00:55:33,080 --> 00:55:36,380
also include easy ways to
represent each of the other

978
00:55:36,380 --> 00:55:38,490
signals that we're
going to send.

979
00:55:38,490 --> 00:55:40,150
But I don't care about
that right now.

980
00:55:40,150 --> 00:55:44,200
All I'm dealing with is this one
hypothesis that came in.

981
00:55:44,200 --> 00:55:48,120
This one signal, a sub m--

982
00:55:50,720 --> 00:55:56,170
oh, the hypothesis m, the
signal a sub m, an the

983
00:55:56,170 --> 00:56:00,060
particular time instant, k, and
this waveform that gets

984
00:56:00,060 --> 00:56:05,490
sent, which can be represented
as the first J terms in this

985
00:56:05,490 --> 00:56:08,860
orthonormal sequence.

986
00:56:08,860 --> 00:56:12,010
OK, so what I'm going
to get then is the

987
00:56:12,010 --> 00:56:14,550
received random process.

988
00:56:14,550 --> 00:56:16,900
Is going to be a sum --

989
00:56:16,900 --> 00:56:19,040
and forgot about the
j prime now--

990
00:56:19,040 --> 00:56:24,890
I can represent it as a sum of
coefficients times these

991
00:56:24,890 --> 00:56:27,240
orthonormal waveforms.

992
00:56:27,240 --> 00:56:31,980
OK, that's what we've done for
arbitrary sequences, and then

993
00:56:31,980 --> 00:56:35,920
we've said we can also do it
for at least well defined

994
00:56:35,920 --> 00:56:38,380
random processes.

995
00:56:38,380 --> 00:56:41,560
I'm going to make, I mean,
instead of making this an

996
00:56:41,560 --> 00:56:45,880
infinite dimensional sum, I
want to make it a finite

997
00:56:45,880 --> 00:56:51,030
dimensional sum where J prime
is very, very large.

998
00:56:51,030 --> 00:56:53,520
Say, 10 to the fiftieth
if you want to.

999
00:56:53,520 --> 00:56:55,660
I don't want to make it
infinite, I want to look at

1000
00:56:55,660 --> 00:56:59,280
what happens when I let it
get bigger or smaller.

1001
00:56:59,280 --> 00:57:05,380
So I'm expanding Y of t over
an orthonormal expansion.

1002
00:57:05,380 --> 00:57:07,700
But I'm not going all the way.

1003
00:57:07,700 --> 00:57:13,160
I'm just going to try to do
maximum likelihood detection

1004
00:57:13,160 --> 00:57:16,270
with this finite set
of observations.

1005
00:57:16,270 --> 00:57:18,190
So I wont do quite as well
as if I have all the

1006
00:57:18,190 --> 00:57:20,360
observations, but I'll
still do pretty well.

1007
00:57:20,360 --> 00:57:22,120
We hope.

1008
00:57:22,120 --> 00:57:26,200
OK, well so Y sub j--

1009
00:57:26,200 --> 00:57:30,940
the output that I see in
this degree of freedom

1010
00:57:30,940 --> 00:57:34,100
corresponding to
phi sub j of t.

1011
00:57:34,100 --> 00:57:37,840
Is going to be xj-- what I
sent in that degree of

1012
00:57:37,840 --> 00:57:40,130
freedom-- plus zj.

1013
00:57:40,130 --> 00:57:43,270
And there are j degrees of--
capital J degrees of freedom

1014
00:57:43,270 --> 00:57:44,270
that I'm using.

1015
00:57:44,270 --> 00:57:50,010
So the outputs in those degrees
of freedom, namely in

1016
00:57:50,010 --> 00:57:55,370
the phi1 of t, phi2 of t, phi3
of t directions in this L2

1017
00:57:55,370 --> 00:57:59,960
space are going to be the
signal plus the noise.

1018
00:57:59,960 --> 00:58:01,760
For all of these dimensions.

1019
00:58:01,760 --> 00:58:05,810
And Yj is just going to
be equal to zj for

1020
00:58:05,810 --> 00:58:08,440
all the other terms.

1021
00:58:08,440 --> 00:58:12,160
OK, now I want to add one
extra thing here.

1022
00:58:12,160 --> 00:58:16,810
What I should be putting in here
is all the other signals

1023
00:58:16,810 --> 00:58:18,820
that are going to
be transmitted.

1024
00:58:18,820 --> 00:58:21,430
I don't know how to do that.

1025
00:58:21,430 --> 00:58:24,100
Notationally it gets
very confusing.

1026
00:58:24,100 --> 00:58:29,190
So what I'm going to say is,
OK Z sub j here is not just

1027
00:58:29,190 --> 00:58:31,420
Gaussian noise.

1028
00:58:31,420 --> 00:58:37,190
Z sub j is Gaussian noise plus
all the signals from other

1029
00:58:37,190 --> 00:58:40,410
time instance that
we're sending.

1030
00:58:40,410 --> 00:58:43,170
Plus all of the signals that
anybody else is sending.

1031
00:58:43,170 --> 00:58:44,910
If we're dealing with wireless
then we have

1032
00:58:44,910 --> 00:58:46,920
interference from them.

1033
00:58:46,920 --> 00:58:50,480
Plus any old other thing you
can think of. z sub j is

1034
00:58:50,480 --> 00:58:55,360
everything but in these other
degrees of freedom.

1035
00:58:55,360 --> 00:58:58,060
In these other coordinates.

1036
00:58:58,060 --> 00:59:01,140
This solves another problem
for us, because when we

1037
00:59:01,140 --> 00:59:05,410
defined white Gaussian noise
we had this problem.

1038
00:59:05,410 --> 00:59:09,300
That we could only say it looked
white over the region

1039
00:59:09,300 --> 00:59:10,470
of interest.

1040
00:59:10,470 --> 00:59:16,220
We could only say it looked
white over some time span.

1041
00:59:16,220 --> 00:59:19,440
Because the earth keeps
changing, you know.

1042
00:59:19,440 --> 00:59:23,700
And over some frequency span
because different frequencies

1043
00:59:23,700 --> 00:59:26,170
behave in different ways.

1044
00:59:26,170 --> 00:59:32,040
So this also allows us to have
different Gaussian random

1045
00:59:32,040 --> 00:59:32,740
variables here.

1046
00:59:32,740 --> 00:59:35,890
So when we have arbitrary random
variables here, they

1047
00:59:35,890 --> 00:59:38,180
can be Gaussian or
non-Gaussian.

1048
00:59:38,180 --> 00:59:40,910
They don't have to have
the same variance.

1049
00:59:40,910 --> 00:59:43,860
They don't have to
have anything.

1050
00:59:43,860 --> 00:59:48,320
What I am going to assume is
that these out of band, out of

1051
00:59:48,320 --> 00:59:53,740
sense, out of view random
variables are all independent

1052
00:59:53,740 --> 00:59:56,880
of the things that
I am looking at.

1053
00:59:56,880 --> 00:59:59,710
And for white Gaussian
noise, that's true.

1054
00:59:59,710 --> 01:00:03,610
All of these random variables
here are independent of all of

1055
01:00:03,610 --> 01:00:05,640
these random variables here.

1056
01:00:05,640 --> 01:00:09,380
For these first capital J
different random variables

1057
01:00:09,380 --> 01:00:10,790
that I'm interested in.

1058
01:00:10,790 --> 01:00:16,280
And all these random variables
are independent of they input

1059
01:00:16,280 --> 01:00:17,910
that I'm using.

1060
01:00:17,910 --> 01:00:18,150
OK?

1061
01:00:18,150 --> 01:00:24,380
In other words, a hypothesis
came into the transmitter that

1062
01:00:24,380 --> 01:00:26,310
generated a signal.

1063
01:00:26,310 --> 01:00:29,870
The signal got turned into a
waveform, which is defined

1064
01:00:29,870 --> 01:00:34,800
solely in terms of these J
degrees of freedom, these J

1065
01:00:34,800 --> 01:00:38,000
orthonormal functions.

1066
01:00:38,000 --> 01:00:43,370
And now everything everywhere
else is independent of these J

1067
01:00:43,370 --> 01:00:47,300
functions that I'm
interested in.

1068
01:00:47,300 --> 01:00:49,130
Why is that a shaky
assumption?

1069
01:00:52,050 --> 01:00:53,790
Anybody think of a
situation where

1070
01:00:53,790 --> 01:00:55,410
that is absolute nonsense?

1071
01:00:59,540 --> 01:00:59,790
Yeah?

1072
01:00:59,790 --> 01:01:08,660
AUDIENCE: The stuff from the
other message had t--

1073
01:01:08,660 --> 01:01:14,940
PROFESSOR: Mm hmm.

1074
01:01:14,940 --> 01:01:16,650
AUDIENCE: You said t of j is
not just Gaussian noise--

1075
01:01:16,650 --> 01:01:21,130
PROFESSOR: It also includes all
those other signals, yes.

1076
01:01:21,130 --> 01:01:23,060
Well, but I want to assume that
those other signals are

1077
01:01:23,060 --> 01:01:27,210
independent of this particular
signal that I'm sending.

1078
01:01:27,210 --> 01:01:30,870
But in fact that is making
a pretty big assumption.

1079
01:01:30,870 --> 01:01:34,660
Because one of the things that a
lot of people like to do is,

1080
01:01:34,660 --> 01:01:38,640
when these bits come into a
channel, the first thing they

1081
01:01:38,640 --> 01:01:42,500
do is they encode the bits
for error correction.

1082
01:01:42,500 --> 01:01:45,440
And then they take those bits
that come out of the error

1083
01:01:45,440 --> 01:01:50,020
correction device, error
encoding device, which are as

1084
01:01:50,020 --> 01:01:52,260
correlated as could be.

1085
01:01:52,260 --> 01:01:55,090
And they're all statistically
very dependent, because we

1086
01:01:55,090 --> 01:01:57,080
want to use that statistical
dependence

1087
01:01:57,080 --> 01:02:00,520
later to correct errors.

1088
01:02:00,520 --> 01:02:03,670
And this assumption that I'm
making here says, "no that's

1089
01:02:03,670 --> 01:02:04,740
not the case.

1090
01:02:04,740 --> 01:02:07,660
I'm assuming that all that other
stuff is independent of

1091
01:02:07,660 --> 01:02:10,290
what I'm transmitting here."
So I'm very specifically

1092
01:02:10,290 --> 01:02:15,090
assuming at this point that all
of that stuff has not been

1093
01:02:15,090 --> 01:02:16,620
encoded first.

1094
01:02:16,620 --> 01:02:20,070
That I'm sending something
which is independent of

1095
01:02:20,070 --> 01:02:21,780
everything else.

1096
01:02:21,780 --> 01:02:25,280
Which is going to enter
this channel.

1097
01:02:25,280 --> 01:02:28,190
We'll just assume that and after
we get done assuming it

1098
01:02:28,190 --> 01:02:30,760
and seeing what the consequence
of it is, we'll go

1099
01:02:30,760 --> 01:02:34,530
back and see what
it all means.

1100
01:02:34,530 --> 01:02:38,750
OK, so for a little
more notation.

1101
01:02:38,750 --> 01:02:43,020
I'm going to call the
vector, Y, the first

1102
01:02:43,020 --> 01:02:45,950
J, capital J, outputs.

1103
01:02:45,950 --> 01:02:50,800
I'm going to call the vector Y
prime all the other outputs.

1104
01:02:50,800 --> 01:02:54,180
Now intuitively what were aiming
at is, we would like to

1105
01:02:54,180 --> 01:02:57,930
say this stuff doesn't have
anything to do with it.

1106
01:02:57,930 --> 01:02:59,900
We're going to base our
decision on this.

1107
01:02:59,900 --> 01:03:04,060
But I want to prove that to you,
and show you why it works

1108
01:03:04,060 --> 01:03:05,300
and why it doesn't.

1109
01:03:05,300 --> 01:03:07,320
And the noise I'm going to
break up the same way.

1110
01:03:07,320 --> 01:03:10,710
Z is this the first J components
of the noise.

1111
01:03:10,710 --> 01:03:14,670
And Z prime is the other
components of noise.

1112
01:03:14,670 --> 01:03:18,565
So what I have is that the
output, the output that I want

1113
01:03:18,565 --> 01:03:21,230
to look at, namely the
output this vector

1114
01:03:21,230 --> 01:03:24,860
of dimension J output.

1115
01:03:24,860 --> 01:03:30,130
Which is equal to a vector of
dimension J input plus of a

1116
01:03:30,130 --> 01:03:35,460
vector of dimension J
noise is equal to--

1117
01:03:35,460 --> 01:03:39,600
well Y is equal X plus Z. And
the out of band stuff, the

1118
01:03:39,600 --> 01:03:44,490
output, is just these noise
and other signals.

1119
01:03:44,490 --> 01:03:46,000
OK?

1120
01:03:46,000 --> 01:03:49,330
And I want to assume that
Z prime, X, and Z are

1121
01:03:49,330 --> 01:03:52,120
statistically independent.

1122
01:03:52,120 --> 01:03:55,360
Question, test your
probability.

1123
01:03:55,360 --> 01:04:00,460
If I assume that Z prime is
independent of Z, and if I

1124
01:04:00,460 --> 01:04:04,400
assume that Z prime is
independent of X, does that

1125
01:04:04,400 --> 01:04:12,880
mean that Z prime is independant
of X and Z?

1126
01:04:12,880 --> 01:04:17,150
If a is independent of b, and
a is independent of c, is a

1127
01:04:17,150 --> 01:04:19,750
necessarily independent
of the pair bc?

1128
01:04:22,800 --> 01:04:24,050
How many think that's true?

1129
01:04:26,630 --> 01:04:29,520
Better go back and
study a little

1130
01:04:29,520 --> 01:04:33,720
elementary probability again.

1131
01:04:33,720 --> 01:04:37,240
And the notes are occasionally
wrong about that, too.

1132
01:04:37,240 --> 01:04:42,860
So you shouldn't feel, you
shouldn't feel badly about it.

1133
01:04:42,860 --> 01:04:45,840
No, the problem is you really
need this joint independence

1134
01:04:45,840 --> 01:04:47,610
between all three of them.

1135
01:04:47,610 --> 01:04:55,200
I could, for example, make X
plus Y be equal to Z. I could

1136
01:04:55,200 --> 01:04:58,470
do this with discrete
random variables.

1137
01:04:58,470 --> 01:05:01,330
Which are equally probably
zero and one.

1138
01:05:01,330 --> 01:05:04,630
And make the plus equal
to a mod 2 operation.

1139
01:05:04,630 --> 01:05:07,890
And if I did that, each pair
would be independent of each

1140
01:05:07,890 --> 01:05:11,890
other, and the triple would be
very, very highly dependant.

1141
01:05:11,890 --> 01:05:14,990
So anyway, I want to assume
that Z prime, X, and Z are

1142
01:05:14,990 --> 01:05:18,770
statistically independent.

1143
01:05:18,770 --> 01:05:21,710
In other words, what I'm doing
is saying, "If I assume that,

1144
01:05:21,710 --> 01:05:24,770
what's the consequence of it?"

1145
01:05:24,770 --> 01:05:29,600
OK, so the likelihood then, the
probability density of the

1146
01:05:29,600 --> 01:05:33,360
output, Y-- this is the output
in the first j dimensions

1147
01:05:33,360 --> 01:05:38,620
given a particular hypothesis
Y-- is equal to the

1148
01:05:38,620 --> 01:05:45,410
probability density of the noise
evaluated at Y minus am.

1149
01:05:45,410 --> 01:05:49,520
Where this is the signal that
goes with this hypothesis,

1150
01:05:49,520 --> 01:05:51,450
well with am.

1151
01:05:51,450 --> 01:05:59,080
Times the probability density
of Y prime for Z prime.

1152
01:05:59,080 --> 01:05:59,600
OK?

1153
01:05:59,600 --> 01:06:01,890
And I don't even have to assume
that this is Gaussian.

1154
01:06:01,890 --> 01:06:04,480
All I've done is to assume that
these random variables

1155
01:06:04,480 --> 01:06:06,760
are independent of these
random variables.

1156
01:06:06,760 --> 01:06:12,400
And therefore this probability
density is multiplied by this

1157
01:06:12,400 --> 01:06:15,370
probability density.

1158
01:06:15,370 --> 01:06:16,660
OK, well that's kind of neat.

1159
01:06:16,660 --> 01:06:22,420
Because if I put a different i
in here in place of m, I get

1160
01:06:22,420 --> 01:06:23,240
this thing.

1161
01:06:23,240 --> 01:06:24,430
Changes all around.

1162
01:06:24,430 --> 01:06:26,420
F sub Z of Y minus a sub i.

1163
01:06:26,420 --> 01:06:29,470
But this stuff, which
is out of band,

1164
01:06:29,470 --> 01:06:32,950
doesn't change at all.

1165
01:06:32,950 --> 01:06:38,030
When I form the likelihood
ratio, what I get then is this

1166
01:06:38,030 --> 01:06:40,380
divided by that.

1167
01:06:40,380 --> 01:06:43,430
What has happened to Y prime?

1168
01:06:43,430 --> 01:06:46,340
Y prime has disappeared.

1169
01:06:46,340 --> 01:06:51,160
In other words, Y1 to Y sub j
are a sufficient statistic for

1170
01:06:51,160 --> 01:06:52,250
this problem.

1171
01:06:52,250 --> 01:06:55,550
We've shown that sufficient
statistics are the only thing

1172
01:06:55,550 --> 01:06:59,900
you need to use to do maximum
likelihood detection.

1173
01:06:59,900 --> 01:07:00,220
OK?

1174
01:07:00,220 --> 01:07:03,150
In other words, all those other
signals, all that other

1175
01:07:03,150 --> 01:07:08,080
noise, all that stuff from out
of band has disappeared.

1176
01:07:08,080 --> 01:07:10,980
Now let's go back to the fact
that we were looking at a

1177
01:07:10,980 --> 01:07:12,620
finite dimensional problem.

1178
01:07:17,010 --> 01:07:20,390
What happens now when I
make j prime bigger?

1179
01:07:20,390 --> 01:07:22,850
When I start enlarging
j prime?

1180
01:07:22,850 --> 01:07:24,930
What happens to all these
probabilities that we're

1181
01:07:24,930 --> 01:07:26,670
talking about?

1182
01:07:26,670 --> 01:07:32,580
This probability density goes
ape because of this term here.

1183
01:07:32,580 --> 01:07:35,610
We're talking about a
probability density here which

1184
01:07:35,610 --> 01:07:39,460
is involving more and
more and more terms.

1185
01:07:39,460 --> 01:07:42,080
I can't talk about that.

1186
01:07:42,080 --> 01:07:44,130
It doesn't go to any limit.

1187
01:07:44,130 --> 01:07:48,980
It goes to absolute nonsense
as j prime gets big.

1188
01:07:48,980 --> 01:07:52,010
But if I form the likelihood
ratio before I go to the

1189
01:07:52,010 --> 01:07:55,200
limit, then I can go to the
limit quite easily.

1190
01:07:55,200 --> 01:07:59,560
Because there isn't any
limit involved there.

1191
01:07:59,560 --> 01:08:03,490
OK, in other words this is the
likelihood ratio between

1192
01:08:03,490 --> 01:08:07,960
hypothesis m and hypothesis i,
if in fact I looked at this

1193
01:08:07,960 --> 01:08:11,980
entire infinite amount
of observation.

1194
01:08:11,980 --> 01:08:17,760
This is all I need to make
the optimal MAP decision.

1195
01:08:17,760 --> 01:08:20,430
OK, so there's a theorem here
which is called the theorem of

1196
01:08:20,430 --> 01:08:22,940
irrelevance.

1197
01:08:22,940 --> 01:08:26,730
This is something that
Wosencraft and Jacobs in their

1198
01:08:26,730 --> 01:08:30,140
book on communication many years
ago stressed a lot in

1199
01:08:30,140 --> 01:08:33,360
trying to come up with a single
space viewpoint of

1200
01:08:33,360 --> 01:08:34,530
communication.

1201
01:08:34,530 --> 01:08:37,400
And you'll see why this really
does give you a single point

1202
01:08:37,400 --> 01:08:39,100
viewpoint of communication.

1203
01:08:39,100 --> 01:08:41,890
It says that assume that Z
prime is statistically

1204
01:08:41,890 --> 01:08:46,740
independent of the pair X and Z.
Then the MAP detection of X

1205
01:08:46,740 --> 01:08:51,570
from the observation of Y and Y
prime depends only on Y. The

1206
01:08:51,570 --> 01:08:56,330
observed sample value of
Y prime is irrelevant.

1207
01:08:56,330 --> 01:08:56,920
OK?

1208
01:08:56,920 --> 01:09:01,050
So you can do all of detection
theory, you can do all of

1209
01:09:01,050 --> 01:09:04,390
communication, simply forgetting
about that

1210
01:09:04,390 --> 01:09:06,690
irrelevant stuff.

1211
01:09:06,690 --> 01:09:09,390
Because of the theorem we can
stick to finite dimensional

1212
01:09:09,390 --> 01:09:13,310
vectors and the other signals
can be viewed

1213
01:09:13,310 --> 01:09:16,220
as part of Z prime.

1214
01:09:16,220 --> 01:09:18,570
So you don't have to
worry about them.

1215
01:09:18,570 --> 01:09:22,770
So long as each signal is
independent of each other--

1216
01:09:22,770 --> 01:09:26,880
which means that these groups of
bits, the first group a bit

1217
01:09:26,880 --> 01:09:30,000
is used to form a sub x sub 1.

1218
01:09:33,100 --> 01:09:36,120
The next group, the form x sub
2, the next group the form x

1219
01:09:36,120 --> 01:09:37,880
sub 3, and so forth.

1220
01:09:37,880 --> 01:09:41,090
So long as those sequences of
bits are independent of each

1221
01:09:41,090 --> 01:09:44,250
other, you're fine.

1222
01:09:44,250 --> 01:09:45,470
Now, suppose they aren't?

1223
01:09:45,470 --> 01:09:46,720
What happens then?

1224
01:09:49,910 --> 01:09:52,900
Interesting question.

1225
01:09:52,900 --> 01:09:56,480
We said that if they are
independent, I can really do

1226
01:09:56,480 --> 01:10:00,180
maximum likelihood detection
on the whole sequence.

1227
01:10:00,180 --> 01:10:04,120
If they aren't independent,
suppose I say, "oh I don't

1228
01:10:04,120 --> 01:10:08,540
care about that." I'm just going
to use this portion of

1229
01:10:08,540 --> 01:10:13,430
the output to make my decision
and not worry about whether

1230
01:10:13,430 --> 01:10:15,570
this is independent
of anything else.

1231
01:10:15,570 --> 01:10:20,900
I can do that, this still is
going to give me the optimal

1232
01:10:20,900 --> 01:10:25,160
maximum likelihood detection in
terms of the observation y1

1233
01:10:25,160 --> 01:10:26,410
up to y sub j.

1234
01:10:28,240 --> 01:10:33,590
So in other words, whether I
have coding done before this

1235
01:10:33,590 --> 01:10:36,170
or not doesn't make
any difference.

1236
01:10:36,170 --> 01:10:39,080
I can still use maximum
likelihood detection on the

1237
01:10:39,080 --> 01:10:41,840
basis of y1 up to y sub j.

1238
01:10:41,840 --> 01:10:47,970
What the theorem says is if
the out of band stuff--

1239
01:10:47,970 --> 01:10:51,120
both these inputs and the
noise-- are independent of

1240
01:10:51,120 --> 01:10:55,130
what I'm trying to detect,
maximum likelihood becomes the

1241
01:10:55,130 --> 01:10:59,100
optimum thing to do for
equally likely inputs.

1242
01:10:59,100 --> 01:11:01,870
And otherwise, it's a perfectly
reasonable thing to

1243
01:11:01,870 --> 01:11:04,520
do but it's not optimal.

1244
01:11:04,520 --> 01:11:06,760
Now, a lot of people--

1245
01:11:06,760 --> 01:11:08,390
and we'll see some
examples of this

1246
01:11:08,390 --> 01:11:10,770
when we look at wireless--

1247
01:11:10,770 --> 01:11:14,980
in fact, use coding.

1248
01:11:14,980 --> 01:11:18,290
Then they use this particular
kind of detection where they

1249
01:11:18,290 --> 01:11:20,140
forget about all of the added

1250
01:11:20,140 --> 01:11:22,260
information from other signals.

1251
01:11:22,260 --> 01:11:28,660
They make a decision on each of
these x sub k, namely each

1252
01:11:28,660 --> 01:11:32,540
of the M-ary signals that
goes in, they make a

1253
01:11:32,540 --> 01:11:34,210
hard decision on it.

1254
01:11:34,210 --> 01:11:37,670
It's called a hard decision, not
because it's difficult it

1255
01:11:37,670 --> 01:11:41,830
because they refuse to ever
go back and change it.

1256
01:11:41,830 --> 01:11:46,590
If they just say likelihoods and
try to put things together

1257
01:11:46,590 --> 01:11:49,540
in the final decoder, it's
called soft decoding.

1258
01:11:49,540 --> 01:11:52,450
Otherwise it's called
hard decoding.

1259
01:11:52,450 --> 01:11:54,470
If you do soft decoding,
it has to work better.

1260
01:11:54,470 --> 01:11:57,310
Because you're making, in a
sense, a better decision

1261
01:11:57,310 --> 01:12:02,450
because eventually you're
using more information.

1262
01:12:02,450 --> 01:12:07,300
So soft decisions are better
than hard decisions.

1263
01:12:07,300 --> 01:12:11,770
Used to be that everybody used
hard decisions because hard

1264
01:12:11,770 --> 01:12:17,810
decisions were easy and soft
decisions were hard.

1265
01:12:17,810 --> 01:12:21,320
Strange, strange thing.

1266
01:12:21,320 --> 01:12:22,720
But anyway that's changed.

1267
01:12:22,720 --> 01:12:23,730
Why?

1268
01:12:23,730 --> 01:12:25,030
Well it ought to
be obvious why.

1269
01:12:25,030 --> 01:12:28,830
Because anything you build now
cost a tenth of what it used

1270
01:12:28,830 --> 01:12:31,140
to cost to build it.

1271
01:12:31,140 --> 01:12:34,490
I heard Irwin Jacobs awhile
ago saying that one of the

1272
01:12:34,490 --> 01:12:37,960
things that they always did when
they were designing new

1273
01:12:37,960 --> 01:12:41,410
pieces of equipment is they
would look at how much it

1274
01:12:41,410 --> 01:12:45,480
would cost to build
these devices.

1275
01:12:45,480 --> 01:12:49,090
And then, as opposed to most
companies which would say

1276
01:12:49,090 --> 01:12:52,440
that's too expensive let's find
the cheaper way to do it,

1277
01:12:52,440 --> 01:12:55,870
they said OK how long is it
going to take for us to do it,

1278
01:12:55,870 --> 01:12:58,530
and what is the price of
components going to be by time

1279
01:12:58,530 --> 01:13:00,280
we got it done?

1280
01:13:00,280 --> 01:13:02,310
And they would usually say, well
it's going to cost a year

1281
01:13:02,310 --> 01:13:04,780
before we can go into
mass production.

1282
01:13:04,780 --> 01:13:08,130
By that time, everything will
cost less than a half of what

1283
01:13:08,130 --> 01:13:09,360
it's costing now.

1284
01:13:09,360 --> 01:13:12,840
So let's go ahead and
do it the right way.

1285
01:13:12,840 --> 01:13:16,520
So again the argument comes
that you can do

1286
01:13:16,520 --> 01:13:19,340
the hard thing now.

1287
01:13:19,340 --> 01:13:22,390
Which is soft decisions, and
that's what most people do at

1288
01:13:22,390 --> 01:13:24,830
this point.

1289
01:13:24,830 --> 01:13:26,840
Let me give you one more picture
to get ready for what

1290
01:13:26,840 --> 01:13:29,200
we're doing next time.

1291
01:13:29,200 --> 01:13:34,960
Because it's a nice picture
of different signal sets.

1292
01:13:34,960 --> 01:13:39,190
Because we've just talked
abstractly of having multiple

1293
01:13:39,190 --> 01:13:46,300
signals viewed as vectors, and
this will give us some idea of

1294
01:13:46,300 --> 01:13:48,870
what all of these mean.

1295
01:13:48,870 --> 01:13:54,110
I can have two signals, a binary
signal set, and I can

1296
01:13:54,110 --> 01:13:57,880
insist on the signals being
orthogonal to each other.

1297
01:13:57,880 --> 01:14:00,920
Which is a nice thing
to do some times.

1298
01:14:00,920 --> 01:14:04,590
But then I can look at it and
I can say, "how can I make

1299
01:14:04,590 --> 01:14:09,540
that a better signal system?"
The trouble with this signal

1300
01:14:09,540 --> 01:14:12,730
system is it's not antipodal.

1301
01:14:12,730 --> 01:14:16,790
It's not antipodal because
somehow by alternating between

1302
01:14:16,790 --> 01:14:20,750
these two orthogonal signals--
there's a mean between them--

1303
01:14:20,750 --> 01:14:23,880
and I'm transmitting that mean
plus the difference.

1304
01:14:23,880 --> 01:14:28,580
And the difference between them
is minus 0.7 and plus 0.7

1305
01:14:28,580 --> 01:14:32,220
in that direction that way.

1306
01:14:32,220 --> 01:14:35,380
If you can, I guess you
can't see it that way.

1307
01:14:35,380 --> 01:14:38,280
In this direction this way.

1308
01:14:42,360 --> 01:14:46,820
Well anyway, OK.

1309
01:14:46,820 --> 01:14:50,435
A better thing to do than this
is this, which is called bi

1310
01:14:50,435 --> 01:14:51,420
orthogonal.

1311
01:14:51,420 --> 01:14:55,980
So you take orthogonal signals
and then you have a signal set

1312
01:14:55,980 --> 01:15:00,490
consisting of four signals and
two dimensional space.

1313
01:15:00,490 --> 01:15:03,010
We then talk about orthogonal
signals and

1314
01:15:03,010 --> 01:15:04,520
higher dimensional space.

1315
01:15:04,520 --> 01:15:08,040
You can talk about three
orthogonal signals in three

1316
01:15:08,040 --> 01:15:10,020
dimensional space here.

1317
01:15:10,020 --> 01:15:11,220
There, there, and there.

1318
01:15:11,220 --> 01:15:15,540
So that's m equals
3 and j equals 3.

1319
01:15:15,540 --> 01:15:18,190
If you do the same thing
that we did here--

1320
01:15:18,190 --> 01:15:20,650
we're going to make this into
an equilateral triangle.

1321
01:15:20,650 --> 01:15:23,900
Namely we're going to center
it around the center.

1322
01:15:23,900 --> 01:15:26,220
If we do the same thing that
we did here we're going to

1323
01:15:26,220 --> 01:15:31,690
turn this into a set of six
waveforms which are still

1324
01:15:31,690 --> 01:15:36,520
using the three degrees of
freedom, but at least get us

1325
01:15:36,520 --> 01:15:39,710
more signals.

1326
01:15:39,710 --> 01:15:42,110
For the same number of
degrees of freedom.

1327
01:15:42,110 --> 01:15:45,560
So you can extend this picture
as far as you want to.

1328
01:15:45,560 --> 01:15:49,060
You can talk about many, many
orthogonal signals going into

1329
01:15:49,060 --> 01:15:52,530
many more degrees of freedom.

1330
01:15:52,530 --> 01:15:54,860
For each one of them you
can come up with a

1331
01:15:54,860 --> 01:15:56,800
simplex set of signals.

1332
01:15:56,800 --> 01:16:00,280
The nice thing about the simplex
set of signals is that

1333
01:16:00,280 --> 01:16:01,690
all of the signals are arranged

1334
01:16:01,690 --> 01:16:03,240
around the center point.

1335
01:16:03,240 --> 01:16:06,030
They're all equally distant
from each other.

1336
01:16:06,030 --> 01:16:09,280
You can get these for every
dimension by starting with

1337
01:16:09,280 --> 01:16:12,090
these and simply taking the mean
out, which loses you one

1338
01:16:12,090 --> 01:16:16,440
dimension and makes this
sort of ideal set.

1339
01:16:16,440 --> 01:16:20,980
Well tomorrow-- on Wednesday
what we're going to do is,

1340
01:16:20,980 --> 01:16:25,640
we're going to talk about these
large sets here, and see

1341
01:16:25,640 --> 01:16:26,450
what happens.

1342
01:16:26,450 --> 01:16:28,070
And we'll see that you in
fact get to channel

1343
01:16:28,070 --> 01:16:29,320
capacity this way.

1344
01:16:31,830 --> 01:16:33,080
OK.