1
00:00:00,000 --> 00:00:00,130

2
00:00:00,130 --> 00:00:02,490
The following content is
provided under a Creative

3
00:00:02,490 --> 00:00:03,650
Commons license.

4
00:00:03,650 --> 00:00:06,920
Your support will help MIT
OpenCourseWare continue offer

5
00:00:06,920 --> 00:00:10,030
high quality educational
resources for free.

6
00:00:10,030 --> 00:00:12,780
To make a donation or to view
additional materials from

7
00:00:12,780 --> 00:00:16,560
hundreds of MIT courses, visit
MIT OpenCourseWare at

8
00:00:16,560 --> 00:00:19,270
ocw.mit.edu.

9
00:00:19,270 --> 00:00:26,460
PROFESSOR: I just started
talking a little bit last time

10
00:00:26,460 --> 00:00:33,230
about viewing L2, namely this
set of functions that are

11
00:00:33,230 --> 00:00:37,190
square integrable as
a vector space.

12
00:00:37,190 --> 00:00:39,230
And I want to reveal a
little bit about why

13
00:00:39,230 --> 00:00:41,370
we want to do that.

14
00:00:41,370 --> 00:00:45,320
Because after spending so long
worrying about all these

15
00:00:45,320 --> 00:00:49,620
questions of measurability and
all that, you must wonder, why

16
00:00:49,620 --> 00:00:53,460
do we want to look at it
a different way now.

17
00:00:53,460 --> 00:00:59,590
Well, a part of it is we want
to be able to look at

18
00:00:59,590 --> 00:01:02,820
orthogonal expansions
geometrically.

19
00:01:02,820 --> 00:01:04,300
In other words, we would
like to be able to

20
00:01:04,300 --> 00:01:06,390
draw pictures of them.

21
00:01:06,390 --> 00:01:09,190
You can draw pictures of a
function, but you're drawing a

22
00:01:09,190 --> 00:01:10,860
picture of one function.

23
00:01:10,860 --> 00:01:14,420
And you can't draw anything
about the relationship between

24
00:01:14,420 --> 00:01:17,180
different functions, except
by drawing the

25
00:01:17,180 --> 00:01:18,360
two different functions.

26
00:01:18,360 --> 00:01:20,990
You get all the detail
there, and you can't

27
00:01:20,990 --> 00:01:22,590
abstract it at all.

28
00:01:22,590 --> 00:01:25,980
So somehow we want to start
to abstract some of this

29
00:01:25,980 --> 00:01:27,740
information.

30
00:01:27,740 --> 00:01:30,650
And we want to be able to draw
pictures which look more like

31
00:01:30,650 --> 00:01:33,490
vector pictures than
like functions.

32
00:01:33,490 --> 00:01:37,250
And you'll see why that becomes
important in a while.

33
00:01:37,250 --> 00:01:40,090
The other thing is, when you
draw a function as a function

34
00:01:40,090 --> 00:01:44,520
of time, the only thing you see
it how it behaves in time.

35
00:01:44,520 --> 00:01:47,490
When you take the Fourier
transform and you draw it in

36
00:01:47,490 --> 00:01:49,960
frequency, the only thing
you see is how

37
00:01:49,960 --> 00:01:51,780
it behaves in frequency.

38
00:01:51,780 --> 00:01:55,110
And, again, you don't see what
the relationship is between

39
00:01:55,110 --> 00:01:56,310
different functions.

40
00:01:56,310 --> 00:01:58,120
So you lose all of that.

41
00:01:58,120 --> 00:02:01,780
When you take this signal space
viewpoint, what you're

42
00:02:01,780 --> 00:02:08,890
trying to do there is to not
stress time or frequency so

43
00:02:08,890 --> 00:02:11,270
much, but more to look
at the relationship

44
00:02:11,270 --> 00:02:13,030
between different functions.

45
00:02:13,030 --> 00:02:14,780
Why do we want to do that?

46
00:02:14,780 --> 00:02:17,390
Well, because as soon as we
start looking at noise and

47
00:02:17,390 --> 00:02:20,620
things like that, we want to
be able to tell something

48
00:02:20,620 --> 00:02:24,340
about how distinguishable
different functions are from

49
00:02:24,340 --> 00:02:25,180
each other.

50
00:02:25,180 --> 00:02:29,320
So the critical question there
you want to ask, when you ask

51
00:02:29,320 --> 00:02:32,380
how different functions are,
is how do you look at two

52
00:02:32,380 --> 00:02:34,870
functions both at
the same time.

53
00:02:34,870 --> 00:02:39,000
So, again, that's --

54
00:02:39,000 --> 00:02:41,450
all of this is coming back
to the same thing.

55
00:02:41,450 --> 00:02:45,050
So we want to be able to draw
pictures of functions which

56
00:02:45,050 --> 00:02:48,270
show how functions are related,
rather than show all

57
00:02:48,270 --> 00:02:50,910
the individual detail
of function.

58
00:02:50,910 --> 00:02:55,640
Finally, what we'll see at the
end of today that it gives us

59
00:02:55,640 --> 00:02:59,230
a lot of capability for
understanding much, much

60
00:02:59,230 --> 00:03:03,040
better what's going on when
we look at conversions of

61
00:03:03,040 --> 00:03:06,170
orthogonal series.

62
00:03:06,170 --> 00:03:09,040
This is something we haven't had
any way to look at before

63
00:03:09,040 --> 00:03:14,500
we talked about the Fourier
series, the discrete time

64
00:03:14,500 --> 00:03:17,510
Fourier transform, and
things like this.

65
00:03:17,510 --> 00:03:22,280
But we haven't been able to
really say something general.

66
00:03:22,280 --> 00:03:24,330
Which, again, is what
we want to do now.

67
00:03:24,330 --> 00:03:28,020

68
00:03:28,020 --> 00:03:31,630
I'm going to go very quickly
through these axioms of a

69
00:03:31,630 --> 00:03:34,410
vector space.

70
00:03:34,410 --> 00:03:37,490
Most of you have seen
them before.

71
00:03:37,490 --> 00:03:41,430
Those of you who haven't seen
axiomatic treatments of

72
00:03:41,430 --> 00:03:44,540
various things in mathematics
are going to be

73
00:03:44,540 --> 00:03:46,020
puzzled by it anyway.

74
00:03:46,020 --> 00:03:49,510
And you're going to have to sit
at home or somewhere on

75
00:03:49,510 --> 00:03:52,690
your own and puzzle this out.

76
00:03:52,690 --> 00:03:55,720
But I just wanted to put them
up so I could start to say

77
00:03:55,720 --> 00:04:00,440
what it is that we're trying
to do with these axioms.

78
00:04:00,440 --> 00:04:05,700
What we're trying to do is to
say everything we know about

79
00:04:05,700 --> 00:04:11,430
n-tuples, which we've been
using all of our lives.

80
00:04:11,430 --> 00:04:15,580
All of these tricks that we use,
most of them we can use

81
00:04:15,580 --> 00:04:18,110
to deal with functions also.

82
00:04:18,110 --> 00:04:20,550
The pictures, we can use.

83
00:04:20,550 --> 00:04:23,900
All of the ideas about
dimension, all of the ideas

84
00:04:23,900 --> 00:04:25,880
about expansions.

85
00:04:25,880 --> 00:04:29,660
All of this stuff becomes
useful again.

86
00:04:29,660 --> 00:04:33,910
But, there are a lot of things
that are true about functions

87
00:04:33,910 --> 00:04:36,740
that aren't true
about vectors.

88
00:04:36,740 --> 00:04:42,150
There are lots of things which
are true about n-dimensional

89
00:04:42,150 --> 00:04:44,750
vectors that aren't true
about functions.

90
00:04:44,750 --> 00:04:48,750
And you want to be able to go
back to these axioms when you

91
00:04:48,750 --> 00:04:53,530
have to, and say, well, is this
property we're looking at

92
00:04:53,530 --> 00:04:56,160
something which is a consequence
of the axioms.

93
00:04:56,160 --> 00:05:00,850
Namely, is this an inherent
property of vector spaces, or

94
00:05:00,850 --> 00:05:05,830
is it in fact something else
which is just because of the

95
00:05:05,830 --> 00:05:08,770
particular kind of thing
we're looking at.

96
00:05:08,770 --> 00:05:10,180
So, vector spaces.

97
00:05:10,180 --> 00:05:13,690
Important thing is, there's
an addition operation.

98
00:05:13,690 --> 00:05:16,100
You can add any two vectors.

99
00:05:16,100 --> 00:05:18,000
You can't multiply
them, by the way.

100
00:05:18,000 --> 00:05:20,230
You can only add them.

101
00:05:20,230 --> 00:05:22,850
You can multiply by scalars,
which we'll talk about in the

102
00:05:22,850 --> 00:05:24,210
next slide.

103
00:05:24,210 --> 00:05:28,360
But you can only add the
vectors themselves.

104
00:05:28,360 --> 00:05:31,090
And the addition is commutative,
just like

105
00:05:31,090 --> 00:05:34,730
ordinary addition of real
numbers or complex numbers is.

106
00:05:34,730 --> 00:05:41,630
It's associative, which says
v, u plus w in parentheses.

107
00:05:41,630 --> 00:05:44,670
Now, you see why we're doing
this is, we've said by

108
00:05:44,670 --> 00:05:50,990
definition that for any two
vectors, u and w, there's

109
00:05:50,990 --> 00:05:54,300
another vector, which
is called u plus w.

110
00:05:54,300 --> 00:05:57,700
In other words, this axiomatic
system says, whenever you add

111
00:05:57,700 --> 00:06:00,110
two vectors you have
to get a vector.

112
00:06:00,110 --> 00:06:01,750
There's no way around that.

113
00:06:01,750 --> 00:06:04,720
So u plus w has to
be a vector.

114
00:06:04,720 --> 00:06:09,430
And that says that v plus u
plus w has to be a vector.

115
00:06:09,430 --> 00:06:13,520
Associativity says that you
get the same vector if you

116
00:06:13,520 --> 00:06:16,690
look at it the other way,
if first adding v and u

117
00:06:16,690 --> 00:06:18,190
and then adding w.

118
00:06:18,190 --> 00:06:21,940
So, anything what you call
a vector space has

119
00:06:21,940 --> 00:06:22,935
to have this property.

120
00:06:22,935 --> 00:06:24,965
Of course, the real numbers
have this property.

121
00:06:24,965 --> 00:06:27,340
The complex numbers have
this property.

122
00:06:27,340 --> 00:06:29,240
All the n-tuples you
deal with all the

123
00:06:29,240 --> 00:06:30,930
time have this property.

124
00:06:30,930 --> 00:06:32,780
Functions have this property.

125
00:06:32,780 --> 00:06:36,780
Sequences, infinite length
sequences have this property,

126
00:06:36,780 --> 00:06:38,370
so there's no big
deal about it.

127
00:06:38,370 --> 00:06:40,500
But that is one of the axioms.

128
00:06:40,500 --> 00:06:44,330
There's a unique vector 0, such
that when you add 0 to a

129
00:06:44,330 --> 00:06:48,300
vector, you get the
same vector again.

130
00:06:48,300 --> 00:06:52,380
Now, this is not the 0 in
the real number system.

131
00:06:52,380 --> 00:06:56,100
It's not the 0 in a complex
number system.

132
00:06:56,100 --> 00:07:00,040
In terms of vectors, if you're
thinking of n-tuples, this is

133
00:07:00,040 --> 00:07:02,590
the n-tuple which is all 0's.

134
00:07:02,590 --> 00:07:05,840
In terms of other vectors,
spaces, it

135
00:07:05,840 --> 00:07:06,820
might be other things.

136
00:07:06,820 --> 00:07:11,480
Like, in terms of functions, 0
means the function which is 0

137
00:07:11,480 --> 00:07:13,850
everywhere.

138
00:07:13,850 --> 00:07:17,400
And finally, there's the unique
vector minus v, which

139
00:07:17,400 --> 00:07:20,620
has the property that v plus
minus v -- in other words, v

140
00:07:20,620 --> 00:07:23,250
minus v -- is equal to 0.

141
00:07:23,250 --> 00:07:25,440
And in a sense that defines

142
00:07:25,440 --> 00:07:27,430
subtraction as well as addition.

143
00:07:27,430 --> 00:07:30,710
So we have addition and
subtraction, but we don't have

144
00:07:30,710 --> 00:07:31,960
multiplication.

145
00:07:31,960 --> 00:07:38,480

146
00:07:38,480 --> 00:07:40,130
And there's scalar
multiplication.

147
00:07:40,130 --> 00:07:44,070
Which means you can multiply
a vector by a scalar.

148
00:07:44,070 --> 00:07:50,330
And the vector spaces we're
looking at here only have two

149
00:07:50,330 --> 00:07:51,400
kinds of scalars.

150
00:07:51,400 --> 00:07:54,730
One is real scalars and the
other is complex scalars.

151
00:07:54,730 --> 00:07:57,790
The two are quite different,
as you know.

152
00:07:57,790 --> 00:08:01,580
And so when we're talking about
a vector space we have

153
00:08:01,580 --> 00:08:06,340
to say what the scalar field is
that we're talking about.

154
00:08:06,340 --> 00:08:10,100
So, for every vector.

155
00:08:10,100 --> 00:08:13,800
And also for every scalar,
there's another vector which

156
00:08:13,800 --> 00:08:15,620
is the scalar times
the vector.

157
00:08:15,620 --> 00:08:18,300
Which means, you have scalar
multiplication.

158
00:08:18,300 --> 00:08:24,510
You can multiply vectors by
scalars in terms of n-tuples.

159
00:08:24,510 --> 00:08:27,750
When you're multiplying a scalar
by an n-tuple, you just

160
00:08:27,750 --> 00:08:30,100
multiply every component
by that scalar.

161
00:08:30,100 --> 00:08:32,840

162
00:08:32,840 --> 00:08:37,430
When you take the scalar 1, and
in a field there's always

163
00:08:37,430 --> 00:08:40,070
an element 1, there's
always an element 0.

164
00:08:40,070 --> 00:08:43,180
When you take the element 1,
and you multiply it by a

165
00:08:43,180 --> 00:08:45,770
vector, you got the
same vector.

166
00:08:45,770 --> 00:08:48,230
Which, of course, is what you
would expect if you were

167
00:08:48,230 --> 00:08:49,920
looking at functions.

168
00:08:49,920 --> 00:08:52,290
You multiply a function.

169
00:08:52,290 --> 00:08:56,100
For every t you multiply it by
1, and you get the same thing

170
00:08:56,100 --> 00:08:57,150
back again.

171
00:08:57,150 --> 00:09:00,910
For an n-tuple, you multiply
every component by 1, you get

172
00:09:00,910 --> 00:09:02,480
the same thing back again.

173
00:09:02,480 --> 00:09:04,860
So that's --

174
00:09:04,860 --> 00:09:06,730
well, it's just another axiom.

175
00:09:06,730 --> 00:09:09,500
Then we have these distributive
laws.

176
00:09:09,500 --> 00:09:11,130
And I won't read them
out to you, they're

177
00:09:11,130 --> 00:09:12,330
just what they say.

178
00:09:12,330 --> 00:09:14,710
I want to talk about them in
a second a little bit.

179
00:09:14,710 --> 00:09:18,500

180
00:09:18,500 --> 00:09:22,630
And as we said, the simplistic
example of a vector space, and

181
00:09:22,630 --> 00:09:27,160
the one you've been using for
years, partly because it saves

182
00:09:27,160 --> 00:09:28,840
you a lot of notation.

183
00:09:28,840 --> 00:09:32,240
Incidentally, one of the reasons
I didn't list for

184
00:09:32,240 --> 00:09:37,800
talking about L2 as a vector
space is, it saves you a lot

185
00:09:37,800 --> 00:09:38,320
of writing.

186
00:09:38,320 --> 00:09:41,720
Also, just like when you're
talking about n-tuples it

187
00:09:41,720 --> 00:09:42,670
saves you writing.

188
00:09:42,670 --> 00:09:46,640
It's a nice, convenient
notational device.

189
00:09:46,640 --> 00:09:50,700
I don't think any part of
mathematics becomes popular

190
00:09:50,700 --> 00:09:55,920
with everyone, unless it saves
notation as well as well as

191
00:09:55,920 --> 00:09:59,200
simplifying things.

192
00:09:59,200 --> 00:10:02,870
So we have these n-tuples, which
is what we mean by r sub

193
00:10:02,870 --> 00:10:04,210
n or c sub n.

194
00:10:04,210 --> 00:10:07,770
In other words, when I talk
about r sub n, I don't mean

195
00:10:07,770 --> 00:10:10,320
just a vector space
of dimension n.

196
00:10:10,320 --> 00:10:13,430
I haven't even defined
dimension yet.

197
00:10:13,430 --> 00:10:18,960
And when you talk in this
generality, you have to think

198
00:10:18,960 --> 00:10:21,000
a little bit about
what it means.

199
00:10:21,000 --> 00:10:25,670
So we're just talking about
n-tuples now. r sub n is

200
00:10:25,670 --> 00:10:29,090
n-tuples of real numbers.
c sub n is

201
00:10:29,090 --> 00:10:32,080
n-tuples of complex numbers.

202
00:10:32,080 --> 00:10:34,980
Addition and scalar
multiplication are component

203
00:10:34,980 --> 00:10:36,915
by component, which
we just said.

204
00:10:36,915 --> 00:10:42,450
In other words, w equals u plus
v means this, for all i.

205
00:10:42,450 --> 00:10:45,350
Scalar multiplication
means this.

206
00:10:45,350 --> 00:10:51,780
The unit vector, e sub i, is a
1 in the i'th position, a 0

207
00:10:51,780 --> 00:10:53,880
everywhere else.

208
00:10:53,880 --> 00:10:57,820
And what that means is that
any vector v which is an

209
00:10:57,820 --> 00:11:02,800
n-tuple, n rn or n cn, can
also be expressed as the

210
00:11:02,800 --> 00:11:06,040
equals summation of these
coefficients,

211
00:11:06,040 --> 00:11:08,430
times these unit factors.

212
00:11:08,430 --> 00:11:12,230
That looks like we're giving up
the simple notation we have

213
00:11:12,230 --> 00:11:14,650
and going to more complex
notation.

214
00:11:14,650 --> 00:11:18,390
This is a very useful
idea here.

215
00:11:18,390 --> 00:11:21,050
That you can take even something
as simple as

216
00:11:21,050 --> 00:11:25,820
n-tuples and express every
vector as a sum of these

217
00:11:25,820 --> 00:11:27,240
simple vectors.

218
00:11:27,240 --> 00:11:30,020
And when you start drawing
pictures, you start to see why

219
00:11:30,020 --> 00:11:31,090
this makes sense.

220
00:11:31,090 --> 00:11:33,930
I'm going to do this
on the next slide.

221
00:11:33,930 --> 00:11:35,950
So let's do it.

222
00:11:35,950 --> 00:11:44,810

223
00:11:44,810 --> 00:11:47,990
And on the slide here, what I've
done is to draw a diagram

224
00:11:47,990 --> 00:11:50,940
which is one you've seen
many times, I'm sure.

225
00:11:50,940 --> 00:11:53,190
Except for the particular
things on it.

226
00:11:53,190 --> 00:11:56,700
Of two-dimensional space, where
you take a 2-tuple and

227
00:11:56,700 --> 00:12:00,930
you think of the first element
in the 2-tuple, v1, a being

228
00:12:00,930 --> 00:12:02,500
the horizontal axis.

229
00:12:02,500 --> 00:12:07,040
The second element, v2, as
being the vertical axis.

230
00:12:07,040 --> 00:12:12,250
And then you can draw any
2-tuple by going over v1 and

231
00:12:12,250 --> 00:12:16,680
then up v2, which is what you've
done all your lives.

232
00:12:16,680 --> 00:12:19,800
The reason I'm drawing this is
to show you that you can take

233
00:12:19,800 --> 00:12:21,380
any two vectors.

234
00:12:21,380 --> 00:12:26,670
First vector, v. Second
vector, u.

235
00:12:26,670 --> 00:12:29,070
One thing that --

236
00:12:29,070 --> 00:12:32,380
I'm sure you all traditionally
do is you view a

237
00:12:32,380 --> 00:12:34,470
vector in two ways.

238
00:12:34,470 --> 00:12:38,610
One, you view it as a point
in two-dimensional space.

239
00:12:38,610 --> 00:12:45,890
And the other is you view it as
a line from 0 up to v. And

240
00:12:45,890 --> 00:12:49,030
when you put a little arrow on
it, it looks like a vector,

241
00:12:49,030 --> 00:12:50,250
and we call it a vector.

242
00:12:50,250 --> 00:12:52,880
So it must be a vector, OK?

243
00:12:52,880 --> 00:12:56,470
So either the point or the
line represents a vector.

244
00:12:56,470 --> 00:12:58,780
Here's another vector, u.

245
00:12:58,780 --> 00:13:01,650
I can take the difference
between two vectors, u minus

246
00:13:01,650 --> 00:13:07,560
v, just by drawing a line
from v up to u.

247
00:13:07,560 --> 00:13:10,530
And thinking of that
as a vector also.

248
00:13:10,530 --> 00:13:16,790
So this vector really means a
vector starting here, going

249
00:13:16,790 --> 00:13:20,610
parallel to this, up to this
point, which is what w is.

250
00:13:20,610 --> 00:13:23,610
But it's very convenient to draw
it this way, which lets

251
00:13:23,610 --> 00:13:24,750
you know what's going on.

252
00:13:24,750 --> 00:13:30,950
Namely, you represent u as
the sum of v plus w.

253
00:13:30,950 --> 00:13:35,420
Or you represent w as a
difference between u and v.

254
00:13:35,420 --> 00:13:37,030
And all of this is trivial.

255
00:13:37,030 --> 00:13:41,090
I just want to say it explicitly
so you start to see

256
00:13:41,090 --> 00:13:46,200
what the connection is between
these axioms, which if I don't

257
00:13:46,200 --> 00:13:48,850
talk about them a little bit,
I'm sure you're just going to

258
00:13:48,850 --> 00:13:53,340
blow them off and decide, eh, I
know all of that, I can look

259
00:13:53,340 --> 00:13:55,750
at things in two dimensions
and I don't have to think

260
00:13:55,750 --> 00:13:57,010
about them at all.

261
00:13:57,010 --> 00:14:03,340
And then, in a few days, when
we're doing the same thing for

262
00:14:03,340 --> 00:14:06,460
functions, you're suddenly going
to become very confused.

263
00:14:06,460 --> 00:14:09,230
So you ought to think about
it in these simple terms.

264
00:14:09,230 --> 00:14:14,730
Now, when I take a scalar
multiple of the vector, in

265
00:14:14,730 --> 00:14:18,740
terms of these diagrams, what
we're doing is taking

266
00:14:18,740 --> 00:14:21,060
something on this
same line here.

267
00:14:21,060 --> 00:14:24,410
I can take scalar multiples
which go all the way up here,

268
00:14:24,410 --> 00:14:25,900
all the way down there.

269
00:14:25,900 --> 00:14:29,130
I can take scalar multiples
of u which go up here.

270
00:14:29,130 --> 00:14:31,610
And down here.

271
00:14:31,610 --> 00:14:37,560
The interesting thing here is
when I scale v down by alpha

272
00:14:37,560 --> 00:14:42,970
and I scale u down by alpha, I
can also scale w down by alpha

273
00:14:42,970 --> 00:14:47,000
and I connect it just like that,
from alpha v to alpha u.

274
00:14:47,000 --> 00:14:48,250
Which axiom is that?

275
00:14:48,250 --> 00:14:54,360

276
00:14:54,360 --> 00:14:57,960
I mean, this is the
canonic example of

277
00:14:57,960 --> 00:15:01,360
one of those axioms.

278
00:15:01,360 --> 00:15:01,990
AUDIENCE: [UNINTELLIGIBLE]

279
00:15:01,990 --> 00:15:03,370
PROFESSOR: What?

280
00:15:03,370 --> 00:15:04,010
AUDIENCE: Distributed.

281
00:15:04,010 --> 00:15:05,110
PROFESSOR: Distributed, good.

282
00:15:05,110 --> 00:15:10,030
This is saying that alpha times
the quantity u minus v

283
00:15:10,030 --> 00:15:15,770
is equal to alpha u minus alpha
v. That's so trivial

284
00:15:15,770 --> 00:15:18,220
that it's hard to see it, but
that's what it's saying.

285
00:15:18,220 --> 00:15:22,540
So that the distributive law
really says the triangles

286
00:15:22,540 --> 00:15:23,790
maintain their shape.

287
00:15:23,790 --> 00:15:26,200

288
00:15:26,200 --> 00:15:27,860
Maybe it's easier to just
say distributive

289
00:15:27,860 --> 00:15:29,110
law, I don't know.

290
00:15:29,110 --> 00:15:35,960

291
00:15:35,960 --> 00:15:40,960
So, for the space of L2 complex
functions, we're going

292
00:15:40,960 --> 00:15:46,790
to define u plus v as w,
where w of t equals u

293
00:15:46,790 --> 00:15:48,100
of t plus v of t.

294
00:15:48,100 --> 00:15:51,470
Now, I've done a bunch of
things here all at once.

295
00:15:51,470 --> 00:15:56,350
One of them is to say what we
used to call a function, u of

296
00:15:56,350 --> 00:16:02,250
t, we're now referring to with
a single letter, boldface u.

297
00:16:02,250 --> 00:16:04,320
What's the advantage of that?

298
00:16:04,320 --> 00:16:08,460
Well, one advantage of it is
when you talk about a function

299
00:16:08,460 --> 00:16:11,090
as u of t, you're really
talking about

300
00:16:11,090 --> 00:16:12,630
two different things.

301
00:16:12,630 --> 00:16:16,530
One, you're talking about the
value of the function at a

302
00:16:16,530 --> 00:16:19,800
particular argument, t.

303
00:16:19,800 --> 00:16:20,950
And the other is,
you're talking

304
00:16:20,950 --> 00:16:22,860
about the whole function.

305
00:16:22,860 --> 00:16:24,530
I mean, sometimes you
want to say a

306
00:16:24,530 --> 00:16:26,130
function has some property.

307
00:16:26,130 --> 00:16:27,770
A function is L2.

308
00:16:27,770 --> 00:16:30,970
Well, u of t at a particular
t is just a number.

309
00:16:30,970 --> 00:16:33,220
It's not a function.

310
00:16:33,220 --> 00:16:36,780
So this gives us a nice way
of distinguishing between

311
00:16:36,780 --> 00:16:40,060
functions and between the
value of functions for

312
00:16:40,060 --> 00:16:42,310
particular arguments.

313
00:16:42,310 --> 00:16:46,060
So that's one more notational
advantage you get by talking

314
00:16:46,060 --> 00:16:49,310
about vectors here.

315
00:16:49,310 --> 00:16:54,250
We're going to define the sum
of two functions just as the

316
00:16:54,250 --> 00:16:55,340
point y sum.

317
00:16:55,340 --> 00:16:57,530
In other words, these
two functions are

318
00:16:57,530 --> 00:16:59,450
defined at each t.

319
00:16:59,450 --> 00:17:04,390
w defined at a t is the
sum of this and that.

320
00:17:04,390 --> 00:17:09,680
The scalar multiplication is
just defined by, at every t, u

321
00:17:09,680 --> 00:17:12,990
of t is equal to alpha
times v of t.

322
00:17:12,990 --> 00:17:14,810
Just what you'd expect.

323
00:17:14,810 --> 00:17:16,510
There's nothing strange here.

324
00:17:16,510 --> 00:17:20,600
I just want to be explicit in
saying how everything follows

325
00:17:20,600 --> 00:17:21,890
from these axioms here.

326
00:17:21,890 --> 00:17:24,260
And I won't say all
of it because

327
00:17:24,260 --> 00:17:27,720
there's too much of it.

328
00:17:27,720 --> 00:17:32,700
With this addition and scalar
multiplication, L2, the set of

329
00:17:32,700 --> 00:17:36,390
finite energy measurable
functions is in fact the

330
00:17:36,390 --> 00:17:38,390
complex vector space.

331
00:17:38,390 --> 00:17:40,950
And it's trivial to go back
and check through all the

332
00:17:40,950 --> 00:17:43,520
axioms with what we said
here, and I'm not

333
00:17:43,520 --> 00:17:45,610
going to do it now.

334
00:17:45,610 --> 00:17:48,510
There's only one of those
axioms which is a bit

335
00:17:48,510 --> 00:17:49,670
questionable.

336
00:17:49,670 --> 00:17:54,490
And that is, when you add up two
functions which are square

337
00:17:54,490 --> 00:17:57,880
integrable, is the sum going to
be square integrable also.

338
00:17:57,880 --> 00:18:00,600

339
00:18:00,600 --> 00:18:04,060
Well, you nod and say yes, but
it's worthwhile proving it

340
00:18:04,060 --> 00:18:06,240
once in your lives.

341
00:18:06,240 --> 00:18:09,790
So the question is, is this
function here less than

342
00:18:09,790 --> 00:18:16,040
infinity if the integral of u of
t squared and the integral

343
00:18:16,040 --> 00:18:18,700
of v of t squared are
both integrable.

344
00:18:18,700 --> 00:18:22,340
Well, it's a useful inequality,
which looks like a

345
00:18:22,340 --> 00:18:26,260
very weak inequality but it,
in fact, is not weak.

346
00:18:26,260 --> 00:18:28,660
It says that u of
t plus v of t.

347
00:18:28,660 --> 00:18:30,975
This is just at a particular
value of t.

348
00:18:30,975 --> 00:18:34,320
So this is just an inequality
for real

349
00:18:34,320 --> 00:18:36,360
numbers and complex numbers.

350
00:18:36,360 --> 00:18:40,860
It says that this u of t plus
v of t, quantity squared,

351
00:18:40,860 --> 00:18:44,370
magnitude squared, is less than
or equal to 2 times u of

352
00:18:44,370 --> 00:18:47,780
t squared plus 2 times
v of t squared.

353
00:18:47,780 --> 00:18:50,990
You wonder, at first, what's
the 2 doing in there.

354
00:18:50,990 --> 00:18:55,310
But then think of making
v of t equal to u of t.

355
00:18:55,310 --> 00:18:59,740
You have 2 times u of t
squared, which is 4

356
00:18:59,740 --> 00:19:01,400
times u of t squared.

357
00:19:01,400 --> 00:19:03,460
Well, that's what you need
here to make this true.

358
00:19:03,460 --> 00:19:06,260
So, in that example
this inequality is

359
00:19:06,260 --> 00:19:09,350
satisfied with equality.

360
00:19:09,350 --> 00:19:12,110
To verify the inequality
in general, you just

361
00:19:12,110 --> 00:19:13,240
multiply this out.

362
00:19:13,240 --> 00:19:17,790
It's u of t squared plus v of t
squared plus two cross-terms

363
00:19:17,790 --> 00:19:19,970
and it all works.

364
00:19:19,970 --> 00:19:23,250

365
00:19:23,250 --> 00:19:27,300
This vector space here is not
quite the vector space we want

366
00:19:27,300 --> 00:19:28,900
to talk about.

367
00:19:28,900 --> 00:19:31,800
But let's put off that question
for a while and we'll

368
00:19:31,800 --> 00:19:34,000
come to it later.

369
00:19:34,000 --> 00:19:37,290
We will come up with a vector
space which is just slightly

370
00:19:37,290 --> 00:19:38,540
different than that.

371
00:19:38,540 --> 00:19:41,650

372
00:19:41,650 --> 00:19:46,620
The main thing you can do with
vector spaces is talk about

373
00:19:46,620 --> 00:19:48,760
their dimension.

374
00:19:48,760 --> 00:19:50,660
Well, there are a lot of other
things you can do, but this is

375
00:19:50,660 --> 00:19:53,460
one of the main things
we can do.

376
00:19:53,460 --> 00:19:57,080
And it's an important thing
which we have to talk about

377
00:19:57,080 --> 00:20:01,000
before going into inner product
spaces, which is what

378
00:20:01,000 --> 00:20:04,400
we're really interested in.

379
00:20:04,400 --> 00:20:06,620
So we need a bunch of
definitions here.

380
00:20:06,620 --> 00:20:10,120
All of this, I'm sure is
familiar to most of you.

381
00:20:10,120 --> 00:20:13,370
For those of you that it's not
familiar to, you just have to

382
00:20:13,370 --> 00:20:15,480
spend a little longer with it.

383
00:20:15,480 --> 00:20:19,650
There's not a whole
lot involved here.

384
00:20:19,650 --> 00:20:25,000
If you have a set of vectors,
which are in a vector space,

385
00:20:25,000 --> 00:20:28,940
you say that they span the
vector space if in fact every

386
00:20:28,940 --> 00:20:32,590
vector in this vector
space is a linear

387
00:20:32,590 --> 00:20:36,870
combination of those vectors.

388
00:20:36,870 --> 00:20:42,890
In other words, any vector, u,
can be made up as some sum of

389
00:20:42,890 --> 00:20:45,420
alpha i times v sub i.

390
00:20:45,420 --> 00:20:50,980
Now, notice we've gone a long
way here beyond those axioms.

391
00:20:50,980 --> 00:20:55,160
Because we're talking about
scalar multiplications.

392
00:20:55,160 --> 00:20:58,340
And then we're talking about
a sum of a lot of scalar

393
00:20:58,340 --> 00:20:59,860
multiplications.

394
00:20:59,860 --> 00:21:02,000
Each scalar multiplication
is a vector.

395
00:21:02,000 --> 00:21:05,330
The sum of a bunch of vectors,
by the associative law is, in

396
00:21:05,330 --> 00:21:06,670
fact, another vector.

397
00:21:06,670 --> 00:21:11,710
So every one of these
sums is a vector.

398
00:21:11,710 --> 00:21:17,110
And by definition, a set of
vectors spans a vector space

399
00:21:17,110 --> 00:21:21,180
if every vector can be
represented as some linear

400
00:21:21,180 --> 00:21:22,356
combination of them.

401
00:21:22,356 --> 00:21:22,530
In

402
00:21:22,530 --> 00:21:26,720
other words, there isn't
something outside of here

403
00:21:26,720 --> 00:21:29,000
sitting there waiting to
be discovered later.

404
00:21:29,000 --> 00:21:32,570
You really understand everything
that's there.

405
00:21:32,570 --> 00:21:36,230
And we say that v is finite
dimensional if it is spanned

406
00:21:36,230 --> 00:21:38,730
by a finite set of vectors.

407
00:21:38,730 --> 00:21:40,060
So that's another definition.

408
00:21:40,060 --> 00:21:42,390
That's what you mean by
finite dimensional.

409
00:21:42,390 --> 00:21:46,030
You have to be able to find a
finite set of vectors such

410
00:21:46,030 --> 00:21:49,020
that linear combinations of
those vectors gives you

411
00:21:49,020 --> 00:21:50,720
everything.

412
00:21:50,720 --> 00:21:52,780
It doesn't mean you have a
finite set of vectors.

413
00:21:52,780 --> 00:21:55,530
You have an infinite set of
vectors because you have an

414
00:21:55,530 --> 00:21:57,480
infinite set of scalars.

415
00:21:57,480 --> 00:21:59,320
In fact, you'd have an
uncountably infinite set of

416
00:21:59,320 --> 00:22:01,820
vectors because of
these scalars.

417
00:22:01,820 --> 00:22:04,330

418
00:22:04,330 --> 00:22:05,300
Second definition.

419
00:22:05,300 --> 00:22:09,940
The vector v1 to vn are linearly
independent -- and

420
00:22:09,940 --> 00:22:15,150
this is a mouthful -- if u
equals the sum of alpha sub i

421
00:22:15,150 --> 00:22:20,280
v sub i equals 0, only
for alpha sub i equal

422
00:22:20,280 --> 00:22:22,780
to 0 for each i.

423
00:22:22,780 --> 00:22:26,900
In other words, you can't take
any linear combination of the

424
00:22:26,900 --> 00:22:32,130
v sub i's and get 0 unless that
linear combination is

425
00:22:32,130 --> 00:22:34,230
using all 0's.

426
00:22:34,230 --> 00:22:37,330
You can't, in any non-trivial
way, add up a bunch of

427
00:22:37,330 --> 00:22:39,870
vectors and get 0.

428
00:22:39,870 --> 00:22:43,660
To put it another way, none of
these basis vectors is a

429
00:22:43,660 --> 00:22:45,290
linear combination
of the others.

430
00:22:45,290 --> 00:22:50,130
That's usually a more convenient
way to put it.

431
00:22:50,130 --> 00:22:52,180
Although it takes
more writing.

432
00:22:52,180 --> 00:22:55,820
Now, we say that a set of
vectors are a basis for v if

433
00:22:55,820 --> 00:22:59,530
they're both linearly
independent and if they span

434
00:22:59,530 --> 00:23:02,850
v. When we first talked about
spanning, we didn't say

435
00:23:02,850 --> 00:23:06,740
anything about these vectors
being linearly independent, so

436
00:23:06,740 --> 00:23:10,110
we might have had many more
of them than we needed.

437
00:23:10,110 --> 00:23:14,360
Now, when we're talking about a
basis, we restrict ourselves

438
00:23:14,360 --> 00:23:18,330
to just the set we need
to span the space.

439
00:23:18,330 --> 00:23:20,920
And then the theorem, which I'm
not going to prove, but

440
00:23:20,920 --> 00:23:28,590
it's standard Theorem One of
any linear algebra book --

441
00:23:28,590 --> 00:23:31,110
well, it's probably Theorem One,
Two, Three and Four all

442
00:23:31,110 --> 00:23:36,990
put together -- but anyway, if
it says if v1 and v sub n span

443
00:23:36,990 --> 00:23:42,100
v, then a subset of these
vectors is the basis of b.

444
00:23:42,100 --> 00:23:45,180
In other words, if you have a
set of vectors which span a

445
00:23:45,180 --> 00:23:50,460
space, you can find the basis by
systematically eliminating

446
00:23:50,460 --> 00:23:53,470
vectors which are linear
combinations of the others,

447
00:23:53,470 --> 00:23:57,040
until you get to a point where
you can't do that any further.

448
00:23:57,040 --> 00:24:00,110
So this theorem has an algorithm
tied into it.

449
00:24:00,110 --> 00:24:03,280
Given any set of vectors which
span a space, you can find the

450
00:24:03,280 --> 00:24:06,840
basis from it by perhaps
throwing out

451
00:24:06,840 --> 00:24:09,510
some of those vectors.

452
00:24:09,510 --> 00:24:14,920
The next part of it is, if v is
a finite dimensional space,

453
00:24:14,920 --> 00:24:17,880
then every basis has
the same size.

454
00:24:17,880 --> 00:24:20,030
This, in fact, is a thing which
takes a little bit of

455
00:24:20,030 --> 00:24:21,940
work proving it.

456
00:24:21,940 --> 00:24:25,460
And, also, any linearly
independent set, v1 to v sub

457
00:24:25,460 --> 00:24:28,420
n, is part of the basis.

458
00:24:28,420 --> 00:24:31,150
In other words, here's another
algorithm you can use.

459
00:24:31,150 --> 00:24:35,230
You have this big, finite
dimensional space.

460
00:24:35,230 --> 00:24:38,960
You have a few vectors which
are linearly independent.

461
00:24:38,960 --> 00:24:43,230
You can build a basis starting
with these, and you just

462
00:24:43,230 --> 00:24:43,870
experiment.

463
00:24:43,870 --> 00:24:48,250
You experiment to find new
vectors, which are not linear

464
00:24:48,250 --> 00:24:50,200
combinations of that set.

465
00:24:50,200 --> 00:24:52,780
As soon as you find one, you
add it to the basis.

466
00:24:52,780 --> 00:24:54,200
You keep on going.

467
00:24:54,200 --> 00:24:56,450
And the theorem says, by
time you get to n of

468
00:24:56,450 --> 00:24:58,380
them, you're done.

469
00:24:58,380 --> 00:25:05,540
So that, in a sense, spanning
sets are too big.

470
00:25:05,540 --> 00:25:08,110
Linearly independent
sets are too small.

471
00:25:08,110 --> 00:25:11,720
And what you want to do is add
the linearly independent sets,

472
00:25:11,720 --> 00:25:15,150
shrink the spanning sets, and
come up with a bases.

473
00:25:15,150 --> 00:25:19,240
And all bases have the same
number of vectors.

474
00:25:19,240 --> 00:25:22,420
There many different bases you
come up with, but they all

475
00:25:22,420 --> 00:25:24,080
have the same number
of vectors.

476
00:25:24,080 --> 00:25:26,980

477
00:25:26,980 --> 00:25:30,060
Not going to talk at all about
infinite dimensional spaces

478
00:25:30,060 --> 00:25:32,750
until the last slide today.

479
00:25:32,750 --> 00:25:36,330
Because the only way I know
to understand infinite

480
00:25:36,330 --> 00:25:42,170
dimensional vector spaces is by
thinking of them, in some

481
00:25:42,170 --> 00:25:46,350
sort of limiting way, as finite
dimensional spaces.

482
00:25:46,350 --> 00:25:50,630
And I think that's the only
way you can do it.

483
00:25:50,630 --> 00:25:57,260
A vector space in itself has
no sense of distance

484
00:25:57,260 --> 00:25:59,320
or angles in it.

485
00:25:59,320 --> 00:26:02,880
When I drew that picture before
for you -- let me put

486
00:26:02,880 --> 00:26:04,130
it up again --

487
00:26:04,130 --> 00:26:11,820

488
00:26:11,820 --> 00:26:19,020
it almost looked like
there was some sense

489
00:26:19,020 --> 00:26:20,250
of distance in here.

490
00:26:20,250 --> 00:26:23,800
Because when we took scalar
multiples, we scaled these

491
00:26:23,800 --> 00:26:26,400
things down.

492
00:26:26,400 --> 00:26:29,540
When I took these triangles, we
scaled down the triangle,

493
00:26:29,540 --> 00:26:31,590
and the triangle
was maintained.

494
00:26:31,590 --> 00:26:35,880
But, in fact, I could do this
just as well if I took v and

495
00:26:35,880 --> 00:26:40,000
moved it down here almost down
on this axis, and if I took u

496
00:26:40,000 --> 00:26:42,770
and moved that almost
down on the axis.

497
00:26:42,770 --> 00:26:48,040
I have the same picture, the
same kind of distributive law.

498
00:26:48,040 --> 00:26:49,040
And everything else.

499
00:26:49,040 --> 00:26:50,700
You can't --

500
00:26:50,700 --> 00:26:55,030
I mean, one of the troubles with
n-dimensional space, to

501
00:26:55,030 --> 00:26:58,670
study what a vector space is
about, is it's very hard to

502
00:26:58,670 --> 00:27:04,650
think of n-tuples without
thinking of distance.

503
00:27:04,650 --> 00:27:07,760
And without thinking of things
being orthogonal to each

504
00:27:07,760 --> 00:27:09,600
other, and of all
of these things.

505
00:27:09,600 --> 00:27:14,520
None of that is talked about at
all in any of these axioms.

506
00:27:14,520 --> 00:27:17,940
And, in fact, you need some new
axioms to be able to talk

507
00:27:17,940 --> 00:27:23,370
about ideas of distance, or
angle, or any of these things.

508
00:27:23,370 --> 00:27:25,000
And that's what we
want to add here.

509
00:27:25,000 --> 00:27:32,800

510
00:27:32,800 --> 00:27:34,860
So we need some new axioms.

511
00:27:34,860 --> 00:27:38,870
And what we need is a new
operation on the vector space,

512
00:27:38,870 --> 00:27:41,730
before the only -- the only
operations we have are

513
00:27:41,730 --> 00:27:43,960
addition and scalar
multiplication.

514
00:27:43,960 --> 00:27:46,880
So that vector spaces
are really

515
00:27:46,880 --> 00:27:48,760
incredibly simple animals.

516
00:27:48,760 --> 00:27:51,050
There's very little you
can do with them.

517
00:27:51,050 --> 00:27:55,120
And this added thing is called
an inner product.

518
00:27:55,120 --> 00:27:58,110
An inner product is
a scalar valued

519
00:27:58,110 --> 00:28:00,330
function of two vectors.

520
00:28:00,330 --> 00:28:05,580
And it's represented by
these little brackets.

521
00:28:05,580 --> 00:28:11,600
And the axioms that these inner
products have to satisfy

522
00:28:11,600 --> 00:28:15,330
is, if you're dealing with
a complex vector space.

523
00:28:15,330 --> 00:28:18,500
In other words, where the
scalars are complex numbers,

524
00:28:18,500 --> 00:28:24,530
then this inner product, when
you switch it around, you have

525
00:28:24,530 --> 00:28:25,340
Hermitian symmetry.

526
00:28:25,340 --> 00:28:31,460
Which means that this inner
product is equal to u v

527
00:28:31,460 --> 00:28:33,910
complex conjugate.

528
00:28:33,910 --> 00:28:35,500
We've already seen that
sort of thing in

529
00:28:35,500 --> 00:28:36,840
taking Fourier series.

530
00:28:36,840 --> 00:28:42,420
And the fact that when you're
dealing with complex numbers,

531
00:28:42,420 --> 00:28:46,340
symmetry doesn't usually hold,
and you usually need some kind

532
00:28:46,340 --> 00:28:51,720
of Hermitian symmetry in most
of the things you do.

533
00:28:51,720 --> 00:28:56,450
The next axiom is something
called bilinearity.

534
00:28:56,450 --> 00:28:59,920
It says that if you take a
vector which is alpha times a

535
00:28:59,920 --> 00:29:04,450
vector v, plus beta times a
vector u, take the inner

536
00:29:04,450 --> 00:29:09,210
product of that with w, it
splits up as alpha times v w

537
00:29:09,210 --> 00:29:11,600
plus beta times u w.

538
00:29:11,600 --> 00:29:13,830
How about if I do it
the other way?

539
00:29:13,830 --> 00:29:19,300
See if you understand what I'm
talking about at all here.

540
00:29:19,300 --> 00:29:30,850
Suppose we take w alpha u plus
beta v. What's that equal to?

541
00:29:30,850 --> 00:29:36,380

542
00:29:36,380 --> 00:29:51,180
Well, it's equal to alpha
something w u plus beta w v.

543
00:29:51,180 --> 00:29:54,450
Except that's wrong, It's right
for real vector spaces,

544
00:29:54,450 --> 00:29:56,780
it's wrong for complex
vector spaces.

545
00:29:56,780 --> 00:30:00,030
What am I missing here?

546
00:30:00,030 --> 00:30:04,770
I need complex conjugates here
and complex conjugates here.

547
00:30:04,770 --> 00:30:07,530
I wanted to talk about that,
because when you're dealing

548
00:30:07,530 --> 00:30:10,760
with inner products, I don't
know whether you're like me,

549
00:30:10,760 --> 00:30:13,990
but every time I start dealing
with inner products and I get

550
00:30:13,990 --> 00:30:16,880
in a hurry writing things down,
I forgot to put those

551
00:30:16,880 --> 00:30:20,110
damn complex conjugates
in them.

552
00:30:20,110 --> 00:30:22,150
And, just be careful.

553
00:30:22,150 --> 00:30:23,430
Because you need them.

554
00:30:23,430 --> 00:30:27,110
At least go back after you're
all done and put the complex

555
00:30:27,110 --> 00:30:27,940
conjugates in.

556
00:30:27,940 --> 00:30:31,160
If you're dealing with real
vectors, of course you don't

557
00:30:31,160 --> 00:30:33,510
need to worry about
any of that.

558
00:30:33,510 --> 00:30:36,720
And all the pictures you draw
are always of real vectors.

559
00:30:36,720 --> 00:30:40,340

560
00:30:40,340 --> 00:30:42,840
Think of trying to draw
this picture.

561
00:30:42,840 --> 00:30:45,520
This is the simplest picture
you can draw.

562
00:30:45,520 --> 00:30:46,890
Of two-dimensional vectors.

563
00:30:46,890 --> 00:30:52,450
This is really a picture of a
vector space of dimension two,

564
00:30:52,450 --> 00:30:54,550
for real vectors.

565
00:30:54,550 --> 00:30:58,700
What happens if you try to
draw a picture of complex

566
00:30:58,700 --> 00:31:00,460
two-dimensional vector space?

567
00:31:00,460 --> 00:31:04,070

568
00:31:04,070 --> 00:31:06,680
Well, it becomes very
difficult to do.

569
00:31:06,680 --> 00:31:17,270
Because you're really talking
about the real part of, if

570
00:31:17,270 --> 00:31:22,050
you're dealing with a basis
which consists of two complex

571
00:31:22,050 --> 00:31:28,550
vectors, then instead of v1, you
need a real part of v1 and

572
00:31:28,550 --> 00:31:30,560
imaginary part of v1.

573
00:31:30,560 --> 00:31:33,800
Instead of v2, you need
real part of v2 and

574
00:31:33,800 --> 00:31:35,650
imaginary part of v2.

575
00:31:35,650 --> 00:31:39,090
And you can always draw this
in four dimensions.

576
00:31:39,090 --> 00:31:41,590
And I even have trouble drawing
in three dimensions,

577
00:31:41,590 --> 00:31:44,990
because somehow my pen doesn't
make marks in three

578
00:31:44,990 --> 00:31:46,060
dimensions.

579
00:31:46,060 --> 00:31:52,520
And in four dimensions,
I'm a blinking 12.

580
00:31:52,520 --> 00:31:56,650
And have no idea
of what to do.

581
00:31:56,650 --> 00:31:58,440
So you have to remember this.

582
00:31:58,440 --> 00:32:01,040

583
00:32:01,040 --> 00:32:07,070
If you're dealing with rn or cn,
almost always you define

584
00:32:07,070 --> 00:32:12,090
the inner product of v and u as
the sum of the components

585
00:32:12,090 --> 00:32:14,900
with the second component
complex conjugated.

586
00:32:14,900 --> 00:32:19,690
This is just a standard thing
that we do all the time.

587
00:32:19,690 --> 00:32:23,690
When we do this, and we use
unit vectors, the inner

588
00:32:23,690 --> 00:32:29,510
product of v with the i'th unit
vector is just v sub i.

589
00:32:29,510 --> 00:32:31,110
That's what this formula says.

590
00:32:31,110 --> 00:32:37,020
Because e sub i is this vector
u, in which there's a 1 only

591
00:32:37,020 --> 00:32:41,220
in the i'th position, and
a 0 everywhere else.

592
00:32:41,220 --> 00:32:46,020
So v e i is always the v i,
and e i v is always v i

593
00:32:46,020 --> 00:32:47,240
complex conjugate.

594
00:32:47,240 --> 00:32:52,160
Again, this Hermitian nonsense
that comes up to

595
00:32:52,160 --> 00:32:54,720
plague us all the time.

596
00:32:54,720 --> 00:32:59,180
And from that, if you make v
equal to e sub j or e sub i,

597
00:32:59,180 --> 00:33:04,060
you get the inner product of two
of these basis vectors is

598
00:33:04,060 --> 00:33:07,050
equal to 0 for i unequal to j.

599
00:33:07,050 --> 00:33:11,240
In other words, the standard way
of drawing pictures when

600
00:33:11,240 --> 00:33:16,060
you make it into an inner
product space, those unit

601
00:33:16,060 --> 00:33:19,200
vectors become orthonormal.

602
00:33:19,200 --> 00:33:20,980
Because that's the way you
like to draw things.

603
00:33:20,980 --> 00:33:23,230
You like to draw one
here and one there.

604
00:33:23,230 --> 00:33:27,680
And that's what we mean by
perpendicular, which the

605
00:33:27,680 --> 00:33:32,160
two-dimensional or
three-dimensional word for

606
00:33:32,160 --> 00:33:33,410
orthogonal.

607
00:33:33,410 --> 00:33:36,600

608
00:33:36,600 --> 00:33:40,010
So we have a couple
of definitions.

609
00:33:40,010 --> 00:33:45,160
The inner product of v with
itself is called inner product

610
00:33:45,160 --> 00:33:51,110
v squared, which is called the
squared norm of the vector.

611
00:33:51,110 --> 00:33:56,080
The squared norm has to be
non-negative, by axiom.

612
00:33:56,080 --> 00:34:00,650
It has to be greater than 0,
unless this vector, v, is in

613
00:34:00,650 --> 00:34:04,330
fact a 0 vector.

614
00:34:04,330 --> 00:34:07,050
And the length is
just the square

615
00:34:07,050 --> 00:34:09,530
root of the norm squared.

616
00:34:09,530 --> 00:34:12,020
In other words, the length and
the norm are the same thing.

617
00:34:12,020 --> 00:34:15,850
The norm of a vector is the
length of the vector.

618
00:34:15,850 --> 00:34:18,570
I've always called it the
length, but a lot of people

619
00:34:18,570 --> 00:34:21,190
like to call it the norm.

620
00:34:21,190 --> 00:34:26,530
v and u are orthogonal if the
inner product of v and u is

621
00:34:26,530 --> 00:34:27,900
equal to 0.

622
00:34:27,900 --> 00:34:29,850
How did I get that?

623
00:34:29,850 --> 00:34:31,730
I defined it that why.

624
00:34:31,730 --> 00:34:34,440
Everybody defines it that way.

625
00:34:34,440 --> 00:34:37,100
That's what you mean
by orthogonality.

626
00:34:37,100 --> 00:34:41,100
Now we have to go back and see
if it makes any sense in terms

627
00:34:41,100 --> 00:34:43,920
of these nice simple diagrams.

628
00:34:43,920 --> 00:34:47,690
But first I'm going to do
something called the

629
00:34:47,690 --> 00:34:50,820
one-dimensional projection
theorem.

630
00:34:50,820 --> 00:34:53,220
Which is something you all know
but you probably have

631
00:34:53,220 --> 00:34:56,150
never thought about.

632
00:34:56,150 --> 00:35:01,670
And what it says is, if you have
to vectors, v and u, you

633
00:35:01,670 --> 00:35:05,140
can always break v up
into two parts.

634
00:35:05,140 --> 00:35:09,530
One of which is on the
same line with u.

635
00:35:09,530 --> 00:35:12,060
In other words, is
colinear with u.

636
00:35:12,060 --> 00:35:15,330
I'm drawing a picture here for
real spaces, but when I say

637
00:35:15,330 --> 00:35:20,270
colinear, when I'm dealing with
complex spaces, I mean

638
00:35:20,270 --> 00:35:24,140
it's u times some scalar,
which could be complex.

639
00:35:24,140 --> 00:35:26,840
So it's somewhere
on this line.

640
00:35:26,840 --> 00:35:32,270
And the other part is
perpendicular to this line.

641
00:35:32,270 --> 00:35:35,620
And this theorem says in any
old inner product space at

642
00:35:35,620 --> 00:35:39,810
all, no matter how many
dimensions you have, infinite

643
00:35:39,810 --> 00:35:42,980
dimensional, finite dimensional,
anything, if it's

644
00:35:42,980 --> 00:35:46,880
an inner product space on either
the scalars r or the

645
00:35:46,880 --> 00:35:52,120
scalars c, you can always take
any old two vectors at all.

646
00:35:52,120 --> 00:35:55,630
And you can break one vector up
into a part that's colinear

647
00:35:55,630 --> 00:35:59,730
with the other, and another
part which is orthogonal.

648
00:35:59,730 --> 00:36:02,150
And you can always draw
a picture of it.

649
00:36:02,150 --> 00:36:06,210
If you don't mind just drawing
the picture for real vectors

650
00:36:06,210 --> 00:36:08,420
instead of complex vectors.

651
00:36:08,420 --> 00:36:11,520
This is an important idea.

652
00:36:11,520 --> 00:36:15,050
Because what we're going to use
it for in a while, is to

653
00:36:15,050 --> 00:36:18,450
be able to talk about functions
which are these

654
00:36:18,450 --> 00:36:21,160
incredibly complicated
objects.

655
00:36:21,160 --> 00:36:23,760
And we're going to talk about
two different functions.

656
00:36:23,760 --> 00:36:27,160
And we want to be able to draw
a picture in which those two

657
00:36:27,160 --> 00:36:30,990
functions are represented
just as points in a

658
00:36:30,990 --> 00:36:33,350
two-dimensional picture.

659
00:36:33,350 --> 00:36:36,060
And we're going to do that by
saying, OK, I take one of

660
00:36:36,060 --> 00:36:37,380
those functions.

661
00:36:37,380 --> 00:36:42,110
And I can represent it as partly
being colinear with

662
00:36:42,110 --> 00:36:43,420
this other function.

663
00:36:43,420 --> 00:36:47,980
And partly being orthogonal
to the other function.

664
00:36:47,980 --> 00:36:51,660
Which says, you can forget about
all of these functions

665
00:36:51,660 --> 00:36:57,010
which extend to infinity, in
time extend to infinity, in

666
00:36:57,010 --> 00:36:59,370
frequency and everything else.

667
00:36:59,370 --> 00:37:02,400
And, so long as you're only
interested in some small set

668
00:37:02,400 --> 00:37:06,310
of functions, you can just deal
with them as a finite

669
00:37:06,310 --> 00:37:08,330
dimensional vector space.

670
00:37:08,330 --> 00:37:12,810
You can get rid of all the mess,
and just think of them

671
00:37:12,810 --> 00:37:14,570
in this very simple sense.

672
00:37:14,570 --> 00:37:19,000
That's really why this vector
space idea, which is called

673
00:37:19,000 --> 00:37:22,070
signal space, is so popular
among engineers.

674
00:37:22,070 --> 00:37:25,420
It lets than get rid of all the
mess and think in terms of

675
00:37:25,420 --> 00:37:27,900
very, very simple things.

676
00:37:27,900 --> 00:37:32,710
So let's see why this
complicated

677
00:37:32,710 --> 00:37:37,870
theorem is in fact true.

678
00:37:37,870 --> 00:37:39,360
Let me state the
theorem first.

679
00:37:39,360 --> 00:37:43,800
It says for any inner products
space, v. And any vectors, u

680
00:37:43,800 --> 00:37:48,350
and v in v, with u
unequal to 0 --

681
00:37:48,350 --> 00:37:53,400
I hope I said that in the notes
-- the vector v can be

682
00:37:53,400 --> 00:37:59,720
broken up into a colinear term
plus an orthogonal term, where

683
00:37:59,720 --> 00:38:04,140
the colinear term is equal
to a scalar times u.

684
00:38:04,140 --> 00:38:05,980
That's what we mean
by colinear.

685
00:38:05,980 --> 00:38:09,070
That's just the definition
of colinear.

686
00:38:09,070 --> 00:38:13,210
And the other vector
is orthogonal to u.

687
00:38:13,210 --> 00:38:18,770
And alpha is uniquely given by
the inner product v u divided

688
00:38:18,770 --> 00:38:22,280
by the norm squared of u.

689
00:38:22,280 --> 00:38:24,400
Now, there's one thing
ugly about this.

690
00:38:24,400 --> 00:38:28,830
You see that norm squared, you
say, what is that doing there.

691
00:38:28,830 --> 00:38:31,730
It just looks like it's making
the formula complicated.

692
00:38:31,730 --> 00:38:33,900
Forgot about that for
the time being.

693
00:38:33,900 --> 00:38:37,480
We will get into it in a minute
and explain why we have

694
00:38:37,480 --> 00:38:39,190
that problem.

695
00:38:39,190 --> 00:38:41,670
But, for the moment, let's
just prove this

696
00:38:41,670 --> 00:38:43,540
and see what it says.

697
00:38:43,540 --> 00:38:48,040
So what we're going to do is
say, well, if I don't look at

698
00:38:48,040 --> 00:38:51,810
what the theorem is saying, what
I'd like to do is look at

699
00:38:51,810 --> 00:39:00,520
some generic element which is
a scalar multiple times u.

700
00:39:00,520 --> 00:39:06,780
So I'll say, OK let v parallel
to u be alpha times u.

701
00:39:06,780 --> 00:39:09,510
I don't know what alpha is yet,
but alpha's going to be

702
00:39:09,510 --> 00:39:11,340
whatever it has to be.

703
00:39:11,340 --> 00:39:15,570
We're going to choose alpha so
that this other vector, v

704
00:39:15,570 --> 00:39:19,960
minus v u, is a thing we're
calling v perp.

705
00:39:19,960 --> 00:39:22,680
So that that, the inner product
of that and u, is

706
00:39:22,680 --> 00:39:23,890
equal to 0.

707
00:39:23,890 --> 00:39:26,010
So what I'm trying to do
is to find a vector --

708
00:39:26,010 --> 00:39:29,570

709
00:39:29,570 --> 00:39:35,070
strategy here, is to take any
old vector along this line and

710
00:39:35,070 --> 00:39:38,810
try to choose the scalar alpha
in such a way that the

711
00:39:38,810 --> 00:39:43,610
difference between this point
and this point is orthogonal

712
00:39:43,610 --> 00:39:45,370
to this line.

713
00:39:45,370 --> 00:39:47,930
That's why I started out
with alpha unknown.

714
00:39:47,930 --> 00:39:50,140
Alpha unknown just
says we have any

715
00:39:50,140 --> 00:39:51,760
point along this line.

716
00:39:51,760 --> 00:39:54,660
Now I'm going to find out
what alpha has to be.

717
00:39:54,660 --> 00:39:58,180
I hope I will find out that it
has to be only one thing, and

718
00:39:58,180 --> 00:40:00,980
it's uniquely chosen.

719
00:40:00,980 --> 00:40:04,550
And that's what we're
going to find.

720
00:40:04,550 --> 00:40:10,230
So v minus this projection
term, this is called a

721
00:40:10,230 --> 00:40:20,260
projection of v on u, is equal
to v u minus a projection

722
00:40:20,260 --> 00:40:22,410
inner product with u.

723
00:40:22,410 --> 00:40:24,330
So it's equal to this
difference here.

724
00:40:24,330 --> 00:40:27,870
This is equal to the
inner product.

725
00:40:27,870 --> 00:40:32,230
Since we have chosen this term
to be alpha times u, we can

726
00:40:32,230 --> 00:40:33,410
bring the alpha out.

727
00:40:33,410 --> 00:40:37,180
So it's alpha times the inner
product of u with itself.

728
00:40:37,180 --> 00:40:39,020
That's the norm squared
of alpha.

729
00:40:39,020 --> 00:40:42,850
So this is inner product of v
and u minus alpha times the

730
00:40:42,850 --> 00:40:44,100
norm squared.

731
00:40:44,100 --> 00:40:46,550

732
00:40:46,550 --> 00:40:50,870
This is 0 if and only if you
set this equal to 0.

733
00:40:50,870 --> 00:40:54,720
And the only value alpha can
have to make this 0 is the

734
00:40:54,720 --> 00:41:01,000
inner product of v and u divided
by the norm squared.

735
00:41:01,000 --> 00:41:10,120
So I would think that if I ask
you to prove this without

736
00:41:10,120 --> 00:41:13,920
knowing the projection theorem,
I would hope that if

737
00:41:13,920 --> 00:41:16,640
you weren't afraid of it or
something, and you just sat

738
00:41:16,640 --> 00:41:18,980
down and tried to do it,
you would all do it in

739
00:41:18,980 --> 00:41:20,750
about half an hour.

740
00:41:20,750 --> 00:41:24,620
It would probably take most of
you longer than that, because

741
00:41:24,620 --> 00:41:27,360
everybody gets screwed up in the
notation when they first

742
00:41:27,360 --> 00:41:28,600
try to do this.

743
00:41:28,600 --> 00:41:30,900
But, in fact, this is not
a complicated thing.

744
00:41:30,900 --> 00:41:33,420
This is not rocket science.

745
00:41:33,420 --> 00:41:38,170

746
00:41:38,170 --> 00:41:38,970
Now.

747
00:41:38,970 --> 00:41:40,860
What is this norm squared
doing here?

748
00:41:40,860 --> 00:41:45,100
The thing we have just proven is
that with any two vectors v

749
00:41:45,100 --> 00:41:51,390
and u, the projection of v on
u, namely that vector, there

750
00:41:51,390 --> 00:41:55,390
which has the property that v
minus v u is perpendicular to

751
00:41:55,390 --> 00:42:01,510
u, we showed, is this inner
product divided by the norm

752
00:42:01,510 --> 00:42:03,320
squared times u.

753
00:42:03,320 --> 00:42:06,070
Now, let me break up
this norm squared.

754
00:42:06,070 --> 00:42:07,790
Which is just some
positive number.

755
00:42:07,790 --> 00:42:10,360
It's a positive real number.

756
00:42:10,360 --> 00:42:12,970
Into the length times
the length.

757
00:42:12,970 --> 00:42:16,670
Namely, the norm u is just
some real number.

758
00:42:16,670 --> 00:42:21,950
And write it this way, as the
inner product of v with u

759
00:42:21,950 --> 00:42:27,930
divided by the length of
u times u divided by

760
00:42:27,930 --> 00:42:29,890
the length of u.

761
00:42:29,890 --> 00:42:33,540
Now, what is the vector u
divided by the length of u?

762
00:42:33,540 --> 00:42:35,860
AUDIENCE: [UNINTELLIGIBLE]

763
00:42:35,860 --> 00:42:36,140
PROFESSOR: What?

764
00:42:36,140 --> 00:42:38,620
AUDIENCE: [UNINTELLIGIBLE]

765
00:42:38,620 --> 00:42:41,220
PROFESSOR: It's the same
direction of u, but

766
00:42:41,220 --> 00:42:43,500
it has length 1.

767
00:42:43,500 --> 00:42:46,360
In other words, this is the
normalized form of u.

768
00:42:46,360 --> 00:42:49,020

769
00:42:49,020 --> 00:42:52,430
And I have the normalized
form of u in here also.

770
00:42:52,430 --> 00:42:59,560
So what this is saying is that
this projection is also equal

771
00:42:59,560 --> 00:43:06,640
to the projection of v on the
normalized form of u, times

772
00:43:06,640 --> 00:43:09,560
the normalized form of u.

773
00:43:09,560 --> 00:43:13,530
Which says that it doesn't make
any difference what the

774
00:43:13,530 --> 00:43:15,300
length of u is.

775
00:43:15,300 --> 00:43:21,670
This projection is a function
only of the direction of u.

776
00:43:21,670 --> 00:43:25,070
I mean, this is obvious from
the picture, isn't it?

777
00:43:25,070 --> 00:43:28,900

778
00:43:28,900 --> 00:43:33,230
But again, since we can't draw
pictures for complex valued

779
00:43:33,230 --> 00:43:36,200
things, it's nice to be able
to see it analytically.

780
00:43:36,200 --> 00:43:41,380
If I shorten u, or lengthen u,
this projection is still going

781
00:43:41,380 --> 00:43:43,080
to be exactly the same thing.

782
00:43:43,080 --> 00:43:46,980

783
00:43:46,980 --> 00:43:49,730
And that's what the norm squared
of u is doing here.

784
00:43:49,730 --> 00:43:54,060
The norm squared of u is simply
sitting there so it

785
00:43:54,060 --> 00:43:57,430
does this normalization
function for us.

786
00:43:57,430 --> 00:44:03,520
It makes this projection equal
to the inner product of v with

787
00:44:03,520 --> 00:44:05,970
the normalized form
of u, times the

788
00:44:05,970 --> 00:44:07,730
normalized vector for u.

789
00:44:07,730 --> 00:44:11,780

790
00:44:11,780 --> 00:44:16,090
The Pythagorean theorem, which
doesn't follow from this, it's

791
00:44:16,090 --> 00:44:20,170
something else, but it's simple
-- in fact, we can do

792
00:44:20,170 --> 00:44:24,660
it right away -- it says if v
and u are orthogonal, then the

793
00:44:24,660 --> 00:44:29,320
norm squared of u plus
v is equal to u

794
00:44:29,320 --> 00:44:31,990
squared plus v squared.

795
00:44:31,990 --> 00:44:33,120
I mean, this is something
you use

796
00:44:33,120 --> 00:44:35,090
geometrically all the time.

797
00:44:35,090 --> 00:44:36,270
And you're familiar with this.

798
00:44:36,270 --> 00:44:39,990
And the argument is, you just
break this up into the norm

799
00:44:39,990 --> 00:44:43,450
squared of u plus the norm
squared of v, plus the two

800
00:44:43,450 --> 00:44:48,440
cross-products, the inner
product of u times the inner

801
00:44:48,440 --> 00:44:52,410
product of u with v, which is 0,
and the inner product of v

802
00:44:52,410 --> 00:44:53,880
with u, which is 0.

803
00:44:53,880 --> 00:44:56,320
So the two cross-terms
go away and you're

804
00:44:56,320 --> 00:44:59,700
left with just this.

805
00:44:59,700 --> 00:45:03,870
And for any v and u, the Schwarz
inequality says that

806
00:45:03,870 --> 00:45:06,720
the inner product, the magnitude
of the inner product

807
00:45:06,720 --> 00:45:12,140
of v and u, is less than or
equal to the length of v times

808
00:45:12,140 --> 00:45:13,390
the length of u.

809
00:45:13,390 --> 00:45:15,990

810
00:45:15,990 --> 00:45:20,290
The Schwarz inequality is
probably -- well, I'm not sure

811
00:45:20,290 --> 00:45:22,670
that it's probably, but it's
perhaps the most used

812
00:45:22,670 --> 00:45:25,680
inequality in mathematics.

813
00:45:25,680 --> 00:45:29,570
Any time you use vectors, you
use this all the time.

814
00:45:29,570 --> 00:45:31,020
And it's extremely useful.

815
00:45:31,020 --> 00:45:34,540
I'm not going to prove it here
because it's in the notes.

816
00:45:34,540 --> 00:45:37,940
It's a two-step proof from
what we've done.

817
00:45:37,940 --> 00:45:43,020
And the trouble is watching
two-step proofs in class.

818
00:45:43,020 --> 00:45:45,970
At a certain point you
saturate on them.

819
00:45:45,970 --> 00:45:48,160
And I have more important things
I want to do later

820
00:45:48,160 --> 00:45:50,860
today, so I don't want you
to saturate on this.

821
00:45:50,860 --> 00:45:53,590
You can read this at
your leisure and

822
00:45:53,590 --> 00:45:54,840
see why this is true.

823
00:45:54,840 --> 00:45:57,610

824
00:45:57,610 --> 00:46:00,620
I did want to say something
about it, though.

825
00:46:00,620 --> 00:46:05,880
If you divide the left side by
the right side, you can write

826
00:46:05,880 --> 00:46:09,660
this as the magnitude of the
inner product of the

827
00:46:09,660 --> 00:46:15,780
normalized form of v with the
normalized form of u.

828
00:46:15,780 --> 00:46:20,610
If we're talking about real
vector space, this in fact is

829
00:46:20,610 --> 00:46:26,390
the cosine of the angle
between v and u.

830
00:46:26,390 --> 00:46:28,170
It's less than or equal to 1.

831
00:46:28,170 --> 00:46:31,410
So for real two-dimensional
vectors, the fact that the

832
00:46:31,410 --> 00:46:35,170
cosine is less than or equal to
1 is really equivalent to

833
00:46:35,170 --> 00:46:36,920
the Schwarz inequality.

834
00:46:36,920 --> 00:46:40,210
And this is the appropriate
generalization for any old

835
00:46:40,210 --> 00:46:41,460
vectors at all.

836
00:46:41,460 --> 00:46:45,900

837
00:46:45,900 --> 00:46:51,500
And the notes would say more
about that if that went a

838
00:46:51,500 --> 00:46:54,900
little too quickly.

839
00:46:54,900 --> 00:47:01,430
OK, the inner product space of
interest to us is this thing

840
00:47:01,430 --> 00:47:02,720
we've called signal space.

841
00:47:02,720 --> 00:47:08,550
Namely, it's a space of
functions which are measurable

842
00:47:08,550 --> 00:47:10,010
n square integrals.

843
00:47:10,010 --> 00:47:15,730
In other words, finite value
when you take the square and

844
00:47:15,730 --> 00:47:18,410
integrate it.

845
00:47:18,410 --> 00:47:20,790
And we want to be able to talk
about the set of either real

846
00:47:20,790 --> 00:47:24,940
or complex L2 functions.

847
00:47:24,940 --> 00:47:28,290
So, either one of them, we're
going to define the inner

848
00:47:28,290 --> 00:47:30,770
product in the same
way for each.

849
00:47:30,770 --> 00:47:33,270
It really is the only natural
way to define an

850
00:47:33,270 --> 00:47:34,260
inner product here.

851
00:47:34,260 --> 00:47:38,640
And you'll see this more later
as we start doing other things

852
00:47:38,640 --> 00:47:40,090
with these inner products.

853
00:47:40,090 --> 00:47:45,070
But just like what when you're
dealing with n-tuples, there's

854
00:47:45,070 --> 00:47:47,830
only one sensible way to define
an inner product.

855
00:47:47,830 --> 00:47:51,880
You can define inner products
in other ways.

856
00:47:51,880 --> 00:47:54,480
But it's just a little
bizarre to do so.

857
00:47:54,480 --> 00:47:57,510
And here it's a little
bizarre also.

858
00:47:57,510 --> 00:48:01,500
There's a big technical
problem here.

859
00:48:01,500 --> 00:48:04,090
And if you look at, it can
anybody spot what it -- no,

860
00:48:04,090 --> 00:48:06,090
no, of course you can't spot
what it is unless you've read

861
00:48:06,090 --> 00:48:09,990
the notes and you
know what it is.

862
00:48:09,990 --> 00:48:17,070
One of the axioms of an inner
product space is that the only

863
00:48:17,070 --> 00:48:22,310
vector in the space which has an
inner product with itself,

864
00:48:22,310 --> 00:48:27,810
a squared norm equal to zero
is the zero vector.

865
00:48:27,810 --> 00:48:30,940
Now, we have all these crazy
vector we've been talking

866
00:48:30,940 --> 00:48:35,880
about, which are zero, except
on a set of measures zero.

867
00:48:35,880 --> 00:48:39,990
In other words, vectors'
functions which have zero

868
00:48:39,990 --> 00:48:45,430
energy but just pop up at
various isolated points and

869
00:48:45,430 --> 00:48:47,400
have values there.

870
00:48:47,400 --> 00:48:50,320
Which we really can't get rid
of if we view a function as

871
00:48:50,320 --> 00:48:54,320
something which is defined
at every value of t.

872
00:48:54,320 --> 00:48:56,760
You have to accept those things
as part of what you're

873
00:48:56,760 --> 00:48:57,800
dealing with.

874
00:48:57,800 --> 00:48:59,780
As soon as you started
integrating things, those

875
00:48:59,780 --> 00:49:01,600
things all disappear.

876
00:49:01,600 --> 00:49:05,930
But the trouble is, those
functions, which are zero

877
00:49:05,930 --> 00:49:08,900
almost everywhere,
are not zero.

878
00:49:08,900 --> 00:49:10,830
They're only zero almost
everywhere.

879
00:49:10,830 --> 00:49:13,240
They're zero for all engineering
purposes.

880
00:49:13,240 --> 00:49:17,700
But they're not zero, they're
only zero almost everywhere.

881
00:49:17,700 --> 00:49:24,330
Well, if you define the inner
product in this way and you

882
00:49:24,330 --> 00:49:28,790
want to satisfy the axioms of an
inner product space, you're

883
00:49:28,790 --> 00:49:30,250
out of luck.

884
00:49:30,250 --> 00:49:31,900
There's no way you can
do it, because this

885
00:49:31,900 --> 00:49:35,530
axiom just gets violated.

886
00:49:35,530 --> 00:49:37,010
So what do you do?

887
00:49:37,010 --> 00:49:39,040
Well, you do what we've
been doing all along.

888
00:49:39,040 --> 00:49:41,940
We've been sort of squinting a
little bit and saying, well,

889
00:49:41,940 --> 00:49:47,060
really, these functions of
measure 0 are really 0 for all

890
00:49:47,060 --> 00:49:49,400
practical purposes.

891
00:49:49,400 --> 00:49:52,820
Mathematically, what we have
to say is, we want to talk

892
00:49:52,820 --> 00:49:56,730
about an equivalence
class of functions.

893
00:49:56,730 --> 00:50:00,510
And two functions are in the
same equivalence class if

894
00:50:00,510 --> 00:50:03,630
their difference has 0 energy.

895
00:50:03,630 --> 00:50:08,340
Which is equivalent to saying
their difference is zero

896
00:50:08,340 --> 00:50:09,820
almost everywhere.

897
00:50:09,820 --> 00:50:12,580
It's one of these bizarre
functions which just jumps up

898
00:50:12,580 --> 00:50:15,890
at isolated points and doesn't
do anything else.

899
00:50:15,890 --> 00:50:19,610
Not impulses at isolated points,
just non-zero at

900
00:50:19,610 --> 00:50:21,450
isolated points.

901
00:50:21,450 --> 00:50:23,850
Impulses are not really
functions at all.

902
00:50:23,850 --> 00:50:27,010
So we're talking about things
that are functions, but

903
00:50:27,010 --> 00:50:30,290
they're these bizarre functions
which we talked

904
00:50:30,290 --> 00:50:31,840
about and we've said they're
unimportant.

905
00:50:31,840 --> 00:50:35,190
So, but they are there.

906
00:50:35,190 --> 00:50:38,450
So the solution is to associate
vectors with

907
00:50:38,450 --> 00:50:40,880
equivalence classes.

908
00:50:40,880 --> 00:50:44,500
And d of t and u of t are
equivalent if the v of t minus

909
00:50:44,500 --> 00:50:47,750
u of t is zero almost
everywhere.

910
00:50:47,750 --> 00:50:51,680
In other words, when we talk
about a vector, u, what we're

911
00:50:51,680 --> 00:50:53,050
talking about is an

912
00:50:53,050 --> 00:50:55,240
equivalence class of functions.

913
00:50:55,240 --> 00:51:01,640
It's the equivalence class of
functions for which two

914
00:51:01,640 --> 00:51:04,700
functions are in the same
equivalence class if they

915
00:51:04,700 --> 00:51:08,020
differ only on a set
of measure zero.

916
00:51:08,020 --> 00:51:10,100
In other words, these are the
things that gave us trouble

917
00:51:10,100 --> 00:51:12,490
when we were talking about
Fourier transforms.

918
00:51:12,490 --> 00:51:14,380
These are the things that gave
us trouble when we were

919
00:51:14,380 --> 00:51:18,090
talking about Fourier series.

920
00:51:18,090 --> 00:51:21,230
When you take anything in the
same equivalence class,

921
00:51:21,230 --> 00:51:24,570
time-limited functions, and
you form a Fourier series,

922
00:51:24,570 --> 00:51:26,830
what happens?

923
00:51:26,830 --> 00:51:30,000
All of the things in the same
equivalence class have the

924
00:51:30,000 --> 00:51:31,250
same Fourier coefficients.

925
00:51:31,250 --> 00:51:33,710

926
00:51:33,710 --> 00:51:36,630
But when you go back from the
Fourier series coefficients

927
00:51:36,630 --> 00:51:40,530
back to the function, then you
might go back in a bunch of

928
00:51:40,530 --> 00:51:41,600
different ways.

929
00:51:41,600 --> 00:51:44,970
So, we started using this limit
in the mean notation and

930
00:51:44,970 --> 00:51:47,250
all of that stuff.

931
00:51:47,250 --> 00:51:50,020
And what we're doing here now
is, for these vectors, we're

932
00:51:50,020 --> 00:51:54,630
just saying, let's represent
a vector as this whole

933
00:51:54,630 --> 00:51:55,880
equivalence class.

934
00:51:55,880 --> 00:52:03,300

935
00:52:03,300 --> 00:52:06,090
While we're talking about
vectors, we're almost always

936
00:52:06,090 --> 00:52:09,460
interested in orthogonal
expansions.

937
00:52:09,460 --> 00:52:14,340
And when we're interested in
orthogonal expansions, the

938
00:52:14,340 --> 00:52:18,400
coefficients in the orthogonal
expansions are found as

939
00:52:18,400 --> 00:52:21,040
integrals with the function.

940
00:52:21,040 --> 00:52:23,520
And the integrals with different
functions in the

941
00:52:23,520 --> 00:52:26,260
same equivalence class
are identical.

942
00:52:26,260 --> 00:52:28,710
In other words, any two
functions in the same

943
00:52:28,710 --> 00:52:33,500
equivalence class have the
same coefficients in any

944
00:52:33,500 --> 00:52:36,430
orthogonal expansion.

945
00:52:36,430 --> 00:52:40,410
So if you talk only about the
orthogonal expansion, and

946
00:52:40,410 --> 00:52:44,645
leave out these detailed notions
of what the function

947
00:52:44,645 --> 00:52:51,310
is doing at individual times,
then you don't have to worry

948
00:52:51,310 --> 00:52:52,860
about equivalence classes.

949
00:52:52,860 --> 00:52:56,190
In other words, when you --

950
00:52:56,190 --> 00:52:59,020
I'm going to say this again more
carefully later, but let

951
00:52:59,020 --> 00:53:01,910
me try to say it now a
little bit crudely.

952
00:53:01,910 --> 00:53:04,500
One of the things we're
interested in doing this

953
00:53:04,500 --> 00:53:08,740
taking this class of functions,
mapping each

954
00:53:08,740 --> 00:53:12,660
function into a set of
coefficients, where the set of

955
00:53:12,660 --> 00:53:14,880
coefficients are the
coefficients in some

956
00:53:14,880 --> 00:53:18,400
particular orthogonal expansion
that we're using.

957
00:53:18,400 --> 00:53:21,990
Namely, that's the whole way
that we're using to get from

958
00:53:21,990 --> 00:53:24,610
waveforms to sequences.

959
00:53:24,610 --> 00:53:27,600
It's the whole --

960
00:53:27,600 --> 00:53:30,620
it's the entire thing we're
doing when we start out on a

961
00:53:30,620 --> 00:53:35,260
channel with a sequence of
binary digits and then a

962
00:53:35,260 --> 00:53:36,790
sequence of symbols.

963
00:53:36,790 --> 00:53:40,110
And we modulate it
into a waveform.

964
00:53:40,110 --> 00:53:46,310
Again, it's the mapping from
sequence to waveform.

965
00:53:46,310 --> 00:53:49,320
Now, the important thing here
about these equivalence

966
00:53:49,320 --> 00:53:54,790
classes is, you can't tell
any of the members of the

967
00:53:54,790 --> 00:53:59,190
equivalence class apart within
the sequence that we're

968
00:53:59,190 --> 00:54:00,900
dealing with.

969
00:54:00,900 --> 00:54:02,760
Everything we're interested
in has to

970
00:54:02,760 --> 00:54:05,680
do with these sequences.

971
00:54:05,680 --> 00:54:08,180
I mean, if we could we'd just
ignore the waveforms

972
00:54:08,180 --> 00:54:09,970
altogether.

973
00:54:09,970 --> 00:54:14,970
Because all the processing that
we do is with sequences.

974
00:54:14,970 --> 00:54:19,340
So the only reason we have these
equivalence classes is

975
00:54:19,340 --> 00:54:23,970
because we need them to really
define what the functions are.

976
00:54:23,970 --> 00:54:27,280
So, we will come back
to that later.

977
00:54:27,280 --> 00:54:31,780

978
00:54:31,780 --> 00:54:33,240
Boy, I think I'm going
to get done today.

979
00:54:33,240 --> 00:54:34,490
That's surprising.

980
00:54:34,490 --> 00:54:37,130

981
00:54:37,130 --> 00:54:39,040
The next idea that
we want to talk

982
00:54:39,040 --> 00:54:41,940
about is vector subspaces.

983
00:54:41,940 --> 00:54:45,130
Again, that's an idea you've
probably heard of,

984
00:54:45,130 --> 00:54:46,390
for the most part.

985
00:54:46,390 --> 00:54:51,250
A subspace of a vector space
is a subset of the vector

986
00:54:51,250 --> 00:54:55,370
space such that that subspace
is a vector

987
00:54:55,370 --> 00:54:58,440
space in its own right.

988
00:54:58,440 --> 00:55:04,230
An equivalent definition is,
for all vectors u and v, in

989
00:55:04,230 --> 00:55:08,060
the subspace alpha
times u plus beta

990
00:55:08,060 --> 00:55:11,390
times v is in s also.

991
00:55:11,390 --> 00:55:15,060
In other words, a subspace is
something which you can't get

992
00:55:15,060 --> 00:55:17,140
out of by linear combinations.

993
00:55:17,140 --> 00:55:19,940

994
00:55:19,940 --> 00:55:25,760
If I take one of these diagrams
back here that I keep

995
00:55:25,760 --> 00:55:29,230
looking at --

996
00:55:29,230 --> 00:55:30,480
I'll use this one.

997
00:55:30,480 --> 00:55:34,590

998
00:55:34,590 --> 00:55:38,820
If I want to form a subspace of
this subspace, if I want to

999
00:55:38,820 --> 00:55:42,160
form a subspace of this
two-dimensional vector space

1000
00:55:42,160 --> 00:55:45,150
here, one of the subspaces
includes

1001
00:55:45,150 --> 00:55:48,500
u, but not v, perhaps.

1002
00:55:48,500 --> 00:55:51,180
Now, if I want to make a
one-dimensional subspace

1003
00:55:51,180 --> 00:55:55,760
including u, what is
that subspace?

1004
00:55:55,760 --> 00:55:57,700
It's just all the
scalars times u.

1005
00:55:57,700 --> 00:56:01,120
In other words, it's this line
that goes through the origin

1006
00:56:01,120 --> 00:56:04,590
and through that vector, u.

1007
00:56:04,590 --> 00:56:07,640
And a subspace has to include
the whole line.

1008
00:56:07,640 --> 00:56:08,490
That's what we're saying.

1009
00:56:08,490 --> 00:56:10,950
It's all scalar multiples
of u.

1010
00:56:10,950 --> 00:56:14,760
If I want a subspace that
includes u and v, where u and

1011
00:56:14,760 --> 00:56:19,790
v are just arbitrary vectors
I've chosen out of my hat,

1012
00:56:19,790 --> 00:56:22,500
then I have this two-dimensional
subspace,

1013
00:56:22,500 --> 00:56:24,880
which is what I've drawn here.

1014
00:56:24,880 --> 00:56:26,480
In other words, this
idea is something

1015
00:56:26,480 --> 00:56:28,900
we've been using already.

1016
00:56:28,900 --> 00:56:32,010
It's just that we didn't need
to be explicit about it.

1017
00:56:32,010 --> 00:56:37,440
The subspace which includes
u and v --

1018
00:56:37,440 --> 00:56:40,500
well, a subspace which includes
u and v, is the

1019
00:56:40,500 --> 00:56:44,610
subspace of all linear
combinations of u and v. And

1020
00:56:44,610 --> 00:56:45,580
nothing else.

1021
00:56:45,580 --> 00:56:47,830
So, it's all vectors
along here.

1022
00:56:47,830 --> 00:56:49,570
It's all vectors along here.

1023
00:56:49,570 --> 00:56:53,050
And you fill it all in with
anything here added to

1024
00:56:53,050 --> 00:56:54,070
anything here.

1025
00:56:54,070 --> 00:56:56,210
So you get this two-dimensional
space.

1026
00:56:56,210 --> 00:56:58,900

1027
00:56:58,900 --> 00:57:03,050
Is 0 always in a subspace?

1028
00:57:03,050 --> 00:57:04,140
Of course it is.

1029
00:57:04,140 --> 00:57:07,120
I mean, you multiply
any vector by 0 and

1030
00:57:07,120 --> 00:57:09,640
you get the 0 vector.

1031
00:57:09,640 --> 00:57:13,220
So you sort of get
it as a linear --

1032
00:57:13,220 --> 00:57:17,410

1033
00:57:17,410 --> 00:57:18,660
what more can I say?

1034
00:57:18,660 --> 00:57:21,530

1035
00:57:21,530 --> 00:57:25,750
If we have a vector space which
is an inner product

1036
00:57:25,750 --> 00:57:28,960
space; in other words, if we
add this inner product

1037
00:57:28,960 --> 00:57:33,310
definition to our vector space,
and I take a subspace

1038
00:57:33,310 --> 00:57:39,880
of it, that can be defined as
an inner product space also,

1039
00:57:39,880 --> 00:57:42,010
with the same definition
of inner

1040
00:57:42,010 --> 00:57:45,400
product that I had before.

1041
00:57:45,400 --> 00:57:49,330
Because I can't get out of it
by linear combinations, and

1042
00:57:49,330 --> 00:57:53,450
the inner product is defined for
every pair of vectors in

1043
00:57:53,450 --> 00:57:54,770
that space.

1044
00:57:54,770 --> 00:57:59,770
So we still have a nice,
well-defined vector space,

1045
00:57:59,770 --> 00:58:05,850
which is an inner product space,
and which is a subspace

1046
00:58:05,850 --> 00:58:09,030
of the space we started with.

1047
00:58:09,030 --> 00:58:12,280
Everything I do from now on, I'm
going to assume that v is

1048
00:58:12,280 --> 00:58:14,650
an inner product space.

1049
00:58:14,650 --> 00:58:18,570
And want to look at how
we normalize vectors.

1050
00:58:18,570 --> 00:58:20,200
We've already talked
about that.

1051
00:58:20,200 --> 00:58:23,170

1052
00:58:23,170 --> 00:58:25,400
If I have a vector in this
vector space that's

1053
00:58:25,400 --> 00:58:29,420
normalized, if its
norm equals 1.

1054
00:58:29,420 --> 00:58:31,950
We already decided how to
normalize a vector.

1055
00:58:31,950 --> 00:58:35,750
We took an arbitrary vector,
u, divided by its norm.

1056
00:58:35,750 --> 00:58:41,380
And as soon as we divide by
its norm, that vector, v,

1057
00:58:41,380 --> 00:58:48,540
divided by the norm of v, the
norm of that is just 1.

1058
00:58:48,540 --> 00:58:52,570
So the projection, what the
projection theorem says, and

1059
00:58:52,570 --> 00:58:55,020
all I need here is a
one-dimensional projection

1060
00:58:55,020 --> 00:59:00,770
theorem, it says that v, in the
direction of this vector

1061
00:59:00,770 --> 00:59:03,910
phi, is equal to the
inner product of u

1062
00:59:03,910 --> 00:59:06,000
with phi times phi.

1063
00:59:06,000 --> 00:59:06,520
That's what we said.

1064
00:59:06,520 --> 00:59:10,180
As soon as we normalized these
vectors, that ugly denominator

1065
00:59:10,180 --> 00:59:11,230
here disappears.

1066
00:59:11,230 --> 00:59:16,020
Because the norm of
phi is equal to --

1067
00:59:16,020 --> 00:59:18,760
because the norm
is equal to 1.

1068
00:59:18,760 --> 00:59:23,920
So, an orthonormal set of
vectors is a set such that

1069
00:59:23,920 --> 00:59:28,270
each pair of vectors is
orthogonal to each other, and

1070
00:59:28,270 --> 00:59:30,330
where each vector
is normalized.

1071
00:59:30,330 --> 00:59:34,230
In other words, it has norm
squared equal to 1.

1072
00:59:34,230 --> 00:59:40,690
So, the inner product of these
vectors is just delta sub j k.

1073
00:59:40,690 --> 00:59:43,310

1074
00:59:43,310 --> 00:59:50,390
If I have an orthogonal set, v
sub j, say, then phi sub j is

1075
00:59:50,390 --> 00:59:54,230
an orthonormal set just by
taking each of these vectors

1076
00:59:54,230 --> 00:59:55,890
and normalizing it.

1077
00:59:55,890 --> 00:59:58,980
In other words, it's no big deal
to take an orthogonal set

1078
00:59:58,980 --> 01:00:01,440
of functions and turn
them into an

1079
01:00:01,440 --> 01:00:02,990
orthonormal set of functions.

1080
01:00:02,990 --> 01:00:07,060
The Fourier series was natural
to define that, and most

1081
01:00:07,060 --> 01:00:12,070
people define it, in such a way
that it's not orthonormal.

1082
01:00:12,070 --> 01:00:15,650
Because we're defining it over
some interval of time, t, that

1083
01:00:15,650 --> 01:00:20,250
has a norm squared of t and,
therefore, you have to divide

1084
01:00:20,250 --> 01:00:23,920
by square root of t
to normalize it.

1085
01:00:23,920 --> 01:00:27,310
If you want to put everything
in a common framework, it's

1086
01:00:27,310 --> 01:00:30,200
nice to deal with orthonormal
series.

1087
01:00:30,200 --> 01:00:33,510
And therefore, that's what we're
going to be stressing

1088
01:00:33,510 --> 01:00:34,760
from now on.

1089
01:00:34,760 --> 01:00:38,360

1090
01:00:38,360 --> 01:00:42,670
So I want to go on so the
real projection theorem.

1091
01:00:42,670 --> 01:00:45,290
Actually, there are three
projection theorems.

1092
01:00:45,290 --> 01:00:48,370
There's the one-dimensional
projection theorem.

1093
01:00:48,370 --> 01:00:50,900
There's the n-dimensional
projection theorem, which is

1094
01:00:50,900 --> 01:00:52,500
what this is.

1095
01:00:52,500 --> 01:00:55,200
And then there's an
infinite-dimensional

1096
01:00:55,200 --> 01:00:58,660
projection theorem, which is
not general for all inner

1097
01:00:58,660 --> 01:01:04,860
product spaces, but is certainly
general for L2.

1098
01:01:04,860 --> 01:01:10,070
So I'm going to assume that
phi 1 to phi sub n is an

1099
01:01:10,070 --> 01:01:13,960
orthonormal basis for an
n-dimensional subspace, s,

1100
01:01:13,960 --> 01:01:17,490
which is a subspace of v. How
do I know there is such an

1101
01:01:17,490 --> 01:01:20,220
orthonormal basis?

1102
01:01:20,220 --> 01:01:22,820
Well, I don't know that yet, and
I'm going to come back to

1103
01:01:22,820 --> 01:01:24,010
that later.

1104
01:01:24,010 --> 01:01:26,550
But for the time being, I'm just
going to assume that as

1105
01:01:26,550 --> 01:01:28,510
part of the theorem.

1106
01:01:28,510 --> 01:01:35,170
Assume I have some particular
subspace which has the

1107
01:01:35,170 --> 01:01:39,320
property that it has an
orthonormal basis.

1108
01:01:39,320 --> 01:01:43,180
So this is an orthonormal basis
for this n-dimensional

1109
01:01:43,180 --> 01:01:50,760
subspace, s and v. For each
vector in v, s now is some

1110
01:01:50,760 --> 01:01:52,290
small subspace.

1111
01:01:52,290 --> 01:01:57,430
v is a big subspace
out around s.

1112
01:01:57,430 --> 01:02:00,340
What I want to do now is, I want
to take some vector in

1113
01:02:00,340 --> 01:02:01,920
the big subspace.

1114
01:02:01,920 --> 01:02:04,170
I want to project it
onto the subspace.

1115
01:02:04,170 --> 01:02:06,800

1116
01:02:06,800 --> 01:02:10,410
By projecting it onto the
subspace, what I mean is, I

1117
01:02:10,410 --> 01:02:18,210
want to find some vector in
the subspace such that the

1118
01:02:18,210 --> 01:02:22,620
difference between v and that
point in the subspace is

1119
01:02:22,620 --> 01:02:25,980
orthogonal to the
subspace itself.

1120
01:02:25,980 --> 01:02:33,020
In other words, it's the same
idea as we used before for

1121
01:02:33,020 --> 01:02:36,950
this over-used picture.

1122
01:02:36,950 --> 01:02:39,910
Which I keep looking at.

1123
01:02:39,910 --> 01:02:43,330
Here, the subspace that I'm
looking at is just the

1124
01:02:43,330 --> 01:02:48,000
subspace of vectors
colinear with u.

1125
01:02:48,000 --> 01:02:52,620
And what I'm trying to do here
is to find, from v I'm trying

1126
01:02:52,620 --> 01:02:56,880
to drop a perpendicular to this
subspace, which is just a

1127
01:02:56,880 --> 01:02:58,370
straight line.

1128
01:02:58,370 --> 01:03:02,490
In general, what I'm trying
to do is, I have an

1129
01:03:02,490 --> 01:03:03,950
n-dimensional subspace.

1130
01:03:03,950 --> 01:03:07,040

1131
01:03:07,040 --> 01:03:09,990
We can sort of visualize a
two-dimensional subspace if

1132
01:03:09,990 --> 01:03:12,750
you think of this in
three dimensions.

1133
01:03:12,750 --> 01:03:16,950
And think of replacing u
with some space, some

1134
01:03:16,950 --> 01:03:20,950
two-dimensional space, which
is going through 0.

1135
01:03:20,950 --> 01:03:23,410
And now I have this vector,
v, which is

1136
01:03:23,410 --> 01:03:25,800
outside of that plane.

1137
01:03:25,800 --> 01:03:29,710
And what I'm trying to do here
is to drop a perpendicular

1138
01:03:29,710 --> 01:03:32,610
from v onto that subspace.

1139
01:03:32,610 --> 01:03:36,850
The projection is where that
perpendicular lands.

1140
01:03:36,850 --> 01:03:42,260
So, in other words, v, minus the
projection, is this v perp

1141
01:03:42,260 --> 01:03:44,410
we're talking about.

1142
01:03:44,410 --> 01:03:47,520
And what I want to do is exactly
the same thing that we

1143
01:03:47,520 --> 01:03:48,510
did before with the

1144
01:03:48,510 --> 01:03:51,590
one-dimensional projection theorem.

1145
01:03:51,590 --> 01:03:54,160
And that's what we're
going to do.

1146
01:03:54,160 --> 01:03:56,620
And it works.

1147
01:03:56,620 --> 01:03:57,770
And it isn't really any more

1148
01:03:57,770 --> 01:04:02,070
complicated, except for notation.

1149
01:04:02,070 --> 01:04:05,550
So the theorem says, assume
you have this orthonormal

1150
01:04:05,550 --> 01:04:09,180
basis for this subspace.

1151
01:04:09,180 --> 01:04:14,610
And then you take any old v in
the entire vector space.

1152
01:04:14,610 --> 01:04:18,710
This can be an infinite
dimensional vector space or

1153
01:04:18,710 --> 01:04:19,470
anything else.

1154
01:04:19,470 --> 01:04:23,470
And what we really want it to
be is some element in L2,

1155
01:04:23,470 --> 01:04:27,330
which is some
infinite-dimensional element.

1156
01:04:27,330 --> 01:04:34,950
It says there's a unique
projection in the subspace s.

1157
01:04:34,950 --> 01:04:41,220
And it's given by the inner
product of v with each of

1158
01:04:41,220 --> 01:04:43,810
those basis vectors.

1159
01:04:43,810 --> 01:04:46,270
That inner product,
times v sub j.

1160
01:04:46,270 --> 01:04:49,160
For the case of a
one-dimensional vector, you

1161
01:04:49,160 --> 01:04:51,800
take the sum out and that
was exactly the

1162
01:04:51,800 --> 01:04:54,190
projection we had before.

1163
01:04:54,190 --> 01:04:57,540
Now we just have the
multi-dimensional projection.

1164
01:04:57,540 --> 01:05:02,820
And it has a property that v
is equal to this projection

1165
01:05:02,820 --> 01:05:06,910
plus the orthogonal thing.

1166
01:05:06,910 --> 01:05:11,600
And the orthogonal thing, the
inner product of that with s

1167
01:05:11,600 --> 01:05:15,790
equal to 0 for all s
in the subspace.

1168
01:05:15,790 --> 01:05:18,940
In other words, it's just what
we got in this picture we were

1169
01:05:18,940 --> 01:05:22,290
just trying to construct, of
a two-dimensional plane.

1170
01:05:22,290 --> 01:05:25,410
You drop a perpendicular
two-dimensional plane.

1171
01:05:25,410 --> 01:05:27,030
And when I drop a perpendicular
to a

1172
01:05:27,030 --> 01:05:31,040
two-dimensional plane, and I
take any old vector in this

1173
01:05:31,040 --> 01:05:34,150
two-dimensional plane,
we still have the

1174
01:05:34,150 --> 01:05:36,890
perpendicularity.

1175
01:05:36,890 --> 01:05:44,530
In other words, you have the
notion of this vector being

1176
01:05:44,530 --> 01:05:48,240
perpendicular to a plane if it's
perpendicular whichever

1177
01:05:48,240 --> 01:05:50,600
way you look at it.

1178
01:05:50,600 --> 01:05:53,470
Let me outline the
proof of this.

1179
01:05:53,470 --> 01:05:55,430
Actually, it's pretty much
a complete proof.

1180
01:05:55,430 --> 01:06:01,970
But with a couple small
details left out.

1181
01:06:01,970 --> 01:06:04,510
We're going to start out the
same way I did before.

1182
01:06:04,510 --> 01:06:08,170
Namely, I don't know how
to choose this vector.

1183
01:06:08,170 --> 01:06:12,390
But I know I want to choose
it to be in this subspace.

1184
01:06:12,390 --> 01:06:14,940
And any element in this
subspace is a linear

1185
01:06:14,940 --> 01:06:17,820
combination of these
p sub i's.

1186
01:06:17,820 --> 01:06:20,560

1187
01:06:20,560 --> 01:06:24,730
So this is just a generic
element in s.

1188
01:06:24,730 --> 01:06:28,040
And I want to find out what
element I have to use.

1189
01:06:28,040 --> 01:06:32,050
I want to find the conditions
on these coefficients here

1190
01:06:32,050 --> 01:06:36,310
such that v minus the projection
-- in other words,

1191
01:06:36,310 --> 01:06:40,160
this v perp, as we've been
calling it -- is orthogonal to

1192
01:06:40,160 --> 01:06:42,680
each phi sub i.

1193
01:06:42,680 --> 01:06:45,980
Now, if it's orthogonal to
each phi sub i, it's also

1194
01:06:45,980 --> 01:06:48,400
orthogonal to each linear
combination

1195
01:06:48,400 --> 01:06:49,980
of the phi sub i's.

1196
01:06:49,980 --> 01:06:53,190
So in fact, that solved
our problem for us.

1197
01:06:53,190 --> 01:06:58,310
So what I want to do is, I want
to set 0 equal to v minus

1198
01:06:58,310 --> 01:07:00,050
this projection.

1199
01:07:00,050 --> 01:07:02,710
Where I don't yet know how to
make the projection, because I

1200
01:07:02,710 --> 01:07:04,890
don't know what the
alpha sub i's are.

1201
01:07:04,890 --> 01:07:09,265
But I'm trying to choose these
so that this, minus the

1202
01:07:09,265 --> 01:07:14,590
projection, the inner product of
that with phi j is equal to

1203
01:07:14,590 --> 01:07:18,410
0 for every j.

1204
01:07:18,410 --> 01:07:25,750
This inner product here is equal
to the inner product of

1205
01:07:25,750 --> 01:07:28,280
v with phi sub j.

1206
01:07:28,280 --> 01:07:32,420
I have this difference here,
so the inner product is the

1207
01:07:32,420 --> 01:07:37,460
inner product of v with phi sub
j minus the inner product

1208
01:07:37,460 --> 01:07:40,590
of this with phi sub j.

1209
01:07:40,590 --> 01:07:44,130
Let me write that here.

1210
01:07:44,130 --> 01:07:47,380

1211
01:07:47,380 --> 01:08:03,020
v minus sum alpha i phi sub i
comma phi sub j is equal to v

1212
01:08:03,020 --> 01:08:15,370
phi sub j minus summation
of i alpha i t sub i.

1213
01:08:15,370 --> 01:08:18,110
phi sub j.

1214
01:08:18,110 --> 01:08:20,730
Which is equal to this.

1215
01:08:20,730 --> 01:08:25,540
All of these terms are 0 except
where j is equal to i.

1216
01:08:25,540 --> 01:08:30,270

1217
01:08:30,270 --> 01:08:34,540
Where i is equal to j.

1218
01:08:34,540 --> 01:08:38,590
So, alpha sub j has to be equal
to the inner product of

1219
01:08:38,590 --> 01:08:41,560
v with this basis vector here.

1220
01:08:41,560 --> 01:08:47,240
And, therefore, this projection
is equal, which we

1221
01:08:47,240 --> 01:08:51,630
said was sum of alpha i phi sub
i, that's really the sum

1222
01:08:51,630 --> 01:08:58,340
of v phi sub j, this inner
product, times p sub j,

1223
01:08:58,340 --> 01:09:01,070
Now, if you really use your
imagination and you really

1224
01:09:01,070 --> 01:09:05,580
think hard about the formula we
were using for the Fourier

1225
01:09:05,580 --> 01:09:11,350
series coefficients, was
really the same formula

1226
01:09:11,350 --> 01:09:12,952
without the normalization
in it.

1227
01:09:12,952 --> 01:09:16,100
It's simplified by being already
normalized for us.

1228
01:09:16,100 --> 01:09:19,470
We don't have that 1 over t in
here, which we had in the

1229
01:09:19,470 --> 01:09:21,880
Fourier series because
now we've gone

1230
01:09:21,880 --> 01:09:24,570
to orthonormal functions.

1231
01:09:24,570 --> 01:09:28,070

1232
01:09:28,070 --> 01:09:30,400
So, in fact that sort of
proves the theorem.

1233
01:09:30,400 --> 01:09:42,200

1234
01:09:42,200 --> 01:09:47,050
If we express v as some linear
combination of these

1235
01:09:47,050 --> 01:09:52,960
orthonormal vectors, then if I
take the norm squared of v,

1236
01:09:52,960 --> 01:09:55,860
this is something we've done
many times already.

1237
01:09:55,860 --> 01:09:59,570
I just express the norm squared,
just by expanding

1238
01:09:59,570 --> 01:10:08,530
this the sum of -- well, here
I've done it this way, so

1239
01:10:08,530 --> 01:10:09,930
let's do it this way again.

1240
01:10:09,930 --> 01:10:13,000
When I take the inner product
of v with all of these terms

1241
01:10:13,000 --> 01:10:17,640
here, I get the sum of alpha
sub j complex conjugated,

1242
01:10:17,640 --> 01:10:21,660
times the inner product
of v with phi sub j.

1243
01:10:21,660 --> 01:10:25,940
But the inner product of v with
phi sub j is just alpha

1244
01:10:25,940 --> 01:10:27,910
sub j times 1.

1245
01:10:27,910 --> 01:10:31,670
So it's a sum of alpha
sub j squared.

1246
01:10:31,670 --> 01:10:34,440
OK, this is this energy
relationship we've

1247
01:10:34,440 --> 01:10:35,630
been using all along.

1248
01:10:35,630 --> 01:10:37,400
We've been using it for
the Fourier series.

1249
01:10:37,400 --> 01:10:41,330
We've been using it for
everything we've been doing.

1250
01:10:41,330 --> 01:10:45,910
It's just a special case of
this relationship here, in

1251
01:10:45,910 --> 01:10:49,120
this n-dimensional projection,
except that there we were

1252
01:10:49,120 --> 01:10:51,050
dealing with infinite dimensions
and here we're

1253
01:10:51,050 --> 01:10:53,160
dealing with finite
dimensions.

1254
01:10:53,160 --> 01:10:56,500
But it's the same formula, and
you'll see how it generalizes

1255
01:10:56,500 --> 01:10:59,340
in a little bit.

1256
01:10:59,340 --> 01:11:02,400
We still have the Pythagorean
theorem, which in this case

1257
01:11:02,400 --> 01:11:07,380
says that the norm squared of
vector v is equal to the norm

1258
01:11:07,380 --> 01:11:11,200
squared of a projection, plus
the norm squared of the

1259
01:11:11,200 --> 01:11:12,690
perpendicular part.

1260
01:11:12,690 --> 01:11:15,550
In other words, when I start
to represent this vector

1261
01:11:15,550 --> 01:11:23,640
outside of the space by a vector
inside the space, by

1262
01:11:23,640 --> 01:11:27,140
this projection, I wind
up with two things.

1263
01:11:27,140 --> 01:11:30,410
I wind up both with the part
that's outside of the space

1264
01:11:30,410 --> 01:11:34,130
entirely, and is orthogonal to
the space, plus the part which

1265
01:11:34,130 --> 01:11:35,510
is in the space.

1266
01:11:35,510 --> 01:11:38,320
And each of those has a certain
amount of energy.

1267
01:11:38,320 --> 01:11:45,630
When I expand this by this
relationship here --

1268
01:11:45,630 --> 01:11:49,160
I'm not doing that yet -- what
I'm doing here is what's

1269
01:11:49,160 --> 01:11:51,930
called a norm bound.

1270
01:11:51,930 --> 01:11:57,230
Which says both of these
terms are non-negative.

1271
01:11:57,230 --> 01:12:00,940
This term is non-negative in
particular, and therefore the

1272
01:12:00,940 --> 01:12:05,780
difference between this and this
is always positive, or

1273
01:12:05,780 --> 01:12:07,450
non-negative.

1274
01:12:07,450 --> 01:12:11,200
Which says that 0 has to be less
than or equal to this,

1275
01:12:11,200 --> 01:12:13,720
because it's non-negative.

1276
01:12:13,720 --> 01:12:17,385
And this has to be less than or
equal to the norm of v. In

1277
01:12:17,385 --> 01:12:22,020
other words, the projection
always has less energy then

1278
01:12:22,020 --> 01:12:25,500
the vector itself.

1279
01:12:25,500 --> 01:12:27,420
Which is not very surprising.

1280
01:12:27,420 --> 01:12:30,000
So the norm bound
is no big deal.

1281
01:12:30,000 --> 01:12:36,500
When I substitute this for the
actual value, what I get is

1282
01:12:36,500 --> 01:12:43,310
the sum j equals 1 to n of the
norm of the inner product of

1283
01:12:43,310 --> 01:12:48,690
v, with each one of these
basis vectors, magnitude

1284
01:12:48,690 --> 01:12:53,280
squared, that's less than or
equal to the energy in v. In

1285
01:12:53,280 --> 01:12:57,370
other words, if we start to
expand, as n gets bigger and

1286
01:12:57,370 --> 01:13:01,670
bigger, and we look at these
terms, we take these inner

1287
01:13:01,670 --> 01:13:03,780
products, square them.

1288
01:13:03,780 --> 01:13:07,400
No matter how many terms I take
here, the sum is always

1289
01:13:07,400 --> 01:13:12,620
less than or equal to the energy
in v. That's called

1290
01:13:12,620 --> 01:13:21,030
Bessel's inequality, and it's a
nice, straightforward thing.

1291
01:13:21,030 --> 01:13:27,436
And finally, the last
of these things --

1292
01:13:27,436 --> 01:13:32,500

1293
01:13:32,500 --> 01:13:38,520
well, I'll use the other
one if I need it.

1294
01:13:38,520 --> 01:13:40,770
The last is this thing
called the mean

1295
01:13:40,770 --> 01:13:42,150
square error property.

1296
01:13:42,150 --> 01:13:44,670

1297
01:13:44,670 --> 01:13:48,220
It says that if you take the
difference between the vector

1298
01:13:48,220 --> 01:13:52,130
and its projection onto this
space, this is less than or

1299
01:13:52,130 --> 01:13:55,290
equal to the difference between
the vector and the

1300
01:13:55,290 --> 01:13:57,095
other s in the space.

1301
01:13:57,095 --> 01:14:00,670
Any other --

1302
01:14:00,670 --> 01:14:07,050
I can always represent v as
being equal to this plus the

1303
01:14:07,050 --> 01:14:09,460
orthogonal component.

1304
01:14:09,460 --> 01:14:13,010
So I wind up with a sum
here of two terms.

1305
01:14:13,010 --> 01:14:18,910
One is the difference between
-- well, it's the --

1306
01:14:18,910 --> 01:14:22,140

1307
01:14:22,140 --> 01:14:29,180
it's the length squared
of the projection.

1308
01:14:29,180 --> 01:14:30,430
Write it out.

1309
01:14:30,430 --> 01:14:33,170

1310
01:14:33,170 --> 01:14:51,410
v minus v s -- let me write this
term out. v minus s is

1311
01:14:51,410 --> 01:15:04,220
equal to v, the projection, plus
v perpendicular to the

1312
01:15:04,220 --> 01:15:06,920
subspace s minus
this vector s.

1313
01:15:06,920 --> 01:15:12,820

1314
01:15:12,820 --> 01:15:19,155
This is perpendicular to this
and this, so -- ah,

1315
01:15:19,155 --> 01:15:20,490
to hell with it.

1316
01:15:20,490 --> 01:15:24,310
Excuse my language.

1317
01:15:24,310 --> 01:15:25,770
I mean, this is proven
in the notes.

1318
01:15:25,770 --> 01:15:30,600
I'm not going to go through it
now because I want to finish

1319
01:15:30,600 --> 01:15:32,480
these other things.

1320
01:15:32,480 --> 01:15:36,410
I don't want to play
around with it.

1321
01:15:36,410 --> 01:15:39,750
We left something out of the

1322
01:15:39,750 --> 01:15:42,690
n-dimensional projection theorem.

1323
01:15:42,690 --> 01:15:46,800
How do you find an orthonormal
basis to start with?

1324
01:15:46,800 --> 01:15:50,630
And there's this neat thing
called Gram-Schmidt, which I

1325
01:15:50,630 --> 01:15:54,640
suspect most of you have
seen before also.

1326
01:15:54,640 --> 01:15:57,310
Which is pretty simple
now in terms of

1327
01:15:57,310 --> 01:15:59,350
the projection theorem.

1328
01:15:59,350 --> 01:16:02,890
Gram-Schmidt is really a
bootstrap operation starting

1329
01:16:02,890 --> 01:16:05,450
with the one-dimensional
projection theorem, working

1330
01:16:05,450 --> 01:16:08,830
your way up to larger and
larger dimensions.

1331
01:16:08,830 --> 01:16:12,630
And each case winding up with an
orthonormal basis for what

1332
01:16:12,630 --> 01:16:13,490
you started with.

1333
01:16:13,490 --> 01:16:16,420
Let's see how that happens.

1334
01:16:16,420 --> 01:16:20,540
I start out with a basis for
an inner product subspace.

1335
01:16:20,540 --> 01:16:24,390
So, s1 up to s sub n
as a basis for this

1336
01:16:24,390 --> 01:16:26,540
inner product space.

1337
01:16:26,540 --> 01:16:29,970
First thing I do is, I
start out with s1.

1338
01:16:29,970 --> 01:16:32,490
I find the normalized
version of s1.

1339
01:16:32,490 --> 01:16:34,500
I call that phi 1.

1340
01:16:34,500 --> 01:16:44,740
So phi 1 is now an orthonormal
basis for the subspace whose

1341
01:16:44,740 --> 01:16:47,670
basis is just phi 1 itself.

1342
01:16:47,670 --> 01:16:50,280

1343
01:16:50,280 --> 01:16:54,210
phi 1 is the basis for the
subspace of all linear

1344
01:16:54,210 --> 01:16:57,470
combinations of s1.

1345
01:16:57,470 --> 01:17:01,490
So it's just a straight
line in space.

1346
01:17:01,490 --> 01:17:04,520
The next thing I do
is, I take s2.

1347
01:17:04,520 --> 01:17:09,310
I find the projection of
s2 on this subspace s1.

1348
01:17:09,310 --> 01:17:11,480
I can do that.

1349
01:17:11,480 --> 01:17:15,720
So I find a part which
is colinear with s1.

1350
01:17:15,720 --> 01:17:18,240
I find the part which
is orthogonal.

1351
01:17:18,240 --> 01:17:21,460
I take the orthogonal
part, and that's

1352
01:17:21,460 --> 01:17:22,880
orthogonal, to phi 1.

1353
01:17:22,880 --> 01:17:25,300
And I normalize it.

1354
01:17:25,300 --> 01:17:29,540
So I then have two vectors, phi
1 and phi 2, which span

1355
01:17:29,540 --> 01:17:32,810
the space of functions
of linear

1356
01:17:32,810 --> 01:17:34,650
combinations of s1 and s2.

1357
01:17:34,650 --> 01:17:37,600

1358
01:17:37,600 --> 01:17:45,040
And I call that subspace
S2, capital S2.

1359
01:17:45,040 --> 01:17:46,700
And then I go on.

1360
01:17:46,700 --> 01:17:50,930
So, given any orthonormal basis,
phi 1 up to phi sub k

1361
01:17:50,930 --> 01:17:55,640
of the subspace s k generated
by s1 to s k, I'm going to

1362
01:17:55,640 --> 01:18:00,730
project s k plus 1 onto this
subspace s sub k, and then I'm

1363
01:18:00,730 --> 01:18:03,170
going to normalize it.

1364
01:18:03,170 --> 01:18:07,210
And by going through this
procedure, I can in fact find

1365
01:18:07,210 --> 01:18:10,780
an orthonormal basis to any set
of vectors that I want to,

1366
01:18:10,780 --> 01:18:12,810
to any subspace that
I want to.

1367
01:18:12,810 --> 01:18:15,730
Why is this important, is this
something you want to do?

1368
01:18:15,730 --> 01:18:20,310
Well, it's something you can
program a computer to do

1369
01:18:20,310 --> 01:18:22,770
almost trivially.

1370
01:18:22,770 --> 01:18:24,980
But that's not why
we want it here.

1371
01:18:24,980 --> 01:18:28,780
The thing we want it here is to
say that there's no reason

1372
01:18:28,780 --> 01:18:31,870
to deal with bases other
than orthonormal bases.

1373
01:18:31,870 --> 01:18:35,280
We can generate orthonormal
bases easily.

1374
01:18:35,280 --> 01:18:38,760
The projection theorem now is
valid for any n-dimensional

1375
01:18:38,760 --> 01:18:42,230
space because for any
n-dimensional space we can

1376
01:18:42,230 --> 01:18:45,670
form this basis that we want.

1377
01:18:45,670 --> 01:18:53,480
Let me just go on and finish
this, so we can start dealing

1378
01:18:53,480 --> 01:18:56,840
with channels next time.

1379
01:18:56,840 --> 01:19:01,370
So far, the projection
theorem is just for

1380
01:19:01,370 --> 01:19:03,810
finite dimensional vectors.

1381
01:19:03,810 --> 01:19:08,660
We want to now extend it to
infinite dimensional vectors.

1382
01:19:08,660 --> 01:19:13,370
To accountably infinite
set of vectors.

1383
01:19:13,370 --> 01:19:16,790
So I'm given any orthogonal set
of functions, status sub

1384
01:19:16,790 --> 01:19:20,580
i, we can first generate
orthonormal functions as phi

1385
01:19:20,580 --> 01:19:23,870
sub i, which are normalized.

1386
01:19:23,870 --> 01:19:26,130
And that's old stuff.

1387
01:19:26,130 --> 01:19:30,750
I can now think of doing the
same thing that we did before.

1388
01:19:30,750 --> 01:19:36,800
Namely, starting out, taking any
old vector I want to, and

1389
01:19:36,800 --> 01:19:39,880
projecting it first on
to the subspace with

1390
01:19:39,880 --> 01:19:41,860
only phi 1 in it.

1391
01:19:41,860 --> 01:19:45,820
Then the subspace generated by
phi 1 and phi 2, then the

1392
01:19:45,820 --> 01:19:51,220
subspace generated by phi 1, phi
2 and phi 3, and so forth.

1393
01:19:51,220 --> 01:19:55,360
When I do that successively,
which is successive

1394
01:19:55,360 --> 01:20:00,420
approximations in a Fourier
expansion, or in any

1395
01:20:00,420 --> 01:20:06,100
orthonormal expansion, what I'm
going to wind up with is

1396
01:20:06,100 --> 01:20:09,910
the following theorem that says,
let phi sub m be a set

1397
01:20:09,910 --> 01:20:12,090
of orthonormal functions.

1398
01:20:12,090 --> 01:20:15,970
Let v be any L2 vector.

1399
01:20:15,970 --> 01:20:20,140
Then there exists an L2 vector,
u, such that v minus u

1400
01:20:20,140 --> 01:20:22,880
is orthogonal to
each phi sub n.

1401
01:20:22,880 --> 01:20:26,720
In other words, this is the
projection theorem, carried on

1402
01:20:26,720 --> 01:20:29,220
as n goes to infinity.

1403
01:20:29,220 --> 01:20:31,690
But I can't quite state it
in the way I did before.

1404
01:20:31,690 --> 01:20:33,180
I need a limit in here.

1405
01:20:33,180 --> 01:20:40,280
Which says the limit as n goes
to infinity of u, namely what

1406
01:20:40,280 --> 01:20:45,610
is now going to be this
projection, minus this term

1407
01:20:45,610 --> 01:20:49,540
here, which is the term
in the subspace of

1408
01:20:49,540 --> 01:20:51,570
these orthonormal functions.

1409
01:20:51,570 --> 01:20:55,180
This difference goes to zero.

1410
01:20:55,180 --> 01:20:56,090
What does this say?

1411
01:20:56,090 --> 01:21:00,360
It doesn't say that I can take
any function, v, and expand it

1412
01:21:00,360 --> 01:21:03,630
in an orthonormal expansion.

1413
01:21:03,630 --> 01:21:06,840
I couldn't say that, because I
have nothing to know whether

1414
01:21:06,840 --> 01:21:10,750
this arbitrary orthonormal
expansion I started with

1415
01:21:10,750 --> 01:21:13,130
actually spans L2 or not.

1416
01:21:13,130 --> 01:21:17,350
And without knowing that,
I can't state a

1417
01:21:17,350 --> 01:21:18,210
theorem like this.

1418
01:21:18,210 --> 01:21:21,450
But what the theorem does say
is, you take any orthonormal

1419
01:21:21,450 --> 01:21:23,250
expansion you want to.

1420
01:21:23,250 --> 01:21:26,900
Like the Fourier series, which
only spans functions which are

1421
01:21:26,900 --> 01:21:28,480
time limited.

1422
01:21:28,480 --> 01:21:30,390
I take an arbitrary function.

1423
01:21:30,390 --> 01:21:32,960
I expand them in this
Fourier series.

1424
01:21:32,960 --> 01:21:36,780
And, bingo, what I get is a
function u, which is the part

1425
01:21:36,780 --> 01:21:40,430
of v, which is within these time
limits and what's left

1426
01:21:40,430 --> 01:21:43,930
over, which is orthogonal, is
the stuff outside of those

1427
01:21:43,930 --> 01:21:46,730
time limits.

1428
01:21:46,730 --> 01:21:51,420
So this is the theorem that says
that in fact you can --

1429
01:21:51,420 --> 01:21:52,650
AUDIENCE: [UNINTELLIGIBLE]

1430
01:21:52,650 --> 01:21:52,950
PROFESSOR: What?

1431
01:21:52,950 --> 01:21:55,550
AUDIENCE: [UNINTELLIGIBLE]

1432
01:21:55,550 --> 01:21:59,290
PROFESSOR: It's similar
to Plancherel.

1433
01:21:59,290 --> 01:22:04,300
Plancherel is done for the
Fourier integral, and in

1434
01:22:04,300 --> 01:22:08,840
Plancherel you need this limit
in the mean on both sides.

1435
01:22:08,840 --> 01:22:13,280
Here we're just dealing
with a series.

1436
01:22:13,280 --> 01:22:18,230
I mean, we still have a limit in
the mean sort of thing, but

1437
01:22:18,230 --> 01:22:20,720
we have a weaker theorem in
the sense that we're not

1438
01:22:20,720 --> 01:22:24,030
asserting that this orthonormal
series actually

1439
01:22:24,030 --> 01:22:26,750
spans all of L2.

1440
01:22:26,750 --> 01:22:29,125
I mean, we found a couple of
orthonormal expansions

1441
01:22:29,125 --> 01:22:32,380
that do span L2.

1442
01:22:32,380 --> 01:22:33,980
So it's lacking in that.

1443
01:22:33,980 --> 01:22:35,380
OK, going to stop there.

1444
01:22:35,380 --> 01:22:37,759