1
00:00:00,000 --> 00:00:00,030


2
00:00:00,030 --> 00:00:02,285
The following content is
provided under a Creative

3
00:00:02,285 --> 00:00:03,610
Commons license.

4
00:00:03,610 --> 00:00:05,460
Your support will help
MIT OpenCourseWare

5
00:00:05,460 --> 00:00:09,940
continue to offer high-quality
educational resources for free.

6
00:00:09,940 --> 00:00:12,530
To make a donation, or to
view additional materials

7
00:00:12,530 --> 00:00:16,130
from hundreds of MIT courses,
visit MIT OpenCourseWare

8
00:00:16,130 --> 00:00:21,070
at ocw.mit.edu.

9
00:00:21,070 --> 00:00:24,930
PROFESSOR STRANG: Well, hope
you had a good Thanksgiving.

10
00:00:24,930 --> 00:00:30,190
So this is partly
review today, even.

11
00:00:30,190 --> 00:00:31,880
Wednesday even more review.

12
00:00:31,880 --> 00:00:36,880
Wednesday evening,
or Wednesday at 4

13
00:00:36,880 --> 00:00:38,900
I'll be here for any questions.

14
00:00:38,900 --> 00:00:44,220
And then the exam is
Thursday at 7:30 in Walker.

15
00:00:44,220 --> 00:00:49,380
Top floor of Walker this
time, not the same 54-100.

16
00:00:49,380 --> 00:00:54,760
OK, and then, no
lectures after that.

17
00:00:54,760 --> 00:00:56,300
Holiday, whatever.

18
00:00:56,300 --> 00:00:57,360
Yes.

19
00:00:57,360 --> 00:01:00,270
Right, you get a
chance to do something.

20
00:01:00,270 --> 00:01:02,520
Catch up with all
those other courses

21
00:01:02,520 --> 00:01:05,510
that are being neglected
in favor of 18.085.

22
00:01:05,510 --> 00:01:06,960
Right.

23
00:01:06,960 --> 00:01:11,240
OK, so here's a bit
of review right away.

24
00:01:11,240 --> 00:01:14,770
We really had four cases.

25
00:01:14,770 --> 00:01:19,370
We started with Fourier series,
that was periodic functions.

26
00:01:19,370 --> 00:01:23,930
And then discrete Fourier
series, also periodic in a way.

27
00:01:23,930 --> 00:01:26,600
Because w^N was one.

28
00:01:26,600 --> 00:01:31,340
So that we have N
numbers and then

29
00:01:31,340 --> 00:01:33,510
we could repeat
them if we wanted.

30
00:01:33,510 --> 00:01:37,500
So those are the
two that repeat.

31
00:01:37,500 --> 00:01:41,020
This is the f(x),
this is all x, so that

32
00:01:41,020 --> 00:01:45,310
would be the Fourier integral
that we did just last week.

33
00:01:45,310 --> 00:01:47,130
Fourier integral transform.

34
00:01:47,130 --> 00:01:51,220
And this was the--
Well, these are all,

35
00:01:51,220 --> 00:01:53,840
this is the discrete
all the way.

36
00:01:53,840 --> 00:02:00,810
So that's-- oh, you can
see these pair off, right?

37
00:02:00,810 --> 00:02:07,260
The periodic function,
the 2pi periodic function

38
00:02:07,260 --> 00:02:10,930
has Fourier
coefficients for all k,

39
00:02:10,930 --> 00:02:14,630
so that's the pair
that we started with.

40
00:02:14,630 --> 00:02:16,010
Section 4.1.

41
00:02:16,010 --> 00:02:20,850
This sort of pairs, I don't
know whether to say with itself.

42
00:02:20,850 --> 00:02:26,990
I mean, we start with N numbers
and we end with N numbers.

43
00:02:26,990 --> 00:02:29,670
We have N numbers
in physical space.

44
00:02:29,670 --> 00:02:33,810
And we have N numbers
in frequency space.

45
00:02:33,810 --> 00:02:39,650
Right, so we call those,
so those went to c_0 up

46
00:02:39,650 --> 00:02:42,630
to c_(N-1).

47
00:02:42,630 --> 00:02:47,360
And this, all x, pair--
for the function,

48
00:02:47,360 --> 00:02:48,820
paired off with itself.

49
00:02:48,820 --> 00:02:55,600
Or with this went to F--
well maybe I used small f.

50
00:02:55,600 --> 00:02:58,650
I guess I did in last week.

51
00:02:58,650 --> 00:03:03,520
So that, and I called its
Fourier transform f hat of k,

52
00:03:03,520 --> 00:03:06,820
all k.

53
00:03:06,820 --> 00:03:10,970
So that's the pairing kind of
inside n-dimensional space,

54
00:03:10,970 --> 00:03:12,670
with the Fourier matrix.

55
00:03:12,670 --> 00:03:16,220
This is the pairing
of the formula for f,

56
00:03:16,220 --> 00:03:19,310
and its similar
formula for f hat,

57
00:03:19,310 --> 00:03:22,390
and these are the guys that
connect with each other.

58
00:03:22,390 --> 00:03:25,580
OK, so that's what we know.

59
00:03:25,580 --> 00:03:31,330
What we haven't done is
anything in two dimensions.

60
00:03:31,330 --> 00:03:34,300
So I would like to
include that today.

61
00:03:34,300 --> 00:03:36,750
I think my real
message about 2-D,

62
00:03:36,750 --> 00:03:39,310
and I'm not going to
include it on the exam,

63
00:03:39,310 --> 00:03:45,870
but you might wonder, OK, can
I have a function of x and y?

64
00:03:45,870 --> 00:03:48,040
And will the whole setup work?

65
00:03:48,040 --> 00:03:49,510
And the answer is yes.

66
00:03:49,510 --> 00:03:54,670
So really, my message is not
to be afraid in any way of 2-D.

67
00:03:54,670 --> 00:04:00,590
It's just the same formulas
with x,y or two indices, k,l.

68
00:04:00,590 --> 00:04:01,090
Yeah.

69
00:04:01,090 --> 00:04:02,190
You'll see that.

70
00:04:02,190 --> 00:04:05,630
OK, now for the new part.

71
00:04:05,630 --> 00:04:09,380
What's a convolution equation?

72
00:04:09,380 --> 00:04:12,710
That's my word for
an equation where

73
00:04:12,710 --> 00:04:16,230
instead of doing a
convolution and finding

74
00:04:16,230 --> 00:04:19,360
the right-hand
side, instead we're

75
00:04:19,360 --> 00:04:21,760
given the right-hand side.

76
00:04:21,760 --> 00:04:26,560
And the unknown is
in the convolution.

77
00:04:26,560 --> 00:04:30,200
So let me write examples
of convolution equation.

78
00:04:30,200 --> 00:04:34,010
Every one of these would
allow a convolution.

79
00:04:34,010 --> 00:04:35,600
So the convolution
equation would

80
00:04:35,600 --> 00:04:42,650
be something the integral
of F(t) u, for the unknown,

81
00:04:42,650 --> 00:04:48,550
at x-t, is-- Oh no, sorry.

82
00:04:48,550 --> 00:04:50,400
F will be the right-hand side.

83
00:04:50,400 --> 00:04:53,430
F of, well, can I--
Yeah, better if I put it

84
00:04:53,430 --> 00:04:55,090
on the right-hand side.

85
00:04:55,090 --> 00:04:59,840
Wouldn't want to call
it the right-hand side.

86
00:04:59,840 --> 00:05:05,500
So this would be some, shall
I call it often K for kernel

87
00:05:05,500 --> 00:05:08,190
is sometimes the word.

88
00:05:08,190 --> 00:05:13,350
So what I'm saying is
equations come this way.

89
00:05:13,350 --> 00:05:19,130
This is really K
convolved with u.

90
00:05:19,130 --> 00:05:24,260
Equals F. You see, the only
novelty is the unknown is here.

91
00:05:24,260 --> 00:05:28,574
So that's why the word
deconvolution is up there.

92
00:05:28,574 --> 00:05:29,990
Because that's
what we have to do.

93
00:05:29,990 --> 00:05:34,120
We have to undo the convolution,
this unknown function

94
00:05:34,120 --> 00:05:39,390
is convolved with a known, K is
known, some known kernel that

95
00:05:39,390 --> 00:05:42,810
tells us the point
spread of the telescope

96
00:05:42,810 --> 00:05:44,060
or whatever we're doing.

97
00:05:44,060 --> 00:05:47,490
And gives us the output
that we're looking at.

98
00:05:47,490 --> 00:05:50,120
And then we have
to find the input.

99
00:05:50,120 --> 00:05:54,760
OK, can I write down the similar
equations for the other three

100
00:05:54,760 --> 00:05:55,360
here?

101
00:05:55,360 --> 00:05:58,770
And then we'll just
think how would we find

102
00:05:58,770 --> 00:06:00,340
u, how would we solve them?

103
00:06:00,340 --> 00:06:07,260
So the equation here
might be that some kernel

104
00:06:07,260 --> 00:06:13,500
circle convolved with
the unknown u is some y.

105
00:06:13,500 --> 00:06:16,940
These are now vectors.

106
00:06:16,940 --> 00:06:19,220
This is known.

107
00:06:19,220 --> 00:06:20,380
This is known.

108
00:06:20,380 --> 00:06:23,710
And those, the N components
of u, are unknown.

109
00:06:23,710 --> 00:06:26,710
OK, so that would be
the same problem here.

110
00:06:26,710 --> 00:06:27,680
What would be here?

111
00:06:27,680 --> 00:06:28,660
Same thing.

112
00:06:28,660 --> 00:06:31,520
Now the integral will go
from-- The only difference is

113
00:06:31,520 --> 00:06:36,420
the integral will go from
minus infinity to infinity,

114
00:06:36,420 --> 00:06:39,990
K(t)u(x-t) dt equal f(x).

115
00:06:39,990 --> 00:06:43,570


116
00:06:43,570 --> 00:06:48,970
And finally regular convolution.

117
00:06:48,970 --> 00:06:50,210
What am I going to call it?

118
00:06:50,210 --> 00:06:55,720
K would be a sequence,
maybe I should call it a,

119
00:06:55,720 --> 00:07:04,141
known, convolved with u,
unknown, is some c, known.

120
00:07:04,141 --> 00:07:04,640
Yeah.

121
00:07:04,640 --> 00:07:07,570
So those would be
four equations.

122
00:07:07,570 --> 00:07:09,210
You might say, wait
a minute, where

123
00:07:09,210 --> 00:07:11,220
is Professor Strang come
up with these problems

124
00:07:11,220 --> 00:07:13,450
at the last week of the course.

125
00:07:13,450 --> 00:07:18,720
But, these are exactly
the type of problems

126
00:07:18,720 --> 00:07:20,520
that we know and love.

127
00:07:20,520 --> 00:07:25,370
These come from
constant-coefficient,

128
00:07:25,370 --> 00:07:31,170
time-invariant, shift-invariant
linear problems.

129
00:07:31,170 --> 00:07:33,620
LTI, linear time-invariant.

130
00:07:33,620 --> 00:07:40,370
And my lecture Wednesday,
just before Thanksgiving, took

131
00:07:40,370 --> 00:07:45,300
a differential equation
for u and found,

132
00:07:45,300 --> 00:07:46,670
and put it in this form.

133
00:07:46,670 --> 00:07:48,640
I'll come back to that.

134
00:07:48,640 --> 00:07:51,750
So suddenly we're
seeing, I mean,

135
00:07:51,750 --> 00:07:53,800
we're actually seeing
some new things

136
00:07:53,800 --> 00:07:56,640
but also it includes
all the old ones.

137
00:07:56,640 --> 00:08:00,170
These are all of the best
problems in the world.

138
00:08:00,170 --> 00:08:02,920
These linear
constant-coefficient problems.

139
00:08:02,920 --> 00:08:05,900
Time invariant, of
any of these types.

140
00:08:05,900 --> 00:08:08,700
This one was an
integral from minus pi

141
00:08:08,700 --> 00:08:12,400
to pi, where this
one went all the way.

142
00:08:12,400 --> 00:08:16,540
So this is not brand new stuff.

143
00:08:16,540 --> 00:08:19,810
But it sort of looks new.

144
00:08:19,810 --> 00:08:26,890
And now the question is, so my
immediate question is, before

145
00:08:26,890 --> 00:08:31,150
doing any example, how would
you solve such an equation.

146
00:08:31,150 --> 00:08:34,820
And I saw on old exams,
some of this sort

147
00:08:34,820 --> 00:08:39,690
for example, let me
focus on this one.

148
00:08:39,690 --> 00:08:42,280
Let me, instead of
K there, I'm not

149
00:08:42,280 --> 00:08:46,080
used to using K for a
vector, I'm used to,

150
00:08:46,080 --> 00:08:48,860
well maybe I'll use c.

151
00:08:48,860 --> 00:08:50,220
For the vector there.

152
00:08:50,220 --> 00:08:59,060
So this is n
equations, n unknowns.

153
00:08:59,060 --> 00:09:01,610
Oops, capital N
is our usual here.

154
00:09:01,610 --> 00:09:03,160
For the number.

155
00:09:03,160 --> 00:09:06,740
N u, N unknown u's.

156
00:09:06,740 --> 00:09:10,670
It's a matrix equation
with a circulant matrix.

157
00:09:10,670 --> 00:09:17,510
So all these equations are
sort of the special best kind.

158
00:09:17,510 --> 00:09:19,540
Because they're convolutions.

159
00:09:19,540 --> 00:09:22,140
And now tell me the main point.

160
00:09:22,140 --> 00:09:24,820
How do we solve
equations like this?

161
00:09:24,820 --> 00:09:30,680
How do we do a deconvolution,
so the unknown is convolved

162
00:09:30,680 --> 00:09:35,330
with c here, it's convolved
with K, it's convolved with a,

163
00:09:35,330 --> 00:09:40,780
how do we deconvolve
it to get u by itself?

164
00:09:40,780 --> 00:09:43,840
So what's the central idea here?

165
00:09:43,840 --> 00:09:47,950
Central idea: go
into frequency space.

166
00:09:47,950 --> 00:09:50,160
Use the convolution rule.

167
00:09:50,160 --> 00:09:57,730
In frequency space,
where these transform,

168
00:09:57,730 --> 00:10:01,190
we're looking at multiplication.

169
00:10:01,190 --> 00:10:04,160
And multiplication, we can undo.

170
00:10:04,160 --> 00:10:06,680
We can de-multiply.

171
00:10:06,680 --> 00:10:10,930
De-multiply is just a big
word for divide, right?

172
00:10:10,930 --> 00:10:12,580
So that's the point.

173
00:10:12,580 --> 00:10:14,030
Get into that space.

174
00:10:14,030 --> 00:10:15,870
That's what we've been
doing all the time.

175
00:10:15,870 --> 00:10:21,490
I better get one example,
the example from the problem

176
00:10:21,490 --> 00:10:23,620
Wednesday, just up here.

177
00:10:23,620 --> 00:10:24,810
Just so you see it.

178
00:10:24,810 --> 00:10:27,280
This won't look like a
convolution equation,

179
00:10:27,280 --> 00:10:33,570
but do you remember that it
was -u'' plus a squared u equal

180
00:10:33,570 --> 00:10:37,410
some f(x)?

181
00:10:37,410 --> 00:10:40,500
So that's a constant, it's
certainly constant-coefficient,

182
00:10:40,500 --> 00:10:42,270
linear, time-invariant.

183
00:10:42,270 --> 00:10:43,100
Right, OK.

184
00:10:43,100 --> 00:10:44,600
And how did we solve that?

185
00:10:44,600 --> 00:10:46,960
We took Fourier transforms.

186
00:10:46,960 --> 00:10:51,770
So this was the
second derivative,

187
00:10:51,770 --> 00:10:52,930
the Fourier transform.

188
00:10:52,930 --> 00:10:56,870
What is the rule for the
Fourier transform of derivative?

189
00:10:56,870 --> 00:11:01,130
Every derivative brings
down an ik in the transform.

190
00:11:01,130 --> 00:11:03,030
So we get ik twice.

191
00:11:03,030 --> 00:11:07,200
So it's k squared. i squared
cancels the minus one.

192
00:11:07,200 --> 00:11:12,380
So that's the transform of -u''.

193
00:11:12,380 --> 00:11:17,010
This is the transform of
ordinary a squared u, just

194
00:11:17,010 --> 00:11:17,840
a squared.

195
00:11:17,840 --> 00:11:19,880
And this is f hat.

196
00:11:19,880 --> 00:11:23,420
So we've got into
frequency space.

197
00:11:23,420 --> 00:11:25,880
Where we are just
seeing a multiplication,

198
00:11:25,880 --> 00:11:34,620
k squared plus a squared, u
hat, of-- this is u hat of k,

199
00:11:34,620 --> 00:11:39,000
equals f hat of k, right?

200
00:11:39,000 --> 00:11:43,190
Oh well, sorry we were-- Yeah,
that's right. f hat of k,

201
00:11:43,190 --> 00:11:45,230
right.

202
00:11:45,230 --> 00:11:48,090
So we're in frequency
space, where we just

203
00:11:48,090 --> 00:11:49,220
see a multiplication.

204
00:11:49,220 --> 00:11:58,010
So again, this is now we
just demultiply, just divide.

205
00:11:58,010 --> 00:12:03,000
And then we have the answer,
but we have its transform.

206
00:12:03,000 --> 00:12:05,550
And then we have
to transform back.

207
00:12:05,550 --> 00:12:10,560
So we have to do the Fourier
transform to get to the,

208
00:12:10,560 --> 00:12:15,460
I'll say the inverse
Fourier transform.

209
00:12:15,460 --> 00:12:19,655
To get back to u(x), the answer.

210
00:12:19,655 --> 00:12:21,160
That's the model.

211
00:12:21,160 --> 00:12:22,110
That's the model.

212
00:12:22,110 --> 00:12:25,410
And that's maybe the
one that we've seen,

213
00:12:25,410 --> 00:12:32,710
now we're able to think
about all these four topics.

214
00:12:32,710 --> 00:12:34,660
Right.

215
00:12:34,660 --> 00:12:38,090
OK, so what was the key idea?

216
00:12:38,090 --> 00:12:40,520
Get into frequency
space and then it's

217
00:12:40,520 --> 00:12:44,160
just a, the equation is
just a multiplication,

218
00:12:44,160 --> 00:12:46,980
so the solution is
just a division.

219
00:12:46,980 --> 00:12:52,050
So can I do that now with
these four examples, just see.

220
00:12:52,050 --> 00:12:54,480
So this is like bring
the pieces together.

221
00:12:54,480 --> 00:12:59,140
OK, and deconvolution is
a very key thing to do.

222
00:12:59,140 --> 00:13:01,370
OK, so I'll take
all four of those

223
00:13:01,370 --> 00:13:04,000
and bring them into
frequency space.

224
00:13:04,000 --> 00:13:07,100
So this will be
maybe, you'll let

225
00:13:07,100 --> 00:13:14,080
me use K hat of, oh, K hat
of k that's not too good.

226
00:13:14,080 --> 00:13:27,390
Well, stuck with it.

227
00:13:27,390 --> 00:13:28,680
What am I doing here?

228
00:13:28,680 --> 00:13:31,040
In this 2pi periodic one?

229
00:13:31,040 --> 00:13:35,250
That's the one I started
with, but now I've got,

230
00:13:35,250 --> 00:13:37,030
I'm using hats and so on.

231
00:13:37,030 --> 00:13:42,180
I didn't do that in Section 4.1.

232
00:13:42,180 --> 00:13:46,980
What the heck am I going to,
what notation am I going to do?

233
00:13:46,980 --> 00:13:51,740
And I really didn't do
convolution that much,

234
00:13:51,740 --> 00:13:52,760
for functions.

235
00:13:52,760 --> 00:13:55,300
So let me jump to here.

236
00:13:55,300 --> 00:13:57,030
I'll come back.

237
00:13:57,030 --> 00:13:59,750
It follows exactly
the same pattern.

238
00:13:59,750 --> 00:14:01,350
So let me jump to this one.

239
00:14:01,350 --> 00:14:05,920
OK, so I have a
convolution equation now.

240
00:14:05,920 --> 00:14:10,700
This is one where you
could do this one.

241
00:14:10,700 --> 00:14:16,185
This could appear on the quiz
because I can do all of it.

242
00:14:16,185 --> 00:14:18,690
So what is this convolution?

243
00:14:18,690 --> 00:14:19,870
OK.

244
00:14:19,870 --> 00:14:21,920
I've got N equations,
N unknowns.

245
00:14:21,920 --> 00:14:23,800
Let me write them
in matrix form,

246
00:14:23,800 --> 00:14:28,880
just so you see it that way
too. c_0, c_1, c_2, c_3.

247
00:14:28,880 --> 00:14:30,970
I'll make N equal four.

248
00:14:30,970 --> 00:14:39,790
And then these are, this
convolution has c_0, c_2, c_1.

249
00:14:39,790 --> 00:14:44,510
c_3, c_2, c_1, c_0.

250
00:14:44,510 --> 00:14:49,960
This'll be c_3, c_3,
c_3, I'm writing down

251
00:14:49,960 --> 00:14:53,830
all the right numbers
in the right places.

252
00:14:53,830 --> 00:14:56,020
So that when I do
that multiplication

253
00:14:56,020 --> 00:15:04,740
with the unknown,
[u 0, u 1,  u 2, u 3],

254
00:15:04,740 --> 00:15:10,530
I get the right-hand, the
known right-hand side.

255
00:15:10,530 --> 00:15:12,350
Maybe b would be
a little better.

256
00:15:12,350 --> 00:15:15,740
Because we're more used
to b as as a known.

257
00:15:15,740 --> 00:15:22,490
It's just an Ax=b problem,
or an Au=b problem.

258
00:15:22,490 --> 00:15:24,550
It looks like a
convolution but now

259
00:15:24,550 --> 00:15:26,520
it's just a matrix
multiplication.

260
00:15:26,520 --> 00:15:30,340
So this is just
[b 0, b 1, b 2,  b 3].

261
00:15:30,340 --> 00:15:32,800
OK.

262
00:15:32,800 --> 00:15:34,920
That's our equation.

263
00:15:34,920 --> 00:15:36,950
Special type of matrix.

264
00:15:36,950 --> 00:15:38,470
Circulant matrix.

265
00:15:38,470 --> 00:15:40,730
So this is just
literally the same

266
00:15:40,730 --> 00:15:46,630
as c circularly convolved
with u equals b.

267
00:15:46,630 --> 00:15:50,640
I just wrote it out
in matrix language.

268
00:15:50,640 --> 00:15:59,640
So you could call MATLAB with
that matrix, and so one way

269
00:15:59,640 --> 00:16:04,960
to answer it would be get
the inverse of the matrix.

270
00:16:04,960 --> 00:16:11,190
But if it was
large, a better way

271
00:16:11,190 --> 00:16:16,190
would be switch over
to frequency space.

272
00:16:16,190 --> 00:16:16,970
Think, now.

273
00:16:16,970 --> 00:16:23,680
What happens when I switch these
vectors to frequency space?

274
00:16:23,680 --> 00:16:25,540
It becomes a multiplication.

275
00:16:25,540 --> 00:16:29,570
So this becomes
a multiplication.

276
00:16:29,570 --> 00:16:36,990
Now so c, I want the Fourier,
from the c's, what am I

277
00:16:36,990 --> 00:16:45,580
going to-- So these are
all in the space where

278
00:16:45,580 --> 00:16:46,820
it's a convolution.

279
00:16:46,820 --> 00:16:49,480
What am I going to call it
where it's in the space where

280
00:16:49,480 --> 00:16:51,270
it's a multiplication?

281
00:16:51,270 --> 00:16:54,050
I just need three new names.

282
00:16:54,050 --> 00:16:59,190
Maybe I'll use c
hat, u hat, and b hat

283
00:16:59,190 --> 00:17:02,950
just because there's no
doubt in anybody's mind

284
00:17:02,950 --> 00:17:05,460
that when you see
that hat, you've

285
00:17:05,460 --> 00:17:07,690
gone into frequency space.

286
00:17:07,690 --> 00:17:11,800
Now, what's the equation
in frequency space?

287
00:17:11,800 --> 00:17:15,760
And then I'll do an example.

288
00:17:15,760 --> 00:17:20,360
It's a multiplication,
but I don't usually

289
00:17:20,360 --> 00:17:27,160
see a vector and nothing there.

290
00:17:27,160 --> 00:17:31,270
What's the multiplication
in frequency space?

291
00:17:31,270 --> 00:17:37,310
It's component by component.

292
00:17:37,310 --> 00:17:48,100
c_0*u_0 equals b_0. c hat
1 u hat 1 equals b hat 1.

293
00:17:48,100 --> 00:17:53,760
c hat 2 u hat 2 equals b hat 2.

294
00:17:53,760 --> 00:18:00,650
And finally, c hat 3 u
hat 3 equals b hat 3.

295
00:18:00,650 --> 00:18:05,040
And there might be, I don't
swear that there isn't, a 1/4

296
00:18:05,040 --> 00:18:06,210
somewhere.

297
00:18:06,210 --> 00:18:07,780
Right?

298
00:18:07,780 --> 00:18:12,830
But the point is, we're
in frequency space now.

299
00:18:12,830 --> 00:18:15,190
We just have a
component by component,

300
00:18:15,190 --> 00:18:21,550
each component of c hat
times each component of u hat

301
00:18:21,550 --> 00:18:23,910
gives us a component
of b hat; now we're

302
00:18:23,910 --> 00:18:26,560
ready for a deconvolution;
just divide.

303
00:18:26,560 --> 00:18:29,860
So now u hat,
obviously I don't have

304
00:18:29,860 --> 00:18:34,940
to write all these,
b hat 0 over c hat 0.

305
00:18:34,940 --> 00:18:35,720
Right?

306
00:18:35,720 --> 00:18:36,850
I just do a division.

307
00:18:36,850 --> 00:18:44,520
So on down to u hat 3 is b hat,
is the third component of b,

308
00:18:44,520 --> 00:18:47,840
divided by the third
component of c.

309
00:18:47,840 --> 00:18:50,860
OK, now don't forget here.

310
00:18:50,860 --> 00:18:53,800
That in going from
here to here, I

311
00:18:53,800 --> 00:18:57,730
had to figure out what
the c hats were, right?

312
00:18:57,730 --> 00:19:03,330
I had to do the Fourier matrix,
or the inverse Fourier matrix

313
00:19:03,330 --> 00:19:07,620
to go from c to c
hat, from b to b hat,

314
00:19:07,620 --> 00:19:10,870
so everything got
Fourier transformed.

315
00:19:10,870 --> 00:19:17,500
But the object was to
make the equation easy.

316
00:19:17,500 --> 00:19:21,720
And of course, now we've
got four trivial equations

317
00:19:21,720 --> 00:19:24,160
that we just solved that way.

318
00:19:24,160 --> 00:19:27,460
Alright, let me see if I
can just pull this down

319
00:19:27,460 --> 00:19:32,730
with some questions.

320
00:19:32,730 --> 00:19:34,000
Here's a good question.

321
00:19:34,000 --> 00:19:39,300
When is a circulant
matrix invertible?

322
00:19:39,300 --> 00:19:42,900
When will this method work?

323
00:19:42,900 --> 00:19:45,520
The circulant matrix could
fail to be invertible.

324
00:19:45,520 --> 00:19:50,090
How would I know that?

325
00:19:50,090 --> 00:19:54,730
If it's singular, and how
would I, if I proceed this way,

326
00:19:54,730 --> 00:19:57,600
here I've got an answer.

327
00:19:57,600 --> 00:19:59,290
But if it's singular
I'm not really

328
00:19:59,290 --> 00:20:00,540
expecting to get an answer.

329
00:20:00,540 --> 00:20:03,260
Let me lift the board a little.

330
00:20:03,260 --> 00:20:14,100
So where would I get, oops,
have to stop this method.

331
00:20:14,100 --> 00:20:16,460
In solving those four equations.

332
00:20:16,460 --> 00:20:20,140
Where would I learn
that it's is singular?

333
00:20:20,140 --> 00:20:22,820
What could go wrong in this?

334
00:20:22,820 --> 00:20:23,330
Yes.

335
00:20:23,330 --> 00:20:25,010
AUDIENCE: [INAUDIBLE]

336
00:20:25,010 --> 00:20:26,380
PROFESSOR STRANG: That's right.

337
00:20:26,380 --> 00:20:32,600
Always in math, the question
is are you dividing by zero.

338
00:20:32,600 --> 00:20:36,210
So the question of whether
the matrix is singular,

339
00:20:36,210 --> 00:20:39,420
is the same as the question
of whether c_0 hat,

340
00:20:39,420 --> 00:20:46,860
c_01, c_02, and c_03-- sorry,
c_0 hat, c_1 hat, c_2 hat,

341
00:20:46,860 --> 00:20:49,740
and c_3 hat, can't be zero.

342
00:20:49,740 --> 00:20:55,780
That's, in fact, even better
those four numbers, those four

343
00:20:55,780 --> 00:21:00,190
c hats, are actually the
eigenvalues of the matrix.

344
00:21:00,190 --> 00:21:06,090
We've switched, what the Fourier
transform did, was switch over

345
00:21:06,090 --> 00:21:08,780
to the eigenvalues
and eigenvectors.

346
00:21:08,780 --> 00:21:12,050
And there, that's the whole
message of those guys is,

347
00:21:12,050 --> 00:21:14,010
you follow each one separately.

348
00:21:14,010 --> 00:21:15,880
Just the way we're doing here.

349
00:21:15,880 --> 00:21:20,150
So this is the component of
the b in the four eigenvector

350
00:21:20,150 --> 00:21:21,200
directions.

351
00:21:21,200 --> 00:21:26,020
Those are the four eigenvalues,
and I have to divide by them.

352
00:21:26,020 --> 00:21:30,620
You see, the idea is, like,
we've diagonalized the matrix.

353
00:21:30,620 --> 00:21:35,570
We've had that
matrix, which is full.

354
00:21:35,570 --> 00:21:41,810
And we take-- By taking the
Fourier transforms, that's

355
00:21:41,810 --> 00:21:46,440
the same thing as as
putting in the eigenvectors,

356
00:21:46,440 --> 00:21:51,660
switching the matrix to
this diagonal matrix, right?

357
00:21:51,660 --> 00:21:56,590
Our problem has become,
like, the diagonalized form

358
00:21:56,590 --> 00:22:01,670
is c_0 hat down to c_3 hat,
sitting on the diagonal.

359
00:22:01,670 --> 00:22:03,420
All zeroes elsewhere.

360
00:22:03,420 --> 00:22:06,920
That's when we switched, when
we did Fourier transform we

361
00:22:06,920 --> 00:22:08,780
were switching to eigenvectors.

362
00:22:08,780 --> 00:22:10,830
OK, so that's the message.

363
00:22:10,830 --> 00:22:18,780
That the test for singularity is
the Fourier, the transform of c

364
00:22:18,780 --> 00:22:21,560
hits zero.

365
00:22:21,560 --> 00:22:22,930
Then we're in trouble.

366
00:22:22,930 --> 00:22:25,350
Let me do an example you know.

367
00:22:25,350 --> 00:22:27,580
Let me do an example you know.

368
00:22:27,580 --> 00:22:33,250
OK here's, so finally now
we get a numerical example.

369
00:22:33,250 --> 00:22:38,900
The example we really
know is this one, right?

370
00:22:38,900 --> 00:22:41,620
As I start writing that,
you may say in your mind,

371
00:22:41,620 --> 00:22:43,280
oh no not again.

372
00:22:43,280 --> 00:22:47,840
But give it to me, one more
week with these matrices.

373
00:22:47,840 --> 00:22:54,710
But it'll be the C matrix, so
it's going to be the circulant.

374
00:22:54,710 --> 00:22:57,900
Recognize this?

375
00:22:57,900 --> 00:23:02,320
And it's got those minus
ones in the corners, too.

376
00:23:02,320 --> 00:23:05,750
OK, let's go back to day one.

377
00:23:05,750 --> 00:23:09,390
Is that matrix invertible?

378
00:23:09,390 --> 00:23:12,310
Yes or no.

379
00:23:12,310 --> 00:23:14,340
Please, no.

380
00:23:14,340 --> 00:23:17,230
Everybody knows that
matrix is not invertible.

381
00:23:17,230 --> 00:23:21,400
And do you remember
what's in the null space?

382
00:23:21,400 --> 00:23:25,440
Yes, what's the vector in the
null space of that matrix?

383
00:23:25,440 --> 00:23:27,350
All ones.

384
00:23:27,350 --> 00:23:30,620
Now, when I take,
just think now.

385
00:23:30,620 --> 00:23:35,250
When I take Fourier
transform, that all ones is

386
00:23:35,250 --> 00:23:38,640
going to transform to what?

387
00:23:38,640 --> 00:23:40,300
It's going to
transform to the delta.

388
00:23:40,300 --> 00:23:45,370
It'll transform to the one
that is like [1, 0, 0, 0].

389
00:23:45,370 --> 00:23:47,400
Or maybe it's [4, 0, 0, 0].

390
00:23:47,400 --> 00:24:01,370
But it's that, well, OK, now I'm
ready to take, so here's my c.

391
00:24:01,370 --> 00:24:02,810
So what's my method now?

392
00:24:02,810 --> 00:24:07,190
I'm going to do this method,
and I'm going to run into,

393
00:24:07,190 --> 00:24:09,520
this thing is going to be zero.

394
00:24:09,520 --> 00:24:11,470
Because that's the
eigenvalue that

395
00:24:11,470 --> 00:24:16,320
goes with the [1, 1, 1, 1, 1]
column, the constant, the zero

396
00:24:16,320 --> 00:24:19,140
frequency in frequency space.

397
00:24:19,140 --> 00:24:20,590
You'll see it happen.

398
00:24:20,590 --> 00:24:24,511
So let's take the Fourier
transform of that.

399
00:24:24,511 --> 00:24:26,260
And then we would have
to take the Fourier

400
00:24:26,260 --> 00:24:28,450
transform of the
right-hand side b, whatever

401
00:24:28,450 --> 00:24:29,380
that happened to be.

402
00:24:29,380 --> 00:24:31,480
But it's always the left side.

403
00:24:31,480 --> 00:24:33,680
The singular or not matrix.

404
00:24:33,680 --> 00:24:35,760
I believe we'll
be singular here.

405
00:24:35,760 --> 00:24:42,470
So, OK, just remind me how do
I take transforms of this guy?

406
00:24:42,470 --> 00:24:44,600
Gosh, we have to
be able to do that.

407
00:24:44,600 --> 00:24:48,430
That's Section 4 point,
well, 4.3 isn't, yeah.

408
00:24:48,430 --> 00:24:53,170
The DFT of that vector.

409
00:24:53,170 --> 00:24:56,960
What do I get?

410
00:24:56,960 --> 00:24:58,570
Yes.

411
00:24:58,570 --> 00:25:02,240
How do I take the
DFT of a vector?

412
00:25:02,240 --> 00:25:05,420
I multiply by the
Fourier matrix, right?

413
00:25:05,420 --> 00:25:06,800
Yes.

414
00:25:06,800 --> 00:25:10,770
So I have to multiply that
thing by the Fourier matrix.

415
00:25:10,770 --> 00:25:17,090
So to get c hat, this was
big C for the matrix, little

416
00:25:17,090 --> 00:25:21,790
c for the vector that goes
into it, into column zero.

417
00:25:21,790 --> 00:25:24,290
And c hat for its transform.

418
00:25:24,290 --> 00:25:29,640
OK, so now here comes the
Fourier matrix that we know, 1,

419
00:25:29,640 --> 00:25:39,770
i, i^2, i^3; 1, i^2, i^4,
i^6; 1, i^3, i^6, and i^9.

420
00:25:39,770 --> 00:25:46,340
So I want to transform that
c to get, to find out c hat.

421
00:25:46,340 --> 00:25:52,300
OK, and what do I get up there?

422
00:25:52,300 --> 00:25:54,840
What's the first component,
the zeroth component

423
00:25:54,840 --> 00:26:02,820
I should say, when I take this
guy, this four, this vector

424
00:26:02,820 --> 00:26:05,070
with four components
and I get back

425
00:26:05,070 --> 00:26:08,910
four components, the
frequency components, what's

426
00:26:08,910 --> 00:26:10,640
the first one?

427
00:26:10,640 --> 00:26:13,570
Ones times this,
what am I getting?

428
00:26:13,570 --> 00:26:14,220
Zero.

429
00:26:14,220 --> 00:26:16,230
That's what I expected.

430
00:26:16,230 --> 00:26:20,100
That tells me the matrix is
not going to be invertible.

431
00:26:20,100 --> 00:26:28,140
Because in a different language,
I'm finding the eigenvalues

432
00:26:28,140 --> 00:26:30,100
and that's one of them.

433
00:26:30,100 --> 00:26:32,610
And if an eigenvalue
is zero, that

434
00:26:32,610 --> 00:26:36,270
means the eigenvector is
getting knocked out completely.

435
00:26:36,270 --> 00:26:40,590
And there's no way a c
inverse could recover

436
00:26:40,590 --> 00:26:42,370
when that eigenvector is gone.

437
00:26:42,370 --> 00:26:43,860
OK, let's do the other ones.

438
00:26:43,860 --> 00:26:49,510
Two minus i, nothing.

439
00:26:49,510 --> 00:26:51,880
What's the other one here?

440
00:26:51,880 --> 00:26:56,560
Two, this is minus i, and
that's plus i, I think.

441
00:26:56,560 --> 00:26:59,040
So I think it's just two.

442
00:26:59,040 --> 00:27:02,910
Alright, this is 2 i squared,
can I write in some of these

443
00:27:02,910 --> 00:27:07,190
just so I have a little,
i squared is minus one,

444
00:27:07,190 --> 00:27:10,170
and i^4 is one, and
that's minus one.

445
00:27:10,170 --> 00:27:15,660
So that's two plus one,
plus one, I think is four.

446
00:27:15,660 --> 00:27:20,100
And then i^3 is the
same as minus i.

447
00:27:20,100 --> 00:27:22,800
And i^9 is the same as plus i.

448
00:27:22,800 --> 00:27:34,900
So I think I'm getting two
plus i, nothing, minus i: two.

449
00:27:34,900 --> 00:27:36,770
So what's my claim?

450
00:27:36,770 --> 00:27:39,280
My claim is that these
are the four eigenvalues,

451
00:27:39,280 --> 00:27:43,080
that the Fourier-- Fourier
diagonalizes these problems.

452
00:27:43,080 --> 00:27:44,800
That's what it comes to.

453
00:27:44,800 --> 00:27:48,700
Fourier diagonalizes all
constant-coefficient,

454
00:27:48,700 --> 00:27:51,850
shift-invariant,
linear problems.

455
00:27:51,850 --> 00:27:55,820
And tells us here
are eigenvalues.

456
00:27:55,820 --> 00:27:57,530
So [0, 2, 4, 2].

457
00:27:57,530 --> 00:27:59,180
Would you like
to, how do I check

458
00:27:59,180 --> 00:28:00,710
the eigenvalues of a matrix?

459
00:28:00,710 --> 00:28:02,940
Let's just remember.

460
00:28:02,940 --> 00:28:06,870
If I give you four numbers and
I say those are the eigenvalues,

461
00:28:06,870 --> 00:28:09,730
and you look at that
matrix, what quick check

462
00:28:09,730 --> 00:28:12,020
does everybody do?

463
00:28:12,020 --> 00:28:14,650
Compute the?

464
00:28:14,650 --> 00:28:15,425
The trace.

465
00:28:15,425 --> 00:28:17,880
Add up the diagonal
of the matrix,

466
00:28:17,880 --> 00:28:21,560
add up the proposed eigenvalues,
they had better be the same.

467
00:28:21,560 --> 00:28:22,330
And they are.

468
00:28:22,330 --> 00:28:23,725
I get eight both ways.

469
00:28:23,725 --> 00:28:26,100
That doesn't mean, of course,
that these four numbers are

470
00:28:26,100 --> 00:28:28,430
right, but I think they are.

471
00:28:28,430 --> 00:28:29,200
Yeah, yeah.

472
00:28:29,200 --> 00:28:31,810
So those added up to
eight, those numbers

473
00:28:31,810 --> 00:28:33,240
added up to eight.

474
00:28:33,240 --> 00:28:36,520
And yep.

475
00:28:36,520 --> 00:28:38,270
And these are real.

476
00:28:38,270 --> 00:28:39,970
They came out
real, and how did I

477
00:28:39,970 --> 00:28:44,530
know that would happen
from the matrix?

478
00:28:44,530 --> 00:28:51,030
What matrices am I certain
to get real eigenvalues for?

479
00:28:51,030 --> 00:28:51,900
Symmetric.

480
00:28:51,900 --> 00:28:52,690
Right.

481
00:28:52,690 --> 00:28:55,300
Now, what about, I
heard the word positive.

482
00:28:55,300 --> 00:28:58,070
Of course, that's the other
question I have to ask.

483
00:28:58,070 --> 00:29:03,760
Is this matrix
positive definite?

484
00:29:03,760 --> 00:29:05,590
OK, everybody, this
is the language

485
00:29:05,590 --> 00:29:06,770
we've learned in 18.085.

486
00:29:06,770 --> 00:29:09,980
Is that matrix positive
definite, yes or no?

487
00:29:09,980 --> 00:29:11,150
No.

488
00:29:11,150 --> 00:29:12,020
What is it?

489
00:29:12,020 --> 00:29:15,260
It's positive semi-definite.

490
00:29:15,260 --> 00:29:18,460
What does that tell
me about eigenvalues?

491
00:29:18,460 --> 00:29:22,100
There, one is zero, that's why
it's not positive definite.

492
00:29:22,100 --> 00:29:24,270
But the others are positive.

493
00:29:24,270 --> 00:29:27,010
So that sure enough,
in other words,

494
00:29:27,010 --> 00:29:32,260
what we've done here, for that
matrix that came on day one

495
00:29:32,260 --> 00:29:38,040
and now we're seeing
it on day N-1 here.

496
00:29:38,040 --> 00:29:42,590
We're we're seeing sort of in
a new way, because at that time

497
00:29:42,590 --> 00:29:47,230
we didn't know these four
were the eigenvectors

498
00:29:47,230 --> 00:29:49,000
of that matrix.

499
00:29:49,000 --> 00:29:50,300
But they are.

500
00:29:50,300 --> 00:29:56,080
And we're coming to
the same conclusion

501
00:29:56,080 --> 00:29:58,010
we came to on day one.

502
00:29:58,010 --> 00:30:03,270
That the matrix is
positive semi-definite

503
00:30:03,270 --> 00:30:05,470
and that we know
its eigenvalues.

504
00:30:05,470 --> 00:30:08,850
And we can,
actually, let me even

505
00:30:08,850 --> 00:30:11,350
take it one more step, just
because this example is

506
00:30:11,350 --> 00:30:13,200
so perfect.

507
00:30:13,200 --> 00:30:17,300
Some right-hand sides we
could solve for, right?

508
00:30:17,300 --> 00:30:21,070
If I have a matrix
that's singular.

509
00:30:21,070 --> 00:30:24,380
Way, way back, even, I think
it was like a worked example

510
00:30:24,380 --> 00:30:32,330
in Section 1.1, I could ask the
question when is Cx=b solvable.

511
00:30:32,330 --> 00:30:34,950
Because there are some
right-hand sides that'll work.

512
00:30:34,950 --> 00:30:37,280
Because if I just take
an x and multiply by C,

513
00:30:37,280 --> 00:30:40,280
I'll get a right-hand
side that works.

514
00:30:40,280 --> 00:30:48,770
But for which vectors,
right-hand sides b,

515
00:30:48,770 --> 00:30:52,000
will my method work?

516
00:30:52,000 --> 00:30:54,470
The ones that have...?

517
00:30:54,470 --> 00:30:56,180
Yeah, the ones that have which?

518
00:30:56,180 --> 00:31:01,830
What do I need with these
c's, c_0, c_1, c_2, and c_3,

519
00:31:01,830 --> 00:31:06,630
for my solution to be possible.

520
00:31:06,630 --> 00:31:09,710
I need b_0 hat.

521
00:31:09,710 --> 00:31:11,540
Equals zero.

522
00:31:11,540 --> 00:31:13,380
I need b_0 hat equals zero.

523
00:31:13,380 --> 00:31:15,180
And then what does that say?

524
00:31:15,180 --> 00:31:18,500
That means that the
b, the vector b,

525
00:31:18,500 --> 00:31:22,790
has no constant term
in the Fourier series.

526
00:31:22,790 --> 00:31:27,950
It means that the vector b is
orthogonal to the [1, 1, 1, 1],

527
00:31:27,950 --> 00:31:29,290
eigenvector.

528
00:31:29,290 --> 00:31:34,050
So this is like a subtle point
but just driving home the point

529
00:31:34,050 --> 00:31:39,120
that what Fourier does is
diagonalize everything.

530
00:31:39,120 --> 00:31:42,990
It diagonalizes all the
important problems of,

531
00:31:42,990 --> 00:31:47,330
all the simplest problems,
of differential equations.

532
00:31:47,330 --> 00:31:52,730
You know, I mean this is like
18.03 looked at from Fourier's

533
00:31:52,730 --> 00:31:53,840
point of view.

534
00:31:53,840 --> 00:31:57,060
OK, what more could I
do with that equation?

535
00:31:57,060 --> 00:32:00,680
I think you really are seeing
all the good stuff here.

536
00:32:00,680 --> 00:32:02,570
You're seeing the matrix.

537
00:32:02,570 --> 00:32:05,050
We're recognizing
it as a circulant.

538
00:32:05,050 --> 00:32:08,860
We're realizing that we could
take its Fourier transform.

539
00:32:08,860 --> 00:32:11,700
We get the eigenvalues.

540
00:32:11,700 --> 00:32:15,270
We're diagonalizing the
matrix, the convolution

541
00:32:15,270 --> 00:32:18,530
becomes a multiplication,
and the solution becomes,

542
00:32:18,530 --> 00:32:22,040
inversion becomes division.

543
00:32:22,040 --> 00:32:26,990
I hope you see that.

544
00:32:26,990 --> 00:32:30,820
That's really a model
problem for this course.

545
00:32:30,820 --> 00:32:33,160
OK. yeah.

546
00:32:33,160 --> 00:32:33,710
Questions.

547
00:32:33,710 --> 00:32:41,417
Good.

548
00:32:41,417 --> 00:32:44,000
AUDIENCE: [INAUDIBLE] PROFESSOR
STRANG: Would I give you a six

549
00:32:44,000 --> 00:32:45,930
by six Fourier matrix on a test?

550
00:32:45,930 --> 00:32:46,730
Probably not.

551
00:32:46,730 --> 00:32:48,080
No.

552
00:32:48,080 --> 00:32:49,630
I just about could.

553
00:32:49,630 --> 00:32:55,220
I mean, it's, six by six, those
are pretty decent numbers.

554
00:32:55,220 --> 00:32:56,730
Right.

555
00:32:56,730 --> 00:32:59,850
Those six roots of
unity, but not quite.

556
00:32:59,850 --> 00:33:00,580
Right, yeah.

557
00:33:00,580 --> 00:33:01,080
Yeah.

558
00:33:01,080 --> 00:33:01,590
Yeah.

559
00:33:01,590 --> 00:33:05,720
So four by four is, five by five
would not be nice, certainly.

560
00:33:05,720 --> 00:33:09,570
Who knows the cosine
of 72 degrees?

561
00:33:09,570 --> 00:33:10,300
Crazy.

562
00:33:10,300 --> 00:33:13,620
But, 60 degrees we could do.

563
00:33:13,620 --> 00:33:17,260
So the Fourier matrix would
be full of square roots

564
00:33:17,260 --> 00:33:20,720
of three over two, and one
over two, and i's, and so on.

565
00:33:20,720 --> 00:33:24,960
But it wouldn't be as nice,
so really four by four

566
00:33:24,960 --> 00:33:26,910
is sort of the model.

567
00:33:26,910 --> 00:33:28,310
Yeah, yeah.

568
00:33:28,310 --> 00:33:31,890
So four by four is that model.

569
00:33:31,890 --> 00:33:33,230
Other questions?

570
00:33:33,230 --> 00:33:35,180
Because this is
really a key example.

571
00:33:35,180 --> 00:33:37,240
Yeah.

572
00:33:37,240 --> 00:33:41,860
When I calculated the
eigenvalues, yeah.

573
00:33:41,860 --> 00:33:42,420
Ah.

574
00:33:42,420 --> 00:33:46,330
Because this matrix, I know
everything about that matrix

575
00:33:46,330 --> 00:33:47,800
when I know its first vector.

576
00:33:47,800 --> 00:33:49,232
AUDIENCE: [INAUDIBLE]

577
00:33:49,232 --> 00:33:50,940
PROFESSOR STRANG:
Yeah, it's because it's

578
00:33:50,940 --> 00:33:52,120
a circulant matrix.

579
00:33:52,120 --> 00:33:57,010
It's because that matrix
is expressing convolution

580
00:33:57,010 --> 00:34:03,170
with this vector.
[2, -1, 0,  -1].

581
00:34:03,170 --> 00:34:06,400
That circulant
matrix essentially

582
00:34:06,400 --> 00:34:08,820
is built from four
numbers, right.

583
00:34:08,820 --> 00:34:09,320
Yeah.

584
00:34:09,320 --> 00:34:11,640
Yeah, and they go in
the zeroth column.

585
00:34:11,640 --> 00:34:12,700
Right, yeah.

586
00:34:12,700 --> 00:34:14,580
Yeah.

587
00:34:14,580 --> 00:34:20,580
Right, so there is an
example where we could like

588
00:34:20,580 --> 00:34:21,780
do everything.

589
00:34:21,780 --> 00:34:27,660
Now, and let me just remember
that with this example,

590
00:34:27,660 --> 00:34:29,440
we could do everything.

591
00:34:29,440 --> 00:34:37,410
So this is an example of, you
could say this type of problem.

592
00:34:37,410 --> 00:34:42,310
But with a very special kernel
there, so it turned out to be,

593
00:34:42,310 --> 00:34:44,470
it looks like an
integral equation here

594
00:34:44,470 --> 00:34:49,010
but if that kernel involves
delta functions and so on then

595
00:34:49,010 --> 00:34:51,530
it can be just a
differential equation.

596
00:34:51,530 --> 00:34:53,420
And then that's
what we got there.

597
00:34:53,420 --> 00:34:59,590
So we took all-- The same
steps we did here, we did here.

598
00:34:59,590 --> 00:35:06,110
We took the Fourier transform,
and I emphasize there,

599
00:35:06,110 --> 00:35:09,850
just to remember Wednesday,
this was a delta function.

600
00:35:09,850 --> 00:35:13,660
When I took the Fourier
transform I got a one,

601
00:35:13,660 --> 00:35:16,020
so this was a one over this.

602
00:35:16,020 --> 00:35:22,190
And I did the inverse transform
and I got back to the function

603
00:35:22,190 --> 00:35:24,260
that I drew.

604
00:35:24,260 --> 00:35:29,170
Which was e^(-ax) over 2a.

605
00:35:29,170 --> 00:35:31,240
And even.

606
00:35:31,240 --> 00:35:38,650
So, yeah. this was
the answer u(x).

607
00:35:38,650 --> 00:35:41,950
So I was able to do that,
I mean this step was easy,

608
00:35:41,950 --> 00:35:43,110
that step is easy.

609
00:35:43,110 --> 00:35:45,960
That step is easy,
the division is easy.

610
00:35:45,960 --> 00:35:55,270
And then I just recognize this
as the transform of this one,

611
00:35:55,270 --> 00:35:57,680
this example that we had done.

612
00:35:57,680 --> 00:35:59,570
Once I divided by 2a.

613
00:35:59,570 --> 00:36:01,420
So you should be
able to do this.

614
00:36:01,420 --> 00:36:05,230
So those are two that you
should really be able to do.

615
00:36:05,230 --> 00:36:07,510
I'm not going to,
obviously I'm not going to,

616
00:36:07,510 --> 00:36:13,600
ask you a 2-D problem on the
exam or even on a homework.

617
00:36:13,600 --> 00:36:16,970
But now if you'll
allow me, I'd like

618
00:36:16,970 --> 00:36:22,360
to spend a few minutes to
get into 2-D. Because really,

619
00:36:22,360 --> 00:36:25,540
you've got the
main thoughts here.

620
00:36:25,540 --> 00:36:28,340
That Fourier is
the same as finding

621
00:36:28,340 --> 00:36:30,910
eigenvectors and eigenvalues.

622
00:36:30,910 --> 00:36:35,520
That's the main thought
for these LTI problems.

623
00:36:35,520 --> 00:36:40,420
OK, now suppose I have, let's
just get the formalities

624
00:36:40,420 --> 00:36:41,560
straight here.

625
00:36:41,560 --> 00:36:45,170
Suppose I have a
function of x and y.

626
00:36:45,170 --> 00:36:48,440
2pi periodic in x, and in y.

627
00:36:48,440 --> 00:36:54,530
So if I bump x by 2pi, or
if I bump y by 2pi -- oh,

628
00:36:54,530 --> 00:36:59,190
I'm using capital F
for the periodic guys.

629
00:36:59,190 --> 00:37:04,410
So let me stay with
capital F -- x, y+2pi.

630
00:37:04,410 --> 00:37:08,470
OK, so I have a function.

631
00:37:08,470 --> 00:37:10,440
This is given.

632
00:37:10,440 --> 00:37:13,610
This is, it's in 2-D now.

633
00:37:13,610 --> 00:37:16,300
And I want to write
its Fourier series.

634
00:37:16,300 --> 00:37:18,850
So I'm just asking
the question what

635
00:37:18,850 --> 00:37:22,770
does the Fourier series look
like for a function of two

636
00:37:22,770 --> 00:37:25,210
variables.

637
00:37:25,210 --> 00:37:28,930
The point is, it's going
to be a nice answer.

638
00:37:28,930 --> 00:37:34,100
And so everything, what you
know how to do in 1-D you can do

639
00:37:34,100 --> 00:37:41,090
in 2-D. So let me write the
complex form, the e^(ik) stuff.

640
00:37:41,090 --> 00:37:44,410
So what would I write,
how would I write this?

641
00:37:44,410 --> 00:37:48,390
I would write that as
a sum, but it'll have,

642
00:37:48,390 --> 00:37:51,630
I'll make it a double sum,
I'll write two sigmas just

643
00:37:51,630 --> 00:37:57,100
to emphasize that we're summing
from k equal minus infinity

644
00:37:57,100 --> 00:38:02,285
to infinity, and from l equal
minus infinity to infinity.

645
00:38:02,285 --> 00:38:07,390
We have coefficients c_kl.

646
00:38:07,390 --> 00:38:11,270
They depend on two indices,
this is the pattern to know.

647
00:38:11,270 --> 00:38:19,130
Multiplying our e^(ikx),
and our e^(ily).

648
00:38:19,130 --> 00:38:23,760
Right, good.

649
00:38:23,760 --> 00:38:25,990
So, alright.

650
00:38:25,990 --> 00:38:27,070
Let me ask you.

651
00:38:27,070 --> 00:38:30,540
How would I find c_23?

652
00:38:30,540 --> 00:38:35,040
Just to know that-- We could
find all these coefficients,

653
00:38:35,040 --> 00:38:36,720
find formulas for them.

654
00:38:36,720 --> 00:38:39,210
We could do examples.

655
00:38:39,210 --> 00:38:41,380
How would I find c_23?

656
00:38:41,380 --> 00:38:49,240
So this is my F. I know
F. I want to find c_23.

657
00:38:49,240 --> 00:38:52,000
What's the magic trick?

658
00:38:52,000 --> 00:38:56,800
And I'm 2pi periodic, so all
integrals, all the integrals--

659
00:38:56,800 --> 00:38:58,480
And I'm giving you
a hint, of course.

660
00:38:58,480 --> 00:39:01,520
I'm going to integrate.

661
00:39:01,520 --> 00:39:04,840
And the integrals will
all go from minus pi to pi

662
00:39:04,840 --> 00:39:06,290
in x and in y.

663
00:39:06,290 --> 00:39:09,650
They'll integrate over
the period square.

664
00:39:09,650 --> 00:39:12,590
Here's the period
square from, there's

665
00:39:12,590 --> 00:39:18,615
the center. x direction, y
direction, goes out to pi

666
00:39:18,615 --> 00:39:21,270
and goes up to pi.

667
00:39:21,270 --> 00:39:26,360
So all integrals
will be over dxdy.

668
00:39:26,360 --> 00:39:28,350
But what do I integrate?

669
00:39:28,350 --> 00:39:33,070
To find c_23.

670
00:39:33,070 --> 00:39:36,770
Well, these guys are orthogonal.

671
00:39:36,770 --> 00:39:38,460
That's what's making
everything work,

672
00:39:38,460 --> 00:39:40,830
they're orthogonal
and very special.

673
00:39:40,830 --> 00:39:43,940
So that by use
orthogonality, what do I do?

674
00:39:43,940 --> 00:39:46,320
I multiply by?

675
00:39:46,320 --> 00:39:50,140
Just tell me what
to multiply by.

676
00:39:50,140 --> 00:39:57,890
By this and integrate.

677
00:39:57,890 --> 00:40:04,260
OK, what is it that I multiply
by if I'm shooting for c_23,

678
00:40:04,260 --> 00:40:15,580
for example? e^(i2x),
is it e^(i2x)?

679
00:40:15,580 --> 00:40:17,150
Minus, right.

680
00:40:17,150 --> 00:40:27,800
I multiply by e^(-i2x),
e^(-i3y), and integrate.

681
00:40:27,800 --> 00:40:28,420
Yeah.

682
00:40:28,420 --> 00:40:31,980
So when I multiply by
that and integrate,

683
00:40:31,980 --> 00:40:35,830
everything will go
except the c_23 term.

684
00:40:35,830 --> 00:40:38,430
Which will be
multiplied by what?

685
00:40:38,430 --> 00:40:43,290
So I'll just have c_23
times probably 2pi squared.

686
00:40:43,290 --> 00:40:46,130
I guess 2pi will come
in from both integrals,

687
00:40:46,130 --> 00:40:51,230
so the formula will be
c_kl-- c_kl will be,

688
00:40:51,230 --> 00:40:52,790
do you want me to
write this formula?

689
00:40:52,790 --> 00:40:55,740
I'll write it here and
then forget it right away.

690
00:40:55,740 --> 00:41:00,570
c_kl will be one
over 2pi squared,

691
00:41:00,570 --> 00:41:07,020
the integral of my
function times my e^(-ikx),

692
00:41:07,020 --> 00:41:10,350
times my e^(-ily) dxdy.

693
00:41:10,350 --> 00:41:15,020


694
00:41:15,020 --> 00:41:17,510
So that just makes the point.

695
00:41:17,510 --> 00:41:22,750
That there's nothing new here,
it's just up a dimension.

696
00:41:22,750 --> 00:41:24,700
But the formulas
all look the same,

697
00:41:24,700 --> 00:41:31,620
and if F was a-- Well, if F is
a delta function, if F is now

698
00:41:31,620 --> 00:41:34,690
a 2-D delta function.

699
00:41:34,690 --> 00:41:38,500
We haven't done delta
functions in 2-D, why don't we?

700
00:41:38,500 --> 00:41:45,340
Suppose F is the delta
function in 2-D. Then

701
00:41:45,340 --> 00:41:49,110
what are the coefficients?

702
00:41:49,110 --> 00:41:50,860
What do you think
you this means,

703
00:41:50,860 --> 00:41:53,230
this delta function in 2-D?

704
00:41:53,230 --> 00:41:57,250
So if I put in the delta
here, and I integrate.

705
00:41:57,250 --> 00:42:01,350
And what do I get then?

706
00:42:01,350 --> 00:42:06,280
So if this guy is a delta, a
two-dimensional delta function,

707
00:42:06,280 --> 00:42:10,990
the rule is that
when I integrate over

708
00:42:10,990 --> 00:42:15,230
a region that includes
the spike, so it's a spike

709
00:42:15,230 --> 00:42:19,610
sitting up above a plane now,
instead of sitting above a line

710
00:42:19,610 --> 00:42:21,250
it's sitting above a plane.

711
00:42:21,250 --> 00:42:25,700
Then I get the value, so
this is the delta function

712
00:42:25,700 --> 00:42:27,030
at the origin.

713
00:42:27,030 --> 00:42:29,690
So I get the value of
this at the origin.

714
00:42:29,690 --> 00:42:32,100
So what answer do I get?

715
00:42:32,100 --> 00:42:34,110
I get one out of the
integral and then

716
00:42:34,110 --> 00:42:35,360
I just have this constant.

717
00:42:35,360 --> 00:42:38,970
So it's constant again.

718
00:42:38,970 --> 00:42:41,460
And it's just one.

719
00:42:41,460 --> 00:42:45,950
So the Fourier coefficients
of the delta function

720
00:42:45,950 --> 00:42:46,600
are constant.

721
00:42:46,600 --> 00:42:49,700
All frequencies
there are the same.

722
00:42:49,700 --> 00:42:53,020
What about a line
of delta functions?

723
00:42:53,020 --> 00:42:54,460
And what does that mean?

724
00:42:54,460 --> 00:43:01,730
What about, yeah let me
try to draw delta(x).

725
00:43:01,730 --> 00:43:06,770
Suppose I have a
function of x and y --

726
00:43:06,770 --> 00:43:12,290
it's just worth imagining
a line of delta functions.

727
00:43:12,290 --> 00:43:21,490
So I'm in the xy-- Let me
look again at this thing.

728
00:43:21,490 --> 00:43:25,480
I have delta functions
all along this line.

729
00:43:25,480 --> 00:43:26,530
Now.

730
00:43:26,530 --> 00:43:29,520
Here is a crazy example,
just to say well there

731
00:43:29,520 --> 00:43:34,140
is something new in 2-D. So
previously my delta function

732
00:43:34,140 --> 00:43:36,210
was just at that point.

733
00:43:36,210 --> 00:43:38,130
And the old integrals
just picked out

734
00:43:38,130 --> 00:43:40,210
the value at that point.

735
00:43:40,210 --> 00:43:43,490
But now think of
a delta function's

736
00:43:43,490 --> 00:43:46,400
a sort of line of spikes.

737
00:43:46,400 --> 00:43:48,820
Going up here, and then
of course it's periodic.

738
00:43:48,820 --> 00:43:51,970
Everything's periodic
so that line continues

739
00:43:51,970 --> 00:43:55,280
and this line appears here,
and this line appears here.

740
00:43:55,280 --> 00:44:02,010
But I only have to focus
on one period square.

741
00:44:02,010 --> 00:44:04,750
What's my answer now?

742
00:44:04,750 --> 00:44:10,930
If this function suddenly
changes from a one-point delta

743
00:44:10,930 --> 00:44:13,210
function to a line
of delta functions?

744
00:44:13,210 --> 00:44:17,390
Now tell me what the
coefficients are.

745
00:44:17,390 --> 00:44:19,240
What are the
Fourier coefficients

746
00:44:19,240 --> 00:44:22,920
in 2-D for a line
of delta functions?

747
00:44:22,920 --> 00:44:28,760
A straight line of delta
functions going up the y axis?

748
00:44:28,760 --> 00:44:32,260
It'll be-- Let's see.

749
00:44:32,260 --> 00:44:34,850
What do I do?

750
00:44:34,850 --> 00:44:39,060
I'm go to integrate--
Oh yeah, what is it?

751
00:44:39,060 --> 00:44:40,600
Good question.

752
00:44:40,600 --> 00:44:42,450
OK.

753
00:44:42,450 --> 00:44:47,260
So what is the--
When do I get zero

754
00:44:47,260 --> 00:44:49,570
and when do I not
get zero out of this?

755
00:44:49,570 --> 00:44:54,600
Yeah, tell me first when do I
get zero out of this integral?

756
00:44:54,600 --> 00:44:56,250
And when do I not?

757
00:44:56,250 --> 00:45:00,170
What am I doing here?

758
00:45:00,170 --> 00:45:01,690
Help me.

759
00:45:01,690 --> 00:45:06,350
I said 2-D was easy and I've
got in over my head here.

760
00:45:06,350 --> 00:45:10,070
So look.

761
00:45:10,070 --> 00:45:12,700
I can do the x integral, right?

762
00:45:12,700 --> 00:45:15,950
We all know how to
do the x integral.

763
00:45:15,950 --> 00:45:18,440
Yes, is that right?

764
00:45:18,440 --> 00:45:22,142
If I integrate with respect
to x, what do I get?

765
00:45:22,142 --> 00:45:24,100
Let's see, I'll keep that
one over 2pi squared.

766
00:45:24,100 --> 00:45:28,680
Now I'm trying to
do this integral.

767
00:45:28,680 --> 00:45:34,710
Do I get a one?

768
00:45:34,710 --> 00:45:37,840
If I get a one from
the x integral.

769
00:45:37,840 --> 00:45:40,410
So then I'm down
to one integral,

770
00:45:40,410 --> 00:45:47,160
just the y integral is
left. e^(-ily) dy, right?

771
00:45:47,160 --> 00:45:52,720
I did the x part, which said,
OK, take the value at x=0,

772
00:45:52,720 --> 00:45:53,990
which was one.

773
00:45:53,990 --> 00:45:57,850
So the x integral was one, good.

774
00:45:57,850 --> 00:45:59,900
And now I've got
down to this part.

775
00:45:59,900 --> 00:46:03,950
Now what is that integral?

776
00:46:03,950 --> 00:46:10,230
It's two-- wait a minute.

777
00:46:10,230 --> 00:46:16,570
Depends on l, doesn't it?

778
00:46:16,570 --> 00:46:21,940
When l-- Yeah, so it's going
to depend on whether l is zero

779
00:46:21,940 --> 00:46:23,080
or not.

780
00:46:23,080 --> 00:46:26,080
Is that right?

781
00:46:26,080 --> 00:46:28,480
Yeah, that's sort
of interesting.

782
00:46:28,480 --> 00:46:33,670
If l is zero, then I'm
getting -- then this is a 2pi.

783
00:46:33,670 --> 00:46:38,820
So the answer, so
I'm getting c_k0,

784
00:46:38,820 --> 00:46:42,530
when l is zero I'm
getting a 2pi out of that.

785
00:46:42,530 --> 00:46:44,820
If l is zero, I'm
integrating one,

786
00:46:44,820 --> 00:46:46,940
I get a 2pi, cancels
one of those.

787
00:46:46,940 --> 00:46:48,020
I get a one over 2pi.

788
00:46:48,020 --> 00:46:51,760


789
00:46:51,760 --> 00:46:59,280
And otherwise the other c_kl's,
when l is not zero, are what?

790
00:46:59,280 --> 00:47:00,270
Just zero, I think.

791
00:47:00,270 --> 00:47:03,120
The integral of this thing,
this is a periodic guy,

792
00:47:03,120 --> 00:47:07,290
if I integrate it from
minus pi to pi it's zero.

793
00:47:07,290 --> 00:47:11,670
What am I-- I'm making a
big deal out of something

794
00:47:11,670 --> 00:47:14,070
that shouldn't be a big deal.

795
00:47:14,070 --> 00:47:20,020
The delta(x) function,
this is just,

796
00:47:20,020 --> 00:47:23,470
its Fourier series is
just the one we know.

797
00:47:23,470 --> 00:47:25,830
Sum of e^(ikx)'s.

798
00:47:25,830 --> 00:47:31,300


799
00:47:31,300 --> 00:47:33,210
Do you see what's happened here?

800
00:47:33,210 --> 00:47:36,850
It was supposed to
be a double sum.

801
00:47:36,850 --> 00:47:41,060
But the ones, when l
wasn't zero, aren't there.

802
00:47:41,060 --> 00:47:42,270
The only ones-- Yeah.

803
00:47:42,270 --> 00:47:48,020
So I'm back to the, for a line
of spikes, a line of deltas,

804
00:47:48,020 --> 00:47:50,410
I'm back to-- So
it only depended

805
00:47:50,410 --> 00:47:54,020
on x, so the Fourier series is
just the one I already know.

806
00:47:54,020 --> 00:47:56,660
All ones.

807
00:47:56,660 --> 00:47:59,470
When there's no--
When l is zero,

808
00:47:59,470 --> 00:48:05,060
all ones or 1/(2pi)'s, all
constants when l is zero,

809
00:48:05,060 --> 00:48:11,340
but there's no y-- There's no
oscillation in the y direction.

810
00:48:11,340 --> 00:48:17,810
OK, I don't know why I
got into that example,

811
00:48:17,810 --> 00:48:21,560
because the conclusion was
just it's the Fourier series

812
00:48:21,560 --> 00:48:25,330
that we already know and
it doesn't depend on l,

813
00:48:25,330 --> 00:48:28,940
because the function
didn't depend on y.

814
00:48:28,940 --> 00:48:32,840
OK, then we could
imagine delta functions

815
00:48:32,840 --> 00:48:36,630
in other positions,
or a general function.

816
00:48:36,630 --> 00:48:44,220
OK, so that's 2-D. Would I
want to tackle a 2-D-- Ha,

817
00:48:44,220 --> 00:48:48,370
we've got two minutes. that's
one dimension a minute.

818
00:48:48,370 --> 00:48:50,500
Right, OK.

819
00:48:50,500 --> 00:48:57,400
What happens, what's a
2-D discrete convolution?

820
00:48:57,400 --> 00:49:00,480
What's a 2-D
discrete convolution?

821
00:49:00,480 --> 00:49:03,180
Now, you might say OK, why
is Professor Strang inventing

822
00:49:03,180 --> 00:49:04,380
these problems?

823
00:49:04,380 --> 00:49:07,240
Because a 2-D
discrete convolution

824
00:49:07,240 --> 00:49:10,170
is the core idea of
image processing.

825
00:49:10,170 --> 00:49:14,440
If I have an image, what
does image processing do?

826
00:49:14,440 --> 00:49:18,360
Image processing
takes my image, it

827
00:49:18,360 --> 00:49:20,860
separates it into
pixels, right, that's

828
00:49:20,860 --> 00:49:23,980
all the image is,
bunch of pixels.

829
00:49:23,980 --> 00:49:28,720
Then many 2-D image
processing algorithms,

830
00:49:28,720 --> 00:49:32,100
will take-- JPEG,
old JPEG for example,

831
00:49:32,100 --> 00:49:36,720
would take an eight by
eight, eight by eight 2-D,

832
00:49:36,720 --> 00:49:39,130
in other words--
Eight by eight, 2-D

833
00:49:39,130 --> 00:49:42,600
is the main point,
set of pixels.

834
00:49:42,600 --> 00:49:43,740
And transform it.

835
00:49:43,740 --> 00:49:46,020
Do a 2-D transform.

836
00:49:46,020 --> 00:49:48,610
So what is a 2-D transform?

837
00:49:48,610 --> 00:49:56,730
What would be the 2-D transform
that would correspond to this?

838
00:49:56,730 --> 00:49:59,420
First of all how
big's the matrix?

839
00:49:59,420 --> 00:50:01,470
Just so we get an idea.

840
00:50:01,470 --> 00:50:04,250
I probably won't get to
the end of this example.

841
00:50:04,250 --> 00:50:10,290
But just, so in 1-D, my
matrix was four by four.

842
00:50:10,290 --> 00:50:15,740
Now I've got, so that was
for four points on a line.

843
00:50:15,740 --> 00:50:22,085
Now I've got a square of points.

844
00:50:22,085 --> 00:50:25,560
So how big is my matrix?

845
00:50:25,560 --> 00:50:26,830
16, right?

846
00:50:26,830 --> 00:50:32,590
16 by 16, because it's
operating on 16 pixels.

847
00:50:32,590 --> 00:50:37,370
It's operating on 16
pixels, where in 1-D it only

848
00:50:37,370 --> 00:50:38,790
had four to act on.

849
00:50:38,790 --> 00:50:42,960
So I'm going to end up with
a 16 by 16 matrix here.

850
00:50:42,960 --> 00:50:48,150
And I think-- Let me
see, what do I need?

851
00:50:48,150 --> 00:50:49,630
Oh, wait a minute.

852
00:50:49,630 --> 00:50:51,330
Uh-oh.

853
00:50:51,330 --> 00:50:54,620
Yeah, I think the
time's up here.

854
00:50:54,620 --> 00:50:59,080
Yeah, because my C, has my
C got to have 16 components?

855
00:50:59,080 --> 00:51:00,140
Yes.

856
00:51:00,140 --> 00:51:03,010
My u has to have 16,
my right-hand side

857
00:51:03,010 --> 00:51:05,060
has got these 16 components.

858
00:51:05,060 --> 00:51:05,600
Yeah.

859
00:51:05,600 --> 00:51:09,900
So I'm up to 16 but a
very special circulant

860
00:51:09,900 --> 00:51:11,610
of a circulant.

861
00:51:11,610 --> 00:51:13,750
It'll be a circulant
of a circulant somehow.

862
00:51:13,750 --> 00:51:17,150
OK, enough for 2-D.
I'll see you Wednesday

863
00:51:17,150 --> 00:51:18,430
and we're back to reality.

864
00:51:18,430 --> 00:51:19,581
OK.

865
00:51:19,581 --> 00:51:20,080