1
00:00:00,000 --> 00:00:00,247


2
00:00:00,247 --> 00:00:02,330
The following content is
provided under a Creative

3
00:00:02,330 --> 00:00:03,610
Commons license.

4
00:00:03,610 --> 00:00:05,450
Your support will help
MIT OpenCourseWare

5
00:00:05,450 --> 00:00:09,940
continue to offer high-quality
educational resources for free.

6
00:00:09,940 --> 00:00:12,530
To make a donation, or to
view additional materials

7
00:00:12,530 --> 00:00:15,840
from hundreds of MIT courses,
visit MIT OpenCourseWare

8
00:00:15,840 --> 00:00:20,290
at ocw.mit.edu.

9
00:00:20,290 --> 00:00:24,140
PROFESSOR STRANG:
OK, so I thought

10
00:00:24,140 --> 00:00:28,770
I'd quickly list the four
central topics for the Fourier

11
00:00:28,770 --> 00:00:30,510
part of the course,
and I would guess

12
00:00:30,510 --> 00:00:33,920
that there'll be a question
on each of those four

13
00:00:33,920 --> 00:00:36,390
in the final quiz.

14
00:00:36,390 --> 00:00:39,340
Final exam.

15
00:00:39,340 --> 00:00:43,520
And we covered 4.1,
Fourier series.

16
00:00:43,520 --> 00:00:50,590
I don't plan to do discussion
in 4.2 of other series

17
00:00:50,590 --> 00:00:53,980
like Bessel and
Legendre and so on.

18
00:00:53,980 --> 00:00:56,170
If I can, I'll
come back to those

19
00:00:56,170 --> 00:00:58,340
in the very last lectures.

20
00:00:58,340 --> 00:01:06,190
But I want to pick up now on
the second key example, which

21
00:01:06,190 --> 00:01:11,355
is the discrete Fourier series
that has only N terms, instead

22
00:01:11,355 --> 00:01:13,615
of this series, as
you're looking at it,

23
00:01:13,615 --> 00:01:15,080
has infinitely many terms.

24
00:01:15,080 --> 00:01:20,340
But I'm going to
cut back to N terms.

25
00:01:20,340 --> 00:01:24,240
So this k, well I
can go one to N,

26
00:01:24,240 --> 00:01:29,590
but the best way is zero to N-1.

27
00:01:29,590 --> 00:01:34,630
So that's the discrete series.

28
00:01:34,630 --> 00:01:40,240
We're going to be dealing with
a vector, c_0 up to c_(N-1).

29
00:01:40,240 --> 00:01:44,110
And a vector of function values.

30
00:01:44,110 --> 00:01:46,350
Vector to vector.

31
00:01:46,350 --> 00:01:49,990
So a matrix is going to be
the key to everything here.

32
00:01:49,990 --> 00:01:51,540
An N by N matrix.

33
00:01:51,540 --> 00:01:52,950
So that's what's coming.

34
00:01:52,950 --> 00:01:57,370
Then, after that
probably by Friday

35
00:01:57,370 --> 00:01:59,770
will be the integral transform.

36
00:01:59,770 --> 00:02:04,470
And early next week, and a
little after Thanksgiving,

37
00:02:04,470 --> 00:02:07,130
would be this key
idea of convolution,

38
00:02:07,130 --> 00:02:11,640
which is really important
in so many applications

39
00:02:11,640 --> 00:02:14,720
and needs to be separated out.

40
00:02:14,720 --> 00:02:17,460
Convolution will
apply to all of these.

41
00:02:17,460 --> 00:02:20,470
And you'll see the point there.

42
00:02:20,470 --> 00:02:24,370
OK, ready for the idea
of the discrete one.

43
00:02:24,370 --> 00:02:28,990
OK, so the key idea is we only
have N terms, so I can't expect

44
00:02:28,990 --> 00:02:30,970
to reproduce a whole function.

45
00:02:30,970 --> 00:02:33,580
I can get N function
values, so I'll

46
00:02:33,580 --> 00:02:37,060
get this at N different points.

47
00:02:37,060 --> 00:02:40,790
And if I'm thinking
of my F(x), on,

48
00:02:40,790 --> 00:02:45,270
let me take zero to 2pi
now, and I'm thinking again

49
00:02:45,270 --> 00:02:47,710
of an F(x) that's periodic.

50
00:02:47,710 --> 00:02:51,800
So it goes, does whatever.

51
00:02:51,800 --> 00:02:56,830
If it was like that,
if that was my F(x),

52
00:02:56,830 --> 00:02:59,530
just remember about
Fourier series,

53
00:02:59,530 --> 00:03:03,240
what could you tell me about the
Fourier series for that F(x)?

54
00:03:03,240 --> 00:03:07,410
Well, when I continue
periodically,

55
00:03:07,410 --> 00:03:09,590
am I going to see a jump?

56
00:03:09,590 --> 00:03:10,600
Yes.

57
00:03:10,600 --> 00:03:18,730
There'll be a jump at every
2pi, so the underlying function

58
00:03:18,730 --> 00:03:28,690
is not continuous, has a jump,
so that our special examples

59
00:03:28,690 --> 00:03:34,360
with jumps and slow decay,
1/k decay rate for the Fourier

60
00:03:34,360 --> 00:03:36,450
coefficients would apply here.

61
00:03:36,450 --> 00:03:39,060
OK, that's remembering
Fourier series.

62
00:03:39,060 --> 00:03:42,070
Now, Fourier integrals,
I'm just going to take,

63
00:03:42,070 --> 00:03:43,070
let me just take N=4.

64
00:03:43,070 --> 00:03:46,040


65
00:03:46,040 --> 00:03:49,607
So I'll take this point, you
know, what four points am I

66
00:03:49,607 --> 00:03:50,190
going to take?

67
00:03:50,190 --> 00:03:54,900
I'm not going to repeat-- Ah.

68
00:03:54,900 --> 00:04:00,130
Just for the heck of it,
let's make it look periodic.

69
00:04:00,130 --> 00:04:02,740
Yeah, let's make
it look periodic.

70
00:04:02,740 --> 00:04:04,860
I never should have
done it otherwise.

71
00:04:04,860 --> 00:04:08,280
OK, so there's one point.

72
00:04:08,280 --> 00:04:12,890
I'm just going to take four
equally spaced, equally spaced

73
00:04:12,890 --> 00:04:17,370
being key, zero,
one, two, three.

74
00:04:17,370 --> 00:04:21,550
So N-1 is three.

75
00:04:21,550 --> 00:04:28,340
And these four values
I can get right.

76
00:04:28,340 --> 00:04:30,490
So those are my four x's.

77
00:04:30,490 --> 00:04:33,970
The four x's that
I'm going to take.

78
00:04:33,970 --> 00:04:36,620
So this is x=0, of course.

79
00:04:36,620 --> 00:04:40,070
And this one is,
what's the x there?

80
00:04:40,070 --> 00:04:45,050
x is 2pi, the whole deal,
the whole interval, but it's

81
00:04:45,050 --> 00:04:51,080
divided into N pieces,
so 2pi/N. So we

82
00:04:51,080 --> 00:04:55,940
are going to see a lot of
fractions involving pi/N.

83
00:04:55,940 --> 00:04:59,710
Or 2pi/N. They're just
going to come up everywhere.

84
00:04:59,710 --> 00:05:03,690
Because that's the
delta x, the h,

85
00:05:03,690 --> 00:05:08,050
the step in the discrete form.

86
00:05:08,050 --> 00:05:08,550
OK.

87
00:05:08,550 --> 00:05:11,550
So those are the x's.

88
00:05:11,550 --> 00:05:22,000
So I'm only going to get
this at x=0, 2pi/N, 4pi/N,

89
00:05:22,000 --> 00:05:28,430
up to whatever it comes
out to, (N-1)*2pi/N.

90
00:05:28,430 --> 00:05:31,420
That doesn't look elegant so
we probably will try to avoid

91
00:05:31,420 --> 00:05:32,830
writing that again.

92
00:05:32,830 --> 00:05:39,220
But this was the three times
2pi/4 that got us to there.

93
00:05:39,220 --> 00:05:46,230
So this is 2pi/8, pi/4.

94
00:05:46,230 --> 00:05:50,240
Sorry, 2pi/4,
4pi/4, 6pi-- Oh, OK.

95
00:05:50,240 --> 00:05:54,500
So we're only going to
get equality at those x's.

96
00:05:54,500 --> 00:05:58,800
So it's those x's that
I should plug in here.

97
00:05:58,800 --> 00:06:00,640
OK, so let me do that.

98
00:06:00,640 --> 00:06:04,500
Let me plug in, what
does this series look

99
00:06:04,500 --> 00:06:09,250
like at these four points?

100
00:06:09,250 --> 00:06:10,260
Let me see.

101
00:06:10,260 --> 00:06:14,440
So writing out all
four equations,

102
00:06:14,440 --> 00:06:20,190
F(0) is going to match the
right side at zero, because zero

103
00:06:20,190 --> 00:06:21,960
is one of my points.

104
00:06:21,960 --> 00:06:30,890
So I get c_0 + c_1 + c_2 + c_3.

105
00:06:30,890 --> 00:06:35,290
At x=0, right, all the e
to the whatevers were one.

106
00:06:35,290 --> 00:06:42,150
Now at the next
x, 2pi/N, now I'm

107
00:06:42,150 --> 00:06:46,760
matching the series with a
function at that value, there.

108
00:06:46,760 --> 00:06:54,050
2pi/N. So now I have k=0,
so what does k=0 give me?

109
00:06:54,050 --> 00:06:58,600
When k is zero, what's
my term look like?

110
00:06:58,600 --> 00:07:03,710
It's just c_0 times one.

111
00:07:03,710 --> 00:07:07,800
Because if k is zero, e to
the whatever is again one.

112
00:07:07,800 --> 00:07:09,510
So this is just c_0.

113
00:07:09,510 --> 00:07:11,900
Now, here's the key.

114
00:07:11,900 --> 00:07:19,470
What is that business
there at x=2pi/N?

115
00:07:19,470 --> 00:07:22,840
Here comes the important
number in this whole business.

116
00:07:22,840 --> 00:07:27,320
The important number this whole
business is, I'll call it w,

117
00:07:27,320 --> 00:07:28,700
is e^(2pi*i/N).

118
00:07:28,700 --> 00:07:33,020


119
00:07:33,020 --> 00:07:38,590
That's a complex
number, and we'll

120
00:07:38,590 --> 00:07:42,270
place it in the complex plane.

121
00:07:42,270 --> 00:07:46,050
So you must get to
know that number, w.

122
00:07:46,050 --> 00:07:49,220
Because when I
plug in x equal-- k

123
00:07:49,220 --> 00:07:53,740
is now one here, so I have c_1.

124
00:07:53,740 --> 00:08:01,820
Times this e to the i one
2pi/N, what is that? w.

125
00:08:01,820 --> 00:08:04,880
So it's w*c_1.

126
00:08:04,880 --> 00:08:08,280
And what do I get from the
next term in this series?

127
00:08:08,280 --> 00:08:11,060
I have c_2*e^(i2x).

128
00:08:11,060 --> 00:08:14,440


129
00:08:14,440 --> 00:08:19,790
x is 2pi/N, do you
see what's happening?

130
00:08:19,790 --> 00:08:23,630
w squared is e^(4pi*i/N).

131
00:08:23,630 --> 00:08:26,470


132
00:08:26,470 --> 00:08:28,590
We're using the beauty
of exponentials,

133
00:08:28,590 --> 00:08:30,590
that's making everything go.

134
00:08:30,590 --> 00:08:32,550
It's w squared.

135
00:08:32,550 --> 00:08:35,250
And this guy would
be w cubed, c_3.

136
00:08:35,250 --> 00:08:38,810


137
00:08:38,810 --> 00:08:41,110
That's the series.

138
00:08:41,110 --> 00:08:45,590
At our points.

139
00:08:45,590 --> 00:08:49,640
So this, like, we'll just see
this once, what are we doing?

140
00:08:49,640 --> 00:08:51,900
But this is what we're
going to live with here.

141
00:08:51,900 --> 00:08:55,520
So I'll rewrite it
all in terms of w,

142
00:08:55,520 --> 00:08:58,710
so you just, you've got the
idea of what are we doing?

143
00:08:58,710 --> 00:09:04,610
We're matching at N points and
now we get the good notation w.

144
00:09:04,610 --> 00:09:12,040
OK, and at the next point,
4pi/N. Well, again, k=0 term,

145
00:09:12,040 --> 00:09:13,000
it's c_0.

146
00:09:13,000 --> 00:09:18,910
The k=1 term, can you tell
me what I get when k is one,

147
00:09:18,910 --> 00:09:30,280
I have something times c_1,
it's e^(i*1*4pi/N), what's that?

148
00:09:30,280 --> 00:09:35,700
w squared. e^(i*4pi/N)
is w squared.

149
00:09:35,700 --> 00:09:39,620
Look, this matrix is
coming out symmetric.

150
00:09:39,620 --> 00:09:44,950
And this will be w^4, and
that'll turn out to be w^6,

151
00:09:44,950 --> 00:09:49,530
and then the last row of the
matrix is going to come from

152
00:09:49,530 --> 00:09:55,610
matching at 6pi/N, and it
will turn out to be c_0,

153
00:09:55,610 --> 00:10:02,130
and it'll be w^3, w^6, and w^9.

154
00:10:02,130 --> 00:10:06,530


155
00:10:06,530 --> 00:10:10,630
So I did the last row
or two a bit fast.

156
00:10:10,630 --> 00:10:15,920
Just because the pattern's
there, and you'll spot it.

157
00:10:15,920 --> 00:10:22,540
Now I want to pull out
of that the matrix.

158
00:10:22,540 --> 00:10:26,580
The Fourier matrix, the four
by four Fourier matrix that

159
00:10:26,580 --> 00:10:29,840
multiplies the c's
and gives me the f's.

160
00:10:29,840 --> 00:10:36,100
Let me call these numbers are
y_0, y_1, y_2, and y_3, just

161
00:10:36,100 --> 00:10:41,240
to have a, so it's a vector
of values of the function.

162
00:10:41,240 --> 00:10:43,430
And this is a vector
of coefficients.

163
00:10:43,430 --> 00:10:47,800
So it's wise to-- The Fourier
transform goes between

164
00:10:47,800 --> 00:10:56,900
y's and c's, and y's.

165
00:10:56,900 --> 00:11:01,680
Connects a vector--
And this is N values,

166
00:11:01,680 --> 00:11:06,160
N function values
in physical space.

167
00:11:06,160 --> 00:11:10,070
These are N coefficients
in frequency space,

168
00:11:10,070 --> 00:11:13,560
and one way is the
discrete Fourier transform

169
00:11:13,560 --> 00:11:17,380
and the other way is the inverse
discrete Fourier transform.

170
00:11:17,380 --> 00:11:21,510
So, and it's a little bit
confused, which is which,

171
00:11:21,510 --> 00:11:22,750
actually.

172
00:11:22,750 --> 00:11:24,960
There was no question
with Fourier series,

173
00:11:24,960 --> 00:11:28,130
it was easy to tell the
function from the coefficients

174
00:11:28,130 --> 00:11:30,600
because the function was
f(x) and the coefficients

175
00:11:30,600 --> 00:11:31,880
were a bunch of numbers.

176
00:11:31,880 --> 00:11:34,020
Here we only have N numbers.

177
00:11:34,020 --> 00:11:40,370
N y's and N c's and we're
transforming back and forth.

178
00:11:40,370 --> 00:11:43,990
So I'm finding right
now the matrix,

179
00:11:43,990 --> 00:11:47,680
this Fourier matrix is going
to take the c's and give me

180
00:11:47,680 --> 00:11:48,270
the y's.

181
00:11:48,270 --> 00:11:52,210
So I'm talking here
about the matrix F.

182
00:11:52,210 --> 00:11:54,080
And let's see what F is.

183
00:11:54,080 --> 00:12:02,890
Can you just see what--
That was y_0 to y_3,

184
00:12:02,890 --> 00:12:06,940
equals this matrix F,
the four by four matrix.

185
00:12:06,940 --> 00:12:10,480
Times c_0 to c_3.

186
00:12:10,480 --> 00:12:14,860
This is really the inverse DFT.

187
00:12:14,860 --> 00:12:20,020


188
00:12:20,020 --> 00:12:26,870
It reconstructs the y's from
the c's, or an easier way

189
00:12:26,870 --> 00:12:32,960
to say that is
add up the series.

190
00:12:32,960 --> 00:12:38,450
So that's the-- The
usual DFT is taking us,

191
00:12:38,450 --> 00:12:41,150
starting with the y's
and giving us the c's.

192
00:12:41,150 --> 00:12:46,930
So this would be the
discrete Fourier transform.

193
00:12:46,930 --> 00:12:50,750
Take a function, produce
its coefficients.

194
00:12:50,750 --> 00:12:54,730
The reverse step is
take the coefficients,

195
00:12:54,730 --> 00:12:59,170
add back, add up the
series, get the function.

196
00:12:59,170 --> 00:13:02,740
And that's what F does.

197
00:13:02,740 --> 00:13:07,030
So I'm going to focus on
F. Which is the ones that

198
00:13:07,030 --> 00:13:09,740
involves this w guy.

199
00:13:09,740 --> 00:13:12,200
OK, can you read off,
peel off from here

200
00:13:12,200 --> 00:13:15,150
what the matrix F is,
the Fourier matrix?

201
00:13:15,150 --> 00:13:20,370
Its first row is all ones.

202
00:13:20,370 --> 00:13:24,910
Its first column is all ones.

203
00:13:24,910 --> 00:13:35,320
And then it's one, w, w^2,
w^3; w^2, w^4, w^6; w^3, w^6,

204
00:13:35,320 --> 00:13:37,370
and w^9.

205
00:13:37,370 --> 00:13:39,640
To the ninth power.

206
00:13:39,640 --> 00:13:43,080
OK, so it's made up
of these powers of w.

207
00:13:43,080 --> 00:13:49,480
Zero, first, second, and third
power and then higher powers.

208
00:13:49,480 --> 00:13:52,150
But higher powers are
all-- Let's draw a picture.

209
00:13:52,150 --> 00:13:54,010
Where is w?

210
00:13:54,010 --> 00:13:57,220
So this is the complex plane.

211
00:13:57,220 --> 00:14:00,630
Here's the real, and here's
the imaginary direction.

212
00:14:00,630 --> 00:14:05,140
And let me take N to
be, here N was four.

213
00:14:05,140 --> 00:14:10,210
Let me take N to be eight in
my picture, just to have some,

214
00:14:10,210 --> 00:14:13,670
then I'll be able to spot
the fours and the eights.

215
00:14:13,670 --> 00:14:16,850
If N is eight,
where would w_8 be?

216
00:14:16,850 --> 00:14:32,240
So w_8 is meant to be
the e^(2pi*i/8), now.

217
00:14:32,240 --> 00:14:38,520
Where's that number
in that picture?

218
00:14:38,520 --> 00:14:43,950
Is it on the circle?

219
00:14:43,950 --> 00:14:46,830
Absolutely.

220
00:14:46,830 --> 00:14:51,340
This discrete Fourier transform
never gets off the circle.

221
00:14:51,340 --> 00:14:55,780
And in fact, it never gets off
eight points on the circle.

222
00:14:55,780 --> 00:14:58,540
Which are the eight
equally spaced points.

223
00:14:58,540 --> 00:15:02,740
This says go 1/8 of the way
of the whole way around,

224
00:15:02,740 --> 00:15:04,260
which will be here.

225
00:15:04,260 --> 00:15:05,050
So there's w_8.

226
00:15:05,050 --> 00:15:08,230


227
00:15:08,230 --> 00:15:12,330
When I square it, what's
the square of w_8?

228
00:15:12,330 --> 00:15:17,800
The angle, what happens to the
angle when I multiply numbers?

229
00:15:17,800 --> 00:15:19,910
I add the exponents.

230
00:15:19,910 --> 00:15:21,830
Here, if I'm
multiplying by itself,

231
00:15:21,830 --> 00:15:23,280
it'll double the exponent.

232
00:15:23,280 --> 00:15:24,570
Double the angle.

233
00:15:24,570 --> 00:15:34,600
It'll be, there's w_8 squared,
so that point is w_8 squared.

234
00:15:34,600 --> 00:15:39,150
And it's the same as w_4 because
it's 1/4 of the way around.

235
00:15:39,150 --> 00:15:45,540
And what is that number
in ordinary language? i.

236
00:15:45,540 --> 00:15:47,830
It's i, right?

237
00:15:47,830 --> 00:15:53,340
It's on the unit
circle at 90 degrees.

238
00:15:53,340 --> 00:15:56,750
And on the imaginary
axis it's just i.

239
00:15:56,750 --> 00:16:04,740
So all this here was i, i^2 i^3;
i^2, i^4, i^6; i^3, i^6, i^9.

240
00:16:04,740 --> 00:16:09,380
The four by four Fourier
matrix is just powers of i.

241
00:16:09,380 --> 00:16:12,260
And the N by N, the eight
by eight Fourier matrix

242
00:16:12,260 --> 00:16:15,220
is powers of w_8.

243
00:16:15,220 --> 00:16:16,970
And let's find all the
rest of the powers.

244
00:16:16,970 --> 00:16:21,020
So there's w_8, w_8
squared, where's w_8 cubed?

245
00:16:21,020 --> 00:16:22,370
Here.

246
00:16:22,370 --> 00:16:25,500
I multiply those guys, I add
the angle, I'm out to here.

247
00:16:25,500 --> 00:16:28,730
Here's the fourth power, here's
the fifth power, sixth power,

248
00:16:28,730 --> 00:16:33,480
seventh power and
finally we get to w_8

249
00:16:33,480 --> 00:16:37,900
to the eighth
power, which is one.

250
00:16:37,900 --> 00:16:40,950
Because if I take the eighth
power here, I've got e^(2pi*i),

251
00:16:40,950 --> 00:16:45,410
and that's one.

252
00:16:45,410 --> 00:16:50,960
So that's the
picture to remember.

253
00:16:50,960 --> 00:16:54,210
And while looking
at that picture,

254
00:16:54,210 --> 00:17:00,780
could you tell me the sum
of those eight numbers?

255
00:17:00,780 --> 00:17:03,340
What do those eight
members add up to?

256
00:17:03,340 --> 00:17:13,280
So I just want to ask what
do one plus w up to w^7,

257
00:17:13,280 --> 00:17:17,680
when I'm doing the
eighth guy, adds up to?

258
00:17:17,680 --> 00:17:20,100
So the sum of those
eight numbers,

259
00:17:20,100 --> 00:17:21,970
well I wouldn't ask
the question if it

260
00:17:21,970 --> 00:17:24,200
didn't have a nice answer.

261
00:17:24,200 --> 00:17:24,900
Zero.

262
00:17:24,900 --> 00:17:29,390
That's of course the
usual answer in math.

263
00:17:29,390 --> 00:17:34,140
We do all this work
and we get zero.

264
00:17:34,140 --> 00:17:37,760
But now, of course
the real math is why.

265
00:17:37,760 --> 00:17:42,310
I mean, so math is,
you get equations

266
00:17:42,310 --> 00:17:45,300
but then always there's
that kicker, why.

267
00:17:45,300 --> 00:17:48,380
And why do they add up to zero?

268
00:17:48,380 --> 00:17:51,920
Here.

269
00:17:51,920 --> 00:17:54,690
Can you see that?

270
00:17:54,690 --> 00:18:00,130
Why those eight numbers
would add to zero?

271
00:18:00,130 --> 00:18:02,850
Well, yeah.

272
00:18:02,850 --> 00:18:04,350
So what do these add up to?

273
00:18:04,350 --> 00:18:06,890
Just those two guys?

274
00:18:06,890 --> 00:18:09,780
One and minus one add to zero.

275
00:18:09,780 --> 00:18:13,470
That and that add to
zero, I can pair them off.

276
00:18:13,470 --> 00:18:15,390
This and this add to zero.

277
00:18:15,390 --> 00:18:17,840
That and that add to zero.

278
00:18:17,840 --> 00:18:19,800
Yeah, so that's one
way of seeing it.

279
00:18:19,800 --> 00:18:24,100
But actually, even if N was
three or five or some odd

280
00:18:24,100 --> 00:18:28,140
number, and I had-- So should
I do the picture for N=3?

281
00:18:28,140 --> 00:18:32,420
Just a little, doesn't need
a very big picture for N=3.

282
00:18:32,420 --> 00:18:35,050
So where would the
roots be for N=3?

283
00:18:35,050 --> 00:18:37,370
One of them is always one.

284
00:18:37,370 --> 00:18:41,610
These are the three
cube roots of one.

285
00:18:41,610 --> 00:18:44,370
That I'm going to draw.

286
00:18:44,370 --> 00:18:48,040
Those were the eight
eighth roots of one.

287
00:18:48,040 --> 00:18:50,420
Because if I take any
one of those numbers

288
00:18:50,420 --> 00:18:55,310
its eighth power
brings me back to one.

289
00:18:55,310 --> 00:18:59,060
So where are these guys, so
this is three numbers equally

290
00:18:59,060 --> 00:19:00,770
spaced around the circle.

291
00:19:00,770 --> 00:19:02,490
Where do you think they are?

292
00:19:02,490 --> 00:19:10,640
Well, it's gotta be at 120
degrees, 240 degrees and 360.

293
00:19:10,640 --> 00:19:14,140
And again, those
will add to zero.

294
00:19:14,140 --> 00:19:16,460
You can't pair them off
because we got an odd number,

295
00:19:16,460 --> 00:19:23,610
but it's safely zero
and we could see why.

296
00:19:23,610 --> 00:19:33,830
OK, so this section, this
topic is all about that matrix.

297
00:19:33,830 --> 00:19:37,720
It's all about that four
by four, or N by N matrix.

298
00:19:37,720 --> 00:19:39,180
Which has powers of w.

299
00:19:39,180 --> 00:19:43,750
Notice that the matrix is,
it hasn't got any zeroes.

300
00:19:43,750 --> 00:19:46,430
Hasn't even got
anything close to zero.

301
00:19:46,430 --> 00:19:50,080
All those numbers are
on the unit circle.

302
00:19:50,080 --> 00:19:56,660
So multiplying by
that matrix, which

303
00:19:56,660 --> 00:20:00,540
is the same as adding up the
Fourier series at the four

304
00:20:00,540 --> 00:20:07,900
points, doing this
calculation, finding these y's

305
00:20:07,900 --> 00:20:12,900
from the c's, reconstructing
the y's, given the coefficients,

306
00:20:12,900 --> 00:20:16,270
has 16 terms.

307
00:20:16,270 --> 00:20:18,050
Right?

308
00:20:18,050 --> 00:20:22,770
The matrix has got 16 entries,
all the entries in the matrix

309
00:20:22,770 --> 00:20:25,810
are showing up here
in powers of w.

310
00:20:25,810 --> 00:20:28,280
We've got 16 multiplications.

311
00:20:28,280 --> 00:20:33,270
Well, maybe a couple of them
were easy, but basically 16

312
00:20:33,270 --> 00:20:36,050
multiplications and additions.

313
00:20:36,050 --> 00:20:39,890
Pretty much N squared work.

314
00:20:39,890 --> 00:20:50,730
And that's not bad.

315
00:20:50,730 --> 00:20:54,260
Except if you want to do
it a million times, right?

316
00:20:54,260 --> 00:20:58,930
Suppose we have a
typical N might be 1024.

317
00:20:58,930 --> 00:21:03,910
So the matrix has then got,
is 1024 squared numbers,

318
00:21:03,910 --> 00:21:05,860
which is about a million.

319
00:21:05,860 --> 00:21:08,540
So if we have a matrix
with a million entries,

320
00:21:08,540 --> 00:21:14,140
a million operations to do, and
then we do it a million times

321
00:21:14,140 --> 00:21:20,160
then it's getting up
to money that Congress

322
00:21:20,160 --> 00:21:23,820
would give to the banks.

323
00:21:23,820 --> 00:21:24,820
Whatever.

324
00:21:24,820 --> 00:21:25,590
Anyway.

325
00:21:25,590 --> 00:21:27,900
Serious money.

326
00:21:27,900 --> 00:21:31,040
Serious computing time.

327
00:21:31,040 --> 00:21:38,010
And the wonderful
thing is that there

328
00:21:38,010 --> 00:21:43,940
is a faster way than N squared.

329
00:21:43,940 --> 00:21:50,050
There's a faster way to do these
16 terms than the obvious way.

330
00:21:50,050 --> 00:21:51,930
You might say, well,
they're pretty nice.

331
00:21:51,930 --> 00:21:53,210
And they are.

332
00:21:53,210 --> 00:21:55,600
But they're nicer
than you can know.

333
00:21:55,600 --> 00:21:58,600
I mean, nicer than you
can see immediately.

334
00:21:58,600 --> 00:22:07,520
This matrix will break
up into simple steps

335
00:22:07,520 --> 00:22:09,950
with lots of zeroes
in each step,

336
00:22:09,950 --> 00:22:16,110
and the final result is
that instead of n squared,

337
00:22:16,110 --> 00:22:20,720
capital N squared, which is
done the old fashioned way,

338
00:22:20,720 --> 00:22:26,890
let's say, that there's
a fast Fourier transform.

339
00:22:26,890 --> 00:22:32,990
So the FFT, I should
maybe find a better space

340
00:22:32,990 --> 00:22:37,060
for the most important algorithm
in the last hundred years,

341
00:22:37,060 --> 00:22:42,030
and here it is.

342
00:22:42,030 --> 00:22:45,320
Right there.

343
00:22:45,320 --> 00:22:47,770
So my one goal
for Wednesday will

344
00:22:47,770 --> 00:22:52,430
be to give you some
insight into the FFT.

345
00:22:52,430 --> 00:22:55,400
But here I'm just
looking at the result.

346
00:22:55,400 --> 00:22:59,500
The number of computations
is, instead of N squared,

347
00:22:59,500 --> 00:23:05,530
it's N times log N. It's
log to the base two,

348
00:23:05,530 --> 00:23:11,350
or maybe 1/2 N log N. So I'll
erase some of the other things

349
00:23:11,350 --> 00:23:14,940
close by so you can see it.

350
00:23:14,940 --> 00:23:19,560
So that, where N squared
was, if N was a thousand,

351
00:23:19,560 --> 00:23:23,100
N squared was a million.

352
00:23:23,100 --> 00:23:26,060
If N is a thousand,
this is a thousand,

353
00:23:26,060 --> 00:23:30,940
and what's the logarithm of
a thousand, to the base two?

354
00:23:30,940 --> 00:23:33,290
That's the ten, right?

355
00:23:33,290 --> 00:23:36,480
The 2^10 gave us that 1,024.

356
00:23:36,480 --> 00:23:39,560
So this is 10^4.

357
00:23:39,560 --> 00:23:46,740
So N is 10^3, or 1,024 more
exactly, the logarithm is ten,

358
00:23:46,740 --> 00:23:48,270
so we're 10^4 instead of 10^6.

359
00:23:48,270 --> 00:23:51,120


360
00:23:51,120 --> 00:23:54,800
So that's a saving of a
factor of 100, that's true.

361
00:23:54,800 --> 00:24:00,570
I mean it just comes from doing
the addition and multiplication

362
00:24:00,570 --> 00:24:02,030
in the right order.

363
00:24:02,030 --> 00:24:05,350
You get this incredible saving.

364
00:24:05,350 --> 00:24:08,620
And it's sort of makes
whole calculations that

365
00:24:08,620 --> 00:24:10,950
were previously
impossible are now

366
00:24:10,950 --> 00:24:18,590
possible because instead of
where it might have been 100

367
00:24:18,590 --> 00:24:24,400
minutes, it's now one minute.

368
00:24:24,400 --> 00:24:29,670
So everything focuses on this
Fourier matrix and its inverse,

369
00:24:29,670 --> 00:24:35,170
which we still have to find
but it'll come out beautifully.

370
00:24:35,170 --> 00:24:38,760
In fact, I could tell
you what F inverse,

371
00:24:38,760 --> 00:24:41,300
just so you see it coming.

372
00:24:41,300 --> 00:24:45,200
F inverse will, so
what happened to w,

373
00:24:45,200 --> 00:24:48,520
that all-important number w?

374
00:24:48,520 --> 00:24:59,310
Well, let me repeat what w is.

375
00:24:59,310 --> 00:25:02,530
I'll put it on this board.

376
00:25:02,530 --> 00:25:08,380
F has powers of w.
w is e^(2pi*i/N).

377
00:25:08,380 --> 00:25:12,040


378
00:25:12,040 --> 00:25:14,220
And what power has it got?

379
00:25:14,220 --> 00:25:19,190
In the j,k position?

380
00:25:19,190 --> 00:25:25,140
So now you can see the
formula for these entries.

381
00:25:25,140 --> 00:25:29,770
Only, you have to let me
start the count at j and k

382
00:25:29,770 --> 00:25:31,020
equals zero.

383
00:25:31,020 --> 00:25:35,790
This is zero-based, where
MATLAB is one-based.

384
00:25:35,790 --> 00:25:40,560
And to do this stuff you always
have to shift things by one.

385
00:25:40,560 --> 00:25:44,870
Some other, more recent,
software like Python

386
00:25:44,870 --> 00:25:50,870
is zero based, because
this is so common.

387
00:25:50,870 --> 00:25:54,670
So here's the formula.

388
00:25:54,670 --> 00:25:59,160
It's w to the power j times k.

389
00:25:59,160 --> 00:26:01,130
That's neat.

390
00:26:01,130 --> 00:26:09,320
That's what goes into the matrix
F. This here is row three.

391
00:26:09,320 --> 00:26:11,510
It looks like row four but
I'm starting with zero,

392
00:26:11,510 --> 00:26:13,050
so that's row three.

393
00:26:13,050 --> 00:26:14,760
That's column three.

394
00:26:14,760 --> 00:26:20,230
This is w three
times three, w^9.

395
00:26:20,230 --> 00:26:23,950
Which by the way is
the same as what?

396
00:26:23,950 --> 00:26:32,160
If w is i in this four by four,
it is. w is i, what's w^9,

397
00:26:32,160 --> 00:26:36,800
what's i to the 9th power?

398
00:26:36,800 --> 00:26:38,970
Same as?

399
00:26:38,970 --> 00:26:41,330
Can you do i^9 in your head?

400
00:26:41,330 --> 00:26:42,760
Sure.

401
00:26:42,760 --> 00:26:45,080
Because you're starting here.

402
00:26:45,080 --> 00:26:50,060
You're taking its ninth power,
so one, two, three, four,

403
00:26:50,060 --> 00:26:53,610
you're back to one. i^4 is one.

404
00:26:53,610 --> 00:26:57,880
One, two, three, four
more, i^8 is one. i^9.

405
00:26:57,880 --> 00:27:02,810
That will actually
be just i. i^9.

406
00:27:02,810 --> 00:27:04,910
So they're all powers of i.

407
00:27:04,910 --> 00:27:06,910
But this is the
good way to see it.

408
00:27:06,910 --> 00:27:10,140
It's three times three, because
it's in-- And this, of course,

409
00:27:10,140 --> 00:27:14,060
is three times two, and two
times two, and three times one.

410
00:27:14,060 --> 00:27:15,761
You see the entries of F?

411
00:27:15,761 --> 00:27:16,260
Yeah.

412
00:27:16,260 --> 00:27:19,120
So now I'd better
write down, I promised

413
00:27:19,120 --> 00:27:22,820
to write this formula
in a better form.

414
00:27:22,820 --> 00:27:26,450
All those formulas
in a better-- Nobody

415
00:27:26,450 --> 00:27:31,590
is going to write this out for
matrices of order 1000, right?

416
00:27:31,590 --> 00:27:36,320
So we've got to write
the decent formula.

417
00:27:36,320 --> 00:27:42,120
So the decent formula
is that the j-th y,

418
00:27:42,120 --> 00:27:44,600
because it's going to
come in equation number j,

419
00:27:44,600 --> 00:27:55,720
will be the sum on k=0 to
N-1, of c_k times w^(jk).

420
00:27:55,720 --> 00:27:58,910


421
00:27:58,910 --> 00:28:02,510
Those are the entries, those
are the c's that it multiplies

422
00:28:02,510 --> 00:28:03,680
and the y's.

423
00:28:03,680 --> 00:28:09,670
This is just y, vector
y, this is y=Fc.

424
00:28:09,670 --> 00:28:12,660
Now we've finally
got a good notation.

425
00:28:12,660 --> 00:28:16,710
So this was horrible notation.

426
00:28:16,710 --> 00:28:19,230
I mean, that's exhausting.

427
00:28:19,230 --> 00:28:21,350
This shows you the
algebra, but you

428
00:28:21,350 --> 00:28:24,690
have to be sort of prepared--
Well, the information is here

429
00:28:24,690 --> 00:28:27,090
because this is telling
you what the entries of F,

430
00:28:27,090 --> 00:28:29,100
of the matrix, are.

431
00:28:29,100 --> 00:28:35,540
But this is with indices, and
this is with whole vectors.

432
00:28:35,540 --> 00:28:39,480
So that would be
y, in MATLAB that

433
00:28:39,480 --> 00:28:49,490
would be the inverse fast
Fourier transform of c.

434
00:28:49,490 --> 00:28:52,880
So you see that
that's our matrix.

435
00:28:52,880 --> 00:28:56,820
Now, I was going
to say, before I

436
00:28:56,820 --> 00:29:01,170
show why, I was going to
say what about the inverse?

437
00:29:01,170 --> 00:29:07,760
Because if the FFT gave us
a fast way to multiply by F,

438
00:29:07,760 --> 00:29:11,370
we want it also to give us a
fast way to do the inverse.

439
00:29:11,370 --> 00:29:14,750
We have to go both ways here.

440
00:29:14,750 --> 00:29:18,350
We get our data, we put
it into frequency space,

441
00:29:18,350 --> 00:29:20,280
we look at it in
frequency space,

442
00:29:20,280 --> 00:29:22,020
we understand what's going on.

443
00:29:22,020 --> 00:29:24,160
By separating out
the frequencies.

444
00:29:24,160 --> 00:29:28,220
We maybe smooth it, we maybe
convolve it, whatever we do.

445
00:29:28,220 --> 00:29:29,440
Compress it.

446
00:29:29,440 --> 00:29:34,090
And then we've got to go
back to physical space.

447
00:29:34,090 --> 00:29:36,290
So we have to go both ways.

448
00:29:36,290 --> 00:29:41,290
And the question is
what about F inverse?

449
00:29:41,290 --> 00:29:52,180
And let me just say,
what goes into F inverse?

450
00:29:52,180 --> 00:29:56,380
The point is that the number
that goes into F inverse

451
00:29:56,380 --> 00:29:59,250
is just like w.

452
00:29:59,250 --> 00:30:04,490
But it's the complex conjugate,
and I'll call it w bar.

453
00:30:04,490 --> 00:30:07,490
And that's e^(-2pi*i/N).

454
00:30:07,490 --> 00:30:10,180


455
00:30:10,180 --> 00:30:13,250
So the thing that's going to
go into the inverse matrix

456
00:30:13,250 --> 00:30:16,540
is the powers of w bar.

457
00:30:16,540 --> 00:30:21,860
The complex conjugate. jk.

458
00:30:21,860 --> 00:30:28,040
Let me just say right away,
that just as there were 2pi's

459
00:30:28,040 --> 00:30:34,640
in the Fourier series
world, that got in our way.

460
00:30:34,640 --> 00:30:39,390
And physicists can't
stand those 2pi's.

461
00:30:39,390 --> 00:30:42,140
They try to get rid of them
but course they can't, they're

462
00:30:42,140 --> 00:30:43,770
there.

463
00:30:43,770 --> 00:30:47,240
So they put it up
into the exponent.

464
00:30:47,240 --> 00:30:50,850
Well, I guess, I'm blaming
physicists, I've done it too.

465
00:30:50,850 --> 00:30:52,310
I've put the 2pi up there.

466
00:30:52,310 --> 00:30:53,730
For the Fourier series.

467
00:30:53,730 --> 00:30:57,910
Anyway whatever, 2pi's
are all in your hair

468
00:30:57,910 --> 00:30:59,280
in the Fourier series.

469
00:30:59,280 --> 00:31:02,210
And in this, for
the discrete one,

470
00:31:02,210 --> 00:31:05,990
the corresponding thing
is an N. There's a factor

471
00:31:05,990 --> 00:31:08,010
N that I have to deal with.

472
00:31:08,010 --> 00:31:13,420
Because look, here's
what I'm saying.

473
00:31:13,420 --> 00:31:16,260
Let me multiply F by F inverse.

474
00:31:16,260 --> 00:31:17,110
OK.

475
00:31:17,110 --> 00:31:23,530
Say, 1, 1, 1, 1; 1 i, i^2,
i^3, this is my F, right?

476
00:31:23,530 --> 00:31:24,600
Whoops.

477
00:31:24,600 --> 00:31:33,270
Don't ever let me do that. i^2,
fourth, sixth; i^3, i^6, i^9.

478
00:31:33,270 --> 00:31:36,700
Now, my claim is
that F inverse is

479
00:31:36,700 --> 00:31:41,620
going to involve the complex
conjugate, w bar. i bar.

480
00:31:41,620 --> 00:31:44,580
What is the complex
conjugate of i?

481
00:31:44,580 --> 00:31:48,610
You see that we really have
to use complex numbers here,

482
00:31:48,610 --> 00:31:50,000
but they're on the unit circle.

483
00:31:50,000 --> 00:31:51,990
You couldn't ask
for a better line.

484
00:31:51,990 --> 00:31:53,650
What's the complex
conjugate that's

485
00:31:53,650 --> 00:31:56,930
going to go into F inverse?

486
00:31:56,930 --> 00:32:03,660
If I take i and I take its
conjugate, what does that mean?

487
00:32:03,660 --> 00:32:06,570
So the conjugate of
that is just, flip it

488
00:32:06,570 --> 00:32:10,600
across the real axis
to the other one.

489
00:32:10,600 --> 00:32:12,030
You see, it's great?

490
00:32:12,030 --> 00:32:16,470
The complex conjugate
of i is minus i.

491
00:32:16,470 --> 00:32:22,700
Here's w_8 here's w bar 8.

492
00:32:22,700 --> 00:32:24,280
It's just across the axis.

493
00:32:24,280 --> 00:32:29,020
So it still has size one.

494
00:32:29,020 --> 00:32:31,380
And, of course, it's e to the?

495
00:32:31,380 --> 00:32:34,770
This angle, which is
just minus this angle.

496
00:32:34,770 --> 00:32:37,610
So that's why we get this minus.

497
00:32:37,610 --> 00:32:41,710
Because when I flip it
across, i changes to minus i,

498
00:32:41,710 --> 00:32:44,120
angle changes to minus angle.

499
00:32:44,120 --> 00:32:46,690
And finally I was going to
do this multiplication just

500
00:32:46,690 --> 00:32:47,580
to see.

501
00:32:47,580 --> 00:32:54,630
So now I'm claiming that this is
1, 1, 1, 1, 1, 1, 1; -i, -i^2,

502
00:32:54,630 --> 00:32:55,880
-i^3.

503
00:32:55,880 --> 00:33:08,690
So on, -i^2, -i^3, fourth,
sixth, sixth, and ninth.

504
00:33:08,690 --> 00:33:11,840
And now, if you do
that multiplication,

505
00:33:11,840 --> 00:33:13,770
you will get the
identity matrix.

506
00:33:13,770 --> 00:33:15,250
That's the great thing.

507
00:33:15,250 --> 00:33:16,700
You get the identity.

508
00:33:16,700 --> 00:33:20,110
1, 1, 1, 1.

509
00:33:20,110 --> 00:33:27,580
Except for a factor N. And
do you see that coming?

510
00:33:27,580 --> 00:33:31,490
When I multiply this
by this, what do I get?

511
00:33:31,490 --> 00:33:34,220
Four.

512
00:33:34,220 --> 00:33:38,640
If they were N by N, I'd
have N ones against N ones,

513
00:33:38,640 --> 00:33:45,070
so I'd have to
expect an N there.

514
00:33:45,070 --> 00:33:51,010
So F times F inverse,
so, in other words

515
00:33:51,010 --> 00:33:57,340
if I want to get the identity,
I'd better divide this by N.

516
00:33:57,340 --> 00:34:03,650
So now I have F F inverse
equal I. So F inverse is just

517
00:34:03,650 --> 00:34:09,240
like F, except complex
conjugate and divide by N.

518
00:34:09,240 --> 00:34:12,670
So if I want this formula
to be really correct,

519
00:34:12,670 --> 00:34:17,740
I have to divide by N. OK?

520
00:34:17,740 --> 00:34:21,280
So and that's what
w and w bar is.

521
00:34:21,280 --> 00:34:26,730
And another letter that
often gets used in this topic

522
00:34:26,730 --> 00:34:27,990
is omega.

523
00:34:27,990 --> 00:34:33,810
And I figure that's a
good way to separate out

524
00:34:33,810 --> 00:34:37,680
the complex conjugates,
let's call this guy w

525
00:34:37,680 --> 00:34:39,880
and call this guy omega.

526
00:34:39,880 --> 00:34:42,420
So this is w and here's omega.

527
00:34:42,420 --> 00:34:44,120
I don't know if that helps.

528
00:34:44,120 --> 00:34:48,000
But you do have to keep
straight whether you're

529
00:34:48,000 --> 00:34:50,220
talking about this
number or that number.

530
00:34:50,220 --> 00:34:54,040
And this number
could be called w,

531
00:34:54,040 --> 00:34:56,100
because omega wouldn't
be easy to type.

532
00:34:56,100 --> 00:35:01,670
So you have to pay attention,
which one you're doing

533
00:35:01,670 --> 00:35:05,610
and when, somewhere you
have to divide by N.

534
00:35:05,610 --> 00:35:08,540
But otherwise it's fantastic.

535
00:35:08,540 --> 00:35:17,340
It's just beautiful.

536
00:35:17,340 --> 00:35:20,770
Can we do that
multiplication, just

537
00:35:20,770 --> 00:35:23,420
to see that it really works?

538
00:35:23,420 --> 00:35:25,570
Well, the ones worked fine.

539
00:35:25,570 --> 00:35:32,560
What about the ones times,
or what about the ones

540
00:35:32,560 --> 00:35:36,780
times that column.

541
00:35:36,780 --> 00:35:42,470
So I'm looking at the (1,2)
entry in this multiplication.

542
00:35:42,470 --> 00:35:45,230
Or rather, the (0,1),
entry I should say.

543
00:35:45,230 --> 00:35:48,610
This is the zeroth row, and
this is the row number one,

544
00:35:48,610 --> 00:35:51,420
and what do I get out of that?

545
00:35:51,420 --> 00:35:54,140
Zero, right.

546
00:35:54,140 --> 00:35:55,640
That was my point.

547
00:35:55,640 --> 00:36:01,380
That this is adding up, oh well,
going the other way around.

548
00:36:01,380 --> 00:36:03,730
Because it's the
complex conjugate,

549
00:36:03,730 --> 00:36:08,540
but it's adding up the four,
same four numbers and giving me

550
00:36:08,540 --> 00:36:12,260
that zero.

551
00:36:12,260 --> 00:36:15,220
And now how about the
ones on the diagonal?

552
00:36:15,220 --> 00:36:20,580
When I do this times
this, so this is the same,

553
00:36:20,580 --> 00:36:23,530
the row times itself.

554
00:36:23,530 --> 00:36:26,320
What am I seeing there?

555
00:36:26,320 --> 00:36:28,480
What are these terms?

556
00:36:28,480 --> 00:36:30,550
That's certainly
one, what is i times

557
00:36:30,550 --> 00:36:35,270
minus i? i times
minus i is one again.

558
00:36:35,270 --> 00:36:38,480
A number's getting
multiplied by its conjugate.

559
00:36:38,480 --> 00:36:40,410
That gives you something
real and positive

560
00:36:40,410 --> 00:36:44,380
when you multiply a
number by its conjugate.

561
00:36:44,380 --> 00:36:47,290
In fact, we get one every
time, so we get four ones,

562
00:36:47,290 --> 00:36:57,240
we divide by four
and we get that.

563
00:36:57,240 --> 00:37:02,230
Maybe you actually see
here, just so you remember.

564
00:37:02,230 --> 00:37:11,650
So F inverse is, I'm concluding
that-- F inverse, I'm

565
00:37:11,650 --> 00:37:13,320
looking now, what is F inverse?

566
00:37:13,320 --> 00:37:18,590
It's F conjugate divided
by this nuisance number

567
00:37:18,590 --> 00:37:22,970
N. I could put square
root of N with F,

568
00:37:22,970 --> 00:37:25,480
and then square root of N
would show up with F inverse,

569
00:37:25,480 --> 00:37:28,840
just the way I could have done,
square root of 2pi with one,

570
00:37:28,840 --> 00:37:30,730
and square root of
2pi with the other.

571
00:37:30,730 --> 00:37:37,060
But let's leave it like so.

572
00:37:37,060 --> 00:37:40,360
So that's not proved in a way.

573
00:37:40,360 --> 00:37:48,740
Just, checked for a
couple of cases with N=4,

574
00:37:48,740 --> 00:37:51,330
but it always works.

575
00:37:51,330 --> 00:37:52,180
That's the beauty.

576
00:37:52,180 --> 00:37:57,310
In other words,
the two directions,

577
00:37:57,310 --> 00:38:01,570
the y to c and the
c to y, are both

578
00:38:01,570 --> 00:38:04,560
going to be speeded
up by the FFT.

579
00:38:04,560 --> 00:38:06,900
The FFT, whatever
it did, whatever

580
00:38:06,900 --> 00:38:10,000
it could do with the
w and powers of w,

581
00:38:10,000 --> 00:38:12,860
it can do the same with w bar.

582
00:38:12,860 --> 00:38:14,280
And powers of w bar.

583
00:38:14,280 --> 00:38:17,690
If the FFT is a fast way
to to multiply by that,

584
00:38:17,690 --> 00:38:21,240
it'll also give me a
fast way to multiply,

585
00:38:21,240 --> 00:38:24,110
to do the other transform.

586
00:38:24,110 --> 00:38:29,610
So the FFT is what makes every--
This would all be important

587
00:38:29,610 --> 00:38:32,500
if the FFT didn't exist.

588
00:38:32,500 --> 00:38:34,800
And of course actually,
Fourier didn't notice it.

589
00:38:34,800 --> 00:38:40,640
Gauss again had the
idea, but he had

590
00:38:40,640 --> 00:38:46,890
so many ideas it wasn't
very clear in his notes.

591
00:38:46,890 --> 00:38:48,230
The FFT idea.

592
00:38:48,230 --> 00:38:58,450
But then Cooley and Tukey,
guys at Bell Labs in the 1950s

593
00:38:58,450 --> 00:38:59,320
discovered it.

594
00:38:59,320 --> 00:39:04,210
In a small paper, I don't
think they had any idea

595
00:39:04,210 --> 00:39:06,880
how important it would be.

596
00:39:06,880 --> 00:39:09,850
But they published
their paper and people

597
00:39:09,850 --> 00:39:12,620
saw what was happening, and
realized that they should,

598
00:39:12,620 --> 00:39:16,890
that that gave the right
way to do the transforms.

599
00:39:16,890 --> 00:39:24,290
OK, I was going to say here, do
you notice that this one, when

600
00:39:24,290 --> 00:39:29,440
I take a vector, that
vectors and its complex

601
00:39:29,440 --> 00:39:33,180
conjugate, what am I getting?

602
00:39:33,180 --> 00:39:35,560
I'm getting the length squared.

603
00:39:35,560 --> 00:39:37,330
I'm getting the
right inner product.

604
00:39:37,330 --> 00:39:41,190
This is the right inner product
when things are complex.

605
00:39:41,190 --> 00:39:42,690
Something times its conjugate.

606
00:39:42,690 --> 00:39:47,330
I'm just reminding you that the
inner product of two vectors,

607
00:39:47,330 --> 00:39:55,080
uv, u dot v or u comma v,
should now be the sum--

608
00:39:55,080 --> 00:39:59,330
It used to be the
sum of u_i*v_i.

609
00:39:59,330 --> 00:40:01,250
But now we're in
the complex case,

610
00:40:01,250 --> 00:40:04,770
so we should take the
conjugate one of them.

611
00:40:04,770 --> 00:40:07,990
And here I guess that's
the v, and this is the u,

612
00:40:07,990 --> 00:40:15,820
and so the point is that
we're taking the right, yeah.

613
00:40:15,820 --> 00:40:22,210
We have, oh yeah. let
me just say it this way.

614
00:40:22,210 --> 00:40:26,400
These columns are orthogonal.

615
00:40:26,400 --> 00:40:29,940
Those four columns, the
columns of the Fourier matrix,

616
00:40:29,940 --> 00:40:31,130
are orthogonal.

617
00:40:31,130 --> 00:40:36,530
In other words, we have
an orthogonal basis.

618
00:40:36,530 --> 00:40:38,990
And it's those four columns.

619
00:40:38,990 --> 00:40:43,820
I almost missed saying that.

620
00:40:43,820 --> 00:40:48,040
But you remember from last week,
the whole point about sines

621
00:40:48,040 --> 00:40:52,380
and cosines and exponentials
in the function case,

622
00:40:52,380 --> 00:40:53,690
was orthogonal.

623
00:40:53,690 --> 00:40:55,170
That's what made
everything work,

624
00:40:55,170 --> 00:40:58,420
and it makes
everything work here.

625
00:40:58,420 --> 00:41:01,180
Here we have vectors,
not functions.

626
00:41:01,180 --> 00:41:04,900
Orthogonal just means
dot products are zero.

627
00:41:04,900 --> 00:41:08,330
But they're complex, so
that in the dot product,

628
00:41:08,330 --> 00:41:11,720
one factor or the other
has to get conjugated.

629
00:41:11,720 --> 00:41:15,080
Anyway, that's an
orthogonal matrix.

630
00:41:15,080 --> 00:41:20,530
Apart from this N,
this capital N thing.

631
00:41:20,530 --> 00:41:25,550
So if I normalize it, if I take
out one over square root of N,

632
00:41:25,550 --> 00:41:28,080
now the length of
every column is one.

633
00:41:28,080 --> 00:41:31,540
So now I have
orthonormal columns.

634
00:41:31,540 --> 00:41:34,940
One squared, one squared, one
squared, one squared is four,

635
00:41:34,940 --> 00:41:38,310
divided by four is one.

636
00:41:38,310 --> 00:41:43,920
And every column has
length, is a vector

637
00:41:43,920 --> 00:41:49,200
in four-dimensional complex
space, of length one.

638
00:41:49,200 --> 00:41:51,070
And orthogonal.

639
00:41:51,070 --> 00:41:52,950
So that means orthonormal.

640
00:41:52,950 --> 00:41:55,820
It's a fantastic matrix.

641
00:41:55,820 --> 00:41:58,730
That's the best
basis we'll ever have

642
00:41:58,730 --> 00:42:00,930
for complex
four-dimensional space.

643
00:42:00,930 --> 00:42:06,870
That basis.

644
00:42:06,870 --> 00:42:10,290
Well, actually when I say the
best basis we'll ever have,

645
00:42:10,290 --> 00:42:13,550
I realize I wrote a
book about wavelets.

646
00:42:13,550 --> 00:42:16,970
And the idea of wavelets
is, there's another basis.

647
00:42:16,970 --> 00:42:18,230
A wavelet basis.

648
00:42:18,230 --> 00:42:20,600
And I hope to tell
you about that

649
00:42:20,600 --> 00:42:24,010
at the very end of the semester.

650
00:42:24,010 --> 00:42:26,050
So wavelets would
be another basis,

651
00:42:26,050 --> 00:42:29,530
would you like to know
the wavelet basis?

652
00:42:29,530 --> 00:42:32,340
Have we got?

653
00:42:32,340 --> 00:42:37,540
Just in case the end of
the semester doesn't come,

654
00:42:37,540 --> 00:42:40,420
can I write down here
the wavelet basis?

655
00:42:40,420 --> 00:42:41,590
OK.

656
00:42:41,590 --> 00:42:45,800
And you can decide you
maybe like it better.

657
00:42:45,800 --> 00:42:49,610
So the four by four
wavelet basis, alright.

658
00:42:49,610 --> 00:42:53,010
So this will be
the wavelet matrix.

659
00:42:53,010 --> 00:42:56,750
Well, this was too
good to lose, right?

660
00:42:56,750 --> 00:42:59,620
That's a terrific vector.

661
00:42:59,620 --> 00:43:02,740
All ones, constant vector.

662
00:43:02,740 --> 00:43:04,960
It's the thing that
gives us the average,

663
00:43:04,960 --> 00:43:13,740
this c_0 is going to again
be the average of the y's.

664
00:43:13,740 --> 00:43:17,500
How do I get that average?

665
00:43:17,500 --> 00:43:20,640
Somehow I'm doing, the
c_0, the constant term

666
00:43:20,640 --> 00:43:22,690
is again I'm going to
take one of everything

667
00:43:22,690 --> 00:43:25,740
and I'll divide by four
and I'll have the average.

668
00:43:25,740 --> 00:43:27,550
So I like that vector.

669
00:43:27,550 --> 00:43:29,770
That's the DC vector,
if I'm talking

670
00:43:29,770 --> 00:43:36,010
about-- direct current vector,
if I'm talking about networks.

671
00:43:36,010 --> 00:43:39,350
In an image, it
would be the whole,

672
00:43:39,350 --> 00:43:41,450
you could call it the
blackboard vector.

673
00:43:41,450 --> 00:43:46,580
The whole image of a empty
blackboard would be constant.

674
00:43:46,580 --> 00:43:49,390
OK, so that's too good to lose.

675
00:43:49,390 --> 00:43:52,100
Now, the obvious thing
orthogonal to that

676
00:43:52,100 --> 00:43:56,910
would be [1, 1, -1, -1].

677
00:43:56,910 --> 00:43:57,580
Right?

678
00:43:57,580 --> 00:44:00,860
Now I've got two
orthogonal columns.

679
00:44:00,860 --> 00:44:05,380
And probably that's pretty close
to something I have over here.

680
00:44:05,380 --> 00:44:08,490
Now, here's the
point about wavelets.

681
00:44:08,490 --> 00:44:10,870
You'll see it with the next guy.

682
00:44:10,870 --> 00:44:14,530
The next one, you saw that
those two are orthogonal.

683
00:44:14,530 --> 00:44:17,140
And real, and
simple, nice numbers.

684
00:44:17,140 --> 00:44:24,930
Now, the next one will
be [1,  -1, 0, 0].

685
00:44:24,930 --> 00:44:28,110
Do you see that
that's orthogonal?

686
00:44:28,110 --> 00:44:31,160
Obviously, because its dot
product with that is zero

687
00:44:31,160 --> 00:44:33,010
and that is zero.

688
00:44:33,010 --> 00:44:35,700
And you want to tell me the
last column of the wavelet,

689
00:44:35,700 --> 00:44:37,940
of the simple wavelet matrix?

690
00:44:37,940 --> 00:44:47,380
This would be the Haar, named
after Haar, the Haar wavelets.

691
00:44:47,380 --> 00:44:50,670
The whole point about
wavelets was a new basis.

692
00:44:50,670 --> 00:44:52,700
Somebody realized,
well actually, Haar

693
00:44:52,700 --> 00:44:56,724
lived a long time ago, but
they got improved and improved,

694
00:44:56,724 --> 00:44:57,890
but here's the simplest one.

695
00:44:57,890 --> 00:45:00,140
What's the good fourth column?

696
00:45:00,140 --> 00:45:05,100
Fourth orthogonal vector here?

697
00:45:05,100 --> 00:45:06,150
What shall I start with?

698
00:45:06,150 --> 00:45:17,170
Zeroes, and then I'll
put one, minus one.

699
00:45:17,170 --> 00:45:23,860
So the beauty of wavelets is,
I've got this averaging vector.

700
00:45:23,860 --> 00:45:27,490
And then I've got this
difference, sort of

701
00:45:27,490 --> 00:45:29,890
on a big scale, and then
I've got differences

702
00:45:29,890 --> 00:45:34,200
on a little scale with zero,
differences here with zeroes

703
00:45:34,200 --> 00:45:39,950
so as I keep going I could
do eight by eight, 16 by 16,

704
00:45:39,950 --> 00:45:44,020
it would-- I get
the averaging guy.

705
00:45:44,020 --> 00:45:47,570
And then I get differencing,
I get large scale differences

706
00:45:47,570 --> 00:45:49,480
and then I get
finer differences.

707
00:45:49,480 --> 00:45:52,610
So coarse differences,
finer differences,

708
00:45:52,610 --> 00:45:54,630
finer finer differences.

709
00:45:54,630 --> 00:46:03,190
And that's a really good
way to-- If your function,

710
00:46:03,190 --> 00:46:10,130
if your signal has jumps, we
saw that Fourier wasn't perfect,

711
00:46:10,130 --> 00:46:11,010
right?

712
00:46:11,010 --> 00:46:13,200
And a lot of signals have jumps.

713
00:46:13,200 --> 00:46:13,700
Right?

714
00:46:13,700 --> 00:46:18,980
Your shocks, whatever
you're trying to represent.

715
00:46:18,980 --> 00:46:21,140
This is a better
basis to represent

716
00:46:21,140 --> 00:46:25,430
a jump, a discontinuity,
than Fourier.

717
00:46:25,430 --> 00:46:29,330
Because Fourier, at a
discontinuity, does its best

718
00:46:29,330 --> 00:46:32,490
but it's not a local basis.

719
00:46:32,490 --> 00:46:37,280
It can't, like, especially
focus on that jump point.

720
00:46:37,280 --> 00:46:42,250
And and the result is the
coefficients decay very slowly

721
00:46:42,250 --> 00:46:44,190
and it's hard to compute with.

722
00:46:44,190 --> 00:46:49,910
Whereas the wavelet is great
for signals that have jumps.

723
00:46:49,910 --> 00:46:52,660
If something happens
suddenly at a certain time.

724
00:46:52,660 --> 00:46:55,720
Anyway, that would be
another, and of course I

725
00:46:55,720 --> 00:46:58,200
haven't made these
unit vectors, I'd

726
00:46:58,200 --> 00:47:01,230
have to divide that by
the square root of four

727
00:47:01,230 --> 00:47:04,140
and divide these guys by
the square root of two just

728
00:47:04,140 --> 00:47:05,810
to normalize them.

729
00:47:05,810 --> 00:47:08,730
But normalizing isn't important.

730
00:47:08,730 --> 00:47:11,670
Orthogonal is what is important.

731
00:47:11,670 --> 00:47:20,770
OK, so that's competition,
you could say.

732
00:47:20,770 --> 00:47:22,310
That's the competition.

733
00:47:22,310 --> 00:47:30,520
OK, back to main guy, so this
is the famous formula that gives

734
00:47:30,520 --> 00:47:34,440
the y's from the c's, and what's
the other formula that gives

735
00:47:34,440 --> 00:47:40,400
the c's from the y's?

736
00:47:40,400 --> 00:47:44,540
So now this is the transform
in the other direction.

737
00:47:44,540 --> 00:47:49,700
So it's going to be a
sum from j=0 to N-1.

738
00:47:49,700 --> 00:47:53,170


739
00:47:53,170 --> 00:47:57,150
And what's going to go in there?

740
00:47:57,150 --> 00:48:01,320
Have you got the idea
of the two transforms?

741
00:48:01,320 --> 00:48:06,330
In one direction this is e^i--
You see why we don't want

742
00:48:06,330 --> 00:48:07,070
to write it.

743
00:48:07,070 --> 00:48:17,030
That is, that number is
e to the 2pi*i*j*k/N.

744
00:48:17,030 --> 00:48:23,620
That's what that number
is, in the Fourier matrix.

745
00:48:23,620 --> 00:48:28,010
So you see using w
makes it much cleaner.

746
00:48:28,010 --> 00:48:34,160
OK, what goes there? w bar.

747
00:48:34,160 --> 00:48:37,530
The complex conjugate, e to
the minus all that stuff.

748
00:48:37,530 --> 00:48:42,310
w bar to the same power, jk.

749
00:48:42,310 --> 00:48:45,690
And then I have to remember
to divide by N somewhere.

750
00:48:45,690 --> 00:48:48,370
I can put it here, or
I can put it there.

751
00:48:48,370 --> 00:48:55,460
But the all ones vector has
length squared equal to N

752
00:48:55,460 --> 00:48:59,130
and I've got to divide
by an N somewhere.

753
00:48:59,130 --> 00:49:01,350
OK, those are the two
formulas that many people

754
00:49:01,350 --> 00:49:02,580
would just remember.

755
00:49:02,580 --> 00:49:05,730
For them that's the
discrete Fourier transform

756
00:49:05,730 --> 00:49:08,660
and the inverse, and maybe they
would put the N over there.

757
00:49:08,660 --> 00:49:11,420
Probably that would
be more common.

758
00:49:11,420 --> 00:49:15,870
I got started with the Fourier
matrix, with this side,

759
00:49:15,870 --> 00:49:24,080
and then the DFT matrix comes
here, and I needed the N.

760
00:49:24,080 --> 00:49:26,330
So those are the formulas.

761
00:49:26,330 --> 00:49:29,330
I don't know how you'd
prefer to remember these.

762
00:49:29,330 --> 00:49:31,500
Somehow you have to commit
something to memory here,

763
00:49:31,500 --> 00:49:33,590
certainly what w is.

764
00:49:33,590 --> 00:49:37,150
And the fact that
it's w and w bar?

765
00:49:37,150 --> 00:49:40,000
And the fact that the
sums start at zero.

766
00:49:40,000 --> 00:49:43,560
And the fact that you have
just these simple expressions

767
00:49:43,560 --> 00:49:46,060
for the entries in the matrix.

768
00:49:46,060 --> 00:49:53,070
And this then would be
c equals F inverse y.

769
00:49:53,070 --> 00:49:59,870
That's what I've written
down there in detail.

770
00:49:59,870 --> 00:50:05,460
OK, so that's the discrete
Fourier transform,

771
00:50:05,460 --> 00:50:06,990
and I think we're out of time.

772
00:50:06,990 --> 00:50:10,510
I've got more to say in a
little bit about the FFT,

773
00:50:10,510 --> 00:50:12,180
how that works.