1
00:00:08,750 --> 00:00:09,310
Okay.

2
00:00:09,310 --> 00:00:13,070
This is a lecture where
complex numbers come in.

3
00:00:13,070 --> 00:00:17,160
It's a -- complex numbers
have slipped into this course

4
00:00:17,160 --> 00:00:23,480
because even a real matrix
can have complex eigenvalues.

5
00:00:23,480 --> 00:00:27,420
So we met complex numbers
there as the eigenvalues

6
00:00:27,420 --> 00:00:30,030
and complex eigenvectors.

7
00:00:30,030 --> 00:00:33,560
And we -- or --

8
00:00:33,560 --> 00:00:36,410
this is probably the last --

9
00:00:36,410 --> 00:00:38,870
we have a lot of other things
to do about eigenvalues

10
00:00:38,870 --> 00:00:40,580
and eigenvectors.

11
00:00:40,580 --> 00:00:43,610
And that will be mostly real.

12
00:00:43,610 --> 00:00:46,090
But at one point
somewhere, we have

13
00:00:46,090 --> 00:00:51,140
to see what you do when the
numbers become complex numbers.

14
00:00:51,140 --> 00:00:54,500
What happens when the
vectors are complex,

15
00:00:54,500 --> 00:00:57,360
when the matrixes are
complex, when the --

16
00:00:57,360 --> 00:01:01,640
what's the inner product of two,
the dot product of two complex

17
00:01:01,640 --> 00:01:03,730
vectors --

18
00:01:03,730 --> 00:01:06,050
we just have to make
the change, just see --

19
00:01:06,050 --> 00:01:09,340
what is the change when
numbers become complex?

20
00:01:09,340 --> 00:01:14,360
Then, can I tell you about
the most important example

21
00:01:14,360 --> 00:01:17,270
of complex matrixes?

22
00:01:17,270 --> 00:01:20,460
It comes in the Fourier matrix.

23
00:01:20,460 --> 00:01:23,480
So the Fourier matrix,
which I'll describe,

24
00:01:23,480 --> 00:01:25,400
is a complex matrix.

25
00:01:25,400 --> 00:01:28,850
It's certainly the most
important complex matrix.

26
00:01:28,850 --> 00:01:34,770
It's the matrix that we
need in Fourier transform.

27
00:01:34,770 --> 00:01:37,930
And the -- really, the special
thing that I want to tell you

28
00:01:37,930 --> 00:01:42,340
about is what's called the
fast Fourier transform,

29
00:01:42,340 --> 00:01:48,000
and everybody refers to it
as the FFT and it's in all

30
00:01:48,000 --> 00:01:50,500
computer and it's used --

31
00:01:50,500 --> 00:01:54,010
it's being used as we
speak in a thousand places,

32
00:01:54,010 --> 00:01:59,510
because it has, like,
transformed whole industries

33
00:01:59,510 --> 00:02:03,090
to be able to do the
Fourier transform fast,

34
00:02:03,090 --> 00:02:09,270
which means multiplying --
how do I multiply fast by that

35
00:02:09,270 --> 00:02:11,730
matrix -- by that n by n matrix?

36
00:02:11,730 --> 00:02:16,380
Normally, multiplications
by an n by n matrix --

37
00:02:16,380 --> 00:02:21,440
would normally be n
squared multiplications,

38
00:02:21,440 --> 00:02:24,460
because I've got n squared
entries and none of them is

39
00:02:24,460 --> 00:02:24,960
zero.

40
00:02:24,960 --> 00:02:27,780
This is a full matrix.

41
00:02:27,780 --> 00:02:30,700
And it's a matrix with
orthogonal columns.

42
00:02:30,700 --> 00:02:34,140
I mean, it's just,
like, the best matrix.

43
00:02:34,140 --> 00:02:37,700
And this fast Fourier
transform idea

44
00:02:37,700 --> 00:02:43,680
reduces this n squared,
which was slowing up

45
00:02:43,680 --> 00:02:47,815
the calculation of Fourier
transforms down to n log(n).

46
00:02:51,000 --> 00:02:54,480
n log(n), log to the
base two, actually.

47
00:02:54,480 --> 00:02:57,940
And it's this --
when that hit --

48
00:02:57,940 --> 00:03:01,930
when that possibility hit,
it made a big difference.

49
00:03:01,930 --> 00:03:08,430
Everybody realized
gradually what, --

50
00:03:08,430 --> 00:03:11,800
that this simple idea -- you'll
see it's just a simple matrix

51
00:03:11,800 --> 00:03:15,060
factorization -- but
it changed everything.

52
00:03:15,060 --> 00:03:15,630
Okay.

53
00:03:15,630 --> 00:03:19,250
So I want to talk about
complex vectors and matrixes

54
00:03:19,250 --> 00:03:22,610
in general, recap a
little bit from last time,

55
00:03:22,610 --> 00:03:27,450
and the Fourier
matrix in particular.

56
00:03:27,450 --> 00:03:28,940
Okay.

57
00:03:28,940 --> 00:03:31,360
So what's the deal?

58
00:03:31,360 --> 00:03:32,210
All right.

59
00:03:32,210 --> 00:03:36,500
The main point is,
what about length?

60
00:03:36,500 --> 00:03:40,560
I'm given a vector,
I have a vector x.

61
00:03:40,560 --> 00:03:44,310
Or let me call it z as a
reminder that it's complex,

62
00:03:44,310 --> 00:03:45,260
for the moment.

63
00:03:45,260 --> 00:03:48,950
But I can -- later I'll
call the components x.

64
00:03:48,950 --> 00:03:50,740
They'll be complex numbers.

65
00:03:50,740 --> 00:03:53,090
But it's a vector --

66
00:03:53,090 --> 00:03:56,500
z1, z2 down to zn.

67
00:03:56,500 --> 00:04:01,590
So the only novelty is
it's not in R^n anymore.

68
00:04:01,590 --> 00:04:04,370
It's in complex n
dimensional space.

69
00:04:04,370 --> 00:04:07,510
Each of those numbers
is a complex number.

70
00:04:07,510 --> 00:04:14,750
So this z,z1 is in C^n, n
dimensional complex space

71
00:04:14,750 --> 00:04:17,700
instead of R^n.

72
00:04:17,700 --> 00:04:20,750
So just a different letter
there, but now the point

73
00:04:20,750 --> 00:04:23,200
about its length is what?

74
00:04:23,200 --> 00:04:29,190
The point about its length is
that z transpose z is no good.

75
00:04:34,800 --> 00:04:38,530
z transpose z -- if I just
put down z transpose here,

76
00:04:38,530 --> 00:04:42,090
it would be z1, z2, to zn.

77
00:04:45,580 --> 00:04:47,990
Doing that multiplication
doesn't give me

78
00:04:47,990 --> 00:04:50,090
the right thing.

79
00:04:50,090 --> 00:04:51,300
W-Why not?

80
00:04:51,300 --> 00:04:57,390
Because the length squared
should be positive.

81
00:04:57,390 --> 00:05:01,930
And if I multiply -- suppose
this is, like, 1 and i.

82
00:05:01,930 --> 00:05:06,560
What's the length of the
vector with components 1 and i?

83
00:05:06,560 --> 00:05:08,550
What if I do this,
so n is just two.

84
00:05:08,550 --> 00:05:11,380
I'm in C^2, two
dimensional space,

85
00:05:11,380 --> 00:05:16,510
complex space with the vector
whose components are 1 and i.

86
00:05:16,510 --> 00:05:17,310
All right.

87
00:05:17,310 --> 00:05:23,320
So if I took one times one
and i times i and added,

88
00:05:23,320 --> 00:05:27,150
z transpose z would be zero.

89
00:05:27,150 --> 00:05:29,310
But I don't -- that
vector is not --

90
00:05:29,310 --> 00:05:32,030
doesn't have length zero -- the
vector with the components 1

91
00:05:32,030 --> 00:05:33,590
and i --

92
00:05:33,590 --> 00:05:41,710
this multiplication -- what I
really want is z1 conjugate z1.

93
00:05:41,710 --> 00:05:46,510
You remember that z1
conjugate z1 is --

94
00:05:46,510 --> 00:05:50,370
so you see that first step
will be z1 conjugate z1,

95
00:05:50,370 --> 00:05:56,820
which is the magnitude of z1
squared, which is what I want.

96
00:05:56,820 --> 00:06:00,550
That's, like, three
squared or five squared.

97
00:06:00,550 --> 00:06:11,170
Now, if it's -- if z1 is i, then
I multiplied by minus i gives

98
00:06:11,170 --> 00:06:19,290
one plus one, so the component
of length -- the component i,

99
00:06:19,290 --> 00:06:21,731
its modulus squared is plus

100
00:06:21,731 --> 00:06:22,230
one.

101
00:06:22,230 --> 00:06:22,990
That's great.

102
00:06:22,990 --> 00:06:25,560
So what I want to do
then is do that --

103
00:06:25,560 --> 00:06:30,680
I want z1 bar z1, z2
bar z2, zn bar zn.

104
00:06:30,680 --> 00:06:33,210
And remember that -- you
remember this complex

105
00:06:33,210 --> 00:06:33,900
conjugate.

106
00:06:33,900 --> 00:06:38,270
So -- so there's the point.

107
00:06:38,270 --> 00:06:41,030
Now I can erase
the no good and put

108
00:06:41,030 --> 00:06:48,800
is good, because that now gives
the answer zero for the zero

109
00:06:48,800 --> 00:06:52,590
vector, of course, but it
gives a positive length

110
00:06:52,590 --> 00:06:55,780
squared for any other vector.

111
00:06:55,780 --> 00:06:58,890
So it's a -- it's the
right definition of length,

112
00:06:58,890 --> 00:07:04,370
and essentially the message is
that we're always going to be

113
00:07:04,370 --> 00:07:06,750
taking --

114
00:07:06,750 --> 00:07:09,400
when we transpose, we also
take complex conjugate.

115
00:07:09,400 --> 00:07:13,670
So let's -- let's find
the length of one --

116
00:07:13,670 --> 00:07:19,890
so the vector one i, that's
z, that's that vector z.

117
00:07:19,890 --> 00:07:23,490
Now I take the conjugate of one
is one, the conjugate of i is

118
00:07:23,490 --> 00:07:24,650
minus i.

119
00:07:24,650 --> 00:07:29,340
I take this vector,
I get one plus one --

120
00:07:29,340 --> 00:07:29,840
I get

121
00:07:29,840 --> 00:07:30,339
two.

122
00:07:33,050 --> 00:07:36,810
So that's a vector and that's a
vector of length -- square root

123
00:07:36,810 --> 00:07:37,740
of two.

124
00:07:37,740 --> 00:07:40,560
Square root of two is the
length and not the zero

125
00:07:40,560 --> 00:07:43,830
that we would have got
from one minus i squared.

126
00:07:43,830 --> 00:07:46,210
Okay.

127
00:07:46,210 --> 00:07:49,250
So the message really is
whenever we transpose,

128
00:07:49,250 --> 00:07:51,660
we also take conjugates.

129
00:07:51,660 --> 00:07:57,300
So here's a symbol --
one symbol to do both.

130
00:07:57,300 --> 00:08:03,890
So that symbol H, it stands
for a guy named Hermite,

131
00:08:03,890 --> 00:08:06,710
who didn't actually
pronounce the H,

132
00:08:06,710 --> 00:08:09,630
but let's pronounce it --

133
00:08:09,630 --> 00:08:14,040
so I would call
that z Hermitian z.

134
00:08:14,040 --> 00:08:20,940
I'll -- let me write that word,
Herm- so his name was Hermite,

135
00:08:20,940 --> 00:08:27,240
and then we make it into
an adjective, Hermitian.

136
00:08:27,240 --> 00:08:32,000
So z Hermitian z. z H z.

137
00:08:32,000 --> 00:08:34,360
Okay.

138
00:08:34,360 --> 00:08:37,510
So, that's the --

139
00:08:37,510 --> 00:08:40,730
that's the, length squared.

140
00:08:40,730 --> 00:08:43,409
Now what's the inner product?

141
00:08:43,409 --> 00:08:45,000
Well, it should match.

142
00:08:45,000 --> 00:08:49,850
The inner product
of two vectors --

143
00:08:49,850 --> 00:08:56,060
so inner product is no longer --

144
00:08:56,060 --> 00:08:59,230
used to be y transpose x.

145
00:08:59,230 --> 00:09:01,680
That's for real vectors.

146
00:09:01,680 --> 00:09:04,970
For complex vectors,
whenever we transpose,

147
00:09:04,970 --> 00:09:07,220
we also take the conjugate.

148
00:09:07,220 --> 00:09:12,110
So it's y Hermitian x.

149
00:09:12,110 --> 00:09:14,160
Of course it's not
real anymore, usually.

150
00:09:14,160 --> 00:09:19,610
That -- the inner product will
usually be complex number.

151
00:09:19,610 --> 00:09:24,100
But if y and x are the
same, if they're the same z,

152
00:09:24,100 --> 00:09:29,240
then we have z -- z H z,
we have the length squared,

153
00:09:29,240 --> 00:09:31,950
and that's what we want, the
inner product of a vector with

154
00:09:31,950 --> 00:09:35,490
itself should be
its length squared.

155
00:09:35,490 --> 00:09:40,230
So this is, like, forced on us
because this is forced on us.

156
00:09:40,230 --> 00:09:43,750
So -- so this z -- this --

157
00:09:43,750 --> 00:09:45,660
everybody's picking
up what this equals.

158
00:09:45,660 --> 00:09:52,170
This is z1 squared
plus zn squared.

159
00:09:52,170 --> 00:09:53,820
That's the length squared.

160
00:09:53,820 --> 00:09:57,080
And that's the inner product
that we have to go with.

161
00:09:57,080 --> 00:10:00,560
So it could be a
complex number now.

162
00:10:00,560 --> 00:10:02,790
One more change.

163
00:10:02,790 --> 00:10:04,750
Well, two more changes.

164
00:10:04,750 --> 00:10:08,780
We've got to change the
idea of a symmetric matrix.

165
00:10:08,780 --> 00:10:11,870
So I'll just recap on
symmetric matrixes.

166
00:10:11,870 --> 00:10:21,610
Symmetric means A transpose
equals A, but not --

167
00:10:21,610 --> 00:10:30,430
no good if A is complex.

168
00:10:30,430 --> 00:10:33,890
So what do we instead --

169
00:10:37,490 --> 00:10:40,360
that applies perfectly
to real matrixes.

170
00:10:40,360 --> 00:10:42,760
But now if my
matrixes were complex,

171
00:10:42,760 --> 00:10:51,910
I want to take the transpose
and the conjugate to equal A.

172
00:10:51,910 --> 00:10:55,430
So there's -- that's the --
the right complex version

173
00:10:55,430 --> 00:10:56,780
of symmetry.

174
00:10:56,780 --> 00:10:59,470
The com- the symmetry
now means when

175
00:10:59,470 --> 00:11:03,180
I transpose it, flip across the
diagonal and take conjugates.

176
00:11:03,180 --> 00:11:07,250
So, for example -- here
would be an example.

177
00:11:07,250 --> 00:11:09,480
On the diagonal, it
had better be real,

178
00:11:09,480 --> 00:11:15,760
because when I flip it, the
diagonal is still there and it

179
00:11:15,760 --> 00:11:18,410
has to -- and then when I take
the complex conjugate it has

180
00:11:18,410 --> 00:11:21,240
to be still there, so it
better be a real number,

181
00:11:21,240 --> 00:11:24,580
let me say two and five.

182
00:11:24,580 --> 00:11:27,790
What about entries
off the diagonal?

183
00:11:27,790 --> 00:11:32,200
If this entry is,
say, three plus i,

184
00:11:32,200 --> 00:11:36,630
then this entry had better be --

185
00:11:36,630 --> 00:11:40,070
because I want whatever
this -- when I transpose,

186
00:11:40,070 --> 00:11:42,790
it'll show up here
and i conjugate.

187
00:11:42,790 --> 00:11:47,000
So I need three minus I there.

188
00:11:47,000 --> 00:11:51,400
So there's a matrix with --

189
00:11:51,400 --> 00:11:55,640
that corresponds to
symmetry, but it's complex.

190
00:11:55,640 --> 00:12:01,550
And those matrixes are
called Hermitian matrixes.

191
00:12:01,550 --> 00:12:03,170
Hermitian matrixes.

192
00:12:03,170 --> 00:12:06,210
A H equals A.

193
00:12:06,210 --> 00:12:08,790
Fine.

194
00:12:08,790 --> 00:12:13,140
Okay, that's -- and those
matrixes have real eigenvalues

195
00:12:13,140 --> 00:12:15,950
and they have
perpendicular eigenvectors.

196
00:12:15,950 --> 00:12:18,760
What does perpendicular mean?

197
00:12:18,760 --> 00:12:21,760
Perpendicular means
the inner product --

198
00:12:21,760 --> 00:12:23,585
so let's go on to perpendicular.

199
00:12:28,180 --> 00:12:31,410
Well, when I had perpendicular
vectors, for example,

200
00:12:31,410 --> 00:12:37,820
they were like q1, q2 up to qn.

201
00:12:37,820 --> 00:12:41,630
That's my -- q is my letter
that I use for perpendicular.

202
00:12:41,630 --> 00:12:43,190
Actually, I usually --

203
00:12:43,190 --> 00:12:45,840
I also mean unit length.

204
00:12:45,840 --> 00:12:49,010
So those are perpendicular
unit vectors.

205
00:12:49,010 --> 00:12:51,030
But now what does
-- so it's a --

206
00:12:51,030 --> 00:12:54,730
orthonormal basis, I'll
still use those words,

207
00:12:54,730 --> 00:12:58,560
but how do I compute
perpendicular?

208
00:12:58,560 --> 00:13:00,770
How do I check perpendicular?

209
00:13:00,770 --> 00:13:07,370
This means that the inner
product of qi with qj --

210
00:13:07,370 --> 00:13:11,510
but now I not only transpose,
I must conjugate, right,

211
00:13:11,510 --> 00:13:20,210
to get zero if i is not
j and one if i is j.

212
00:13:20,210 --> 00:13:25,950
So it's a unit vector, meaning
unit length, orthogonal --

213
00:13:25,950 --> 00:13:28,930
all the angles are right
angles, but these are angles

214
00:13:28,930 --> 00:13:33,020
in complex n dimensional space.

215
00:13:33,020 --> 00:13:37,155
So it's q1, q on-
qi bar transpose.

216
00:13:39,790 --> 00:13:44,310
Or, for short, qi H qj.

217
00:13:44,310 --> 00:13:46,650
So it will still be
true -- so let me --

218
00:13:46,650 --> 00:13:51,230
again I'll create a
matrix out of those guys.

219
00:13:51,230 --> 00:13:56,090
The matrix will have these
q-s in its columns, q2 to qn.

220
00:14:00,320 --> 00:14:03,400
And I want to turn that
into matrix language,

221
00:14:03,400 --> 00:14:05,130
just like before.

222
00:14:05,130 --> 00:14:06,690
What does that mean?

223
00:14:06,690 --> 00:14:08,800
That means I want all
these inner products,

224
00:14:08,800 --> 00:14:13,920
so I take these columns of
Q, multiply by their rows --

225
00:14:13,920 --> 00:14:17,250
so it was -- it used to be Q
-- it used to be Q transpose Q

226
00:14:17,250 --> 00:14:18,160
equals I, right?

227
00:14:20,840 --> 00:14:22,535
This was an orthogonal matrix.

228
00:14:27,100 --> 00:14:30,490
But what's changed?

229
00:14:30,490 --> 00:14:33,290
These are now complex vectors.

230
00:14:33,290 --> 00:14:37,730
Their inner products are --
involve conjugating the first

231
00:14:37,730 --> 00:14:39,920
factor.

232
00:14:39,920 --> 00:14:44,240
So it's not -- it's the
conjugate of Q transpose.

233
00:14:44,240 --> 00:14:47,050
It's Q bar transpose Q.

234
00:14:47,050 --> 00:14:48,590
Q H.

235
00:14:48,590 --> 00:14:55,240
So can I call this -- let me
call it Q H Q, which is I.

236
00:14:55,240 --> 00:14:57,260
So that's our new --

237
00:14:57,260 --> 00:14:59,550
you -- you see I'm just
translating, and the --

238
00:14:59,550 --> 00:15:07,300
the book h- on one page gives a
little dictionary of the right

239
00:15:07,300 --> 00:15:12,900
words in the real case, R^n,
and the corresponding words

240
00:15:12,900 --> 00:15:17,120
in the complex case for
the vector space C^n.

241
00:15:17,120 --> 00:15:19,600
Of course, C^n is
a vector space,

242
00:15:19,600 --> 00:15:22,650
the numbers we multiply
are now complex numbers --

243
00:15:22,650 --> 00:15:27,770
we're just moving into
complex n dimensional space.

244
00:15:27,770 --> 00:15:29,700
Okay.

245
00:15:29,700 --> 00:15:34,450
Now -- actually, I have to say
we changed the word symmet-

246
00:15:34,450 --> 00:15:38,500
symmetric to Hermitian
for those matrixes.

247
00:15:38,500 --> 00:15:43,300
People also change this word
orthogonal into another word

248
00:15:43,300 --> 00:15:52,730
that happens to be unitary,
as a word that applies --

249
00:15:52,730 --> 00:15:56,540
that signals that we might be
dealing with a complex matrix

250
00:15:56,540 --> 00:15:57,320
here.

251
00:15:57,320 --> 00:15:59,940
So what's a unitary matrix?

252
00:15:59,940 --> 00:16:02,960
It's a -- it's just like
an orthogonal matrix.

253
00:16:02,960 --> 00:16:10,800
It's a square, n by n matrix
with orthonormal columns,

254
00:16:10,800 --> 00:16:13,970
perpendicular columns,
unit vectors --

255
00:16:13,970 --> 00:16:20,010
unit vectors computed by --
and perpendicularity computed

256
00:16:20,010 --> 00:16:23,820
by remembering that
there's a conjugate as well

257
00:16:23,820 --> 00:16:25,250
as a transpose.

258
00:16:25,250 --> 00:16:26,270
Okay.

259
00:16:26,270 --> 00:16:28,520
So those are the words.

260
00:16:28,520 --> 00:16:32,880
Now I'm ready to get into the
substance of the lecture which

261
00:16:32,880 --> 00:16:38,290
is the most famous complex
matrix, which happens

262
00:16:38,290 --> 00:16:41,500
to be one of these guys.

263
00:16:41,500 --> 00:16:50,240
It has orthogonal columns,
and it's named after Fourier

264
00:16:50,240 --> 00:16:52,800
because it comes into
the Fourier transform,

265
00:16:52,800 --> 00:16:55,380
so it's the matrix
that's all around us.

266
00:16:55,380 --> 00:16:55,880
Okay.

267
00:16:58,450 --> 00:17:05,660
Let me tell you what it is
first of all in the n by n case.

268
00:17:05,660 --> 00:17:09,750
Then often I'll let n be four
because four is a good size

269
00:17:09,750 --> 00:17:11,150
to work with.

270
00:17:11,150 --> 00:17:13,369
But here's the n by
n Fourier matrix.

271
00:17:17,790 --> 00:17:20,319
Its first column is
the vector of ones.

272
00:17:24,310 --> 00:17:26,319
It's n by n, of course.

273
00:17:26,319 --> 00:17:30,420
Its second column is
the powers, the --

274
00:17:30,420 --> 00:17:35,370
actually, better if I move
from the math department

275
00:17:35,370 --> 00:17:39,910
to EE for this one half
hour and then, please,

276
00:17:39,910 --> 00:17:42,100
let me move back again.

277
00:17:42,100 --> 00:17:43,870
Okay.

278
00:17:43,870 --> 00:17:46,170
What's the difference between
those two departments?

279
00:17:46,170 --> 00:17:50,640
It's just math starts
counting with one

280
00:17:50,640 --> 00:17:56,180
and electrical engineers
start counting at zero.

281
00:17:56,180 --> 00:17:57,840
Actually, they're
probably right.

282
00:17:57,840 --> 00:18:00,720
So anyway, we'll give
them -- humor them.

283
00:18:00,720 --> 00:18:05,090
So this is really
the zeroes column.

284
00:18:05,090 --> 00:18:08,030
And the first column
up to the n-1, that's

285
00:18:08,030 --> 00:18:11,780
the one inconvenient spot
in electrical engineering.

286
00:18:11,780 --> 00:18:14,760
All these expressions
start at zero, no problem,

287
00:18:14,760 --> 00:18:16,720
but they end at n-1.

288
00:18:16,720 --> 00:18:24,000
Well, that's -- that's the
difficulty of Course 6.

289
00:18:24,000 --> 00:18:27,750
So what's -- they're the powers
of a number that I'm going

290
00:18:27,750 --> 00:18:28,660
to call W --

291
00:18:28,660 --> 00:18:34,840
W squared, W cubed, W to the
-- now what is the W here?

292
00:18:34,840 --> 00:18:36,060
What's the power?

293
00:18:36,060 --> 00:18:39,000
This was the zeroes power,
first power, second power,

294
00:18:39,000 --> 00:18:43,290
this will be n
minus first power.

295
00:18:43,290 --> 00:18:45,130
That's the column.

296
00:18:45,130 --> 00:18:46,500
What's the next column?

297
00:18:46,500 --> 00:18:51,920
It's the powers of W squared, W
to the fourth, W to the sixth,

298
00:18:51,920 --> 00:18:55,870
W to the two n-1.

299
00:18:55,870 --> 00:18:58,010
And then more columns
and more columns

300
00:18:58,010 --> 00:19:00,060
and more columns and
what's the last column?

301
00:19:02,790 --> 00:19:07,361
It's the powers of --

302
00:19:07,361 --> 00:19:07,860
let's see.

303
00:19:07,860 --> 00:19:10,860
We -- actually, if we
look around rows, w-

304
00:19:10,860 --> 00:19:12,520
this matrix is symmetric.

305
00:19:12,520 --> 00:19:17,220
It's symmetric in the old
not quite perfect way,

306
00:19:17,220 --> 00:19:21,100
not perfect because these
numbers are complex.

307
00:19:21,100 --> 00:19:25,370
And so it's -- that
first row is all ones.

308
00:19:25,370 --> 00:19:28,740
One W W squared up
to W to the n-1.

309
00:19:28,740 --> 00:19:34,500
That's the last column is
the powers of W to the n-1,

310
00:19:34,500 --> 00:19:38,841
so this guy matches that, and
finally we get W to something

311
00:19:38,841 --> 00:19:39,340
here.

312
00:19:43,060 --> 00:19:46,670
I guess we could actually figure
out what that something is.

313
00:19:46,670 --> 00:19:48,610
What are the entries
of this matrix?

314
00:19:48,610 --> 00:19:57,490
The i j entry of
this matrix are --

315
00:19:57,490 --> 00:20:00,420
I going to -- are you going to
allow me to let i go from zero

316
00:20:00,420 --> 00:20:01,890
to n minus one?

317
00:20:01,890 --> 00:20:07,180
So i and g go from zero to n-1.

318
00:20:07,180 --> 00:20:10,870
So the one -- the zero
zero entry is a one --

319
00:20:10,870 --> 00:20:15,670
it's just this same W guy
to the power i times j.

320
00:20:21,960 --> 00:20:22,460
Let's see.

321
00:20:22,460 --> 00:20:24,690
I'm jumping into formulas
here and I have to tell you

322
00:20:24,690 --> 00:20:27,903
what W is and then you know
everything about this matrix.

323
00:20:30,960 --> 00:20:34,520
So W is the -- well,
shall we finish here?

324
00:20:34,520 --> 00:20:37,890
What was this -- this is
the (n-1) (n-1) entry.

325
00:20:37,890 --> 00:20:41,380
This is W to the n-1 squared.

326
00:20:41,380 --> 00:20:46,140
Everything's looking like a
mess here, because we have --

327
00:20:46,140 --> 00:20:51,890
not too bad, because all
the entries are powers of W.

328
00:20:51,890 --> 00:20:53,830
There -- none of them are zero.

329
00:20:53,830 --> 00:20:56,600
This is a full matrix.

330
00:20:56,600 --> 00:20:59,840
But W is a very special number.

331
00:20:59,840 --> 00:21:08,720
W is the special number
whose n-th power is one.

332
00:21:08,720 --> 00:21:12,780
In fact -- well, actually,
there are n numbers like that.

333
00:21:12,780 --> 00:21:14,620
One of them is one, of course.

334
00:21:14,620 --> 00:21:19,850
But the one we --
the W we want is --

335
00:21:19,850 --> 00:21:23,470
the angle is two pi over n.

336
00:21:28,060 --> 00:21:29,130
Is that what I mean?

337
00:21:29,130 --> 00:21:32,740
n over two pi.

338
00:21:32,740 --> 00:21:34,310
No, two pi over n.

339
00:21:34,310 --> 00:21:42,540
W is E to the I and the
angle is two pi over n.

340
00:21:42,540 --> 00:21:43,550
Right.

341
00:21:43,550 --> 00:21:47,590
Where is this W in
the complex plane?

342
00:21:47,590 --> 00:21:49,870
It's -- it's on the
unit circle, right?

343
00:21:49,870 --> 00:21:56,160
This is -- it's the cosine
of two pi over n plus I times

344
00:21:56,160 --> 00:21:59,620
the sine of two pi over n.

345
00:21:59,620 --> 00:22:02,840
But actually, forget this.

346
00:22:02,840 --> 00:22:09,050
It's never good to work with
the real and imaginary parts,

347
00:22:09,050 --> 00:22:14,130
the rectangular coordinates,
when we're taking powers.

348
00:22:14,130 --> 00:22:17,770
To take that to the tenth power,
we can't see what we're doing.

349
00:22:17,770 --> 00:22:19,890
To take this form
to the tenth power,

350
00:22:19,890 --> 00:22:22,350
we see immediately
what we're doing.

351
00:22:22,350 --> 00:22:26,480
It would be e to
the i 20 pi over n.

352
00:22:26,480 --> 00:22:30,570
So when our matrix is full of
powers -- so it's this formula,

353
00:22:30,570 --> 00:22:32,690
and where is this on
the complex plain?

354
00:22:32,690 --> 00:22:36,830
Here are the real numbers,
here's the imaginary axis,

355
00:22:36,830 --> 00:22:40,410
here's the unit
circle of radius one,

356
00:22:40,410 --> 00:22:44,240
and this number is
on the unit circle

357
00:22:44,240 --> 00:22:48,860
at this angle, which is one
n-th of the full way round.

358
00:22:48,860 --> 00:22:52,700
So if I drew, for
example, n equals six,

359
00:22:52,700 --> 00:22:56,490
this would be e to the
two pi, two pi over six,

360
00:22:56,490 --> 00:22:59,160
it would be one sixth of the way
around, it would be 60 degrees.

361
00:23:03,250 --> 00:23:06,090
And where is W squared?

362
00:23:06,090 --> 00:23:13,120
So I -- my W is e to the two
pi I over six in this case,

363
00:23:13,120 --> 00:23:16,760
in this six by -- for the
six by six Fourier transform,

364
00:23:16,760 --> 00:23:22,800
it's totally constructed out
of this number and its powers.

365
00:23:22,800 --> 00:23:24,480
So what are its powers?

366
00:23:24,480 --> 00:23:27,470
Well, its powers are on
the unit circle, right?

367
00:23:27,470 --> 00:23:32,680
Because when I square a
number, a complex number,

368
00:23:32,680 --> 00:23:37,740
I square its absolute value,
which gives me one again.

369
00:23:37,740 --> 00:23:40,500
All the powers have --
are on the unit circle.

370
00:23:40,500 --> 00:23:43,250
And the -- the angle
gets doubled to a hundred

371
00:23:43,250 --> 00:23:46,180
and twenty, so there's W
squared, there's W cubed,

372
00:23:46,180 --> 00:23:50,660
there's W to the fourth, there's
W to the fifth and there is W

373
00:23:50,660 --> 00:23:57,070
to the sixth, as we hoped, W to
the sixth coming back to one.

374
00:23:57,070 --> 00:24:01,080
So those are the six --

375
00:24:01,080 --> 00:24:02,620
can I say this on TV?

376
00:24:02,620 --> 00:24:07,130
The six sixth roots
of one, and it's

377
00:24:07,130 --> 00:24:14,560
this one, the primitive one we
say, the first one, which is W.

378
00:24:14,560 --> 00:24:17,840
Okay, so what -- let
me change -- let me --

379
00:24:17,840 --> 00:24:21,440
I said I would probably
switch to n equal four.

380
00:24:21,440 --> 00:24:24,130
What's W for that?

381
00:24:24,130 --> 00:24:25,970
It's the fourth root of one.

382
00:24:25,970 --> 00:24:28,590
W to the fourth will be one.

383
00:24:28,590 --> 00:24:33,410
W will be e to the two
pi i over four now.

384
00:24:38,820 --> 00:24:41,040
What's that?

385
00:24:41,040 --> 00:24:43,270
This is e to the i pi over two.

386
00:24:43,270 --> 00:24:46,180
This is a quarter of the
way around the unit circle,

387
00:24:46,180 --> 00:24:53,930
and that's exactly i, a
quarter of the way around.

388
00:24:53,930 --> 00:24:58,660
And sure enough, the
powers are i, i squared,

389
00:24:58,660 --> 00:25:01,670
which is minus
one, i cubed, which

390
00:25:01,670 --> 00:25:08,780
is minus i and finally i to
the fourth which is one, right.

391
00:25:08,780 --> 00:25:12,760
So there's W, W squared, W
cubed, W to the fourth --

392
00:25:12,760 --> 00:25:15,050
I'm really ready to
write down this Fourier

393
00:25:15,050 --> 00:25:18,780
matrix for the
four by four case,

394
00:25:18,780 --> 00:25:21,790
just so we see that clearly.

395
00:25:21,790 --> 00:25:22,890
Let me do it here.

396
00:25:22,890 --> 00:25:34,760
F4 is -- all right, one one one
one one one one W -- it's I.

397
00:25:34,760 --> 00:25:35,590
I squared.

398
00:25:35,590 --> 00:25:36,770
That's minus one.

399
00:25:36,770 --> 00:25:38,520
i cubed is minus i.

400
00:25:41,350 --> 00:25:43,490
I'll -- I could write
i squared and i cubed.

401
00:25:43,490 --> 00:25:46,140
Why don't I, just so we
see the pattern for sure.

402
00:25:46,140 --> 00:25:54,040
i squared, i cubed, i squared,
i cubed, i fourth, i sixth --

403
00:25:54,040 --> 00:25:58,690
i fourth, i sixth and i ninth.

404
00:25:58,690 --> 00:26:01,970
You see the exponents
fall in this nice --

405
00:26:01,970 --> 00:26:05,760
the exponent is the row number
times the column number,

406
00:26:05,760 --> 00:26:08,770
always starting at zero.

407
00:26:08,770 --> 00:26:09,560
Okay.

408
00:26:09,560 --> 00:26:12,910
And now I can put in those
numbers if you like --

409
00:26:12,910 --> 00:26:20,520
one one one one, one i minus
one minus i, one minus one,

410
00:26:20,520 --> 00:26:28,710
one minus one and one
minus i minus one i.

411
00:26:28,710 --> 00:26:29,410
No.

412
00:26:29,410 --> 00:26:31,111
Yes.

413
00:26:31,111 --> 00:26:31,610
Right.

414
00:26:38,260 --> 00:26:40,965
What's -- why do I think
that matrix is so remarkable?

415
00:26:44,520 --> 00:26:49,030
It's the four by four matrix
that comes into the four point

416
00:26:49,030 --> 00:26:52,020
Fourier transform.

417
00:26:52,020 --> 00:26:55,700
When we want to find the Fourier
transform, the four point

418
00:26:55,700 --> 00:27:01,380
Fourier transform of a
vector with four components,

419
00:27:01,380 --> 00:27:05,540
we want to multiply
by this F4 or we

420
00:27:05,540 --> 00:27:08,920
want to multiply by F4 inverse.

421
00:27:08,920 --> 00:27:10,840
One way we're taking
the transform,

422
00:27:10,840 --> 00:27:13,820
one way we're taking
the inverse transform.

423
00:27:13,820 --> 00:27:17,950
Actually, they're so close that
it's easy to confuse the two.

424
00:27:17,950 --> 00:27:21,990
The inverse of this matrix
will be a nice matrix also.

425
00:27:25,250 --> 00:27:29,190
So -- and that's, of
course, what makes it --

426
00:27:29,190 --> 00:27:32,370
that -- I guess
Fourier knew that.

427
00:27:32,370 --> 00:27:34,200
He knew the inverse
of this matrix.

428
00:27:36,970 --> 00:27:39,890
A- as you'll see, it just comes
from the fact that the columns

429
00:27:39,890 --> 00:27:41,640
are orthogonal --

430
00:27:41,640 --> 00:27:44,050
from the fact that the
columns are orthogonal,

431
00:27:44,050 --> 00:27:51,220
we will quickly figure
out what is the inverse.

432
00:27:51,220 --> 00:27:53,910
What Fourier didn't
know -- didn't notice --

433
00:27:53,910 --> 00:27:58,180
I think Gauss noticed it but
didn't make a point of it

434
00:27:58,180 --> 00:28:00,020
and then it turned
out to be really

435
00:28:00,020 --> 00:28:04,770
important was the fact that this
matrix is so special that you

436
00:28:04,770 --> 00:28:10,340
can break it up into nice pieces
with lots of zeroes, factors

437
00:28:10,340 --> 00:28:13,360
that have lots of
zeroes and multiply

438
00:28:13,360 --> 00:28:16,650
by it or by its inverse
very, very fast.

439
00:28:16,650 --> 00:28:17,150
Okay.

440
00:28:19,740 --> 00:28:22,850
But how did it get into
this lecture first?

441
00:28:22,850 --> 00:28:26,340
Because the columns
are orthogonal.

442
00:28:26,340 --> 00:28:29,130
Can I just check that the
columns of this matrix

443
00:28:29,130 --> 00:28:30,265
are orthogonal?

444
00:28:33,480 --> 00:28:38,360
So the inner product of
that column with that column

445
00:28:38,360 --> 00:28:41,260
is zero.

446
00:28:41,260 --> 00:28:48,250
The inner product of column
one with column three is zero.

447
00:28:48,250 --> 00:28:55,850
How about the inner
product of two and four?

448
00:28:55,850 --> 00:29:01,280
Can I take the inner product
of column two with column four?

449
00:29:01,280 --> 00:29:04,860
Or even the inner product of two
with three, let's -- let's see,

450
00:29:04,860 --> 00:29:06,820
does that --

451
00:29:06,820 --> 00:29:07,751
let me do two and

452
00:29:07,751 --> 00:29:08,250
four.

453
00:29:15,330 --> 00:29:17,260
Okay.

454
00:29:17,260 --> 00:29:19,110
What -- oh, I see, yes, hmm.

455
00:29:19,110 --> 00:29:19,610
Hmm.

456
00:29:19,610 --> 00:29:30,060
Let's see, I believe that those
two columns are orthogonal.

457
00:29:30,060 --> 00:29:33,520
So let me take their inner
product and hope to get zero.

458
00:29:33,520 --> 00:29:36,310
Okay, now if you hadn't
listened to the first half

459
00:29:36,310 --> 00:29:39,890
of this lecture, when you
took the inner product of that

460
00:29:39,890 --> 00:29:42,760
with that, you would have
multiplied one by one,

461
00:29:42,760 --> 00:29:48,090
i by minus i, and that
would have given you one,

462
00:29:48,090 --> 00:29:51,370
minus one by minus one
would have given you

463
00:29:51,370 --> 00:29:55,940
another one minus I by I would
have been minus I squared,

464
00:29:55,940 --> 00:29:56,810
that's another one.

465
00:30:00,870 --> 00:30:04,240
So do I conclude that the
inner product of columns --

466
00:30:04,240 --> 00:30:05,920
I said columns two
and four, that's

467
00:30:05,920 --> 00:30:08,550
because I forgot those
are columns one and three.

468
00:30:12,110 --> 00:30:13,670
I'm interested in
their inner product

469
00:30:13,670 --> 00:30:17,570
and I'm hoping it's zero, but
it doesn't look like zero.

470
00:30:17,570 --> 00:30:19,090
Nevertheless, it is zero.

471
00:30:19,090 --> 00:30:20,580
Those columns are perpendicular.

472
00:30:20,580 --> 00:30:21,500
Why?

473
00:30:21,500 --> 00:30:25,390
Because the inner product --

474
00:30:25,390 --> 00:30:26,520
we conjugate.

475
00:30:26,520 --> 00:30:29,150
Do you remember that the --
one of the vectors in the inner

476
00:30:29,150 --> 00:30:31,510
product has to get conjugated.

477
00:30:31,510 --> 00:30:34,750
So when I conjugated, it
changes that i to a minus i,

478
00:30:34,750 --> 00:30:37,900
changes this to a plus
i, changes those --

479
00:30:37,900 --> 00:30:44,060
that second sine and that
fourth sine and I do get zero.

480
00:30:44,060 --> 00:30:46,490
So those columns are orthogonal.

481
00:30:46,490 --> 00:30:49,130
So columns are orthogonal.

482
00:30:51,780 --> 00:30:55,630
They're not quite orthonormal.

483
00:30:55,630 --> 00:30:58,020
But I could fix that easily.

484
00:30:58,020 --> 00:31:02,710
They -- all those
columns have length two.

485
00:31:02,710 --> 00:31:07,450
Length squared is four, like
this -- the four I had there --

486
00:31:07,450 --> 00:31:10,260
this length squared, one plus
-- one squared one squared one

487
00:31:10,260 --> 00:31:13,910
squared one squared is
four, square root is two --

488
00:31:13,910 --> 00:31:17,420
so if I really wanted them --
suppose I really wanted to fix

489
00:31:17,420 --> 00:31:21,300
life perfectly, I
could divide by two,

490
00:31:21,300 --> 00:31:26,653
and now I have columns that
are actually orthonormal.

491
00:31:33,420 --> 00:31:35,560
So what?

492
00:31:35,560 --> 00:31:38,040
So I can invert
right away, right?

493
00:31:38,040 --> 00:31:41,970
O- orthonormal columns means --
now I'm keeping this one half

494
00:31:41,970 --> 00:31:43,820
in here for the moment --

495
00:31:43,820 --> 00:31:48,060
c- means F4 Hermitian,
can I use that,

496
00:31:48,060 --> 00:31:51,620
conjugate transpose times F4 is

497
00:31:51,620 --> 00:31:52,120
i.

498
00:31:56,230 --> 00:31:59,150
So I see what the inverse is.

499
00:31:59,150 --> 00:32:01,840
The inverse of F4 is
-- it's just like an --

500
00:32:01,840 --> 00:32:03,180
an orthogonal matrix.

501
00:32:03,180 --> 00:32:06,230
The inverse is the transpose
-- here the inverse is

502
00:32:06,230 --> 00:32:08,990
the conjugate transpose.

503
00:32:08,990 --> 00:32:10,250
So, fine.

504
00:32:10,250 --> 00:32:15,000
That -- that tells me that
anything good that I learn

505
00:32:15,000 --> 00:32:19,610
about F4 I'll know the same --

506
00:32:19,610 --> 00:32:21,710
I'll know a similar
fact about its inverse,

507
00:32:21,710 --> 00:32:25,620
because its inverse is just
its conjugate transpose.

508
00:32:25,620 --> 00:32:28,080
Okay, now -- so what's good?

509
00:32:28,080 --> 00:32:30,430
Well, first, the
columns are orthogonal.

510
00:32:30,430 --> 00:32:33,870
That's a key fact, then.

511
00:32:33,870 --> 00:32:36,670
That's the thing that
makes the inverse easy.

512
00:32:36,670 --> 00:32:40,740
But what property is it that
leads to the fast Fourier

513
00:32:40,740 --> 00:32:41,460
transform?

514
00:32:41,460 --> 00:32:44,000
So now I'm going to talk,
in these last minutes,

515
00:32:44,000 --> 00:32:48,010
about the fast
Fourier transform.

516
00:32:48,010 --> 00:32:49,290
What -- here's the idea.

517
00:32:51,980 --> 00:32:56,030
F6, our six by six
matrix, will c-

518
00:32:56,030 --> 00:33:02,250
there's a neat connection
to F3, half as big.

519
00:33:02,250 --> 00:33:05,200
There's a connection
of F8 to F4.

520
00:33:05,200 --> 00:33:09,690
There's a connection
of F(64) to F(32).

521
00:33:09,690 --> 00:33:12,070
Shall I write down what
that connection is?

522
00:33:12,070 --> 00:33:15,190
What's the connection
of F(64) to F(32)?

523
00:33:15,190 --> 00:33:22,950
So F(64) is a 64 by
64 matrix whose W

524
00:33:22,950 --> 00:33:26,500
is the 64th root of one.

525
00:33:26,500 --> 00:33:31,870
So it's one 64th of
the way round in F(64).

526
00:33:31,870 --> 00:33:35,959
And it -- do- and F(32)
is a 32 by 32 matrix.

527
00:33:35,959 --> 00:33:37,375
Remember, they're
different sizes.

528
00:33:40,450 --> 00:33:45,680
And the W in that 32 by 32
matrix is the 32nd root of one,

529
00:33:45,680 --> 00:33:50,220
which is twice as far -- that
-- you sh- see that key point --

530
00:33:50,220 --> 00:33:55,850
that's the -- that's how 32
and 64 are connected in the Ws.

531
00:33:55,850 --> 00:33:59,380
The W for 64 is one
64th of the way --

532
00:33:59,380 --> 00:34:05,610
so all I'm saying is
that if I square the W --

533
00:34:05,610 --> 00:34:09,219
W(64), that's what I'm using
for the one over -- the --

534
00:34:09,219 --> 00:34:15,070
W sixty f- this Wn is either
the i two pi over n --

535
00:34:15,070 --> 00:34:18,690
so W(64) is one 64th
of the way around it.

536
00:34:18,690 --> 00:34:24,139
When I square that,
what do I get but W(32)?

537
00:34:24,139 --> 00:34:25,969
Right?

538
00:34:25,969 --> 00:34:31,060
If I square this matrix,
I double the angle --

539
00:34:31,060 --> 00:34:36,300
if I square this number, I
double the angle, I get, the --

540
00:34:36,300 --> 00:34:40,120
the W(32).

541
00:34:40,120 --> 00:34:43,239
So somehow there's
a little hope here

542
00:34:43,239 --> 00:34:46,440
to connect F(64) with F(32).

543
00:34:46,440 --> 00:34:49,400
And here's the connection.

544
00:34:49,400 --> 00:34:49,980
Okay.

545
00:34:49,980 --> 00:34:53,530
Let me -- let me go back, --

546
00:34:53,530 --> 00:34:54,170
yes, let me --

547
00:34:54,170 --> 00:34:58,780
I'll do it here.

548
00:34:58,780 --> 00:35:00,520
Here's the connection.

549
00:35:00,520 --> 00:35:01,780
F(64).

550
00:35:01,780 --> 00:35:06,530
The 64 by 64 Fourier
matrix is connected

551
00:35:06,530 --> 00:35:10,070
to two copies of F(32).

552
00:35:10,070 --> 00:35:13,120
Let me leave a little
space for the connection.

553
00:35:13,120 --> 00:35:15,990
So this is 64 by 64.

554
00:35:15,990 --> 00:35:18,980
Here's a matrix
of that same size,

555
00:35:18,980 --> 00:35:25,640
because it's got two copies of
F(32) and two zero matrixes.

556
00:35:25,640 --> 00:35:31,500
Those zero matrixes are the key,
because when I multiply by this

557
00:35:31,500 --> 00:35:35,570
matrix, just as it is, regular
multiplication, I would take --

558
00:35:35,570 --> 00:35:37,160
need 64 --

559
00:35:37,160 --> 00:35:40,320
I would -- I would have 64
squared little multiplications

560
00:35:40,320 --> 00:35:41,470
to do.

561
00:35:41,470 --> 00:35:45,000
But this matrix is half zero.

562
00:35:45,000 --> 00:35:46,600
Well, of course, the
two aren't equal.

563
00:35:46,600 --> 00:35:52,010
I'm going to put an equals sign,
but there has to be some fix up

564
00:35:52,010 --> 00:35:53,570
factors --

565
00:35:53,570 --> 00:35:57,640
one there and one there --

566
00:35:57,640 --> 00:36:00,620
to make it true.

567
00:36:00,620 --> 00:36:05,260
The beauty is that these fix
up factors will be really --

568
00:36:05,260 --> 00:36:07,550
almost all zeroes.

569
00:36:07,550 --> 00:36:11,470
So that as soon as we
get this formula right,

570
00:36:11,470 --> 00:36:15,670
we've got a great idea for
how to get from the sixty-

571
00:36:15,670 --> 00:36:21,180
from the 64 squared calculations
-- so this original --

572
00:36:21,180 --> 00:36:25,110
originally we have 64 squared
calculations from there,

573
00:36:25,110 --> 00:36:29,570
but this one will give us
-- this is -- this will --

574
00:36:29,570 --> 00:36:34,660
we don't need that many -- we
only need two times 32 squared,

575
00:36:34,660 --> 00:36:38,120
because we've got that twice.

576
00:36:38,120 --> 00:36:42,440
And -- plus the fix-up.

577
00:36:42,440 --> 00:36:47,570
So I have to tell you what's
in this fix-up matrix.

578
00:36:47,570 --> 00:36:50,370
The one on the right is
actually a permutation matrix,

579
00:36:50,370 --> 00:36:55,820
a very simple odds and evens
permutation matrix, the --

580
00:36:55,820 --> 00:36:58,260
ones show up --

581
00:36:58,260 --> 00:37:00,820
I haven't put enough
ones, I really need a --

582
00:37:00,820 --> 00:37:05,106
32 of these guys at --
double space and then --

583
00:37:09,210 --> 00:37:12,400
you see it's -- it's
a permutation matrix.

584
00:37:12,400 --> 00:37:14,070
What it does --

585
00:37:14,070 --> 00:37:18,430
shall I call it P for
permutation matrix?

586
00:37:18,430 --> 00:37:23,920
So what that P does when
it multiplies a vector,

587
00:37:23,920 --> 00:37:28,770
it takes the odd -- the even
numbered components first

588
00:37:28,770 --> 00:37:30,580
and then the odds.

589
00:37:30,580 --> 00:37:33,640
You see this -- this one
skipping every time is going

590
00:37:33,640 --> 00:37:40,260
to pick out x0, x2, x4, x6 and
then below that will come --

591
00:37:40,260 --> 00:37:42,335
will pick out x1, x3, x5.

592
00:37:45,850 --> 00:37:49,650
And of course, that can be
hard wired in the computer

593
00:37:49,650 --> 00:37:52,350
to be instantaneous.

594
00:37:52,350 --> 00:37:56,570
So that says -- so
far, what have we said?

595
00:37:56,570 --> 00:38:00,960
We're saying that the 64 by
64 Fourier matrix is really

596
00:38:00,960 --> 00:38:04,540
separated into -- separate
your vector into the odd --

597
00:38:04,540 --> 00:38:07,430
into the even components
and the odd components,

598
00:38:07,430 --> 00:38:12,820
then do a 32 size Fourier
transform onto those

599
00:38:12,820 --> 00:38:17,420
separately, and then put
the pieces together again.

600
00:38:17,420 --> 00:38:21,890
So the pieces -- putting them
together turns them out to be I

601
00:38:21,890 --> 00:38:25,350
and a diagonal matrix
and I and a minus,

602
00:38:25,350 --> 00:38:26,615
that same diagonal matrix.

603
00:38:29,330 --> 00:38:33,540
So the fix-up cost is really
the cost of multiplying by D,

604
00:38:33,540 --> 00:38:38,120
this diagonal matrix, because
there's essentially no cost

605
00:38:38,120 --> 00:38:39,020
in --

606
00:38:39,020 --> 00:38:42,630
in the I part or in
the permutation part,

607
00:38:42,630 --> 00:38:50,720
so really it's -- the fix-up
cost is essentially because D

608
00:38:50,720 --> 00:38:52,950
is diagonal --

609
00:38:52,950 --> 00:38:54,160
is 32 multiplications.

610
00:38:58,930 --> 00:39:00,430
That's the -- there
you're seeing --

611
00:39:00,430 --> 00:39:02,888
of course we didn't check the
formula or we didn't even say

612
00:39:02,888 --> 00:39:05,030
what D is yet, but I will --

613
00:39:05,030 --> 00:39:08,900
this diagonal matrix
D is powers of W --

614
00:39:08,900 --> 00:39:13,345
one W W squared down
to W to the 31st.

615
00:39:19,720 --> 00:39:22,630
So you see that when I --
to do a multiplication by D,

616
00:39:22,630 --> 00:39:25,920
I need to do 32 multiplications.

617
00:39:25,920 --> 00:39:27,780
There they are.

618
00:39:27,780 --> 00:39:33,030
Then -- but the other, the more
serious work is to do the F(32)

619
00:39:33,030 --> 00:39:34,910
twice on the --

620
00:39:34,910 --> 00:39:37,300
separately on the
even numbered and odd

621
00:39:37,300 --> 00:39:41,600
numbered components,
so twice 32 squared.

622
00:39:41,600 --> 00:39:45,970
So 64 squared is gone now.

623
00:39:45,970 --> 00:39:47,990
And that's the new count.

624
00:39:50,580 --> 00:39:54,500
Okay, great, but what next?

625
00:39:54,500 --> 00:39:58,730
So that's -- I -- we
now have the key idea --

626
00:39:58,730 --> 00:40:01,110
we would have to
check the algebra,

627
00:40:01,110 --> 00:40:06,770
but it's just checking a lot of
sums that come out correctly.

628
00:40:06,770 --> 00:40:10,290
This is right -- the right
way to see the fast Fourier

629
00:40:10,290 --> 00:40:15,010
transform, or one
right way to see it.

630
00:40:15,010 --> 00:40:17,240
Then you've got to see
what's the next idea.

631
00:40:17,240 --> 00:40:21,390
The next idea is to
break the 32s down.

632
00:40:21,390 --> 00:40:23,450
Break those 32s down.

633
00:40:23,450 --> 00:40:27,430
So we have this factor,
and now we have the F(32),

634
00:40:27,430 --> 00:40:32,630
but that breaks into
some guy here --

635
00:40:32,630 --> 00:40:36,510
F thirty- F six- F(16) --

636
00:40:36,510 --> 00:40:37,610
F(16).

637
00:40:37,610 --> 00:40:43,320
Each -- each F(32) is breaking
into two copies of F(16),

638
00:40:43,320 --> 00:40:47,950
and then we have a
permutation and then the --

639
00:40:47,950 --> 00:40:52,210
so this is a -- like, this
was a 64 size permutation,

640
00:40:52,210 --> 00:40:55,410
this is a 32 size permutation --

641
00:40:55,410 --> 00:40:57,252
I guess I've got it twice.

642
00:40:57,252 --> 00:40:57,960
So it's -- I'm --

643
00:40:57,960 --> 00:41:01,120
I'm just using the same
idea recursively --

644
00:41:01,120 --> 00:41:05,270
recursion is the key word --
that on each of those F(32)s --

645
00:41:05,270 --> 00:41:09,465
so here's zero zero -- it's
just -- to get F(32) --

646
00:41:13,810 --> 00:41:16,980
this is the odd
even permutations --

647
00:41:16,980 --> 00:41:20,580
so you see, we're --
the combination of those

648
00:41:20,580 --> 00:41:24,210
permutations, what's it doing?

649
00:41:24,210 --> 00:41:28,070
This guy separates into odds
-- in -- into evens and odds,

650
00:41:28,070 --> 00:41:34,160
and then this guy separates
the evens into the ones --

651
00:41:34,160 --> 00:41:37,430
the numbers that are
mult- the even evens,

652
00:41:37,430 --> 00:41:41,080
which means zero
four eight sixteen --

653
00:41:41,080 --> 00:41:48,580
and even odds, which means
two, six, ten, fourteen --

654
00:41:48,580 --> 00:41:52,240
and then odd evens and odd odds.

655
00:41:52,240 --> 00:41:55,930
You see, together these
permutations then break it --

656
00:41:55,930 --> 00:42:00,630
break our vector down into
x, even even and three other

657
00:42:00,630 --> 00:42:02,090
pieces.

658
00:42:02,090 --> 00:42:05,070
Those are the four pieces
that separately get multiplied

659
00:42:05,070 --> 00:42:06,650
by F(16) --

660
00:42:06,650 --> 00:42:12,808
separately fixed up by these Is
and Ds and Is and minus Ds --

661
00:42:15,680 --> 00:42:19,340
so this count is now reduced.

662
00:42:19,340 --> 00:42:23,220
This count is now --
what's it reduced to?

663
00:42:23,220 --> 00:42:28,490
So that's going to be gone,
because 32 squared -- that's --

664
00:42:28,490 --> 00:42:30,680
that's the change
I'm making, right?

665
00:42:30,680 --> 00:42:35,130
The 32 squared -- w- so -- so
it's this that's now reduced.

666
00:42:35,130 --> 00:42:40,150
So I still have two times it,
but now what's 32 squared?

667
00:42:40,150 --> 00:42:47,710
It's gone in favor of two
sixteen squareds plus sixteen.

668
00:42:47,710 --> 00:42:54,055
That's -- and then the
original 32 to fix.

669
00:43:00,170 --> 00:43:01,910
Maybe you see what's happening.

670
00:43:01,910 --> 00:43:04,580
Even easier than this
formula is w- what's --

671
00:43:04,580 --> 00:43:09,410
when I do the recursion
more and more times,

672
00:43:09,410 --> 00:43:12,670
I get simpler and simpler
factors in the middle.

673
00:43:12,670 --> 00:43:15,330
Eventually I'll be down
to two point or one point

674
00:43:15,330 --> 00:43:18,320
Fourier transforms.

675
00:43:18,320 --> 00:43:21,170
But I get more and more
factors piling up on the right

676
00:43:21,170 --> 00:43:21,720
and left.

677
00:43:21,720 --> 00:43:23,650
On the right, I'm just
getting permutation

678
00:43:23,650 --> 00:43:24,450
matrixes.

679
00:43:24,450 --> 00:43:29,300
On the left, I'm getting
these guys, these Is and Ds,

680
00:43:29,300 --> 00:43:32,490
so that there was
a 32 there and --

681
00:43:32,490 --> 00:43:35,320
each one of these is costing 32.

682
00:43:35,320 --> 00:43:39,330
Each one of those is costing
And how many will there be?

683
00:43:42,450 --> 00:43:45,080
So you see the 32
for this original fix

684
00:43:45,080 --> 00:43:49,850
up, because D had 32
numbers, 32 for this next fix

685
00:43:49,850 --> 00:43:54,750
up, because D has
16 and 16 more.

686
00:43:54,750 --> 00:43:56,920
I keep going.

687
00:43:56,920 --> 00:44:00,320
So the count in the
middle goes down to zip,

688
00:44:00,320 --> 00:44:04,080
but these fix up counts
are all that I'm left with,

689
00:44:04,080 --> 00:44:10,820
and how many factors -- how many
fix-ups have I got -- log in --

690
00:44:10,820 --> 00:44:15,740
from 64, one step to 32, one
step to 16, one step to eight,

691
00:44:15,740 --> 00:44:17,660
four, two and one.

692
00:44:17,660 --> 00:44:19,030
Six steps.

693
00:44:19,030 --> 00:44:20,550
So I have six fix-up --

694
00:44:20,550 --> 00:44:23,540
six fix up factors.

695
00:44:23,540 --> 00:44:33,380
Finally I get to six times
the That's my final count.

696
00:44:33,380 --> 00:44:41,430
Instead of 64 squared, this is
log to the base two of 64 times

697
00:44:41,430 --> 00:44:42,960
64 --

698
00:44:42,960 --> 00:44:45,190
actually, half of 64.

699
00:44:45,190 --> 00:44:51,600
So actually, the final count is
n log to the base two of n --

700
00:44:51,600 --> 00:44:52,815
that's the 32 --

701
00:44:55,380 --> 00:44:57,430
a half.

702
00:44:57,430 --> 00:45:02,160
So can I put a box
around that wonderful,

703
00:45:02,160 --> 00:45:07,210
extremely important and
satisfying conclusion --

704
00:45:07,210 --> 00:45:12,400
that the fast Fourier transform
multiplies by an n by n matrix,

705
00:45:12,400 --> 00:45:16,110
but it does it not in n
squared steps, but in one

706
00:45:16,110 --> 00:45:19,100
half n log n steps.

707
00:45:19,100 --> 00:45:21,130
And if we just --

708
00:45:21,130 --> 00:45:25,070
complete by doing a
count, let's suppose --

709
00:45:25,070 --> 00:45:31,680
suppose -- a typical
case would be two to the

710
00:45:31,680 --> 00:45:32,180
tenth.

711
00:45:35,540 --> 00:45:39,760
Now n squared is
bigger than a million.

712
00:45:44,560 --> 00:45:47,910
So it's a thousand twenty four
times a thousand twenty four.

713
00:45:47,910 --> 00:45:51,390
But what is n --
what is one half --

714
00:45:51,390 --> 00:45:55,690
what is the new count,
done the right way?

715
00:45:55,690 --> 00:46:02,550
It's n -- the thousand
twenty four times one half,

716
00:46:02,550 --> 00:46:04,640
and what's the logarithm?

717
00:46:04,640 --> 00:46:06,180
It's ten.

718
00:46:06,180 --> 00:46:08,540
So times ten over two.

719
00:46:08,540 --> 00:46:12,880
So it's five times -- it's five
times a thousand twenty four,

720
00:46:12,880 --> 00:46:16,690
where this one was a thousand
twenty four times a thousand

721
00:46:16,690 --> 00:46:17,460
twenty four.

722
00:46:21,450 --> 00:46:25,470
We've reduced the calculation
by a factor of 200 just

723
00:46:25,470 --> 00:46:28,830
by factoring the
matrix properly.

724
00:46:28,830 --> 00:46:34,580
This was a thousand times n,
we're now down to five times n.

725
00:46:34,580 --> 00:46:38,570
So we can do 200
Fourier transforms,

726
00:46:38,570 --> 00:46:42,160
where before we
could do one, and

727
00:46:42,160 --> 00:46:45,720
in real scientific calculations
where Fourier transforms are

728
00:46:45,720 --> 00:46:48,200
happening all the
time, we're saving

729
00:46:48,200 --> 00:46:52,120
a factor of in one
of the major steps

730
00:46:52,120 --> 00:46:55,960
of modern scientific computing.

731
00:46:55,960 --> 00:46:58,280
So that's the idea of the
fast Fourier transform,

732
00:46:58,280 --> 00:47:00,020
and you see the
whole thing hinged

733
00:47:00,020 --> 00:47:07,360
on being a special matrix
with orthonormal columns.

734
00:47:07,360 --> 00:47:13,020
Okay, that's actually
it for complex numbers.

735
00:47:13,020 --> 00:47:16,960
I'm back next time really to --

736
00:47:16,960 --> 00:47:22,560
to real numbers,
eigenvalues and eigenvectors

737
00:47:22,560 --> 00:47:27,190
and the key idea of
positive definite matrixes

738
00:47:27,190 --> 00:47:28,790
is going to show up.

739
00:47:28,790 --> 00:47:30,660
What's a positive
definite matrix?

740
00:47:30,660 --> 00:47:32,550
And it's terrific
that this course

741
00:47:32,550 --> 00:47:35,520
is going to reach
positive definiteness,

742
00:47:35,520 --> 00:47:37,840
because those are
the matrixes that you

743
00:47:37,840 --> 00:47:40,320
see the most in applications.

744
00:47:40,320 --> 00:47:41,965
Okay, see you next time.

745
00:47:46,240 --> 00:47:47,790
Thanks.