1
00:00:00,090 --> 00:00:02,500
The following content is
provided under a Creative

2
00:00:02,500 --> 00:00:04,019
Commons license.

3
00:00:04,019 --> 00:00:06,360
Your support will help
MIT OpenCourseWare

4
00:00:06,360 --> 00:00:10,730
continue to offer high-quality
educational resources for free.

5
00:00:10,730 --> 00:00:13,330
To make a donation or
view additional materials

6
00:00:13,330 --> 00:00:17,217
from hundreds of MIT courses,
visit MIT OpenCourseWare

7
00:00:17,217 --> 00:00:17,842
at ocw.mit.edu.

8
00:00:21,240 --> 00:00:24,360
ERIK DEMAINE: All right,
let's get started.

9
00:00:24,360 --> 00:00:25,510
I am Erik Demaine.

10
00:00:25,510 --> 00:00:27,130
You can call me Erik.

11
00:00:27,130 --> 00:00:29,020
Today we're going
to do another divide

12
00:00:29,020 --> 00:00:31,970
and conquer algorithm called
the fast Fourier transform.

13
00:00:31,970 --> 00:00:35,870
It's probably the most
taught algorithm at MIT.

14
00:00:35,870 --> 00:00:39,300
It's used in all
sorts of contexts,

15
00:00:39,300 --> 00:00:41,440
especially digital
signal processing

16
00:00:41,440 --> 00:00:45,046
like MP3 compression,
and all sorts of things.

17
00:00:45,046 --> 00:00:46,670
But we're going to
think about it today

18
00:00:46,670 --> 00:00:51,560
in the context of divide
and conquer and polynomials.

19
00:00:51,560 --> 00:00:55,540
So let me remind you-- I
mean, this class is all

20
00:00:55,540 --> 00:00:59,160
about polynomial time, but
usually with polynomial time

21
00:00:59,160 --> 00:01:01,250
we only care about
the lead term.

22
00:01:01,250 --> 00:01:04,250
Today-- and today
only, pretty much--

23
00:01:04,250 --> 00:01:09,670
we're going to be thinking about
all the terms in a polynomial.

24
00:01:09,670 --> 00:01:13,080
So I'll talk about
polynomials mostly a and b.

25
00:01:13,080 --> 00:01:15,480
You have a constant
term, and then

26
00:01:15,480 --> 00:01:22,170
a linear term, and a quadratic
term, and so on up to-- I

27
00:01:22,170 --> 00:01:24,160
will say that there are
n terms-- which means

28
00:01:24,160 --> 00:01:28,150
the last one is a n minus 1.

29
00:01:28,150 --> 00:01:32,320
Normally the degree the
polynomial here is n minus 1.

30
00:01:32,320 --> 00:01:35,819
I wish the degree was defined
to be n here, but whatever.

31
00:01:35,819 --> 00:01:37,860
That's-- I'm not-- I can't
change the definitions

32
00:01:37,860 --> 00:01:39,110
in algebra.

33
00:01:39,110 --> 00:01:42,060
So this is the
traditional algebraic way

34
00:01:42,060 --> 00:01:44,680
of thinking about a polynomial.

35
00:01:44,680 --> 00:01:48,640
Of course, you can write
it with summation notation.

36
00:01:48,640 --> 00:01:52,990
akx to the k, k
equals 0 to n minus 1.

37
00:01:52,990 --> 00:01:54,790
We'll jump back and
forth between them.

38
00:01:54,790 --> 00:01:59,350
I'm also going to introduce a
vector notation for polynomials

39
00:01:59,350 --> 00:02:06,770
because-- so the ai's are
real numbers, typically.

40
00:02:06,770 --> 00:02:09,620
We might change
that at some point,

41
00:02:09,620 --> 00:02:11,904
but usually-- a common
reason-- maybe you

42
00:02:11,904 --> 00:02:13,320
don't care much
about polynomials,

43
00:02:13,320 --> 00:02:15,260
but you definitely
care about vectors.

44
00:02:15,260 --> 00:02:17,180
Any kind of
one-dimensional data set

45
00:02:17,180 --> 00:02:20,780
is a string of real
numbers, like if you're

46
00:02:20,780 --> 00:02:22,530
sampling audio--
like right now we're

47
00:02:22,530 --> 00:02:25,240
recording this
microphone-- you're

48
00:02:25,240 --> 00:02:29,800
seeing lots of different--
the movement of the membrane

49
00:02:29,800 --> 00:02:32,120
in this microphone over time.

50
00:02:32,120 --> 00:02:35,875
You're sampling-- whatever--
40,000 times the second.

51
00:02:35,875 --> 00:02:37,500
Each one you're
measuring a real number

52
00:02:37,500 --> 00:02:38,860
about where that thing is.

53
00:02:38,860 --> 00:02:40,910
That is a sequence
of real numbers.

54
00:02:40,910 --> 00:02:43,610
Now, you can convert it into
a polynomial if you want.

55
00:02:43,610 --> 00:02:44,740
They're the same thing.

56
00:02:44,740 --> 00:02:46,990
x is not necessarily
meaningful here.

57
00:02:46,990 --> 00:02:49,770
We really care about
other coefficients.

58
00:02:49,770 --> 00:02:52,000
Now, given such a
polynomial there

59
00:02:52,000 --> 00:02:55,330
are three typical
things we want to do.

60
00:02:55,330 --> 00:02:58,670
So these are the
operations on polynomials.

61
00:03:04,730 --> 00:03:08,800
I'll say why we want
to do them in a second.

62
00:03:08,800 --> 00:03:11,740
So the obvious one, if
you do care about x,

63
00:03:11,740 --> 00:03:14,650
is some kind of evaluation.

64
00:03:14,650 --> 00:03:18,140
So maybe I give you
a polynomial-- a

65
00:03:18,140 --> 00:03:23,390
of x-- and I give you a
number-- let's call it x0--

66
00:03:23,390 --> 00:03:26,660
and I want to compute
what is a of x0.

67
00:03:26,660 --> 00:03:29,600
So if I plug in-- x here
is a general variable,

68
00:03:29,600 --> 00:03:31,490
but if I give you an
actual real number,

69
00:03:31,490 --> 00:03:36,060
say, for the x's, what
does that add up to?

70
00:03:39,070 --> 00:03:41,390
So, how would you
solve evaluation

71
00:03:41,390 --> 00:03:43,220
before we go to the
other operations?

72
00:03:48,230 --> 00:03:50,300
There's an obvious way.

73
00:03:50,300 --> 00:03:53,580
We, you know-- you compute
all the terms and add them up,

74
00:03:53,580 --> 00:03:59,190
but if you do that
naively, computing

75
00:03:59,190 --> 00:04:03,700
x to the k-- in this form,
maybe-- computing x to the k

76
00:04:03,700 --> 00:04:06,540
maybe takes k
multiplications, and so

77
00:04:06,540 --> 00:04:08,139
the total runtime
of the quadratic,

78
00:04:08,139 --> 00:04:09,430
but we can do better than that.

79
00:04:15,307 --> 00:04:17,640
I know it's 11 o'clock-- too
early in the morning think.

80
00:04:25,850 --> 00:04:26,654
Yeah.

81
00:04:26,654 --> 00:04:28,820
AUDIENCE: When you've already
calculated x to the i,

82
00:04:28,820 --> 00:04:31,030
you can calculate x to the
i plus 1 by multiplying it.

83
00:04:31,030 --> 00:04:31,390
ERIK DEMAINE: Good.

84
00:04:31,390 --> 00:04:33,230
Once you've computed
x to the k, you

85
00:04:33,230 --> 00:04:36,920
can compute x to the k plus
1 with one multiplication,

86
00:04:36,920 --> 00:04:42,150
and so you can compute all
x to n's in linear time

87
00:04:42,150 --> 00:04:48,300
and then you're basically
doing a dot product between x

88
00:04:48,300 --> 00:04:51,330
to the k's and the a vector.

89
00:04:51,330 --> 00:04:52,785
Cool, my first Frisbee.

90
00:04:56,510 --> 00:04:59,550
So, that's one way to do it.

91
00:04:59,550 --> 00:05:01,360
There's a slightly
slicker way to write

92
00:05:01,360 --> 00:05:02,850
it called Horner's
Rule, but it's

93
00:05:02,850 --> 00:05:04,641
doing exactly the same
thing that you said.

94
00:05:08,410 --> 00:05:11,950
It's a-- this is just a nice
algebraic way of writing it.

95
00:05:11,950 --> 00:05:18,580
a of x is the same as a0 plus
x times a1 plus x times a2

96
00:05:18,580 --> 00:05:21,340
plus, and so on.

97
00:05:21,340 --> 00:05:28,250
x times a n minus 1, and then
lots of-- close parentheses.

98
00:05:28,250 --> 00:05:31,380
So this is, of course,
equivalent to that expression

99
00:05:31,380 --> 00:05:36,440
by the law of distribution,
and this is essentially doing

100
00:05:36,440 --> 00:05:38,940
the x-products one at a time.

101
00:05:38,940 --> 00:05:41,890
So, this is clearly order n
additions and multiplication,

102
00:05:41,890 --> 00:05:45,530
so we get order n time.

103
00:05:45,530 --> 00:05:48,260
In this lecture,
time is the number

104
00:05:48,260 --> 00:05:50,065
of arithmetic operations.

105
00:05:50,065 --> 00:05:50,950
That's our model.

106
00:05:50,950 --> 00:05:54,850
Assume it takes constant
time to multiply or add

107
00:05:54,850 --> 00:05:55,675
two real numbers.

108
00:06:00,971 --> 00:06:01,470
OK.

109
00:06:04,550 --> 00:06:05,050
Cool.

110
00:06:05,050 --> 00:06:07,850
So, evaluation-- that's easy.

111
00:06:07,850 --> 00:06:10,330
Linear time, for today, is good.

112
00:06:10,330 --> 00:06:12,260
Quadratic is bad.

113
00:06:12,260 --> 00:06:14,200
We want to beat quadratic.

114
00:06:14,200 --> 00:06:16,690
OK, second thing you might
want to do with polynomials

115
00:06:16,690 --> 00:06:18,430
is add them.

116
00:06:18,430 --> 00:06:21,280
The third thing
is multiply them.

117
00:06:21,280 --> 00:06:25,220
So, we're given two
polynomials-- a of x and b

118
00:06:25,220 --> 00:06:29,950
of x-- and we want to compute
a new polynomial-- c of x--

119
00:06:29,950 --> 00:06:31,730
that is the summation.

120
00:06:31,730 --> 00:06:33,900
How do you define the summation?

121
00:06:33,900 --> 00:06:36,570
Well, you would like
c of x to equal a of x

122
00:06:36,570 --> 00:06:39,390
plus b of x for all x.

123
00:06:39,390 --> 00:06:41,010
That's the definition.

124
00:06:41,010 --> 00:06:43,570
Of course, we can do it
algebraically as well

125
00:06:43,570 --> 00:06:47,110
because these are
numbers in the end--

126
00:06:47,110 --> 00:06:49,610
for any x this
evaluates to a number--

127
00:06:49,610 --> 00:06:51,970
so if we add two
polynomials of this form--

128
00:06:51,970 --> 00:06:56,660
one with ai's, one with the b
i's-- all we're doing is adding

129
00:06:56,660 --> 00:06:59,470
corresponding ai's and b i's.

130
00:06:59,470 --> 00:07:02,180
So, this is easy.

131
00:07:02,180 --> 00:07:09,860
We just need ck to equal
ak plus bk for all k.

132
00:07:09,860 --> 00:07:14,050
So, again, linear
time-- no problem.

133
00:07:14,050 --> 00:07:22,450
Third operation is the exciting
one, the hard one to get good,

134
00:07:22,450 --> 00:07:26,840
otherwise this lecture would be
over in a couple more minutes.

135
00:07:26,840 --> 00:07:28,930
So multiplication-- same deal.

136
00:07:28,930 --> 00:07:31,730
We're given a of x
and b of x, and we

137
00:07:31,730 --> 00:07:36,640
want to convert that into
some c of x that, for all x,

138
00:07:36,640 --> 00:07:41,225
evaluates to the product
of those two polynomials.

139
00:07:44,720 --> 00:07:47,750
How do we do this?

140
00:07:47,750 --> 00:07:53,040
We can't just multiply
corresponding ak's and bk's.

141
00:07:53,040 --> 00:07:54,830
In fact, if you take
a big thing like this

142
00:07:54,830 --> 00:07:57,520
and you multiply it by
corresponding big thing-- eh,

143
00:07:57,520 --> 00:07:58,030
let's do it.

144
00:08:11,328 --> 00:08:12,453
This doesn't look like fun.

145
00:08:17,800 --> 00:08:19,020
We get-- let's see.

146
00:08:19,020 --> 00:08:22,560
So, the constant term is just
the product-- that's easy,

147
00:08:22,560 --> 00:08:25,070
the constant terms--
but then, if I

148
00:08:25,070 --> 00:08:28,360
take this product or this
product, I get linear terms.

149
00:08:28,360 --> 00:08:36,330
So it's going to be
a1b0 plus a0b1 times x,

150
00:08:36,330 --> 00:08:38,370
and then there's
a quadratic term

151
00:08:38,370 --> 00:08:44,960
which I get from-- switch
colors-- this and this

152
00:08:44,960 --> 00:08:46,170
and this.

153
00:08:46,170 --> 00:08:49,330
So there's three
things times x squared,

154
00:08:49,330 --> 00:08:51,280
and that's where I get tired.

155
00:08:51,280 --> 00:08:55,875
I'm going to switch to
the summation notation.

156
00:08:55,875 --> 00:08:58,500
I didn't go to high school, but
I assume in high school algebra

157
00:08:58,500 --> 00:09:00,720
you learn this.

158
00:09:00,720 --> 00:09:10,180
ck is the sum of
j equals 0 to k.

159
00:09:10,180 --> 00:09:13,770
ajbk minus j.

160
00:09:13,770 --> 00:09:17,180
That's the general form
because aj came from an x

161
00:09:17,180 --> 00:09:21,290
to the j term, bk minus j came
from x to the k minus j term.

162
00:09:21,290 --> 00:09:23,970
When you multiply those
together, you get x to the k,

163
00:09:23,970 --> 00:09:26,980
so this is the
coefficient of x to the k.

164
00:09:26,980 --> 00:09:28,830
Cool?

165
00:09:28,830 --> 00:09:31,060
So, that's what we'd
like to compute.

166
00:09:31,060 --> 00:09:34,340
Given a and b we want to
compute this polynomial c.

167
00:09:34,340 --> 00:09:35,280
How long does it take?

168
00:09:38,816 --> 00:09:41,780
We have to do this for all k.

169
00:09:41,780 --> 00:09:45,210
So, to compute the k-th
term takes order k time,

170
00:09:45,210 --> 00:09:46,660
so the total time is n squared.

171
00:09:52,370 --> 00:09:55,620
So, that's not good
for this lecture.

172
00:09:55,620 --> 00:09:57,240
We want to do better.

173
00:09:57,240 --> 00:10:03,920
In fact, today we
will achieve n log n.

174
00:10:06,560 --> 00:10:11,230
That's our goal-- polynomial
multiplication in n log n.

175
00:10:11,230 --> 00:10:14,120
Why do we care about
polynomial multiplication?

176
00:10:14,120 --> 00:10:16,770
Because it's equivalent
to another operation which

177
00:10:16,770 --> 00:10:20,900
we use all the time in
digital signal processing,

178
00:10:20,900 --> 00:10:23,530
image editing, all sorts
of different things,

179
00:10:23,530 --> 00:10:24,790
which is convolution.

180
00:10:32,910 --> 00:10:35,640
Convolution is usually thought
of as an operation on vectors.

181
00:10:44,450 --> 00:10:48,079
So, remember this
vector notation,

182
00:10:48,079 --> 00:10:50,120
where we're just thinking
about the coefficients.

183
00:10:50,120 --> 00:10:51,420
x's are kind of irrelevant.

184
00:10:51,420 --> 00:10:54,380
We're just thinking about
a sequence of real numbers.

185
00:10:54,380 --> 00:10:58,790
So, maybe that sequence of
real numbers for a represents

186
00:10:58,790 --> 00:10:59,930
waveform.

187
00:10:59,930 --> 00:11:03,080
Maybe this is the
audio I'm speaking now.

188
00:11:03,080 --> 00:11:05,940
And then I take
some other waveform.

189
00:11:05,940 --> 00:11:09,890
Here I have a Gaussian
function-- e to the minus

190
00:11:09,890 --> 00:11:15,080
x squared-- and I want to
take-- for all possible shifts

191
00:11:15,080 --> 00:11:17,290
of this Gaussian
I want to compute

192
00:11:17,290 --> 00:11:19,650
the dot product
between the blackboard

193
00:11:19,650 --> 00:11:21,325
and the piece of paper.

194
00:11:21,325 --> 00:11:22,950
That's some kind of
smoothing function,

195
00:11:22,950 --> 00:11:26,342
if I wanted to clean up
noise or something like that.

196
00:11:26,342 --> 00:11:28,550
You can do the same thing
on a two-dimensional image.

197
00:11:28,550 --> 00:11:30,060
It's a little harder
to think about,

198
00:11:30,060 --> 00:11:31,370
but you can map a
two-dimensional image

199
00:11:31,370 --> 00:11:32,750
to a one-dimensional vector.

200
00:11:32,750 --> 00:11:34,458
And you have a
two-dimensional Gaussian--

201
00:11:34,458 --> 00:11:36,920
if you ever do Gaussian
blur in Photoshop

202
00:11:36,920 --> 00:11:39,530
this is what you're
doing-- a convolution--

203
00:11:39,530 --> 00:11:43,910
and it's used to pretend that
your lens is out of focus

204
00:11:43,910 --> 00:11:45,640
when you do that.

205
00:11:45,640 --> 00:11:50,090
It's done in audio processing,
and all sorts of things.

206
00:11:50,090 --> 00:11:54,210
So, formally, you're
given two vectors

207
00:11:54,210 --> 00:11:57,480
and you want to take all
possible shifts of one vector

208
00:11:57,480 --> 00:12:00,200
and take the dot product
with the other one.

209
00:12:00,200 --> 00:12:01,880
I have that written down.

210
00:12:01,880 --> 00:12:07,140
I just-- dot product, same
thing as inner product,

211
00:12:07,140 --> 00:12:11,244
which just means multiply
corresponding positions

212
00:12:11,244 --> 00:12:11,910
and add them up.

213
00:12:22,260 --> 00:12:24,544
And, if you ignore
this minus sign,

214
00:12:24,544 --> 00:12:25,960
that's exactly
what this is doing.

215
00:12:25,960 --> 00:12:30,810
This is taking aj versus
bk, pretend it's plus j.

216
00:12:30,810 --> 00:12:34,650
So, that's the bj vector, but
with all possible shifts k.

217
00:12:34,650 --> 00:12:38,120
We compute this for all k.

218
00:12:38,120 --> 00:12:39,780
That's really cool.

219
00:12:39,780 --> 00:12:42,120
We're going to compute
it in n log n time.

220
00:12:42,120 --> 00:12:45,309
All different n shifts of b will
take the dot product with a.

221
00:12:45,309 --> 00:12:46,850
It's kind of magical
because it looks

222
00:12:46,850 --> 00:12:48,280
like you're doing
n squared work,

223
00:12:48,280 --> 00:12:50,030
but we will do it
in n log n time.

224
00:12:53,160 --> 00:12:56,210
The only issue is we
have to reverse b.

225
00:12:56,210 --> 00:12:58,809
Then the minus signs
turn into plus signs.

226
00:12:58,809 --> 00:13:00,350
And there's some
boundary conditions,

227
00:13:00,350 --> 00:13:01,710
but it's basically the same.

228
00:13:01,710 --> 00:13:03,860
If we can solve
polynomial multiplication,

229
00:13:03,860 --> 00:13:05,253
we can solve convolution.

230
00:13:05,253 --> 00:13:07,040
Cool?

231
00:13:07,040 --> 00:13:09,430
So, that's why we care
about multiplication.

232
00:13:12,040 --> 00:13:15,190
So, how are we
going to solve this?

233
00:13:15,190 --> 00:13:18,760
What I'd like to do is talk
about alternate representations

234
00:13:18,760 --> 00:13:19,650
of polynomials.

235
00:13:19,650 --> 00:13:21,270
That's the next thing here.

236
00:13:21,270 --> 00:13:22,730
We just did operations.

237
00:13:22,730 --> 00:13:25,140
Let's talk about
different representations.

238
00:13:25,140 --> 00:13:27,460
So, we talked about this
one representation--

239
00:13:27,460 --> 00:13:30,177
it's one way to
represent polynomial--

240
00:13:30,177 --> 00:13:31,260
but it's not the only one.

241
00:13:31,260 --> 00:13:32,310
You probably know others.

242
00:13:35,490 --> 00:13:43,890
So, on the one hand we
have representation a

243
00:13:43,890 --> 00:13:45,920
is a coefficient vector.

244
00:13:49,530 --> 00:13:52,280
So we can write down the ai's.

245
00:13:52,280 --> 00:13:56,900
That was just one way to
represent a polynomial.

246
00:13:56,900 --> 00:13:59,730
Can anyone give me another
way to represent a polynomial?

247
00:13:59,730 --> 00:14:01,420
I have two ways in mind.

248
00:14:01,420 --> 00:14:01,920
Yeah.

249
00:14:01,920 --> 00:14:03,174
AUDIENCE: Generating function?

250
00:14:03,174 --> 00:14:04,590
ERIK DEMAINE:
Generating function.

251
00:14:04,590 --> 00:14:10,150
Isn't that the same-- oh,
I guess in principle you

252
00:14:10,150 --> 00:14:12,900
could imagine writing a
recurrence on the generating

253
00:14:12,900 --> 00:14:14,320
function or something.

254
00:14:14,320 --> 00:14:14,970
It's plausible.

255
00:14:14,970 --> 00:14:16,480
In general, generating
functions are

256
00:14:16,480 --> 00:14:19,580
polynomials if you know
what that-- they are cool.

257
00:14:19,580 --> 00:14:21,430
So it doesn't quite
answer the question.

258
00:14:21,430 --> 00:14:21,930
Yeah?

259
00:14:21,930 --> 00:14:22,829
AUDIENCE: [INAUDIBLE]

260
00:14:22,829 --> 00:14:23,620
ERIK DEMAINE: Sure.

261
00:14:23,620 --> 00:14:24,690
AUDIENCE: Representation.

262
00:14:24,690 --> 00:14:25,523
ERIK DEMAINE: Sorry?

263
00:14:25,523 --> 00:14:26,272
AUDIENCE: Points?

264
00:14:26,272 --> 00:14:27,980
ERIK DEMAINE: Point
representation, yeah.

265
00:14:27,980 --> 00:14:30,860
I call them samples.

266
00:14:30,860 --> 00:14:36,740
I'm going to put that under
C. A bunch of samples, a bunch

267
00:14:36,740 --> 00:14:39,240
of points on the polynomial.

268
00:14:39,240 --> 00:14:52,050
So like xk, yk for-- how many
do we need-- I think n minus 1

269
00:14:52,050 --> 00:14:53,500
should do it.

270
00:14:53,500 --> 00:14:54,550
We'll check.

271
00:14:54,550 --> 00:14:55,780
Yep.

272
00:14:55,780 --> 00:15:00,700
And if we are told
that a of xk equals yk,

273
00:15:00,700 --> 00:15:07,450
and we're told that all
the xk's are distinct,

274
00:15:07,450 --> 00:15:10,860
then this uniquely
determines the polynomial.

275
00:15:10,860 --> 00:15:13,780
You have a degree n
minus one polynomial,

276
00:15:13,780 --> 00:15:15,680
and you have n samples.

277
00:15:15,680 --> 00:15:17,410
There's only one
polynomial that passes

278
00:15:17,410 --> 00:15:20,620
through all those points.

279
00:15:20,620 --> 00:15:24,890
It's a consequence of the
fundamental theorem of algebra

280
00:15:24,890 --> 00:15:26,490
that gives you uniqueness.

281
00:15:26,490 --> 00:15:30,097
Existence, I think, was
proved by Legendre in 1700s,

282
00:15:30,097 --> 00:15:33,059
1800s-- a long time ago.

283
00:15:33,059 --> 00:15:33,600
This is good.

284
00:15:33,600 --> 00:15:35,310
This is what we're going to use.

285
00:15:35,310 --> 00:15:36,603
There's another answer.

286
00:15:36,603 --> 00:15:39,970
I should give you
a Frisbee first.

287
00:15:39,970 --> 00:15:42,330
You look so excited.

288
00:15:42,330 --> 00:15:46,140
Just wait till I hit you,
then you'll be less excited.

289
00:15:46,140 --> 00:15:47,420
OK, so.

290
00:15:47,420 --> 00:15:50,780
Samples, coefficients,
anything else?

291
00:15:50,780 --> 00:15:51,280
Yeah.

292
00:15:51,280 --> 00:15:52,043
AUDIENCE: Roots?

293
00:15:52,043 --> 00:15:53,126
ERIK DEMAINE: Roots, yeah.

294
00:15:56,320 --> 00:15:58,790
Roots is the other
answer I was looking for,

295
00:15:58,790 --> 00:16:01,951
but it's not going to be so good
algorithmically, as we'll see.

296
00:16:01,951 --> 00:16:02,450
Sorry.

297
00:16:05,076 --> 00:16:06,700
So, I can give you
a sequence of roots.

298
00:16:06,700 --> 00:16:08,450
This is the fundamental
theorem of algebra

299
00:16:08,450 --> 00:16:11,590
that every polynomial
is uniquely determined

300
00:16:11,590 --> 00:16:12,980
by its set of roots.

301
00:16:12,980 --> 00:16:15,680
If you allow roots
with multiplicity then

302
00:16:15,680 --> 00:16:19,420
every polynomial of degree
n has exactly n roots.

303
00:16:19,420 --> 00:16:27,686
So, this would be some sequence
of r1 up to rn minus 1,

304
00:16:27,686 --> 00:16:29,810
and the polynomial would
be given as-- you actually

305
00:16:29,810 --> 00:16:36,920
need a constant multiplier--
but x minus r0, x minus r1.

306
00:16:36,920 --> 00:16:39,800
That would be polynomial.

307
00:16:39,800 --> 00:16:41,990
The trouble with roots
is that if I give you

308
00:16:41,990 --> 00:16:45,290
a coefficient vector and I
want to compute the roots,

309
00:16:45,290 --> 00:16:48,620
not only is it hard to do, it's
impossible to do in our model.

310
00:16:48,620 --> 00:16:51,020
If you're only allowed to
add, subtract, multiply,

311
00:16:51,020 --> 00:16:54,340
divide, take square
roots, take k-th roots,

312
00:16:54,340 --> 00:16:57,770
there is no way to solve
a polynomial of degree

313
00:16:57,770 --> 00:16:58,930
5 or larger.

314
00:16:58,930 --> 00:17:01,690
There's the quadratic formula,
cubic formula, quartic formula.

315
00:17:01,690 --> 00:17:03,060
There is no quintic formula.

316
00:17:03,060 --> 00:17:05,250
That's an old result. 1800s.

317
00:17:05,250 --> 00:17:08,420
So, going from coefficient
vector to roots

318
00:17:08,420 --> 00:17:10,119
takes infinite time.

319
00:17:10,119 --> 00:17:11,530
It's not so good.

320
00:17:11,530 --> 00:17:14,859
And, in particular, if we
think about our operations,

321
00:17:14,859 --> 00:17:18,000
addition becomes
really difficult.

322
00:17:18,000 --> 00:17:22,621
Multiplication is easy if I
have two polynomials represented

323
00:17:22,621 --> 00:17:23,579
as a sequence of roots.

324
00:17:23,579 --> 00:17:24,579
I want to multiply them.

325
00:17:24,579 --> 00:17:27,540
That's just concatenating
the vectors--

326
00:17:27,540 --> 00:17:31,676
taking union the vectors
of their root lists.

327
00:17:31,676 --> 00:17:32,300
So that's cool.

328
00:17:32,300 --> 00:17:34,150
I'm multiplying
the c's, I guess.

329
00:17:34,150 --> 00:17:36,190
But addition is really
hard because addition

330
00:17:36,190 --> 00:17:39,670
is sort of fundamentally
about coefficient vectors.

331
00:17:39,670 --> 00:17:42,455
And then, once
you go there-- you

332
00:17:42,455 --> 00:17:44,830
can go from roots to coefficient
vectors and add them up,

333
00:17:44,830 --> 00:17:46,746
but then there's no
relation between the roots

334
00:17:46,746 --> 00:17:49,193
of the sum of the
polynomials versus the roots

335
00:17:49,193 --> 00:17:51,975
of the original.

336
00:17:51,975 --> 00:17:53,850
I don't know for sure
that that's impossible.

337
00:17:53,850 --> 00:17:56,505
It's definitely very, very
hard, probably impossible.

338
00:18:02,200 --> 00:18:04,940
Let me draw a little table.

339
00:18:04,940 --> 00:18:08,170
Each of these representations
has some advantages

340
00:18:08,170 --> 00:18:12,700
and a disadvantage in terms
of these three operations.

341
00:18:12,700 --> 00:18:18,670
So, on the one hand,
we have the algorithms

342
00:18:18,670 --> 00:18:38,790
we care about-- evaluation,
addition, and multiplication--

343
00:18:38,790 --> 00:18:42,400
and on the other axis we have
our representations, which

344
00:18:42,400 --> 00:18:52,074
are coefficient vectors,
roots, and samples.

345
00:18:52,074 --> 00:18:53,990
You'll see why I chose
this order in a moment.

346
00:18:53,990 --> 00:18:55,448
It makes for a
nice, pretty matrix.

347
00:19:03,950 --> 00:19:07,260
We've talked about almost
every cell in this matrix,

348
00:19:07,260 --> 00:19:08,950
but let me just summarize.

349
00:19:08,950 --> 00:19:12,310
We started out just thinking
about coefficient vector,

350
00:19:12,310 --> 00:19:16,320
and evaluation was linear time.

351
00:19:16,320 --> 00:19:18,600
Addition was linear time.

352
00:19:18,600 --> 00:19:20,760
Multiplication, so
far, was quadratic,

353
00:19:20,760 --> 00:19:23,820
although our goal
is to make n log n.

354
00:19:23,820 --> 00:19:28,670
For roots, I just said
multiplication is easy.

355
00:19:28,670 --> 00:19:34,381
That's linear time,
addition is really hard--

356
00:19:34,381 --> 00:19:38,920
like, infinite time--
and evaluation,

357
00:19:38,920 --> 00:19:41,910
I guess that's linear time.

358
00:19:41,910 --> 00:19:43,560
In fact, the way it's
written, you only

359
00:19:43,560 --> 00:19:45,070
have a linear number
of subtractions

360
00:19:45,070 --> 00:19:49,330
and multiplications, so it's
really easy to evaluate.

361
00:19:49,330 --> 00:19:55,100
And then sample vectors-- we
haven't talked much about that.

362
00:19:55,100 --> 00:19:58,160
The idea is, suppose you're
given two polynomials with

363
00:19:58,160 --> 00:19:59,840
the same xk's.

364
00:19:59,840 --> 00:20:00,980
We're going to fix xk's.

365
00:20:00,980 --> 00:20:02,521
All we need is that
they're distinct.

366
00:20:02,521 --> 00:20:04,650
So xk could equal
k, for example,

367
00:20:04,650 --> 00:20:06,230
just a bunch of integers.

368
00:20:06,230 --> 00:20:10,270
And then we are told what
polynomial a evaluates

369
00:20:10,270 --> 00:20:12,330
to at every xk, and we're
told what polynomial

370
00:20:12,330 --> 00:20:14,940
b evaluates to at every xk.

371
00:20:14,940 --> 00:20:18,790
So, we're given some yk's
and some zk's, and then we

372
00:20:18,790 --> 00:20:23,970
want to compute, say, the sum
or the product of those two

373
00:20:23,970 --> 00:20:24,850
vectors.

374
00:20:24,850 --> 00:20:26,580
What do we do?

375
00:20:26,580 --> 00:20:32,030
Just add or multiply the
corresponding yk and zk's,

376
00:20:32,030 --> 00:20:36,290
because if we're
told we want c of x

377
00:20:36,290 --> 00:20:39,390
to equal a of x times b of
x, or c of x to equal a of x

378
00:20:39,390 --> 00:20:42,760
plus b of x for all x, Well, now
we know what x's we care about.

379
00:20:42,760 --> 00:20:44,600
We just do it at the xk's.

380
00:20:44,600 --> 00:20:46,590
That's what we're
told for a and for b,

381
00:20:46,590 --> 00:20:50,275
and so to compute c of
xk it's just the sum

382
00:20:50,275 --> 00:20:53,290
or the product of yk and zk.

383
00:20:53,290 --> 00:20:56,180
So multiplication is really
easy in the sample view,

384
00:20:56,180 --> 00:20:59,460
and this is why we are
going to use this view.

385
00:20:59,460 --> 00:21:02,270
We're also going to use this
view because, as we'll see,

386
00:21:02,270 --> 00:21:04,270
there's a problem.

387
00:21:04,270 --> 00:21:08,800
Addition is easy,
multiplication is easy,

388
00:21:08,800 --> 00:21:13,130
evaluation is annoying.

389
00:21:13,130 --> 00:21:19,010
I can evaluate a of
x at xk for any k,

390
00:21:19,010 --> 00:21:23,880
but I can't evaluate it at
some arbitrary value of x.

391
00:21:23,880 --> 00:21:25,500
That's annoying.

392
00:21:25,500 --> 00:21:27,210
I'm told at these
finite sample points,

393
00:21:27,210 --> 00:21:29,100
but now I have to
somehow interpolate.

394
00:21:29,100 --> 00:21:31,170
This is called
polynomial interpolation,

395
00:21:31,170 --> 00:21:33,620
well studied in numerical
analysis and so on.

396
00:21:33,620 --> 00:21:38,960
You can do it, but it takes
quadratic time, in general.

397
00:21:38,960 --> 00:21:40,800
The best known
algorithms are quadratic.

398
00:21:43,890 --> 00:21:52,020
So, this is bad, this
is bad, and this is bad,

399
00:21:52,020 --> 00:21:56,380
so no representation is perfect.

400
00:21:56,380 --> 00:21:59,100
Life sucks.

401
00:21:59,100 --> 00:22:00,789
What we'd like is to
get the best-- now

402
00:22:00,789 --> 00:22:03,330
this one is really hard to work
with because converting inter

403
00:22:03,330 --> 00:22:08,210
roots is impossible in
an arithmetic model,

404
00:22:08,210 --> 00:22:11,530
so we're going to focus
on column A and column

405
00:22:11,530 --> 00:22:15,860
C. We kind of like to take
the min of those two columns.

406
00:22:15,860 --> 00:22:17,440
We won't quite get that.

407
00:22:17,440 --> 00:22:20,620
What we will get is an
algorithm for converting

408
00:22:20,620 --> 00:22:23,720
between these two
representations in n log n

409
00:22:23,720 --> 00:22:28,110
time-- it's not quite
linear, but close--

410
00:22:28,110 --> 00:22:31,080
and once we can do that,
if we want to multiply two

411
00:22:31,080 --> 00:22:35,090
things in the coefficient land
we can convert to sample land,

412
00:22:35,090 --> 00:22:38,070
do it in linear time,
and then convert back.

413
00:22:38,070 --> 00:22:41,230
So that's the magical
transformation

414
00:22:41,230 --> 00:22:43,861
we're going to cover, and it
is called the fast Fourier

415
00:22:43,861 --> 00:22:44,360
transfer.

416
00:22:44,360 --> 00:22:46,390
Fast Fourier transform
is the algorithm.

417
00:22:46,390 --> 00:22:49,691
Discrete Fourier transform
is that transformation

418
00:22:49,691 --> 00:22:50,316
mathematically.

419
00:22:53,172 --> 00:22:54,150
Cool.

420
00:22:54,150 --> 00:22:56,990
So the whole name of
the game is converting

421
00:22:56,990 --> 00:23:00,720
from coefficient representation
to samples, or vice versa.

422
00:23:00,720 --> 00:23:02,180
Turns out they're
almost the same,

423
00:23:02,180 --> 00:23:04,150
though that won't be
obvious for a long time--

424
00:23:04,150 --> 00:23:05,920
till the end of the class.

425
00:23:05,920 --> 00:23:08,834
Any questions before we proceed?

426
00:23:08,834 --> 00:23:09,808
Yeah.

427
00:23:09,808 --> 00:23:11,680
AUDIENCE: [INAUDIBLE]
multiply repetitions,

428
00:23:11,680 --> 00:23:15,564
why not we evaluate a and
b first then multiply?

429
00:23:15,564 --> 00:23:16,550
ERIK DEMAINE: Ah.

430
00:23:16,550 --> 00:23:20,700
So, OK, the question
is, if I want

431
00:23:20,700 --> 00:23:22,730
to-- we'll get
there in a second--

432
00:23:22,730 --> 00:23:25,110
but if I want to
multiply, and it's

433
00:23:25,110 --> 00:23:28,480
so easy to do in sample land,
why don't I just sample a and b

434
00:23:28,480 --> 00:23:29,550
and then multiply them?

435
00:23:29,550 --> 00:23:32,690
That's right, but [? effect ?]
sampling is not so easy.

436
00:23:32,690 --> 00:23:34,520
It takes quadratic time.

437
00:23:34,520 --> 00:23:35,770
Let's go there now.

438
00:23:43,290 --> 00:23:47,950
Because we have n samples to do,
each one will cost linear time.

439
00:23:47,950 --> 00:23:51,160
Remember, to evaluate a
polynomial takes linear time.

440
00:23:51,160 --> 00:23:53,600
If you want to think
of it in a matrix--

441
00:23:53,600 --> 00:24:00,725
let's enter the matrix--
then we get a big matrix.

442
00:24:13,920 --> 00:24:16,510
So we're given the
xi's and we just

443
00:24:16,510 --> 00:24:20,120
want to evaluate a given
polynomial whose coefficients

444
00:24:20,120 --> 00:24:27,060
are given by a0,
a1, a2, and minus 1.

445
00:24:27,060 --> 00:24:33,310
Our goal is to compute
the yi's-- y0, y1, y2,

446
00:24:33,310 --> 00:24:38,880
to yn minus 1-- and if you
know a matrix-vector product,

447
00:24:38,880 --> 00:24:40,974
you take this row
with that column.

448
00:24:40,974 --> 00:24:42,640
You take the dot
product that multiplies

449
00:24:42,640 --> 00:24:44,110
corresponding entries.

450
00:24:44,110 --> 00:24:45,130
You get y0.

451
00:24:45,130 --> 00:24:47,960
That is the definition of
the polynomial evaluation.

452
00:24:47,960 --> 00:24:51,342
I'm just going to write
a bunch of these rows

453
00:24:51,342 --> 00:24:52,300
so you get the pattern.

454
00:24:58,527 --> 00:25:01,200
Pretty simple.

455
00:25:09,580 --> 00:25:13,270
This is called the
Vandermonde matrix.

456
00:25:20,560 --> 00:25:26,698
I'll call it V. And in general--
I don't have room for it,

457
00:25:26,698 --> 00:25:27,281
so let me go--

458
00:25:40,560 --> 00:25:47,260
In general, if we look at
V ij, it's just-- sorry.

459
00:25:47,260 --> 00:25:49,800
You may notice I'm not
using the letter i.

460
00:25:49,800 --> 00:25:52,090
We will get to why in a moment.

461
00:25:52,090 --> 00:25:53,860
V jk.

462
00:25:53,860 --> 00:25:55,960
Row j, column k.

463
00:25:55,960 --> 00:25:59,670
That's going to be x
sub j to the power k.

464
00:25:59,670 --> 00:26:02,710
That's the Vandermonde matrix.

465
00:26:02,710 --> 00:26:04,599
We can compute it
in quadratic time.

466
00:26:04,599 --> 00:26:05,640
It has quadratic entries.

467
00:26:05,640 --> 00:26:07,730
We can use the trick
we suggested earlier--

468
00:26:07,730 --> 00:26:10,590
compute each term
from the previous one

469
00:26:10,590 --> 00:26:14,470
by multiplying by
xj-- and then we

470
00:26:14,470 --> 00:26:17,910
want to compute this
matrix-vector product,

471
00:26:17,910 --> 00:26:20,820
and you can clearly do
it in quadratic time--

472
00:26:20,820 --> 00:26:23,800
I'm just computing each
thing correspondingly--

473
00:26:23,800 --> 00:26:25,310
and that's sort of
the best you can

474
00:26:25,310 --> 00:26:26,790
do without any
further assumptions.

475
00:26:26,790 --> 00:26:30,090
So this takes-- if
I want to compute

476
00:26:30,090 --> 00:26:35,770
this product, that's the
coefficients to samples

477
00:26:35,770 --> 00:26:38,210
problem.

478
00:26:38,210 --> 00:26:45,860
This is the same thing as
computing V times the A vector,

479
00:26:45,860 --> 00:26:48,440
so this is a matrix-vector
multiplication,

480
00:26:48,440 --> 00:26:50,630
which takes n squared time.

481
00:26:54,100 --> 00:26:55,000
OK?

482
00:26:55,000 --> 00:26:58,110
On the other hand--
so, that's a problem

483
00:26:58,110 --> 00:27:00,730
because we're trying to beat
quadratic multiplication,

484
00:27:00,730 --> 00:27:02,880
so if we spend quadratic
time to convert over here

485
00:27:02,880 --> 00:27:04,740
it doesn't matter if
this is linear time.

486
00:27:04,740 --> 00:27:05,450
There are two problems.

487
00:27:05,450 --> 00:27:07,033
One is that conversion
costs too much.

488
00:27:07,033 --> 00:27:09,970
The other is we don't yet
know how to convert backwards.

489
00:27:09,970 --> 00:27:11,860
But this matrix
field gives us also

490
00:27:11,860 --> 00:27:13,620
the reverse transformation.

491
00:27:13,620 --> 00:27:21,310
If we want to convert
samples to coefficients,

492
00:27:21,310 --> 00:27:26,190
this is-- the best notation
I know is from MATLAB.

493
00:27:26,190 --> 00:27:28,036
How many people know MATLAB?

494
00:27:28,036 --> 00:27:28,820
A bunch.

495
00:27:28,820 --> 00:27:32,780
So for you, it's V
backslash A, but usually

496
00:27:32,780 --> 00:27:34,760
in linear algebra
like 18.06 you see

497
00:27:34,760 --> 00:27:40,540
you have some matrix V
times some unknown vector--

498
00:27:40,540 --> 00:27:42,525
usually it's called x,
here it's called a--

499
00:27:42,525 --> 00:27:44,380
and you know the
right-hand side.

500
00:27:44,380 --> 00:27:45,710
You want to solve for this.

501
00:27:45,710 --> 00:27:46,420
How do you do it?

502
00:27:49,619 --> 00:27:50,990
AUDIENCE: [INAUDIBLE]

503
00:27:50,990 --> 00:27:52,250
ERIK DEMAINE: Sorry?

504
00:27:52,250 --> 00:27:53,930
AUDIENCE: Multiply
by the inverse?

505
00:27:53,930 --> 00:27:55,050
ERIK DEMAINE: Multiply
by the inverse.

506
00:27:55,050 --> 00:27:55,549
Yeah.

507
00:27:55,549 --> 00:27:57,350
How do you do it in
computer science?

508
00:27:57,350 --> 00:27:58,130
AUDIENCE: Gaussian elimination.

509
00:27:58,130 --> 00:27:59,250
ERIK DEMAINE:
Gaussian elimination.

510
00:27:59,250 --> 00:28:01,215
Turns out inverse is
the right answer here,

511
00:28:01,215 --> 00:28:04,020
but Gaussian elimination
would be the standard way

512
00:28:04,020 --> 00:28:06,360
to solve a linear
system like that.

513
00:28:06,360 --> 00:28:09,250
The trouble with
Gaussian elimination

514
00:28:09,250 --> 00:28:16,080
is it takes cubic time
in its normal form.

515
00:28:16,080 --> 00:28:17,780
In this case it's a
little bit special

516
00:28:17,780 --> 00:28:19,830
because this matrix
is essentially fixed.

517
00:28:19,830 --> 00:28:22,130
The xi's don't need to change.

518
00:28:22,130 --> 00:28:26,720
It could be xi is just i
or something, so we can--

519
00:28:26,720 --> 00:28:28,160
in this case it's
a little better

520
00:28:28,160 --> 00:28:30,320
to compute the inverse first.

521
00:28:30,320 --> 00:28:35,451
So we could also just
do v inverse times a.

522
00:28:35,451 --> 00:28:37,700
From a numerical analysis
standpoint this is very bad,

523
00:28:37,700 --> 00:28:39,120
but don't worry
about it for now.

524
00:28:39,120 --> 00:28:41,070
We're going to get a
better algorithm today.

525
00:28:41,070 --> 00:28:43,780
Anyway, it doesn't
involve matrices at all,

526
00:28:43,780 --> 00:28:45,980
but the nice thing
is, if computing

527
00:28:45,980 --> 00:28:47,657
the inverse, that
takes n cubed time,

528
00:28:47,657 --> 00:28:48,990
but you only have to do it once.

529
00:28:48,990 --> 00:28:51,150
So if you have to
do this many times

530
00:28:51,150 --> 00:28:53,840
you can do this product
in n squared time.

531
00:28:53,840 --> 00:28:59,300
You just have to maintain that
v inverse once and for all.

532
00:28:59,300 --> 00:29:00,250
OK, great.

533
00:29:00,250 --> 00:29:01,820
So, we've got
quadratic algorithms

534
00:29:01,820 --> 00:29:03,960
to go back and forth between
these representations.

535
00:29:03,960 --> 00:29:06,110
That, at least,
tells us it's doable,

536
00:29:06,110 --> 00:29:09,600
but we, of course,
need better than

537
00:29:09,600 --> 00:29:14,526
quadratic to improve on the
naive multiplication algorithm,

538
00:29:14,526 --> 00:29:15,900
so that's what
we're going to do.

539
00:29:18,740 --> 00:29:20,730
In general, we
can't do any better,

540
00:29:20,730 --> 00:29:24,960
but we have one freedom,
which is we have said nothing

541
00:29:24,960 --> 00:29:26,430
about the x case.

542
00:29:26,430 --> 00:29:29,770
We can choose them to be
whatever we want them to be.

543
00:29:29,770 --> 00:29:31,800
I keep saying xk equals k.

544
00:29:31,800 --> 00:29:32,930
That seems fine.

545
00:29:32,930 --> 00:29:36,200
It's actually really bad choice
for a reason we will get to,

546
00:29:36,200 --> 00:29:39,210
but there is a choice where,
magically, this transformation

547
00:29:39,210 --> 00:29:43,910
becomes easy and you can
do it in n log n time.

548
00:29:43,910 --> 00:29:48,310
Before we get there,
I want to give you

549
00:29:48,310 --> 00:29:54,160
some motivation for how
this could possibly work.

550
00:30:15,140 --> 00:30:18,680
As you might expect, even just
from that n log n running time,

551
00:30:18,680 --> 00:30:23,930
we're going to be using
divide and conquer,

552
00:30:23,930 --> 00:30:28,780
so let's just think about how
divide and conquer could work.

553
00:30:28,780 --> 00:30:31,190
I'm going to show
you an idea and then

554
00:30:31,190 --> 00:30:33,760
we'll figure out how that
idea could possibly work.

555
00:30:33,760 --> 00:30:36,190
It doesn't work at
the moment, but we

556
00:30:36,190 --> 00:30:40,210
will be able to choose
the xk so that it works.

557
00:30:40,210 --> 00:30:45,490
So, let's say the goal-- I mean,
what we want to do is compute

558
00:30:45,490 --> 00:30:47,880
this v times a.

559
00:30:47,880 --> 00:30:54,110
I'm going to convert-- think of
that back into polynomial land.

560
00:30:54,110 --> 00:31:01,270
So our goal is to compute a
of x for all x in some set x.

561
00:31:05,050 --> 00:31:10,420
This is taking a
bunch of samples.

562
00:31:10,420 --> 00:31:13,580
Set x is just a set
of the xk's, but I'm

563
00:31:13,580 --> 00:31:19,410
going to change that set in
a moment using recursion.

564
00:31:19,410 --> 00:31:22,390
So the input to this algorithm
is a polynomial a of x,

565
00:31:22,390 --> 00:31:26,520
and it's a set
capital X of positions

566
00:31:26,520 --> 00:31:28,420
that I'd like to evaluate
that polynomial at.

567
00:31:28,420 --> 00:31:30,295
This is clearly more
general than the problem

568
00:31:30,295 --> 00:31:33,450
we're trying to solve, and I'm
going to solve it with divide

569
00:31:33,450 --> 00:31:33,950
and conquer.

570
00:31:33,950 --> 00:31:35,783
In divide and conquer
there are three steps.

571
00:31:35,783 --> 00:31:39,440
Divide, conquer, and combine.

572
00:31:39,440 --> 00:31:40,640
Let's start with divide.

573
00:31:43,660 --> 00:31:44,610
Here's the big idea.

574
00:31:47,997 --> 00:31:49,580
I would say there
are two natural ways

575
00:31:49,580 --> 00:31:50,500
to divide a vector.

576
00:31:50,500 --> 00:31:52,470
One is in the middle,
that's what we've always

577
00:31:52,470 --> 00:31:58,390
seen with merge sort and
convex hull from last time,

578
00:31:58,390 --> 00:32:01,000
but there's another way
which will work better here,

579
00:32:01,000 --> 00:32:05,480
which is the even entries
and the odd entries.

580
00:32:05,480 --> 00:32:08,820
So I'm going to divide into
even and odd coefficients.

581
00:32:15,410 --> 00:32:18,150
Let me write that down.

582
00:32:18,150 --> 00:32:21,860
One of them is called a sub
even of x-- that's a polynomial.

583
00:32:21,860 --> 00:32:24,440
It's going to have half
the degree, so it's

584
00:32:24,440 --> 00:32:31,750
going to be sum from k
equals 0 to-- I wrote n here,

585
00:32:31,750 --> 00:32:36,150
but I think I want something
like n over 2 minus 1,

586
00:32:36,150 --> 00:32:45,910
n minus 1 over 2, one of those
things-- of a to k x to the k.

587
00:32:45,910 --> 00:32:49,470
So, really, what I want--
which is easier to write-- is,

588
00:32:49,470 --> 00:32:54,762
in the vector notation, I
want all the even entries.

589
00:32:54,762 --> 00:32:56,720
I won't try to figure
out what the last one is,

590
00:32:56,720 --> 00:32:58,120
but it's roughly n over 2-ish.

591
00:33:00,750 --> 00:33:02,220
I'll just write that.

592
00:33:02,220 --> 00:33:03,470
You can go a little bit extra.

593
00:33:03,470 --> 00:33:07,210
It's fine if you define a sub
n to be 0, and a sub n plus 1

594
00:33:07,210 --> 00:33:10,700
to be zero, and all those to be
0, those terms will disappear.

595
00:33:10,700 --> 00:33:13,170
So the key thing here is
I'm taking the even entries,

596
00:33:13,170 --> 00:33:18,040
but I don't have 2k up here.

597
00:33:18,040 --> 00:33:20,640
This is the x to the 0 term.

598
00:33:20,640 --> 00:33:21,930
This is the x to the 1 term.

599
00:33:21,930 --> 00:33:24,020
This is the x to the 2 term.

600
00:33:24,020 --> 00:33:29,601
So there's a difference between
x to the k's and the sub 2k,

601
00:33:29,601 --> 00:33:31,600
but, I mean, just think
about it in vector form.

602
00:33:31,600 --> 00:33:33,620
Don't worry about
the algebra for now.

603
00:33:33,620 --> 00:33:35,703
We're going to have to
worry about it in a second,

604
00:33:35,703 --> 00:33:38,050
but, intuitively,
what I want to do

605
00:33:38,050 --> 00:33:42,620
is extract from the vector
of all the ai's these two

606
00:33:42,620 --> 00:33:48,090
vectors-- the odd
coefficients in order

607
00:33:48,090 --> 00:33:50,830
and the even
coefficients in order--

608
00:33:50,830 --> 00:33:53,500
but I'm going to need the
algebraic form in a moment

609
00:33:53,500 --> 00:33:54,750
for the combined step.

610
00:33:58,260 --> 00:34:01,950
It should be 2k plus 1 x to k.

611
00:34:06,684 --> 00:34:07,350
That's step one.

612
00:34:10,290 --> 00:34:10,812
Easy to do.

613
00:34:10,812 --> 00:34:11,770
Linear time, of course.

614
00:34:14,510 --> 00:34:16,510
Let's jump ahead to
step three, combine.

615
00:34:20,739 --> 00:34:24,560
In order to compute
a of x from-- what

616
00:34:24,560 --> 00:34:26,690
I'd like to do is
recursively compute a even

617
00:34:26,690 --> 00:34:29,082
of x and a odd of x
for some values x.

618
00:34:29,082 --> 00:34:30,290
It's not going to be x and x.

619
00:34:30,290 --> 00:34:32,469
It's going to be some other set.

620
00:34:32,469 --> 00:34:35,300
Let's think about
how I can compute

621
00:34:35,300 --> 00:34:40,739
a of x given some solutions
to a even of x and a odd of x.

622
00:34:40,739 --> 00:34:49,604
So this is step three, combine.

623
00:34:52,780 --> 00:35:00,040
So I would like
a of x over here,

624
00:35:00,040 --> 00:35:10,880
and I want a even of something
and a odd of something

625
00:35:10,880 --> 00:35:13,330
and something in here.

626
00:35:13,330 --> 00:35:15,680
Anyone see the algebra?

627
00:35:19,260 --> 00:35:20,190
Maybe start with this?

628
00:35:25,130 --> 00:35:26,366
I hear a mumble.

629
00:35:26,366 --> 00:35:26,866
Yeah.

630
00:35:26,866 --> 00:35:27,360
AUDIENCE: x squared.

631
00:35:27,360 --> 00:35:28,250
ERIK DEMAINE: x squared.

632
00:35:28,250 --> 00:35:28,750
Exactly.

633
00:35:32,138 --> 00:35:33,086
Time for a purple one.

634
00:35:36,404 --> 00:35:38,070
I'm getting better.

635
00:35:38,070 --> 00:35:38,970
Why x squared?

636
00:35:38,970 --> 00:35:41,740
Because we have
this mismatch here.

637
00:35:41,740 --> 00:35:43,290
We have a sub 2k.

638
00:35:43,290 --> 00:35:45,446
We want x to the 2k.

639
00:35:45,446 --> 00:35:46,320
How could we do that?

640
00:35:46,320 --> 00:35:54,190
Well, we could put x squared to
the 2k, and x squared to the k

641
00:35:54,190 --> 00:35:58,670
is the same thing
as x to the 2k.

642
00:35:58,670 --> 00:36:03,760
And so magically this
transforms into the even entries

643
00:36:03,760 --> 00:36:04,790
of a of x.

644
00:36:04,790 --> 00:36:06,110
That's half of them.

645
00:36:06,110 --> 00:36:11,260
We do the same thing for the
odd ones and we're almost there.

646
00:36:11,260 --> 00:36:18,180
Now we have a sub 2k plus 1
times x to the 2k, no plus 1.

647
00:36:18,180 --> 00:36:20,617
So how can I add a plus 1?

648
00:36:20,617 --> 00:36:21,900
AUDIENCE: Multiply by x.

649
00:36:21,900 --> 00:36:25,320
ERIK DEMAINE:
Multiply by x here.

650
00:36:25,320 --> 00:36:27,700
Take the whole
thing, multiply by x,

651
00:36:27,700 --> 00:36:31,010
then I get all of the
odd terms of a of x.

652
00:36:31,010 --> 00:36:32,880
I add these together.

653
00:36:32,880 --> 00:36:35,727
I get a of x.

654
00:36:35,727 --> 00:36:37,560
I mean, you could prove
this more carefully,

655
00:36:37,560 --> 00:36:42,980
but that's just algebra to
see that this is correct.

656
00:36:42,980 --> 00:36:45,700
Once you have this, it
tells you what I need to do

657
00:36:45,700 --> 00:36:50,940
is compute a even of x
squared for all x in x,

658
00:36:50,940 --> 00:36:54,080
so this is 4x and x.

659
00:36:54,080 --> 00:36:55,620
There's a for loop for you.

660
00:36:55,620 --> 00:36:57,120
So that's going to
take linear time.

661
00:36:57,120 --> 00:37:00,350
If I already know this value
and I already know this value,

662
00:37:00,350 --> 00:37:03,140
I do one multiplication,
one addition, and boom.

663
00:37:03,140 --> 00:37:05,760
I get a of x.

664
00:37:05,760 --> 00:37:20,580
So in the conquer step I
want to recursively compute--

665
00:37:20,580 --> 00:37:33,460
I think I'll call it-- a even
of y, and a odd of y for y

666
00:37:33,460 --> 00:37:35,440
in x squared.

667
00:37:35,440 --> 00:37:39,695
x squared is the set of
squares of all numbers in x.

668
00:37:43,220 --> 00:37:44,700
So I'm changing my set x.

669
00:37:44,700 --> 00:37:48,240
I started with a
polynomial a and a set x.

670
00:37:48,240 --> 00:37:51,310
Recursively, I'm doing
a different polynomial

671
00:37:51,310 --> 00:37:57,000
of half the degree-- half
the number of terms--

672
00:37:57,000 --> 00:38:02,650
but with a different
set of the same size.

673
00:38:02,650 --> 00:38:04,500
I started with x.

674
00:38:04,500 --> 00:38:08,300
x squared has the
same size as x, right?

675
00:38:08,300 --> 00:38:12,995
So let's try to figure out how
fast or slow this algorithm is.

676
00:38:15,504 --> 00:38:16,920
But that isn't
divide and conquer.

677
00:38:16,920 --> 00:38:20,290
That is going to be
our golden ticket.

678
00:38:20,290 --> 00:38:26,070
It's pretty simple, but we're
going to need another trick.

679
00:38:26,070 --> 00:38:29,540
So I'm going to
write a recurrence.

680
00:38:29,540 --> 00:38:32,720
Now, this recurrence
depends on two things.

681
00:38:32,720 --> 00:38:37,610
One is how many terms are there
in a that we've been calling n?

682
00:38:37,610 --> 00:38:40,170
And the other is how many
numbers are there in x?

683
00:38:40,170 --> 00:38:43,150
How many different places do I
have to evaluate my polynomial?

684
00:38:43,150 --> 00:38:47,450
So we've got t of n, and I
always call it size of x.

685
00:38:51,680 --> 00:38:55,620
So divide and conquer goes
hand-in-hand with recurrences.

686
00:38:55,620 --> 00:38:58,310
Generally you've got
the recursive part,

687
00:38:58,310 --> 00:39:00,440
so that's just how big
are the subproblems.

688
00:39:00,440 --> 00:39:01,280
How many are there?

689
00:39:01,280 --> 00:39:03,100
There are two subproblems.

690
00:39:03,100 --> 00:39:05,910
They have half the
size in terms of n,

691
00:39:05,910 --> 00:39:08,920
but they have the same
size in terms of x.

692
00:39:08,920 --> 00:39:14,190
So, all right, 2
times-- because there's

693
00:39:14,190 --> 00:39:24,610
two subproblems-- each with
size n over 2 and size x, plus

694
00:39:24,610 --> 00:39:28,240
what goes here is however much
it costs to do the divide step,

695
00:39:28,240 --> 00:39:30,690
plus however much it costs
to do the combined step-- all

696
00:39:30,690 --> 00:39:32,620
the non-recursive parts.

697
00:39:32,620 --> 00:39:35,640
So this is just partitioning
the vectors-- linear scan,

698
00:39:35,640 --> 00:39:37,290
linear time.

699
00:39:37,290 --> 00:39:39,480
This is-- we talked
about it-- it's

700
00:39:39,480 --> 00:39:43,560
a constant number of arithmetic
operations for each x.

701
00:39:43,560 --> 00:39:48,390
So this cause-order x time,
this cause-order n time.

702
00:39:48,390 --> 00:39:55,560
So, in general, we get n plus x.

703
00:39:55,560 --> 00:39:58,252
Now, this is, again,
not a recurrent solvable

704
00:39:58,252 --> 00:40:00,335
by the master method because
it has two variables.

705
00:40:03,130 --> 00:40:05,500
So usually when you're faced
with this sort of thing,

706
00:40:05,500 --> 00:40:08,600
you want to do back of
the envelope picture.

707
00:40:08,600 --> 00:40:10,110
Draw a recursion tree.

708
00:40:10,110 --> 00:40:11,600
That's a good way to go.

709
00:40:11,600 --> 00:40:16,720
So at the root-- now, I know
that initially x equals n.

710
00:40:16,720 --> 00:40:18,540
When I start out I
have n coefficients,

711
00:40:18,540 --> 00:40:21,270
I have n different positions
I want to evaluate them

712
00:40:21,270 --> 00:40:23,270
at because that's
what I want to do

713
00:40:23,270 --> 00:40:26,490
to do this conversion from
coefficients to samples.

714
00:40:26,490 --> 00:40:29,330
So at the root of the recursion
tree I'm just going to write n.

715
00:40:29,330 --> 00:40:33,360
Order n work to get started
and to do the recursions.

716
00:40:33,360 --> 00:40:35,610
There are two recursive calls.

717
00:40:35,610 --> 00:40:39,242
One has som-- both have
size n over 2 in terms of a,

718
00:40:39,242 --> 00:40:42,670
and they have the same
x-- which is also known

719
00:40:42,670 --> 00:40:45,340
as n-- in those two recursions.

720
00:40:45,340 --> 00:40:48,740
So in fact, the linear
work here will be n and n,

721
00:40:48,740 --> 00:40:50,680
and then we'll get n and n.

722
00:40:50,680 --> 00:40:53,100
x never goes down,
so x always remains

723
00:40:53,100 --> 00:40:56,120
n-- the original value of n.

724
00:40:56,120 --> 00:40:59,240
This is a bad recurrence.

725
00:40:59,240 --> 00:41:01,855
There are n-- sorry,
there are log n levels.

726
00:41:01,855 --> 00:41:02,730
That's the good news.

727
00:41:02,730 --> 00:41:06,300
Once we get down to constant
size we can kind of stop.

728
00:41:06,300 --> 00:41:08,020
When there's only
one coefficient

729
00:41:08,020 --> 00:41:11,740
I know how to evaluate the
polynomial with just a 0.

730
00:41:11,740 --> 00:41:13,090
That's easy.

731
00:41:13,090 --> 00:41:17,010
So down at the bottom
here, at the last level--

732
00:41:17,010 --> 00:41:20,550
this is the height log n--
the last level is just, again,

733
00:41:20,550 --> 00:41:23,440
going to be a whole bunch
of n's-- all the same n.

734
00:41:23,440 --> 00:41:25,740
Here n is the original
value of x because we

735
00:41:25,740 --> 00:41:27,246
haven't changed that.

736
00:41:27,246 --> 00:41:28,620
How many n's are
there down here?

737
00:41:33,629 --> 00:41:34,670
AUDIENCE: 2 to the log n.

738
00:41:34,670 --> 00:41:36,586
ERIK DEMAINE: 2 to the
log n, also known as n.

739
00:41:40,681 --> 00:41:41,180
Good.

740
00:41:41,180 --> 00:41:44,410
So we had-- because
we had log n levels,

741
00:41:44,410 --> 00:41:47,830
we had binary branching, so
it's 2 to the number of levels,

742
00:41:47,830 --> 00:41:49,100
which is just n.

743
00:41:49,100 --> 00:41:54,180
So this is n squared.

744
00:41:54,180 --> 00:41:56,095
All this work, still n squared.

745
00:42:01,540 --> 00:42:06,500
Clearly what we need is
for x to get smaller, too.

746
00:42:06,500 --> 00:42:09,750
If x-- if, in this
recursion-- let

747
00:42:09,750 --> 00:42:12,365
me, in red, draw the recursion
I would like to have.

748
00:42:15,980 --> 00:42:18,480
If x became x over
2 here, that's

749
00:42:18,480 --> 00:42:20,740
the only change we'd need.

750
00:42:20,740 --> 00:42:24,640
Then n and x change in
exactly the same way,

751
00:42:24,640 --> 00:42:26,700
and so then we can
just forget about x--

752
00:42:26,700 --> 00:42:28,310
it's going to be the same as n.

753
00:42:28,310 --> 00:42:33,040
Then we get 2 times n
over 2 plus order n.

754
00:42:33,040 --> 00:42:34,440
Look familiar?

755
00:42:34,440 --> 00:42:36,140
It is our bread and
butter recurrence--

756
00:42:36,140 --> 00:42:37,500
merge sort recurrence.

757
00:42:37,500 --> 00:42:39,247
That's n log n.

758
00:42:39,247 --> 00:42:40,330
That's what we need to do.

759
00:42:40,330 --> 00:42:48,050
Somehow, when we convert
our set x to x squared,

760
00:42:48,050 --> 00:42:49,520
I want x to get smaller.

761
00:42:52,100 --> 00:42:53,595
Is that at all plausible?

762
00:42:53,595 --> 00:42:54,470
Let's think about it.

763
00:43:04,150 --> 00:43:06,680
What's the base case?

764
00:43:06,680 --> 00:43:13,210
To keep things simple let's say
the base case when x equals 1,

765
00:43:13,210 --> 00:43:19,231
I'll just let x be-- let's say
I want to compute my a at 1.

766
00:43:19,231 --> 00:43:19,855
Keep it simple.

767
00:43:22,810 --> 00:43:26,007
What if I want two values in x?

768
00:43:26,007 --> 00:43:27,590
I'd like to have the
feature that when

769
00:43:27,590 --> 00:43:32,080
I square all the values in
x-- so I want two values,

770
00:43:32,080 --> 00:43:34,820
but when I square them
I only have one value.

771
00:43:38,240 --> 00:43:39,110
Solve for x.

772
00:43:44,346 --> 00:43:44,846
Yeah.

773
00:43:44,846 --> 00:43:45,971
AUDIENCE: Negative 1 and 1?

774
00:43:45,971 --> 00:43:49,320
ERIK DEMAINE: Negative 1 and 1.

775
00:43:49,320 --> 00:43:51,264
Whew, tough one.

776
00:43:51,264 --> 00:43:52,240
Ehh.

777
00:43:52,240 --> 00:43:53,470
Not bad.

778
00:43:53,470 --> 00:43:55,720
Good catch.

779
00:43:55,720 --> 00:43:56,710
Negative 1 and 1.

780
00:43:56,710 --> 00:43:58,977
That could work.

781
00:43:58,977 --> 00:44:00,060
What are negative 1 and 1?

782
00:44:00,060 --> 00:44:02,750
They're the square roots of 1.

783
00:44:02,750 --> 00:44:05,870
Negative 1 squared
is 1, 1 squared is 1.

784
00:44:05,870 --> 00:44:07,920
There's two square
roots for every number.

785
00:44:07,920 --> 00:44:09,750
Two square roots
for every number.

786
00:44:09,750 --> 00:44:11,480
Interesting.

787
00:44:11,480 --> 00:44:14,300
So that means, if I just
keep taking square roots,

788
00:44:14,300 --> 00:44:19,190
when I square them it
collapses by a factor of 2.

789
00:44:19,190 --> 00:44:28,100
Let me go to another board
and define something.

790
00:44:34,942 --> 00:44:36,900
You're all anticipating
what's going to happen,

791
00:44:36,900 --> 00:44:41,840
but I'm going to
say, a collapsing

792
00:44:41,840 --> 00:44:52,720
set x-- or, a set is collapsing
if either the size of x squared

793
00:44:52,720 --> 00:44:58,265
is the size of x divided
by 2, and, recursively,

794
00:44:58,265 --> 00:44:59,265
x squared is collapsing.

795
00:45:04,810 --> 00:45:07,310
So I need this to work all
the way down the recursion.

796
00:45:07,310 --> 00:45:11,020
Or I need a base case,
which is just x equals 1.

797
00:45:11,020 --> 00:45:12,750
There's a single item in x.

798
00:45:12,750 --> 00:45:15,610
So I happened to start
with x equals the item 1.

799
00:45:15,610 --> 00:45:16,720
It didn't have to be 1.

800
00:45:16,720 --> 00:45:18,980
It could have been 7.

801
00:45:18,980 --> 00:45:22,930
It couldn't be 0 because 0--
you won't get two numbers.

802
00:45:22,930 --> 00:45:24,780
There's only one
square root of 0.

803
00:45:24,780 --> 00:45:26,310
OK, so I lied a little bit.

804
00:45:26,310 --> 00:45:30,850
Other than 0, every number
has exactly two square roots,

805
00:45:30,850 --> 00:45:35,063
so what's the square
root of negative 1?

806
00:45:35,063 --> 00:45:35,563
AUDIENCE: i.

807
00:45:35,563 --> 00:45:37,450
ERIK DEMAINE: i.

808
00:45:37,450 --> 00:45:41,150
So complex numbers.

809
00:45:41,150 --> 00:45:47,040
If I take square roots of these
guys I get i and negative i,

810
00:45:47,040 --> 00:45:51,640
and again I get minus 1 and 1.

811
00:45:51,640 --> 00:45:53,880
That's when x equals 4.

812
00:45:53,880 --> 00:45:56,550
Turns out this is only going
to work for powers of 2,

813
00:45:56,550 --> 00:45:58,260
but, hey, if n isn't
a power of 2 just

814
00:45:58,260 --> 00:45:59,680
round up to the next power of 2.

815
00:45:59,680 --> 00:46:03,370
That only hurts me
by a factor of 2.

816
00:46:03,370 --> 00:46:05,100
Complex numbers.

817
00:46:05,100 --> 00:46:07,740
Every time I said real
number in the past,

818
00:46:07,740 --> 00:46:09,587
pretend I said complex number.

819
00:46:09,587 --> 00:46:10,920
Everything I said is still true.

820
00:46:10,920 --> 00:46:13,440
Actually, the root
thing is only true

821
00:46:13,440 --> 00:46:14,820
when you allow complex numbers.

822
00:46:14,820 --> 00:46:17,420
Some polynomials
have complex roots,

823
00:46:17,420 --> 00:46:19,780
so pretend I said complex.

824
00:46:19,780 --> 00:46:21,280
We're going to need
complex numbers.

825
00:46:21,280 --> 00:46:22,100
This is why.

826
00:46:22,100 --> 00:46:24,330
Because, when we'd start
taking square roots,

827
00:46:24,330 --> 00:46:27,890
we immediately get
complex numbers.

828
00:46:27,890 --> 00:46:29,512
Next would be x equals 8.

829
00:46:34,400 --> 00:46:37,116
Square root of i.

830
00:46:37,116 --> 00:46:38,990
Square root of i-- let's
see if I can do it--

831
00:46:38,990 --> 00:46:42,430
should be root 2 over
2 times 1 plus i.

832
00:46:45,420 --> 00:46:51,470
And then this one is just
the-- square root of negative i

833
00:46:51,470 --> 00:46:56,890
is going to be root 2
over 2 times 1 minus i,

834
00:46:56,890 --> 00:46:58,610
and then we have
all our old guys.

835
00:47:03,584 --> 00:47:05,250
Oh, and then I could
write plus or minus

836
00:47:05,250 --> 00:47:06,610
in front of each of these.

837
00:47:06,610 --> 00:47:08,485
I was like, there weren't
enough terms there.

838
00:47:08,485 --> 00:47:11,190
Now I've got eight-- sorry,
I've got four numbers here.

839
00:47:11,190 --> 00:47:12,465
I've got four numbers there.

840
00:47:12,465 --> 00:47:15,020
How in the world
could I remember this?

841
00:47:15,020 --> 00:47:16,050
Maybe I memorized it.

842
00:47:16,050 --> 00:47:17,530
No, I didn't memorize it.

843
00:47:17,530 --> 00:47:21,330
It's actually really
easy to figure this out

844
00:47:21,330 --> 00:47:23,715
if you know geometry.

845
00:47:23,715 --> 00:47:25,500
Geometry.

846
00:47:25,500 --> 00:47:27,710
Let's do geometry over here.

847
00:47:36,090 --> 00:47:36,861
It's convenient.

848
00:47:36,861 --> 00:47:37,860
I'm actually a geometer.

849
00:47:43,180 --> 00:47:45,380
You know, complex numbers
have two parts, right?

850
00:47:45,380 --> 00:47:47,570
The real part and
the complex part.

851
00:47:47,570 --> 00:47:50,560
I'm going to draw that in what's
called the complex plane, where

852
00:47:50,560 --> 00:47:53,690
we draw the real
part here, and I

853
00:47:53,690 --> 00:47:57,260
guess it's usually called the
imaginary part on the y-axis.

854
00:47:57,260 --> 00:48:00,190
Every point in this plane is
a complex number, and vice

855
00:48:00,190 --> 00:48:01,190
versa-- the same thing.

856
00:48:04,320 --> 00:48:05,870
So what did we start with?

857
00:48:05,870 --> 00:48:08,160
We started with the number 1.

858
00:48:08,160 --> 00:48:12,510
Number 1 will be here.

859
00:48:12,510 --> 00:48:14,240
It has no imaginary
part, so it's

860
00:48:14,240 --> 00:48:17,061
on the x-axis-- this is
the real line down here--

861
00:48:17,061 --> 00:48:19,310
and it's at position 1, which
I'm going to just define

862
00:48:19,310 --> 00:48:21,547
to be right there.

863
00:48:21,547 --> 00:48:22,630
Then we've got negative 1.

864
00:48:22,630 --> 00:48:24,650
That's over here.

865
00:48:24,650 --> 00:48:26,860
Then we got i.

866
00:48:26,860 --> 00:48:28,940
That's here, 1 times i.

867
00:48:28,940 --> 00:48:30,440
Then we've got negative i.

868
00:48:30,440 --> 00:48:33,380
That's here.

869
00:48:33,380 --> 00:48:36,660
Then we've got root 2
over 2 times i plus 1.

870
00:48:36,660 --> 00:48:40,470
That's here.

871
00:48:40,470 --> 00:48:43,060
Root 2 over 2 by root 2 over 2.

872
00:48:43,060 --> 00:48:46,920
What is a property of root 2
over 2 comma root 2 over 2?

873
00:48:46,920 --> 00:48:50,850
It has distance exactly
1 to the origin.

874
00:48:50,850 --> 00:48:57,012
If I draw this triangle,
root 2 over 2, root 2 over 2.

875
00:48:57,012 --> 00:48:58,290
I square this.

876
00:48:58,290 --> 00:48:58,990
I get a half.

877
00:48:58,990 --> 00:49:00,720
I square this I get a half.

878
00:49:00,720 --> 00:49:02,900
I add them together, I get what?

879
00:49:02,900 --> 00:49:06,870
Take the square root, I still
get 1, so this distance is 1.

880
00:49:06,870 --> 00:49:08,270
Interesting.

881
00:49:08,270 --> 00:49:11,070
And then I got the negative
of that, which is over here,

882
00:49:11,070 --> 00:49:12,670
and a negative of that.

883
00:49:12,670 --> 00:49:17,095
Then this is the-- with a
negative-- did I get it wrong?

884
00:49:17,095 --> 00:49:18,870
Yep, sorry.

885
00:49:18,870 --> 00:49:25,082
It doesn't matter, but I'll
think of it as i minus-- killed

886
00:49:25,082 --> 00:49:30,201
a chalk-- i minus 1.

887
00:49:30,201 --> 00:49:32,200
It's the same because I
have the plus and minus,

888
00:49:32,200 --> 00:49:34,150
but I like the geometry.

889
00:49:34,150 --> 00:49:40,380
So this point is negative root
2 over 2 by root 2 over 2,

890
00:49:40,380 --> 00:49:43,724
and then there's the
negative over here.

891
00:49:43,724 --> 00:49:45,415
What property do
these points have?

892
00:49:50,770 --> 00:49:51,574
I heard a word.

893
00:49:51,574 --> 00:49:52,490
AUDIENCE: Unit circle?

894
00:49:52,490 --> 00:49:53,573
ERIK DEMAINE: Unit circle.

895
00:49:57,690 --> 00:49:58,650
Good.

896
00:49:58,650 --> 00:50:00,660
Who said unit circle?

897
00:50:00,660 --> 00:50:02,160
Nice.

898
00:50:02,160 --> 00:50:03,890
It's a unit circle, right?

899
00:50:03,890 --> 00:50:05,390
Clearly that deserves
a Frisbee end.

900
00:50:09,970 --> 00:50:11,200
Unit circle.

901
00:50:11,200 --> 00:50:13,150
Hm.

902
00:50:13,150 --> 00:50:14,980
Circle.

903
00:50:14,980 --> 00:50:17,020
It seem good.

904
00:50:17,020 --> 00:50:24,100
What's going on here
is I took this number.

905
00:50:24,100 --> 00:50:28,000
I claimed it was
the square root of i

906
00:50:28,000 --> 00:50:30,470
because it turns out, if you
take points on the unit circle

907
00:50:30,470 --> 00:50:32,360
in the complex plane,
when you square

908
00:50:32,360 --> 00:50:36,260
a number it's like doubling the
angle relative to the x-axis.

909
00:50:36,260 --> 00:50:37,560
This is angle 0.

910
00:50:37,560 --> 00:50:40,900
This is angle-- what do
you call it-- 45 degrees.

911
00:50:40,900 --> 00:50:42,550
This is angle 90 degrees.

912
00:50:42,550 --> 00:50:45,360
So when I square this
number I get 90 degrees.

913
00:50:45,360 --> 00:50:49,774
That's why this number
is a square root of i.

914
00:50:49,774 --> 00:50:51,648
I probably should have
labeled some of these.

915
00:50:51,648 --> 00:50:53,022
This is i.

916
00:50:53,022 --> 00:50:54,770
This is minus i.

917
00:50:54,770 --> 00:51:00,320
This is minus 1, and this is 1.

918
00:51:00,320 --> 00:51:12,520
In general, we get something
called-- so these are

919
00:51:12,520 --> 00:51:15,040
called the n-th roots of unity.

920
00:51:19,040 --> 00:51:23,330
Unity is just a
fancy word for 1.

921
00:51:23,330 --> 00:51:27,380
1 is here, and first we
computed the square roots of 1.

922
00:51:27,380 --> 00:51:28,920
They were minus 1 and 1.

923
00:51:28,920 --> 00:51:32,280
Then we computed the
fourth roots of 1.

924
00:51:32,280 --> 00:51:37,910
All of these numbers, if
you take the fourth power,

925
00:51:37,910 --> 00:51:39,590
you get 1.

926
00:51:39,590 --> 00:51:41,970
Then we computed the
eighth roots of 1.

927
00:51:41,970 --> 00:51:45,160
All of these numbers, if
you take the 8th power,

928
00:51:45,160 --> 00:51:46,140
you get 1 again.

929
00:51:46,140 --> 00:51:49,074
So, in general,
n-th roots of unity.

930
00:51:49,074 --> 00:51:50,740
We're going to assume
n is a power of 2,

931
00:51:50,740 --> 00:51:53,730
but this notion actually
makes sense for any n.

932
00:51:53,730 --> 00:51:56,000
And they're just uniformly
spaced around the circle,

933
00:51:56,000 --> 00:51:59,640
and if you know some geometry
and how it relates to trig,

934
00:51:59,640 --> 00:52:01,820
you know that a general
point on the circle

935
00:52:01,820 --> 00:52:04,610
is cos theta comma sine theta.

936
00:52:04,610 --> 00:52:06,480
x-coordinate is cos theta.

937
00:52:06,480 --> 00:52:07,650
y-coordinate is sine theta.

938
00:52:07,650 --> 00:52:09,990
This is also a funny notation
for complex numbers--

939
00:52:09,990 --> 00:52:13,020
not so funny, this is the
geometric interpretation

940
00:52:13,020 --> 00:52:21,510
of cos theta plus i sine theta.

941
00:52:21,510 --> 00:52:23,810
And if I want them uniformly
spaced around the circle,

942
00:52:23,810 --> 00:52:27,060
and I want to include
this point-- also known

943
00:52:27,060 --> 00:52:30,510
as theta equals 0-- because
when theta equals 0,

944
00:52:30,510 --> 00:52:32,425
cos theta is 1, sine theta 0.

945
00:52:35,070 --> 00:52:40,380
So I want to say 4
theta equals to 0,

946
00:52:40,380 --> 00:52:46,630
and then-- here I'm going
to get fancy-- tau over n

947
00:52:46,630 --> 00:52:54,280
to tau over n up to n
minus 1 over n times tau.

948
00:52:54,280 --> 00:52:55,637
What's tau?

949
00:52:55,637 --> 00:52:56,220
AUDIENCE: 2pi.

950
00:52:56,220 --> 00:52:57,720
ERIK DEMAINE: 2pi, thank you.

951
00:53:00,590 --> 00:53:04,390
This is modern notation just
over the last couple years.

952
00:53:04,390 --> 00:53:09,790
I believe in tau so much I
got it tattooed on my arm.

953
00:53:09,790 --> 00:53:12,260
Tau is the fundamental constant.

954
00:53:12,260 --> 00:53:14,310
Screw pi.

955
00:53:14,310 --> 00:53:16,000
None of that 3 and change.

956
00:53:16,000 --> 00:53:19,460
Six and change is where it's at.

957
00:53:19,460 --> 00:53:21,196
So tau.

958
00:53:21,196 --> 00:53:22,320
Clearly this is much nicer.

959
00:53:22,320 --> 00:53:24,840
Tau over n, not 2pi over n.

960
00:53:24,840 --> 00:53:28,870
Tau is a whole circle.

961
00:53:28,870 --> 00:53:32,062
This is one n-th of a circle,
2n-ths of a circle, n minus 1

962
00:53:32,062 --> 00:53:33,020
over n-ths of a circle.

963
00:53:33,020 --> 00:53:35,340
I didn't do n n-ths of a
circle because that's also

964
00:53:35,340 --> 00:53:36,240
the same as 0.

965
00:53:39,020 --> 00:53:47,290
Now, why did I
introduce this notation?

966
00:53:47,290 --> 00:53:49,540
Because there's this
other great thing

967
00:53:49,540 --> 00:53:55,460
called Euler's formula,
which is that this

968
00:53:55,460 --> 00:54:02,810
equals e to the i theta.

969
00:54:07,130 --> 00:54:09,380
Double check.

970
00:54:09,380 --> 00:54:14,050
It's so rare that I get
to do real calculus.

971
00:54:14,050 --> 00:54:16,460
This is Euler's
formula-- e for Euler--

972
00:54:16,460 --> 00:54:18,840
another number-- 2 and change.

973
00:54:18,840 --> 00:54:21,810
e to the i-- this is funny
because it's complex-- times

974
00:54:21,810 --> 00:54:24,780
theta is equal to cos
theta plus i sine theta.

975
00:54:24,780 --> 00:54:28,980
This is the relation between
exponentials and trigonometry.

976
00:54:28,980 --> 00:54:31,670
That's a big thing Euler did.

977
00:54:31,670 --> 00:54:33,080
Cool.

978
00:54:33,080 --> 00:54:33,660
So what?

979
00:54:33,660 --> 00:54:36,080
Because this lets
us understand how

980
00:54:36,080 --> 00:54:42,640
squares work-- not squares the
shape, squares the operation.

981
00:54:50,580 --> 00:54:57,060
When I take squares-- so
if I take e to the i theta,

982
00:54:57,060 --> 00:55:00,020
this is one of my
roots of unity.

983
00:55:00,020 --> 00:55:02,380
Let me expand out what theta is.

984
00:55:02,380 --> 00:55:07,460
So theta is some k
times tau over n.

985
00:55:07,460 --> 00:55:09,010
Let's do it this way first.

986
00:55:09,010 --> 00:55:17,070
So in reality we have k tau
over n, but when I square it,

987
00:55:17,070 --> 00:55:19,790
that's the same thing as
putting the 2 right here.

988
00:55:19,790 --> 00:55:24,390
This is the same thing
as e to i times 2 theta.

989
00:55:24,390 --> 00:55:24,890
Bingo.

990
00:55:24,890 --> 00:55:28,200
I get what I was claiming, that
if I start at some angle theta

991
00:55:28,200 --> 00:55:31,450
relative to the x-axis,
when I square the number

992
00:55:31,450 --> 00:55:32,950
I just double the angle.

993
00:55:32,950 --> 00:55:35,370
This is why.

994
00:55:35,370 --> 00:55:39,770
This is obvious just
from regular algebra.

995
00:55:39,770 --> 00:55:41,820
And then this thing,
Euler's formula,

996
00:55:41,820 --> 00:55:45,680
tells me that corresponds to
doubling the angle on a circle.

997
00:55:45,680 --> 00:55:47,970
So it only works with
points on a circle.

998
00:55:47,970 --> 00:55:58,610
So when I go here I get e to
the i2k times tau over n--

999
00:55:58,610 --> 00:56:02,080
twice as far around circle.

1000
00:56:02,080 --> 00:56:03,850
All right.

1001
00:56:03,850 --> 00:56:06,530
Fine.

1002
00:56:06,530 --> 00:56:10,520
What happens if I take this
number and I square it?

1003
00:56:10,520 --> 00:56:12,030
This has a really big angle.

1004
00:56:12,030 --> 00:56:20,500
This is 1/2 plus 1/8, whatever
that is-- 5/8 times tau.

1005
00:56:20,500 --> 00:56:28,230
When I double that
angle I go to here.

1006
00:56:28,230 --> 00:56:31,910
Now, this, you might
call it, 10/8, or you

1007
00:56:31,910 --> 00:56:36,540
might also call it 1/4 because
when you go around the circle

1008
00:56:36,540 --> 00:56:37,684
you stay on the circle.

1009
00:56:37,684 --> 00:56:39,100
So there's another
thing going on,

1010
00:56:39,100 --> 00:56:47,580
which is really-- this is e to
the i times 2 theta mod tau.

1011
00:56:47,580 --> 00:56:49,720
Usually we think of mods
relative to integers,

1012
00:56:49,720 --> 00:56:54,040
but what I mean is every time
I add a multiple of tau nothing

1013
00:56:54,040 --> 00:56:54,540
changes.

1014
00:56:54,540 --> 00:56:56,998
If I go around the circle five
times and then do something,

1015
00:56:56,998 --> 00:56:59,290
it's the same as just
doing the something.

1016
00:56:59,290 --> 00:57:01,620
So I kind of need that.

1017
00:57:01,620 --> 00:57:08,340
And this is true because
e to the i tau equals 1.

1018
00:57:08,340 --> 00:57:12,500
You may know it as e to
the i pi equals negative 1,

1019
00:57:12,500 --> 00:57:15,990
but clearly this is a superior
formula-- so superior I got

1020
00:57:15,990 --> 00:57:17,220
it tattooed on my other arm.

1021
00:57:17,220 --> 00:57:20,549
[LAUGHTER]

1022
00:57:20,549 --> 00:57:22,590
It's amazing what you can
do with a laser printer

1023
00:57:22,590 --> 00:57:24,260
and a temporary tattoo kit.

1024
00:57:24,260 --> 00:57:31,440
Sadly, these won't last, but
definitely try it at home.

1025
00:57:31,440 --> 00:57:32,090
Cool.

1026
00:57:32,090 --> 00:57:36,390
So e to the i tau equals
1, so going around

1027
00:57:36,390 --> 00:57:38,550
in circles-- same thing as not.

1028
00:57:38,550 --> 00:57:39,860
All right.

1029
00:57:39,860 --> 00:57:42,580
So you can draw on this
picture for every number, what

1030
00:57:42,580 --> 00:57:44,440
is its square?

1031
00:57:44,440 --> 00:57:47,957
And, in general, if you
look at these four guys,

1032
00:57:47,957 --> 00:57:50,040
their squares are just
going to be these two guys.

1033
00:57:50,040 --> 00:57:52,510
If you look at these
four guys, their squares

1034
00:57:52,510 --> 00:57:55,000
are going to be among
these four guys.

1035
00:57:55,000 --> 00:57:56,440
So I started with 8 guys.

1036
00:57:56,440 --> 00:57:57,530
I square them, I get 4.

1037
00:57:57,530 --> 00:57:58,500
I square them, I get 2.

1038
00:57:58,500 --> 00:57:59,510
I square them, I get 1.

1039
00:57:59,510 --> 00:58:01,640
That's how we
constructed it, but you

1040
00:58:01,640 --> 00:58:05,100
can see it works not
only for this 8-point set

1041
00:58:05,100 --> 00:58:07,800
that we constructed
sort of by hand,

1042
00:58:07,800 --> 00:58:09,770
but it works for the
n-th roots of unity.

1043
00:58:09,770 --> 00:58:14,260
As long as n is a power
of 2 this set of points

1044
00:58:14,260 --> 00:58:15,055
will be collapsing.

1045
00:58:17,690 --> 00:58:23,240
So if n is the power of
2 for some integer k,

1046
00:58:23,240 --> 00:58:42,210
then n-th roots of
unity are collapsing

1047
00:58:42,210 --> 00:58:44,700
according to this definition.

1048
00:58:44,700 --> 00:58:46,480
And that's what we want.

1049
00:58:46,480 --> 00:58:50,140
Then this divide and conquer
algorithm runs in n log n time

1050
00:58:50,140 --> 00:58:53,000
because every time
we square the set x,

1051
00:58:53,000 --> 00:58:55,510
we reduce its size
by a factor of 2.

1052
00:58:55,510 --> 00:58:58,990
We get t of n equals 2
times t of n over 2 plus n.

1053
00:58:58,990 --> 00:59:01,190
Plus order n and
that's order n log n.

1054
00:59:01,190 --> 00:59:05,000
And so this whole thing we
compute-- in other words,

1055
00:59:05,000 --> 00:59:13,280
we set xk to be e to
the ik tau over n.

1056
00:59:13,280 --> 00:59:15,780
Now, I guess I should
say how to compute that,

1057
00:59:15,780 --> 00:59:18,500
but let's just say that's
given to you for free.

1058
00:59:18,500 --> 00:59:21,586
It takes constant time to
compute each roots of unity.

1059
00:59:21,586 --> 00:59:23,210
In fact, again, we
only need to do this

1060
00:59:23,210 --> 00:59:27,000
once and for all
for each value of n,

1061
00:59:27,000 --> 00:59:29,680
so you can think of it as just
being part of the algorithm.

1062
00:59:29,680 --> 00:59:30,990
These are the xk's we use.

1063
00:59:30,990 --> 00:59:33,550
I said xk could be anything
we want as long as they're all

1064
00:59:33,550 --> 00:59:35,290
different, so I'm
going to choose them

1065
00:59:35,290 --> 00:59:41,830
to be n uniformly spaced points
around the complex unit circle.

1066
00:59:41,830 --> 00:59:46,610
And then magically this
algorithm runs in n log n time.

1067
00:59:46,610 --> 00:59:49,109
It's pretty cool.

1068
00:59:49,109 --> 00:59:51,150
Intuitively, you think of
real numbers x squared,

1069
00:59:51,150 --> 00:59:54,360
of course it has
the same sides as x.

1070
00:59:54,360 --> 00:59:56,580
But once you go
to complex numbers

1071
00:59:56,580 --> 00:59:58,420
there's this nifty
trick where you

1072
00:59:58,420 --> 01:00:00,920
start with one
n-th of the circle

1073
01:00:00,920 --> 01:00:03,345
and you'd uniformly space
points after the first level

1074
01:00:03,345 --> 01:00:03,920
of recursion.

1075
01:00:03,920 --> 01:00:06,000
You only have to be
dealing with n over two

1076
01:00:06,000 --> 01:00:08,580
of those points,
namely the even ones,

1077
01:00:08,580 --> 01:00:11,179
also known as the n over
[? 2th ?] roots of unity.

1078
01:00:11,179 --> 01:00:12,720
And after the next
level of recursion

1079
01:00:12,720 --> 01:00:14,220
is the n over fourth
roots of unity.

1080
01:00:14,220 --> 01:00:14,870
And so on.

1081
01:00:14,870 --> 01:00:16,930
So this recursion
is well defined.

1082
01:00:16,930 --> 01:00:19,030
When you have a vector
of size n you just

1083
01:00:19,030 --> 01:00:21,670
have to deal with the n-th roots
of unity, and you're happy.

1084
01:00:25,710 --> 01:00:28,090
That is fast Fourier transform.

1085
01:00:28,090 --> 01:00:33,750
That algorithm with
these xi's is FFT.

1086
01:00:33,750 --> 01:00:35,230
So let me write this somewhere.

1087
01:00:44,590 --> 01:00:46,040
It kind of snuck up on us.

1088
01:00:46,040 --> 01:00:50,690
This is the algorithm
we were aiming to find.

1089
01:00:50,690 --> 01:01:05,549
Fast Fourier transform is that
divide and conquer algorithm

1090
01:01:05,549 --> 01:01:06,090
on the right.

1091
01:01:12,570 --> 01:01:14,860
I'll write it abstractly.

1092
01:01:14,860 --> 01:01:18,080
For something called the
DFT-- the discrete Fourier

1093
01:01:18,080 --> 01:01:29,570
transform-- is the corresponding
mathematical transformation.

1094
01:01:29,570 --> 01:01:31,310
The fast is about an algorithm.

1095
01:01:31,310 --> 01:01:34,840
Discrete is about discrete.

1096
01:01:34,840 --> 01:01:40,040
So DFT is this thing that
we wanted to compute,

1097
01:01:40,040 --> 01:01:45,660
which was basically
the product v times a.

1098
01:01:45,660 --> 01:01:48,310
Remember, v was the
Vandermonde matrix,

1099
01:01:48,310 --> 01:01:52,430
and it depended
on all these xk's.

1100
01:01:52,430 --> 01:02:00,240
And we're going to set xk to
be this e to the ik tau over n.

1101
01:02:03,400 --> 01:02:14,810
So if you remember the vjk
here was xj to the k-th power,

1102
01:02:14,810 --> 01:02:23,650
so this just becomes e
to the ijk tau over n.

1103
01:02:23,650 --> 01:02:26,120
It's a little funny
these are all consecutive

1104
01:02:26,120 --> 01:02:27,640
because this is a
totally different,

1105
01:02:27,640 --> 01:02:31,620
but-- that's the matrix.

1106
01:02:31,620 --> 01:02:35,044
If you take that
matrix times a vector,

1107
01:02:35,044 --> 01:02:36,835
that is called a discrete
Fourier transform

1108
01:02:36,835 --> 01:02:41,780
of the vector, and FFT is a way
to compute it in n log n time.

1109
01:02:41,780 --> 01:02:44,680
You just run this algorithm
and for those xk's it'll

1110
01:02:44,680 --> 01:02:46,104
just work in n log n time.

1111
01:02:48,880 --> 01:02:50,480
Cool.

1112
01:02:50,480 --> 01:02:53,560
But if you remember way back to
the beginning, what we needed

1113
01:02:53,560 --> 01:02:56,300
is a way-- this converts
a coefficient vector

1114
01:02:56,300 --> 01:02:58,390
into a sample vector.

1115
01:02:58,390 --> 01:03:04,890
Then we can multiply
the polynomials,

1116
01:03:04,890 --> 01:03:08,150
then we need to transform the
sample vector that results back

1117
01:03:08,150 --> 01:03:09,820
into a coefficient vector.

1118
01:03:09,820 --> 01:03:11,750
So we're only half done.

1119
01:03:11,750 --> 01:03:13,840
I only have 15 minutes.

1120
01:03:13,840 --> 01:03:17,040
Luckily, the other half is
almost identical to this half,

1121
01:03:17,040 --> 01:03:19,420
so let's do that next.

1122
01:03:19,420 --> 01:03:20,390
Maybe over here.

1123
01:03:28,160 --> 01:03:31,130
So we've got our great
divide and conquer algorithm,

1124
01:03:31,130 --> 01:03:33,290
what we need now--
let me give you

1125
01:03:33,290 --> 01:03:35,630
the polynomial
multiplication algorithm.

1126
01:03:38,520 --> 01:03:42,660
Let's call this-- sound more
exotic-- fast polynomial

1127
01:03:42,660 --> 01:03:45,565
multiplication, fast
meaning n log n.

1128
01:03:48,830 --> 01:03:53,400
Let's say that we're
given two polynomials, a

1129
01:03:53,400 --> 01:03:56,460
and b, represented
in coefficient form.

1130
01:04:00,770 --> 01:04:06,280
What we need is-- I'll
call it a star, which is

1131
01:04:06,280 --> 01:04:10,230
the result of running FFT on a.

1132
01:04:10,230 --> 01:04:15,320
It's the discrete Fourier
transform of a. b star,

1133
01:04:15,320 --> 01:04:17,430
do the same thing.

1134
01:04:17,430 --> 01:04:20,250
So we know what this
means is convert

1135
01:04:20,250 --> 01:04:23,040
a from a coefficient vector
into a sample vector.

1136
01:04:23,040 --> 01:04:25,970
Convert b from a coefficient
vector into a sample vector.

1137
01:04:25,970 --> 01:04:31,880
Now I have the samples of a star
at the n-th roots of unity--

1138
01:04:31,880 --> 01:04:36,400
these guys-- and
I have the samples

1139
01:04:36,400 --> 01:04:40,250
of b at the exact same points--
the n-th roots of unity--

1140
01:04:40,250 --> 01:04:45,900
so I can compute c star, the
transform version-- the DFT

1141
01:04:45,900 --> 01:04:51,920
of c-- is just the,
I mean, c star k

1142
01:04:51,920 --> 01:04:55,900
equals a star k times b star k.

1143
01:04:55,900 --> 01:05:01,160
This is the multiplication
algorithm for sample vectors.

1144
01:05:01,160 --> 01:05:02,440
We started with that.

1145
01:05:02,440 --> 01:05:03,990
That's linear time.

1146
01:05:03,990 --> 01:05:06,420
And then the missing
piece is I need

1147
01:05:06,420 --> 01:05:10,640
to re-compute c, which is
the inverse fast Fourier

1148
01:05:10,640 --> 01:05:14,005
transform of c star.

1149
01:05:17,940 --> 01:05:21,030
So this is the missing
link, so to speak.

1150
01:05:21,030 --> 01:05:22,477
We need to be able
to go backwards

1151
01:05:22,477 --> 01:05:23,435
in this transformation.

1152
01:05:32,910 --> 01:05:36,630
Good news is the algorithm
is not going to change.

1153
01:05:36,630 --> 01:05:40,104
What we're computing
isn't going to change.

1154
01:05:40,104 --> 01:05:41,770
All that's going to
change are the xk's.

1155
01:05:45,230 --> 01:05:46,740
Why?

1156
01:05:46,740 --> 01:05:51,210
Because, remember
from the top-- right

1157
01:05:51,210 --> 01:05:53,940
now we know how to compute
v times a, but what we now

1158
01:05:53,940 --> 01:05:57,230
need to compute is
the inverse times a.

1159
01:05:57,230 --> 01:06:00,470
So the only question
is, what is the inverse?

1160
01:06:00,470 --> 01:06:05,660
This matrix has a super special
structure-- it's symmetric,

1161
01:06:05,660 --> 01:06:08,340
lots of points on a
circle-- maybe the inverse

1162
01:06:08,340 --> 01:06:10,560
has a nice structure.

1163
01:06:10,560 --> 01:06:13,070
And it does.

1164
01:06:13,070 --> 01:06:25,220
Claim is the inverse is v
complex conjugate divided by n.

1165
01:06:25,220 --> 01:06:27,340
What's complex conjugate?

1166
01:06:27,340 --> 01:06:30,220
For a geometer, it's
reflection through the x-axis.

1167
01:06:30,220 --> 01:06:35,130
For an algebraic
person, a plus ib,

1168
01:06:35,130 --> 01:06:40,000
the complex conjugate
is a minus ib.

1169
01:06:40,000 --> 01:06:43,560
So just apply that to
every entry in the matrix,

1170
01:06:43,560 --> 01:06:45,647
and then divide all
the entries by n.

1171
01:06:45,647 --> 01:06:46,480
You get the inverse.

1172
01:06:50,130 --> 01:06:51,490
Cool.

1173
01:06:51,490 --> 01:06:55,610
Very cool because what
this tells us is we

1174
01:06:55,610 --> 01:06:58,850
run exactly the same
algorithm and do exactly

1175
01:06:58,850 --> 01:07:01,320
the same transformation.

1176
01:07:01,320 --> 01:07:04,410
If we want to do the
inverse we can actually

1177
01:07:04,410 --> 01:07:08,310
just use v, but with a
different choice of xk.

1178
01:07:08,310 --> 01:07:13,315
Namely, for the inverse,
we'll call it xk inverse.

1179
01:07:16,280 --> 01:07:19,230
We just take the
complex conjugate

1180
01:07:19,230 --> 01:07:24,040
of this thing, which
turns out to be

1181
01:07:24,040 --> 01:07:32,534
e to the minus ijk
tau over n, and then

1182
01:07:32,534 --> 01:07:33,700
divide the whole thing by n.

1183
01:07:36,610 --> 01:07:39,550
I'm using a fact here, which
is that the complex conjugate

1184
01:07:39,550 --> 01:07:42,990
of this number is
actually, just,

1185
01:07:42,990 --> 01:07:45,090
you put a minus sign here.

1186
01:07:45,090 --> 01:07:45,940
Why does that hold?

1187
01:07:45,940 --> 01:07:46,815
Because geometry.

1188
01:07:49,350 --> 01:07:53,040
Theta is usually measuring
the counterclockwise angle

1189
01:07:53,040 --> 01:07:54,990
from the x-axis.

1190
01:07:54,990 --> 01:07:57,890
If you take the complex
conjugate you go from up here

1191
01:07:57,890 --> 01:08:00,395
to down here-- the
reflection to the x-axis.

1192
01:08:00,395 --> 01:08:02,270
That's the same thing
as if you measure theta

1193
01:08:02,270 --> 01:08:05,380
as a clockwise angle
from the x-axis.

1194
01:08:05,380 --> 01:08:07,790
That's the same thing
as negating the angle.

1195
01:08:07,790 --> 01:08:09,361
So that's just a
little geometry.

1196
01:08:09,361 --> 01:08:11,110
You can prove it
algebraically, although I

1197
01:08:11,110 --> 01:08:13,850
don't know how off-hand.

1198
01:08:13,850 --> 01:08:15,830
It's not hard.

1199
01:08:15,830 --> 01:08:18,370
So if I want to take
some angle and then

1200
01:08:18,370 --> 01:08:21,189
flip it through the x-axis,
it's the same as the negative

1201
01:08:21,189 --> 01:08:21,750
of the angle.

1202
01:08:21,750 --> 01:08:24,439
So the complex
conjugate of this number

1203
01:08:24,439 --> 01:08:26,997
is the same thing
with a minus sign.

1204
01:08:26,997 --> 01:08:28,330
And then we have to divide by n.

1205
01:08:28,330 --> 01:08:31,570
If I just-- did I lie?

1206
01:08:34,979 --> 01:08:37,279
I can't divide by n, sorry.

1207
01:08:37,279 --> 01:08:38,640
It's in the wrong spot.

1208
01:08:38,640 --> 01:08:43,670
I use these xk's and then
I apply the transform.

1209
01:08:43,670 --> 01:08:45,670
I take this thing.

1210
01:08:45,670 --> 01:08:49,939
I get not quite the inverse, but
I end up getting n the inverse.

1211
01:08:49,939 --> 01:08:53,649
I multiply that by my a star.

1212
01:08:53,649 --> 01:08:55,702
That's going to
give me n times n.

1213
01:09:02,000 --> 01:09:05,289
So then I just take that
vector and divided by n.

1214
01:09:05,289 --> 01:09:06,830
Boom, I've got the
inverse transform.

1215
01:09:06,830 --> 01:09:13,200
So this is how you do inverse
fast Fourier transform.

1216
01:09:13,200 --> 01:09:17,000
You just flip the sign and
the exponent in the xk's,

1217
01:09:17,000 --> 01:09:20,270
then you do the same
matrix-vector product,

1218
01:09:20,270 --> 01:09:21,495
and then you divide by n.

1219
01:09:21,495 --> 01:09:22,870
And then you've
done the inverse.

1220
01:09:22,870 --> 01:09:25,757
So that's how you do-- if you
believe this claim-- that's

1221
01:09:25,757 --> 01:09:26,840
how you do this last step.

1222
01:09:26,840 --> 01:09:27,340
Question?

1223
01:09:27,340 --> 01:09:30,180
AUDIENCE: What's the j
in the x j [INAUDIBLE].

1224
01:09:30,180 --> 01:09:33,330
ERIK DEMAINE:
Whoops, no j, sorry.

1225
01:09:33,330 --> 01:09:37,180
I was imagining these guys.

1226
01:09:37,180 --> 01:09:38,680
Just k.

1227
01:09:38,680 --> 01:09:40,700
Thank you.

1228
01:09:40,700 --> 01:09:43,720
Other questions?

1229
01:09:43,720 --> 01:09:48,390
So what remains is
to prove this claim.

1230
01:09:48,390 --> 01:09:54,420
I'd be very happy if I-- I
should probably have better--

1231
01:09:54,420 --> 01:09:57,570
anyway, that's not the best.

1232
01:09:57,570 --> 01:10:01,600
I'm going to call this xk prime.

1233
01:10:01,600 --> 01:10:06,850
Then when I plug that in, I get
a different matrix, v prime.

1234
01:10:06,850 --> 01:10:12,040
And what this claim
says is that v prime is

1235
01:10:12,040 --> 01:10:16,330
equal to n times the inverse.

1236
01:10:16,330 --> 01:10:18,810
So I apply the same
f of t algorithm,

1237
01:10:18,810 --> 01:10:21,510
but with v prime
instead of v, then I

1238
01:10:21,510 --> 01:10:23,390
get not quite the
product I want,

1239
01:10:23,390 --> 01:10:25,230
but just n times
the product I want.

1240
01:10:25,230 --> 01:10:29,165
Divide every term by n
and we get the inverse.

1241
01:10:29,165 --> 01:10:31,040
This is the cool thing
about complex numbers.

1242
01:10:31,040 --> 01:10:33,580
Here we get another cool thing.

1243
01:10:33,580 --> 01:10:36,120
All right, but we have
to prove this claim.

1244
01:10:36,120 --> 01:10:38,420
So let's do a little
bit of algebra.

1245
01:10:38,420 --> 01:10:39,240
No pain, no gain.

1246
01:10:41,851 --> 01:10:42,350
Good.

1247
01:10:45,460 --> 01:10:49,885
So let's look at vjk prime.

1248
01:10:55,240 --> 01:10:55,740
Sorry.

1249
01:11:04,690 --> 01:11:12,460
So let's look at p,
the product of v,

1250
01:11:12,460 --> 01:11:18,550
and v complex conjugate-- what
I was calling v prime up there.

1251
01:11:18,550 --> 01:11:22,500
I claim that this thing
is n times the identity

1252
01:11:22,500 --> 01:11:24,355
matrix, with 1's
down the diagonal

1253
01:11:24,355 --> 01:11:26,860
and 0's everywhere else.

1254
01:11:26,860 --> 01:11:28,990
So let's look at this product.

1255
01:11:28,990 --> 01:11:35,370
In general, let's look at the
the jk-th item in the product.

1256
01:11:35,370 --> 01:11:43,670
That's going to come from row
j of V, dot product with column

1257
01:11:43,670 --> 01:11:47,620
k of the complex conjugate.

1258
01:11:47,620 --> 01:11:49,120
Now the matrices
here are symmetric,

1259
01:11:49,120 --> 01:11:50,890
so actually rows and
columns are the same,

1260
01:11:50,890 --> 01:11:53,267
but that's the general
definition of the cell

1261
01:11:53,267 --> 01:11:54,600
and the product of two matrices.

1262
01:11:57,330 --> 01:12:01,170
So let's write it
out and a summation.

1263
01:12:01,170 --> 01:12:05,560
We have-- from m equals
0-- it's so hard not

1264
01:12:05,560 --> 01:12:07,450
to use i for my summations.

1265
01:12:07,450 --> 01:12:10,270
It's the only class I have to do
it because i is already taken,

1266
01:12:10,270 --> 01:12:12,040
but I guess I can
still use capital I,

1267
01:12:12,040 --> 01:12:18,120
but we'll use m because i
is complex number today.

1268
01:12:18,120 --> 01:12:30,570
So we have e to the i
tau jm over n times e

1269
01:12:30,570 --> 01:12:36,010
to the minus i tau mk k over n.

1270
01:12:36,010 --> 01:12:37,924
I don't know why I
changed the order.

1271
01:12:37,924 --> 01:12:40,090
I put the tau here instead
of there, but same thing.

1272
01:12:42,720 --> 01:12:46,040
This is just the
for every position

1273
01:12:46,040 --> 01:12:47,980
m in the cell and
corresponding position

1274
01:12:47,980 --> 01:12:51,020
m in-- sorry, m in the
row, corresponding position

1275
01:12:51,020 --> 01:12:55,929
m in the column, I have
jm and I have mk-- again,

1276
01:12:55,929 --> 01:12:56,970
the order doesn't matter.

1277
01:12:56,970 --> 01:12:59,600
It's symmetric, but I'm
getting it right here.

1278
01:12:59,600 --> 01:13:03,520
This is-- we're
using this formula.

1279
01:13:03,520 --> 01:13:09,600
I put a minus sign here because
this is the complex conjugate.

1280
01:13:09,600 --> 01:13:11,440
So now I just do some algebra.

1281
01:13:11,440 --> 01:13:12,230
These share a lot.

1282
01:13:12,230 --> 01:13:13,520
They share i.

1283
01:13:13,520 --> 01:13:20,500
They share m, and they
share the divided by n.

1284
01:13:20,500 --> 01:13:28,910
So this is sum m equals 0,
n minus 1 e to the i-- oh,

1285
01:13:28,910 --> 01:13:40,060
they also share tau-- tau
m over n times j minus k.

1286
01:13:40,060 --> 01:13:43,450
Please correct me if
I make any mistakes.

1287
01:13:43,450 --> 01:13:43,950
Cool.

1288
01:13:43,950 --> 01:13:48,660
So it depends how and
k relate, of course.

1289
01:13:48,660 --> 01:13:54,410
If j equals k, that's also known
as something on the diagonal.

1290
01:13:54,410 --> 01:13:59,090
I want the matrix
n, n, n, n, 0, 0.

1291
01:13:59,090 --> 01:14:00,590
So j equals k.

1292
01:14:00,590 --> 01:14:01,510
That's the diagonal.

1293
01:14:01,510 --> 01:14:04,240
That's where I want to get n,
and, indeed, if j equals k,

1294
01:14:04,240 --> 01:14:05,320
this becomes 0.

1295
01:14:05,320 --> 01:14:09,410
So all this becomes
0. e to the 0 is 1,

1296
01:14:09,410 --> 01:14:11,920
and so I'm summing up 1 n times.

1297
01:14:11,920 --> 01:14:14,260
I get n.

1298
01:14:14,260 --> 01:14:15,060
Cool.

1299
01:14:15,060 --> 01:14:15,810
That's one case.

1300
01:14:15,810 --> 01:14:21,110
This is n if j equals k, and
somehow I claim that everywhere

1301
01:14:21,110 --> 01:14:23,300
else I get 0's.

1302
01:14:23,300 --> 01:14:24,750
So let's prove that.

1303
01:14:43,810 --> 01:14:51,030
So if j does not
equal k-- I'm going

1304
01:14:51,030 --> 01:14:54,070
to rewrite this a little bit.

1305
01:14:54,070 --> 01:14:57,010
j and k are fixed.

1306
01:14:57,010 --> 01:15:01,500
m is changing in the sum,
so I want to write this

1307
01:15:01,500 --> 01:15:17,880
as sum m equals 0 to n minus
1, e to the i tau j minus k,

1308
01:15:17,880 --> 01:15:22,510
over m, times m.

1309
01:15:22,510 --> 01:15:27,430
In other words, this thing
raised to the n-th power.

1310
01:15:27,430 --> 01:15:28,569
What is this series?

1311
01:15:33,460 --> 01:15:34,836
One word.

1312
01:15:34,836 --> 01:15:35,760
AUDIENCE: Geometric.

1313
01:15:35,760 --> 01:15:36,760
ERIK DEMAINE: Geometric.

1314
01:15:36,760 --> 01:15:37,435
Thank you.

1315
01:15:41,100 --> 01:15:44,820
This is a geometric
series, and I

1316
01:15:44,820 --> 01:15:48,010
guess I should have waited to
give you the Frisbee until you

1317
01:15:48,010 --> 01:15:50,380
tell me how do you
solve a geometric series

1318
01:15:50,380 --> 01:15:51,660
with a finite term?

1319
01:15:51,660 --> 01:15:55,050
So think of this as z to the m.

1320
01:15:55,050 --> 01:15:56,080
Do you know the formula?

1321
01:15:56,080 --> 01:15:59,315
AUDIENCE: That's
the e to the i tau--

1322
01:15:59,315 --> 01:16:00,190
ERIK DEMAINE: Just z.

1323
01:16:00,190 --> 01:16:03,295
ERIK DEMAINE: Oh,
z to the n minus 1.

1324
01:16:03,295 --> 01:16:04,670
ERIK DEMAINE: Z
to the n minus 1.

1325
01:16:04,670 --> 01:16:05,290
Almost.

1326
01:16:05,290 --> 01:16:06,332
AUDIENCE: Over z minus 1.

1327
01:16:06,332 --> 01:16:07,540
ERIK DEMAINE: Over z minus 1.

1328
01:16:07,540 --> 01:16:08,250
Yep.

1329
01:16:08,250 --> 01:16:14,210
That's-- If you have sum of z to
the k, k equals 0 to n minus 1.

1330
01:16:14,210 --> 01:16:14,830
That's that.

1331
01:16:14,830 --> 01:16:17,290
It's in the appendix
of your textbook.

1332
01:16:17,290 --> 01:16:21,340
So we just plug
that in and we get--

1333
01:16:21,340 --> 01:16:29,380
oh boy-- e to the-- better
chalk-- e to the i tau

1334
01:16:29,380 --> 01:16:39,224
j minus k over n to the n-th
power minus 1 over e to the i.

1335
01:16:39,224 --> 01:16:40,890
Actually, the denominator
doesn't really

1336
01:16:40,890 --> 01:16:43,660
matter because this
is supposed to be 0,

1337
01:16:43,660 --> 01:16:48,360
so it's all about the numerator,
but it's the same thing.

1338
01:16:48,360 --> 01:16:49,130
Minus 1.

1339
01:16:53,130 --> 01:16:54,306
Where's my red?

1340
01:16:54,306 --> 01:16:56,590
Red.

1341
01:16:56,590 --> 01:17:00,280
The n cancels with the n.

1342
01:17:00,280 --> 01:17:07,090
This is an integer not equal to
zero, so we have e to the i tau

1343
01:17:07,090 --> 01:17:09,020
to an integer power.

1344
01:17:09,020 --> 01:17:12,270
What's e to the i tau?

1345
01:17:12,270 --> 01:17:12,770
1.

1346
01:17:15,430 --> 01:17:17,170
Convenient that I
have that there.

1347
01:17:17,170 --> 01:17:21,011
1 minus 1 is 0, so we get 0.

1348
01:17:21,011 --> 01:17:21,510
Satisfying.

1349
01:17:24,670 --> 01:17:26,700
So that proves this claim.

1350
01:17:26,700 --> 01:17:28,780
We prove that, on the
diagonal, we get n

1351
01:17:28,780 --> 01:17:30,370
because we had n copies of 1.

1352
01:17:30,370 --> 01:17:34,110
Off the diagonal we
get 0, like this.

1353
01:17:34,110 --> 01:17:40,530
Therefore, v complex conjugate
times n is the v inverse.

1354
01:17:40,530 --> 01:17:42,320
Therefore, this
algorithm actually

1355
01:17:42,320 --> 01:17:46,520
computes the inverse
fast Fourier transform.

1356
01:17:46,520 --> 01:17:49,460
So that's it for
algorithms today,

1357
01:17:49,460 --> 01:17:52,730
but let me quickly tell you
about some applications.

1358
01:17:52,730 --> 01:17:54,560
You've probably
taken other classes

1359
01:17:54,560 --> 01:17:56,560
that use applications of
fast Fourier transform,

1360
01:17:56,560 --> 01:17:58,510
so I will just summarize.

1361
01:17:58,510 --> 01:18:02,380
If you've ever edited
audio, you probably

1362
01:18:02,380 --> 01:18:08,490
did it in-- unless you're just
pasting audio clips together--

1363
01:18:08,490 --> 01:18:11,160
you probably did it in what's
called frequency space.

1364
01:18:11,160 --> 01:18:13,910
So you know that-- as I talked
about in the beginning--

1365
01:18:13,910 --> 01:18:16,480
when we're measuring where the
membrane on this microphone

1366
01:18:16,480 --> 01:18:21,550
goes over time, that is in
the time domain for every time

1367
01:18:21,550 --> 01:18:23,850
we sample where,
physically, this thing is.

1368
01:18:23,850 --> 01:18:26,277
If you apply-- I think
the way I defined it here

1369
01:18:26,277 --> 01:18:27,860
is the inverse fast
Fourier transform,

1370
01:18:27,860 --> 01:18:32,430
usually it's called Fourier
transform-- to that time domain

1371
01:18:32,430 --> 01:18:34,950
vector, you get a new vector.

1372
01:18:34,950 --> 01:18:36,826
Now it's a complex vector.

1373
01:18:36,826 --> 01:18:38,450
You may have started
with real numbers.

1374
01:18:38,450 --> 01:18:40,000
You get complex numbers.

1375
01:18:40,000 --> 01:18:43,850
So for every position
in the vector--

1376
01:18:43,850 --> 01:18:47,060
what it corresponds to-- the
x-axis is no longer time.

1377
01:18:47,060 --> 01:18:48,360
Now it's frequency.

1378
01:18:48,360 --> 01:18:51,390
For every frequency
you're measuring,

1379
01:18:51,390 --> 01:18:53,510
essentially, you're
viewing this vector--

1380
01:18:53,510 --> 01:18:57,130
the waveform-- as a bunch
of trigonometric functions--

1381
01:18:57,130 --> 01:19:01,201
say, sine of
something times theta.

1382
01:19:01,201 --> 01:19:03,200
If you look at one of the
entries in the vector,

1383
01:19:03,200 --> 01:19:04,810
and it's a complex
number-- if you compute

1384
01:19:04,810 --> 01:19:06,351
the magnitude of
the complex number--

1385
01:19:06,351 --> 01:19:08,640
the length of the
vector, of the length

1386
01:19:08,640 --> 01:19:13,695
of that two-coordinate vector--
that is how much stuff--

1387
01:19:13,695 --> 01:19:15,900
of that frequency-- you have.

1388
01:19:15,900 --> 01:19:18,630
And then the angle
of the vector, in 2D,

1389
01:19:18,630 --> 01:19:23,670
is how that trigonometric
function shifted in time.

1390
01:19:23,670 --> 01:19:27,900
So if you take a pure note,
like if I was playing a bell

1391
01:19:27,900 --> 01:19:32,980
and it's exactly C major,
it looks really wavy.

1392
01:19:32,980 --> 01:19:35,780
It's actually a
nice perfect sine

1393
01:19:35,780 --> 01:19:39,127
curve-- some offset
depending on when I hit it--

1394
01:19:39,127 --> 01:19:41,460
and then if you apply the
Fourier transform what you get

1395
01:19:41,460 --> 01:19:44,790
is 0's everywhere except for the
one frequency that's appearing,

1396
01:19:44,790 --> 01:19:48,540
and there you get 1, and
everywhere else you get 0.

1397
01:19:48,540 --> 01:19:52,650
Well, 1 possibly rotated,
depending on the phase.

1398
01:19:52,650 --> 01:19:55,410
And you can take
any audio stream,

1399
01:19:55,410 --> 01:19:57,292
convert it by a
Fourier transform,

1400
01:19:57,292 --> 01:19:58,250
do manipulations there.

1401
01:19:58,250 --> 01:20:01,095
For example, you've probably
heard of high-pass filters that

1402
01:20:01,095 --> 01:20:02,470
removes all the
high frequencies,

1403
01:20:02,470 --> 01:20:04,400
or low-pass filters remove
all of the low frequencies.

1404
01:20:04,400 --> 01:20:06,225
You just convert to
this space and zero out

1405
01:20:06,225 --> 01:20:08,100
the parts you want, and
then you convert back

1406
01:20:08,100 --> 01:20:10,230
with inverse Fourier transform.

1407
01:20:10,230 --> 01:20:11,980
If you've used Adobe
Audition or Audacity,

1408
01:20:11,980 --> 01:20:14,140
they can all do these things.

1409
01:20:14,140 --> 01:20:15,720
And there are tons
of contexts where

1410
01:20:15,720 --> 01:20:18,500
converting to Fourier
space makes life easy.

1411
01:20:18,500 --> 01:20:20,790
And, in general, if you
have any time-based signal

1412
01:20:20,790 --> 01:20:23,220
you should always think about
what do you get with FFT

1413
01:20:23,220 --> 01:20:26,714
when you transform to
frequency-based signal,

1414
01:20:26,714 --> 01:20:28,130
and you can do
lots of cool things

1415
01:20:28,130 --> 01:20:29,213
you couldn't do otherwise.

1416
01:20:29,213 --> 01:20:30,790
And it only takes n log n time.

1417
01:20:30,790 --> 01:20:32,360
Plus people do it in hardware.

1418
01:20:32,360 --> 01:20:33,860
There's a fast
implementation called

1419
01:20:33,860 --> 01:20:37,640
FFTW-- the fastest Fourier
transform in the west-- which

1420
01:20:37,640 --> 01:20:40,360
is made here at MIT
a bunch of years ago,

1421
01:20:40,360 --> 01:20:43,390
and is still the best software
implementation of FFT.

1422
01:20:43,390 --> 01:20:46,150
So people use it everywhere.

1423
01:20:46,150 --> 01:20:49,720
Your noise-cancelling headsets
probably use it, MP3 uses it.

1424
01:20:49,720 --> 01:20:51,146
It's a cool algorithm.