1
00:00:00,080 --> 00:00:02,500
The following content is
provided under a Creative

2
00:00:02,500 --> 00:00:04,010
Commons license.

3
00:00:04,010 --> 00:00:06,360
Your support will help
MIT OpenCourseWare

4
00:00:06,360 --> 00:00:10,730
continue to offer high-quality
educational resources for free.

5
00:00:10,730 --> 00:00:13,340
To make a donation or
view additional materials

6
00:00:13,340 --> 00:00:17,236
from hundreds of MIT courses,
visit MIT OpenCourseWare

7
00:00:17,236 --> 00:00:17,861
at ocw.mit.edu.

8
00:00:21,720 --> 00:00:25,520
PROFESSOR: All right, welcome
to the final lecture of 6046.

9
00:00:25,520 --> 00:00:28,617
Today we continue our theme
of cache oblivious algorithms.

10
00:00:28,617 --> 00:00:30,200
We're going to look
at two of the most

11
00:00:30,200 --> 00:00:32,259
basic problems in
computer science--

12
00:00:32,259 --> 00:00:36,080
searching and sorting,
a little bit of each.

13
00:00:36,080 --> 00:00:38,510
And then I'll tell you a
little bit about what class

14
00:00:38,510 --> 00:00:41,450
you might take after this one.

15
00:00:41,450 --> 00:00:44,970
So brief recap of the model,
we introduced two models

16
00:00:44,970 --> 00:00:48,430
of computation although one was
just a variation of the other.

17
00:00:48,430 --> 00:00:51,020
The base model is an
external memory model.

18
00:00:51,020 --> 00:00:53,820
This is a two-level
memory hierarchy.

19
00:00:53,820 --> 00:00:55,890
CPU and cache, we view as one.

20
00:00:55,890 --> 00:00:58,240
So there's instant
communication between them,

21
00:00:58,240 --> 00:01:00,760
which means what
you're computing on

22
00:01:00,760 --> 00:01:05,469
can involve this cache of size
m-- total size of the cache

23
00:01:05,469 --> 00:01:07,320
is m-- words.

24
00:01:07,320 --> 00:01:11,110
The cache is divided into
these blocks of size b each.

25
00:01:11,110 --> 00:01:14,110
So they're m over b blocks.

26
00:01:14,110 --> 00:01:17,710
And your problem doesn't
fit here, presumably,

27
00:01:17,710 --> 00:01:19,730
or the problem's not
very interesting.

28
00:01:19,730 --> 00:01:22,990
So your problem size n is
going to require storing

29
00:01:22,990 --> 00:01:24,220
your information on disk.

30
00:01:24,220 --> 00:01:26,400
So the input is really
provided over here.

31
00:01:26,400 --> 00:01:28,370
Disk is basically
infinite in size.

32
00:01:28,370 --> 00:01:30,170
It's also partitioned
into blocks.

33
00:01:30,170 --> 00:01:32,930
And you can't access
individual items here.

34
00:01:32,930 --> 00:01:34,480
You can only access
entire blocks.

35
00:01:34,480 --> 00:01:37,390
So the model is you say,
I want to read this block

36
00:01:37,390 --> 00:01:38,670
and put it here.

37
00:01:38,670 --> 00:01:40,667
I want to write this
block out and put it here.

38
00:01:40,667 --> 00:01:43,250
That's what you're allowed to
do in the external memory model.

39
00:01:43,250 --> 00:01:46,609
And what we count is how many
block-memory transfers we do.

40
00:01:46,609 --> 00:01:47,900
We call those memory transfers.

41
00:01:47,900 --> 00:01:49,520
So you want to minimize that.

42
00:01:49,520 --> 00:01:51,450
And usually you
don't worry too much

43
00:01:51,450 --> 00:01:53,575
about what happens in here,
although you could also

44
00:01:53,575 --> 00:01:57,010
minimize regular running
time as we usually do.

45
00:01:57,010 --> 00:02:00,260
The cache oblivious variation
is that the algorithm is not

46
00:02:00,260 --> 00:02:01,870
allowed to know the
cache parameters.

47
00:02:01,870 --> 00:02:04,150
It's not allowed to
know the block size.

48
00:02:04,150 --> 00:02:06,700
Sorry, it's also block
size b in the disk.

49
00:02:06,700 --> 00:02:08,509
So they match.

50
00:02:08,509 --> 00:02:13,150
And you're not allowed to
know the cache size, m.

51
00:02:13,150 --> 00:02:16,150
Because of that, all the
block reads and writes

52
00:02:16,150 --> 00:02:17,160
are done automatically.

53
00:02:17,160 --> 00:02:20,190
So the model is, whenever
you access an item,

54
00:02:20,190 --> 00:02:22,650
you view the disk
as just written row

55
00:02:22,650 --> 00:02:25,570
by row-- sequentially
block by block.

56
00:02:25,570 --> 00:02:30,610
So in linear eyes, it looks like
this, partitioned into blocks.

57
00:02:30,610 --> 00:02:32,780
And so whenever
you touch an item,

58
00:02:32,780 --> 00:02:35,330
the system automatically
loads that block.

59
00:02:35,330 --> 00:02:37,520
If it's not already in
cache, it loads it in.

60
00:02:37,520 --> 00:02:40,490
If it's already in
cache, it's free.

61
00:02:40,490 --> 00:02:43,042
When you load a block
in, you probably

62
00:02:43,042 --> 00:02:44,250
already have something there.

63
00:02:44,250 --> 00:02:46,570
So if the cache is
already full, you

64
00:02:46,570 --> 00:02:48,250
have to decide
which one to evict.

65
00:02:48,250 --> 00:02:49,800
And we had a couple
of strategies.

66
00:02:49,800 --> 00:02:53,130
But the one I defined was the
least-recently-used block.

67
00:02:53,130 --> 00:02:55,290
So whichever one in the
cache that's least recently

68
00:02:55,290 --> 00:02:57,320
been used by the
CPU, that's the one

69
00:02:57,320 --> 00:02:58,970
that gets written
out, back to disk

70
00:02:58,970 --> 00:03:01,460
where it originally came from.

71
00:03:01,460 --> 00:03:02,062
And that's it.

72
00:03:02,062 --> 00:03:03,830
That's the model.

73
00:03:03,830 --> 00:03:08,970
OK, this is a pretty good
model of how real caches work.

74
00:03:08,970 --> 00:03:12,410
Although this last part is
not how all real caches work.

75
00:03:12,410 --> 00:03:14,280
It's close.

76
00:03:14,280 --> 00:03:16,560
And at the very end, I
mentioned this theorem

77
00:03:16,560 --> 00:03:20,610
that Why LRU is good.

78
00:03:20,610 --> 00:03:24,830
And if you take
the number of block

79
00:03:24,830 --> 00:03:28,930
evictions-- the
number of block reads,

80
00:03:28,930 --> 00:03:34,530
equivalently-- then LRU has
to do on a cache of size m.

81
00:03:34,530 --> 00:03:37,280
Then that's going to be,
at most, twice whatever

82
00:03:37,280 --> 00:03:40,260
the best possible
thing you could do

83
00:03:40,260 --> 00:03:43,370
is given a cache
of size m over 2.

84
00:03:43,370 --> 00:03:44,860
So we're restricting OPT.

85
00:03:44,860 --> 00:03:47,680
We're kind of tying OPT's hands
behind his back a little bit

86
00:03:47,680 --> 00:03:50,100
by decreasing m
by a factor of 2.

87
00:03:50,100 --> 00:03:53,150
But then we get a factor of
2 approximation, basically.

88
00:03:53,150 --> 00:03:55,150
So this was the
resource augmentation.

89
00:03:55,150 --> 00:03:57,380
And this is regular
approximation algorithms.

90
00:03:57,380 --> 00:04:02,880
In general, this is a world
called online algorithms, which

91
00:04:02,880 --> 00:04:06,360
is a whole field.

92
00:04:06,360 --> 00:04:09,430
I'm just going to
mention it briefly here.

93
00:04:09,430 --> 00:04:12,270
The distinction here
is LRU, or whatever

94
00:04:12,270 --> 00:04:14,370
we implement in a
real system, has

95
00:04:14,370 --> 00:04:19,360
to make a decision based only
on the past of what's happened.

96
00:04:19,360 --> 00:04:22,610
The system, we're assuming,
doesn't know the future.

97
00:04:22,610 --> 00:04:25,790
So in a compiler, maybe you
could try to predict the future

98
00:04:25,790 --> 00:04:26,870
and do something.

99
00:04:26,870 --> 00:04:28,830
But on a CPU, it
doesn't know what

100
00:04:28,830 --> 00:04:32,040
instruction's going to come
next, 10 steps in the future.

101
00:04:32,040 --> 00:04:34,220
So you just have to
make a decision now,

102
00:04:34,220 --> 00:04:35,220
sort of your best guess.

103
00:04:35,220 --> 00:04:38,490
And least recently used
is a good best guess.

104
00:04:38,490 --> 00:04:42,340
OPT, on the other hand,
we're giving a lot of power.

105
00:04:42,340 --> 00:04:44,245
This is what we call
an offline algorithm.

106
00:04:48,550 --> 00:04:50,880
It's like the Q in Star
Trek: Next Generation

107
00:04:50,880 --> 00:04:52,920
or some other mythical being.

108
00:04:52,920 --> 00:04:54,640
It lives outside
of the timeline.

109
00:04:54,640 --> 00:04:58,910
It can see all of time and say,
I think I'll evict this block.

110
00:04:58,910 --> 00:05:00,830
This is like the waste
of Q's resources.

111
00:05:00,830 --> 00:05:02,340
But I'll evict this
block because I

112
00:05:02,340 --> 00:05:05,880
know it's going to be used
for this in the future.

113
00:05:05,880 --> 00:05:10,060
LRU is evicting the thing that
was used farthest in the past.

114
00:05:10,060 --> 00:05:11,380
There's a difference there.

115
00:05:11,380 --> 00:05:13,350
And it could be
a big difference.

116
00:05:13,350 --> 00:05:15,620
But it turns out they're
related in this way.

117
00:05:15,620 --> 00:05:17,830
So this is what we call
an online algorithm,

118
00:05:17,830 --> 00:05:20,737
meaning you have to make
decisions as you go.

119
00:05:20,737 --> 00:05:22,570
The offline algorithm
gets to see the future

120
00:05:22,570 --> 00:05:24,000
and optimize accordingly.

121
00:05:24,000 --> 00:05:26,813
Both are computable, but this
one's only computable if you

122
00:05:26,813 --> 00:05:29,170
know the future, which we don't.

123
00:05:31,910 --> 00:05:33,660
What I haven't done
is prove this theorem.

124
00:05:33,660 --> 00:05:37,015
It's actually really easy proof.

125
00:05:37,015 --> 00:05:37,640
So let's do it.

126
00:05:41,690 --> 00:05:51,612
I want to take the timeline
and divide it into phases.

127
00:05:51,612 --> 00:05:52,765
Phases sounds cool.

128
00:06:12,312 --> 00:06:13,770
So this is going
to be an analysis.

129
00:06:13,770 --> 00:06:15,340
And in an analysis,
we're allowed

130
00:06:15,340 --> 00:06:17,631
to know the future because
we're trying to imagine what

131
00:06:17,631 --> 00:06:19,200
OPT could do relative to LRU.

132
00:06:19,200 --> 00:06:20,450
So we're fixing the algorithm.

133
00:06:20,450 --> 00:06:21,965
It's obviously not
using the future.

134
00:06:21,965 --> 00:06:24,340
When we analyze it, we're
assuming we do know the future.

135
00:06:24,340 --> 00:06:26,009
We know the entire timeline.

136
00:06:26,009 --> 00:06:27,800
So all the algorithms
we covered last time,

137
00:06:27,800 --> 00:06:30,341
all the ones we covered today
you can think of as just making

138
00:06:30,341 --> 00:06:32,470
a sequence of accesses.

139
00:06:32,470 --> 00:06:35,010
They're making sequences
of accesses to elements.

140
00:06:35,010 --> 00:06:38,130
But if we assume
we know what b is,

141
00:06:38,130 --> 00:06:41,000
that's just a sequence
of accesses to blocks.

142
00:06:41,000 --> 00:06:43,590
OK so you can just
think of the timeline

143
00:06:43,590 --> 00:06:47,010
as a sequence of block IDs.

144
00:06:47,010 --> 00:06:50,680
And if you access a block that's
currently stored in cache,

145
00:06:50,680 --> 00:06:51,400
it's free.

146
00:06:51,400 --> 00:06:53,380
Otherwise, you pay 1.

147
00:06:53,380 --> 00:06:55,970
All right, so I'm just going
to look at the timeline of all

148
00:06:55,970 --> 00:07:01,530
these accesses and say, well,
take a prefix of the accesses

149
00:07:01,530 --> 00:07:08,640
until I get to m over b
distinct blocks, block IDs.

150
00:07:08,640 --> 00:07:11,900
Keep going until,
if I went one more,

151
00:07:11,900 --> 00:07:14,230
I'd have m over b plus
1 distinct blocks.

152
00:07:14,230 --> 00:07:17,780
So it's a maximal prefix of
m over b distinct blocks.

153
00:07:17,780 --> 00:07:19,080
Cut there.

154
00:07:19,080 --> 00:07:19,940
And then repeat.

155
00:07:19,940 --> 00:07:20,960
So start over.

156
00:07:20,960 --> 00:07:22,810
Start counting at zero.

157
00:07:22,810 --> 00:07:26,400
Extend until I have m over
b distinct block accesses.

158
00:07:26,400 --> 00:07:29,580
And if I went one more, I'd
have m over b plus 1 and so on.

159
00:07:29,580 --> 00:07:32,160
So the timeline gets divided.

160
00:07:32,160 --> 00:07:32,660
Who knows?

161
00:07:32,660 --> 00:07:34,660
It could be totally irregular.

162
00:07:34,660 --> 00:07:37,120
If you access the same
blocks many times,

163
00:07:37,120 --> 00:07:40,150
you could get along for a
very long time and only access

164
00:07:40,150 --> 00:07:41,220
m over b distinct blocks.

165
00:07:41,220 --> 00:07:41,790
Who knows?

166
00:07:41,790 --> 00:07:43,410
The algorithm
definitely doesn't know

167
00:07:43,410 --> 00:07:44,930
because it doesn't know m or b.

168
00:07:44,930 --> 00:07:46,570
But from an analysis
perspective,

169
00:07:46,570 --> 00:07:47,920
we can just count these things.

170
00:07:47,920 --> 00:07:55,990
So each of these has exactly
m over b distinct accesses,

171
00:07:55,990 --> 00:07:57,860
distinct block IDs.

172
00:07:57,860 --> 00:08:05,610
So I have two claims
about such a phase.

173
00:08:05,610 --> 00:08:17,940
First claim is that LRU with
a cache of size m on one phase

174
00:08:17,940 --> 00:08:19,120
is, at most, what?

175
00:08:24,410 --> 00:08:25,260
It's easy.

176
00:08:28,585 --> 00:08:29,497
STUDENT: M over b.

177
00:08:29,497 --> 00:08:30,330
PROFESSOR: M over b.

178
00:08:33,900 --> 00:08:36,690
The claim is, at most,
m over b basically

179
00:08:36,690 --> 00:08:39,070
because the LRU
is not brain dead.

180
00:08:41,789 --> 00:08:45,360
Well, you're accessing
these blocks.

181
00:08:45,360 --> 00:08:47,480
And they've all been
accessed more recently.

182
00:08:47,480 --> 00:08:49,044
I mean, let's look
at this phase.

183
00:08:49,044 --> 00:08:50,460
All the blocks
that you touch here

184
00:08:50,460 --> 00:08:52,570
have been accessed more recently
than whatever came before.

185
00:08:52,570 --> 00:08:54,290
That's the definition
of this timeline.

186
00:08:54,290 --> 00:08:55,980
This is an order by time.

187
00:08:55,980 --> 00:08:57,980
So anything you
load in here, you

188
00:08:57,980 --> 00:09:00,314
will keep preferentially
over the things that

189
00:09:00,314 --> 00:09:02,480
are not in the phase because
everything in the phase

190
00:09:02,480 --> 00:09:04,190
has been accessed more recently.

191
00:09:04,190 --> 00:09:07,980
So maybe, eventually, you
load all m over b blocks

192
00:09:07,980 --> 00:09:09,270
that are in the phase.

193
00:09:09,270 --> 00:09:12,480
Everything else you touch,
by definition of a phase,

194
00:09:12,480 --> 00:09:13,440
are the same blocks.

195
00:09:13,440 --> 00:09:15,400
So they will remain in cache.

196
00:09:15,400 --> 00:09:17,810
And that's all it
will cost, m over b

197
00:09:17,810 --> 00:09:19,820
memory transfers per phase.

198
00:09:19,820 --> 00:09:22,351
So this is basically ignoring
any carry over from phase

199
00:09:22,351 --> 00:09:22,850
to phase.

200
00:09:22,850 --> 00:09:25,270
This is a conservative
upper bounds.

201
00:09:25,270 --> 00:09:28,237
But it's an upper bounds.

202
00:09:28,237 --> 00:09:30,320
And then the other question
is, what could OPT do?

203
00:09:37,260 --> 00:09:42,160
So OPT-- remember, we're tying
its hands behind its back.

204
00:09:42,160 --> 00:09:44,834
It only has a cache
of size m over 2.

205
00:09:44,834 --> 00:09:46,375
And then we're
evaluating on a phase.

206
00:09:50,290 --> 00:09:54,140
I want to claim that OPT is,
well, at least half of that

207
00:09:54,140 --> 00:09:55,650
if I want to get a factor of 2.

208
00:10:00,260 --> 00:10:03,700
So I claim it's at
least 1/2 m over b.

209
00:10:03,700 --> 00:10:06,180
Why?

210
00:10:06,180 --> 00:10:08,330
Now we have to think
about carry over.

211
00:10:08,330 --> 00:10:11,019
So OPT did something
in this phase.

212
00:10:11,019 --> 00:10:13,560
And then we're wondering what
happens in the very next phase.

213
00:10:13,560 --> 00:10:16,520
So some of these blocks may
be shared with these blocks.

214
00:10:16,520 --> 00:10:17,150
We don't know.

215
00:10:17,150 --> 00:10:18,608
I mean, there's
some set of blocks.

216
00:10:18,608 --> 00:10:21,590
We know that this very first
block was not in the set,

217
00:10:21,590 --> 00:10:23,540
otherwise the phase
would have been longer.

218
00:10:23,540 --> 00:10:25,160
But maybe some
later block happens

219
00:10:25,160 --> 00:10:27,105
to repeat some block
that's over there.

220
00:10:27,105 --> 00:10:27,980
We don't really know.

221
00:10:27,980 --> 00:10:30,060
There could be some carry over.

222
00:10:30,060 --> 00:10:31,930
So how lucky could OPT be?

223
00:10:31,930 --> 00:10:34,770
At this moment in
time, at the beginning

224
00:10:34,770 --> 00:10:36,970
of the phase we're
looking at, it

225
00:10:36,970 --> 00:10:41,060
could be the entire cache
has things that we want,

226
00:10:41,060 --> 00:10:44,730
has blocks that
appear in this phase.

227
00:10:44,730 --> 00:10:47,800
That's the maximum carry
over, the entire cache.

228
00:10:47,800 --> 00:10:56,750
So sort of the best case for
OPT is that the entire cache

229
00:10:56,750 --> 00:11:08,067
is useful, meaning it
contains blocks in the phase

230
00:11:08,067 --> 00:11:10,150
that we're interested in--
the phase we're looking

231
00:11:10,150 --> 00:11:12,340
at-- at the start of the phase.

232
00:11:16,845 --> 00:11:17,720
That's the best case.

233
00:11:17,720 --> 00:11:23,640
But because we gave up only m
over 2, that means, at most,

234
00:11:23,640 --> 00:11:27,120
one half m over b blocks.

235
00:11:27,120 --> 00:11:28,090
This was cache size.

236
00:11:28,090 --> 00:11:30,140
This is the number of
blocks in the cache.

237
00:11:33,370 --> 00:11:36,790
At most, this many
blocks will be free,

238
00:11:36,790 --> 00:11:39,460
won't cost anything for OPT.

239
00:11:39,460 --> 00:11:43,242
But by definition, the phase
has m over b distinct blocks.

240
00:11:43,242 --> 00:11:44,450
So half of them will be free.

241
00:11:44,450 --> 00:11:47,850
The other half, OPT is
going to have to load in.

242
00:11:47,850 --> 00:11:50,500
So it's a kind of
trivial analysis.

243
00:11:50,500 --> 00:11:54,040
It's amazing this
proof is so simple.

244
00:11:54,040 --> 00:11:56,520
It's all about setting
things up right.

245
00:11:56,520 --> 00:11:59,890
If you define phases
that are good for LRU,

246
00:11:59,890 --> 00:12:01,545
then they're also
bad for OPT when

247
00:12:01,545 --> 00:12:03,730
it has cache at half the size.

248
00:12:03,730 --> 00:12:05,460
And so OPT has to pay
at least half what

249
00:12:05,460 --> 00:12:07,560
LRU is definitely paying.

250
00:12:07,560 --> 00:12:09,060
Here we can forget
about carry over.

251
00:12:09,060 --> 00:12:10,940
Here we're bounding
the carry over just

252
00:12:10,940 --> 00:12:12,920
by making the cache smaller.

253
00:12:12,920 --> 00:12:13,420
That's it.

254
00:12:13,420 --> 00:12:16,140
So this is a most twice that.

255
00:12:16,140 --> 00:12:19,230
And so we get the theorem.

256
00:12:19,230 --> 00:12:21,480
I mean, changing the cache
size could dramatically

257
00:12:21,480 --> 00:12:25,010
change the number of cache
reads that you have to do

258
00:12:25,010 --> 00:12:29,300
or disk reads it you
have to do into cache.

259
00:12:29,300 --> 00:12:32,360
But in all of the
algorithms we will cover,

260
00:12:32,360 --> 00:12:34,510
we're giving some
bound in terms of m.

261
00:12:34,510 --> 00:12:36,470
That bound will
always be, at most,

262
00:12:36,470 --> 00:12:38,850
some polynomial dependence in m.

263
00:12:38,850 --> 00:12:41,985
Usually it's like a 1 over
m, 1 over square root of m,

264
00:12:41,985 --> 00:12:45,090
1 over log m,
something like that.

265
00:12:45,090 --> 00:12:46,940
All of those bounds
will only be affected

266
00:12:46,940 --> 00:12:50,610
by a constant factor when you
change m by a constant factor.

267
00:12:50,610 --> 00:12:54,065
So this is good enough for
cache oblivious algorithms.

268
00:12:56,820 --> 00:12:59,990
All right, so that's
sort of review of why

269
00:12:59,990 --> 00:13:02,580
this model is reasonable.

270
00:13:02,580 --> 00:13:04,520
LRU is good.

271
00:13:04,520 --> 00:13:08,030
So now we're going to talk about
two basic problems-- searching

272
00:13:08,030 --> 00:13:13,986
for stuff in array, sorting an
array in both of these models.

273
00:13:13,986 --> 00:13:15,610
We won't be able to
do everything cache

274
00:13:15,610 --> 00:13:16,430
obliviously today.

275
00:13:16,430 --> 00:13:18,370
But they're are all possible.

276
00:13:18,370 --> 00:13:21,510
It just takes more
time than we have.

277
00:13:21,510 --> 00:13:24,150
We'll give you more of a flavor
of how these things work.

278
00:13:24,150 --> 00:13:28,141
Again, the theme is going to
be divide and conquer, my glass

279
00:13:28,141 --> 00:13:28,640
class.

280
00:13:37,240 --> 00:13:40,590
So let's say we have n elements.

281
00:13:40,590 --> 00:13:43,007
Let's say, for simplicity,
we're in the comparison models.

282
00:13:43,007 --> 00:13:44,798
So all we can really
do with those elements

283
00:13:44,798 --> 00:13:47,020
is compare them-- less
than, greater than, equal.

284
00:13:49,670 --> 00:13:52,620
And let's say we
want to do search

285
00:13:52,620 --> 00:13:55,601
in the comparison model, which
I'll think of as a predecessor

286
00:13:55,601 --> 00:13:56,100
search.

287
00:14:01,770 --> 00:14:03,770
So given a new element
x, I want to find,

288
00:14:03,770 --> 00:14:05,880
what is the previous element?

289
00:14:05,880 --> 00:14:09,520
What's the largest element
smaller than x in my set?

290
00:14:09,520 --> 00:14:14,130
I'm thinking of these n
elements as static, let's say.

291
00:14:14,130 --> 00:14:15,990
You can generalize
everything I say to have

292
00:14:15,990 --> 00:14:17,640
insertions and deletions.

293
00:14:17,640 --> 00:14:22,030
But let's not worry
about that for now.

294
00:14:22,030 --> 00:14:24,190
I just want to
store them somehow

295
00:14:24,190 --> 00:14:26,210
in order to enable search.

296
00:14:26,210 --> 00:14:30,810
So any suggestions in external
memory model or cache oblivious

297
00:14:30,810 --> 00:14:31,410
model?

298
00:14:31,410 --> 00:14:32,950
How would you do this?

299
00:14:32,950 --> 00:14:34,450
[STUDENT COUGHS]

300
00:14:38,490 --> 00:14:41,020
This may sound
easy, but it's not.

301
00:14:41,020 --> 00:14:41,860
But that's OK.

302
00:14:41,860 --> 00:14:44,370
You know, I like easy
answers, simple answers.

303
00:14:48,156 --> 00:14:49,280
There's two simple answers.

304
00:14:49,280 --> 00:14:51,160
One is correct, one is wrong.

305
00:14:51,160 --> 00:14:55,240
But I like both, so I
want to analyze both.

306
00:14:55,240 --> 00:14:55,740
Yeah?

307
00:14:55,740 --> 00:14:58,740
STUDENT: Store them
sorted in order?

308
00:14:58,740 --> 00:15:00,940
PROFESSOR: Store them
sorted in order, good.

309
00:15:00,940 --> 00:15:03,620
That's how we'd normally
solve this problem.

310
00:15:03,620 --> 00:15:04,850
So let's see how it does.

311
00:15:09,040 --> 00:15:10,540
I thought I had
solution, too, here.

312
00:15:10,540 --> 00:15:13,180
But that's OK.

313
00:15:13,180 --> 00:15:25,960
Binary search in a sorted array,
sort the elements in order.

314
00:15:25,960 --> 00:15:28,810
And then to do a query,
binary search on it.

315
00:15:28,810 --> 00:15:30,610
So you remember binary search.

316
00:15:30,610 --> 00:15:32,050
You've got an array.

317
00:15:32,050 --> 00:15:33,690
You start in the middle.

318
00:15:33,690 --> 00:15:36,480
Then let's say the element
looking for is way over here.

319
00:15:36,480 --> 00:15:40,930
So then we'll go over this way
and go there, this way, there,

320
00:15:40,930 --> 00:15:44,870
this way, log n time.

321
00:15:44,870 --> 00:15:47,039
I mean, binary search
is, in a certain sense,

322
00:15:47,039 --> 00:15:48,330
a divide and conquer algorithm.

323
00:15:48,330 --> 00:15:51,920
You only recurse on one side,
but it's divide and conquer.

324
00:15:51,920 --> 00:15:53,180
So divide and conquer is good.

325
00:15:53,180 --> 00:15:55,770
Surely binary search is good.

326
00:15:55,770 --> 00:15:58,190
If only it were that simple.

327
00:15:58,190 --> 00:16:01,290
So sort of orthogonal
to this picture--

328
00:16:01,290 --> 00:16:03,620
maybe I'll just draw
it on one side--

329
00:16:03,620 --> 00:16:06,105
there's a division into blocks.

330
00:16:06,105 --> 00:16:07,480
And in a cache
oblivious setting,

331
00:16:07,480 --> 00:16:10,120
we don't know where that falls.

332
00:16:10,120 --> 00:16:13,930
But the point is,
for the most part,

333
00:16:13,930 --> 00:16:17,345
every one of these
accesses we do

334
00:16:17,345 --> 00:16:21,420
as we go farther and farther to
the right-- almost all of them

335
00:16:21,420 --> 00:16:22,860
will be in a different block.

336
00:16:22,860 --> 00:16:25,670
The middle one is very far
away from the 3/4 mark.

337
00:16:25,670 --> 00:16:28,800
It is very far away from
the 7/8 mark, and so on,

338
00:16:28,800 --> 00:16:30,397
up until the very end.

339
00:16:30,397 --> 00:16:31,980
Let's say we're
searching for the max.

340
00:16:31,980 --> 00:16:33,800
So this will hold
for all of them.

341
00:16:33,800 --> 00:16:38,425
At the end, once we're within
a problem of size order b,

342
00:16:38,425 --> 00:16:40,300
then there's only a
constant number of blocks

343
00:16:40,300 --> 00:16:41,180
that we're touching.

344
00:16:41,180 --> 00:16:44,440
And so from then on, everything
will be free, basically.

345
00:16:44,440 --> 00:16:48,880
So if you think
about it carefully,

346
00:16:48,880 --> 00:16:51,040
the obvious upper
bound-- this is

347
00:16:51,040 --> 00:16:59,074
our usual recurrence for binary
search-- would be constant.

348
00:16:59,074 --> 00:17:01,490
And what we hope to gain here
is, basically, a better base

349
00:17:01,490 --> 00:17:01,890
case.

350
00:17:01,890 --> 00:17:03,973
And I claim that all you
get in terms of base case

351
00:17:03,973 --> 00:17:08,349
here is t of b equals order 1.

352
00:17:08,349 --> 00:17:10,640
And, if you think
about it, this just

353
00:17:10,640 --> 00:17:16,369
solves to log n
minus log b, which

354
00:17:16,369 --> 00:17:22,089
is the same thing as
log n over b, which

355
00:17:22,089 --> 00:17:25,557
is a small improvement
over just regular log n

356
00:17:25,557 --> 00:17:26,640
but not a big improvement.

357
00:17:26,640 --> 00:17:29,992
I claim we can do better.

358
00:17:29,992 --> 00:17:31,575
You've actually seen
how to do better.

359
00:17:34,360 --> 00:17:37,140
But maybe we didn't tell you.

360
00:17:37,140 --> 00:17:41,690
So it's a data structure you've
seen already-- b tree, yeah.

361
00:17:47,810 --> 00:17:51,750
So because we weren't thinking
about this memory hierarchy

362
00:17:51,750 --> 00:17:54,300
business when we
said b tree, we meant

363
00:17:54,300 --> 00:18:00,520
like 2-4 trees or 5-10
trees or some constant bound

364
00:18:00,520 --> 00:18:02,330
on the degree of each node.

365
00:18:02,330 --> 00:18:08,740
But if you make the degree of
the node b-- or some theta b,

366
00:18:08,740 --> 00:18:12,560
b approximate-- so you allow
a big branching factor.

367
00:18:12,560 --> 00:18:16,340
It's got to be somewhere, let's
say, between b over 2 and b.

368
00:18:16,340 --> 00:18:19,430
Then we can store all of these
pointers and all of these keys

369
00:18:19,430 --> 00:18:21,790
in a constant number of blocks.

370
00:18:21,790 --> 00:18:26,780
And so if we're doing just
search, as we navigate down

371
00:18:26,780 --> 00:18:30,220
the b tree we'll spend
order 1 block reads

372
00:18:30,220 --> 00:18:33,210
to load in this node and then
figure out which way to go.

373
00:18:33,210 --> 00:18:34,960
And then let's
say it's this way.

374
00:18:34,960 --> 00:18:37,320
And then we'll spend
order 1 memory transfers

375
00:18:37,320 --> 00:18:39,782
to read this node then
figure out which way to go.

376
00:18:39,782 --> 00:18:41,990
So the cost is going to be
proportional to the height

377
00:18:41,990 --> 00:18:48,050
of the tree, which is
just log base b of n

378
00:18:48,050 --> 00:18:51,360
up to the constant
factors because we're

379
00:18:51,360 --> 00:18:53,090
between b over 2 and b.

380
00:18:53,090 --> 00:18:56,150
But that will affect
this by a factor of 2.

381
00:18:58,760 --> 00:19:02,470
So we can do search in
a b tree in the log base

382
00:19:02,470 --> 00:19:04,420
b of n memory transfers.

383
00:19:11,900 --> 00:19:19,310
OK, remember, log base b of
n is log n divided by log b.

384
00:19:19,310 --> 00:19:20,400
So this is a lot better.

385
00:19:20,400 --> 00:19:22,650
Here we had log n minus log b.

386
00:19:22,650 --> 00:19:24,910
Now we have log n
divided by log b.

387
00:19:24,910 --> 00:19:27,200
And this turns
out to be optimal.

388
00:19:27,200 --> 00:19:30,810
In the comparison model, this
is the best you can hope to do,

389
00:19:30,810 --> 00:19:33,500
so good news.

390
00:19:33,500 --> 00:19:36,054
The bad news is we kind
of critically needed

391
00:19:36,054 --> 00:19:36,845
to know what b was.

392
00:19:41,540 --> 00:19:46,230
B trees really only make
sense if you what b is.

393
00:19:46,230 --> 00:19:48,310
You need to know the
branching factor.

394
00:19:48,310 --> 00:19:50,562
So this is not a cache
oblivious data structure.

395
00:19:50,562 --> 00:19:51,770
But it has other nice things.

396
00:19:51,770 --> 00:19:53,770
We can actually do inserts
and deletes, as well.

397
00:19:53,770 --> 00:19:56,190
So I said static, but if
you want dynamic insert

398
00:19:56,190 --> 00:19:58,850
and deleting elements, you
can also do those in log base

399
00:19:58,850 --> 00:20:01,410
b of n memory transfers using
exactly the algorithms we've

400
00:20:01,410 --> 00:20:03,400
seen with splits and merges.

401
00:20:03,400 --> 00:20:04,700
So all that's good.

402
00:20:08,390 --> 00:20:11,760
But I want to do it cache
obviously-- just the search

403
00:20:11,760 --> 00:20:12,260
for now.

404
00:20:17,400 --> 00:20:19,890
And this is not obvious.

405
00:20:19,890 --> 00:20:22,580
But it's our good
friend van Emde Boas.

406
00:20:31,420 --> 00:20:35,360
So despite the name, this
is not a data structure

407
00:20:35,360 --> 00:20:37,720
that van Emde Boas.

408
00:20:37,720 --> 00:20:41,150
But it's inspired by the data
structure that we covered.

409
00:20:41,150 --> 00:20:44,350
And it's actually a solution
by Harold [INAUDIBLE],

410
00:20:44,350 --> 00:20:47,010
who did the m-edge
thesis on this work.

411
00:20:47,010 --> 00:20:48,909
In the conclusion, it's
like, oh, by the way,

412
00:20:48,909 --> 00:20:49,950
here's how you do search.

413
00:20:49,950 --> 00:20:54,030
It seems like the best
page of that thesis.

414
00:20:54,030 --> 00:20:56,330
And then I think we called
it van Emde Boas because we

415
00:20:56,330 --> 00:20:58,870
thought it was reminiscent.

416
00:20:58,870 --> 00:21:01,960
So here's the idea.

417
00:21:01,960 --> 00:21:03,927
Take all the items
you want to store.

418
00:21:03,927 --> 00:21:06,260
And you're really tempted to
store them in sorted order,

419
00:21:06,260 --> 00:21:07,820
but I'm not going to do that.

420
00:21:07,820 --> 00:21:10,400
I'm going to use some other
divide and conquer order.

421
00:21:10,400 --> 00:21:12,650
First thing I'm going to
do is take those elements,

422
00:21:12,650 --> 00:21:16,280
put them in a perfectly
balanced binary search tree.

423
00:21:16,280 --> 00:21:21,780
So this is a BSTT-- not a
b tree, just a binary tree

424
00:21:21,780 --> 00:21:24,050
because I don't know what b is.

425
00:21:24,050 --> 00:21:26,380
So then maybe the
median's up here.

426
00:21:26,380 --> 00:21:29,200
And then there's two
children and so on.

427
00:21:29,200 --> 00:21:30,340
OK, the mean's over here.

428
00:21:30,340 --> 00:21:32,860
The max is over
here, a regular BST.

429
00:21:32,860 --> 00:21:34,360
Now we know how to
search in a tree.

430
00:21:34,360 --> 00:21:37,290
You just walk down.

431
00:21:37,290 --> 00:21:39,240
The big question
is, in what order

432
00:21:39,240 --> 00:21:40,755
should I store these nodes?

433
00:21:40,755 --> 00:21:42,380
If I just store them
in a random order,

434
00:21:42,380 --> 00:21:45,790
this is going to be super
bad-- log n memory transfers.

435
00:21:45,790 --> 00:21:48,180
But I claim, if I
do a clever order,

436
00:21:48,180 --> 00:21:52,320
I can achieve log base b
of n, which is optimal.

437
00:21:52,320 --> 00:21:59,220
So van Emde Boas suggests
cutting this tree

438
00:21:59,220 --> 00:22:01,331
in the middle.

439
00:22:01,331 --> 00:22:02,080
Why in the middle?

440
00:22:02,080 --> 00:22:04,730
This was n nodes over here.

441
00:22:04,730 --> 00:22:08,930
And we're breaking it up
into a square root of n nodes

442
00:22:08,930 --> 00:22:14,530
at the top because the height
of this overall tree is log n.

443
00:22:14,530 --> 00:22:16,940
If we split it in half,
the height of the tree

444
00:22:16,940 --> 00:22:19,010
is half log n.

445
00:22:19,010 --> 00:22:21,310
2 to the half log n is root n.

446
00:22:21,310 --> 00:22:22,850
I'm losing some
constant factors,

447
00:22:22,850 --> 00:22:24,710
but let's just call it root n.

448
00:22:24,710 --> 00:22:27,650
Then we've got, at the bottom,
everything looks the same.

449
00:22:27,650 --> 00:22:30,920
We're going to have a whole
bunch of trees of size

450
00:22:30,920 --> 00:22:35,780
square root of n, hopefully.

451
00:22:35,780 --> 00:22:41,340
OK, that's what happens when
I cut in the middle level.

452
00:22:41,340 --> 00:22:43,560
Then I recurse.

453
00:22:43,560 --> 00:22:44,700
And what am I recursing?

454
00:22:44,700 --> 00:22:45,550
What am I doing?

455
00:22:45,550 --> 00:22:47,060
This is a layout.

456
00:22:47,060 --> 00:22:51,020
Last time, we did a very
similar thing with matrices.

457
00:22:51,020 --> 00:22:52,210
We had an n by n matrix.

458
00:22:52,210 --> 00:22:55,350
We divided it into four n
over 2 by n over 2 matrices.

459
00:22:55,350 --> 00:23:00,530
We recursively laid out the
1/4, wrote those out in order

460
00:23:00,530 --> 00:23:01,830
so it was consecutive.

461
00:23:01,830 --> 00:23:04,840
Then we laid out the next
quarter, next quarter,

462
00:23:04,840 --> 00:23:05,430
next quarter.

463
00:23:05,430 --> 00:23:07,240
The order of the
quarters didn't matter.

464
00:23:07,240 --> 00:23:09,239
What mattered is that
each quarter of the matrix

465
00:23:09,239 --> 00:23:11,900
was stored as a consecutive
unit so when recursed,

466
00:23:11,900 --> 00:23:13,150
good things happened.

467
00:23:13,150 --> 00:23:15,900
Same thing here, except now
I have roughly square root

468
00:23:15,900 --> 00:23:18,720
of n plus 1.

469
00:23:18,720 --> 00:23:20,760
Chunks, little
triangles-- I'm going

470
00:23:20,760 --> 00:23:22,185
to recursively lay them out.

471
00:23:32,590 --> 00:23:35,190
And then I'm going to
concatenate those layouts.

472
00:23:35,190 --> 00:23:38,880
So this one, I'm going
to recursively figure out

473
00:23:38,880 --> 00:23:41,300
what order to store those
nodes and then put those all as

474
00:23:41,300 --> 00:23:43,180
consecutive in the array.

475
00:23:43,180 --> 00:23:45,750
And then this one goes here.

476
00:23:45,750 --> 00:23:47,380
This one goes here.

477
00:23:47,380 --> 00:23:49,100
Actually, the order
doesn't matter.

478
00:23:49,100 --> 00:23:51,260
But you might as well
preserve the order.

479
00:23:51,260 --> 00:23:55,260
So do the top one, then
the bottom ones in order.

480
00:23:55,260 --> 00:23:56,930
And so, recursively,
each of these ones

481
00:23:56,930 --> 00:23:58,400
is going to get
cut in the middle.

482
00:23:58,400 --> 00:24:00,420
Recursively lay out the
top, then the next one.

483
00:24:00,420 --> 00:24:01,253
Let's do an example.

484
00:24:11,940 --> 00:24:14,510
Let's do an actual tree.

485
00:24:14,510 --> 00:24:20,350
This is actually my favorite
diagram to draw or something.

486
00:24:20,350 --> 00:24:27,580
My most frequently drawn
diagram, complete binary tree

487
00:24:27,580 --> 00:24:30,410
on eight children, eight leaves.

488
00:24:30,410 --> 00:24:33,156
So this is 15 nodes.

489
00:24:33,156 --> 00:24:35,780
It happens to have a
height that's a power of 2,

490
00:24:35,780 --> 00:24:37,640
so this algorithm
works especially well.

491
00:24:37,640 --> 00:24:39,800
So I'm going to
split it in half,

492
00:24:39,800 --> 00:24:41,630
then recursively
lay out the top.

493
00:24:41,630 --> 00:24:43,720
To lay out the top, I'm
going to split it in half.

494
00:24:43,720 --> 00:24:45,260
Then I'm going to
recursively lay out the top.

495
00:24:45,260 --> 00:24:47,676
Well, single node-- it's pretty
clear what order to put it

496
00:24:47,676 --> 00:24:48,960
in with respect to itself.

497
00:24:48,960 --> 00:24:53,740
So that goes first,
then this, then this.

498
00:24:53,740 --> 00:24:55,740
Then I finish the
first recursion.

499
00:24:55,740 --> 00:24:59,800
Next, I'm going to
recursively lay out this thing

500
00:24:59,800 --> 00:25:01,610
by cutting it in
half, laying out

501
00:25:01,610 --> 00:25:04,540
the top, then the bottom parts.

502
00:25:04,540 --> 00:25:09,150
OK, then I'm going to
recursively layout this-- 7, 8,

503
00:25:09,150 --> 00:25:16,530
9, 10, 11, 12, 13, 14, 15.

504
00:25:16,530 --> 00:25:18,410
It gets even more exciting
the next level up.

505
00:25:18,410 --> 00:25:22,030
But it would take a
long time to draw this.

506
00:25:22,030 --> 00:25:23,840
But just imagine this repeated.

507
00:25:23,840 --> 00:25:26,920
So that would be just the
top half of some tree.

508
00:25:26,920 --> 00:25:29,450
Cut here, and then you
do the same thing here

509
00:25:29,450 --> 00:25:30,200
and here and here.

510
00:25:30,200 --> 00:25:32,320
This is very different
from in-order traversal

511
00:25:32,320 --> 00:25:34,200
or any other order
that we've seen.

512
00:25:34,200 --> 00:25:37,160
This is the van Emde Boas order.

513
00:25:37,160 --> 00:25:39,142
And this numbering is
supposed to be the order

514
00:25:39,142 --> 00:25:40,100
that I store the nodes.

515
00:25:40,100 --> 00:25:42,300
So when I write
this into memory,

516
00:25:42,300 --> 00:25:46,960
it's going to look like this--
just the nodes in order.

517
00:25:46,960 --> 00:25:48,757
And when I'm drawing
a circle-- wow,

518
00:25:48,757 --> 00:25:49,965
this is going to get tedious.

519
00:25:53,250 --> 00:25:54,732
And then there's pointers here.

520
00:25:54,732 --> 00:25:56,190
Every time I draw
a circle, there's

521
00:25:56,190 --> 00:25:57,648
a left pointer and
a right pointer.

522
00:25:57,648 --> 00:26:00,380
So 1's going to
point to 2 and 3.

523
00:26:00,380 --> 00:26:04,857
2 is going to point to 4 and 7.

524
00:26:04,857 --> 00:26:06,690
So just take the regular
binary search tree,

525
00:26:06,690 --> 00:26:08,940
but store it in this
really weird order.

526
00:26:08,940 --> 00:26:13,750
I claim this will work really
well, log base b of n search.

527
00:26:13,750 --> 00:26:14,690
Let's analyze it.

528
00:26:18,850 --> 00:26:22,220
Good first time, this is
a cache oblivious layout.

529
00:26:22,220 --> 00:26:23,330
I didn't use b at all.

530
00:26:23,330 --> 00:26:24,380
There's no b here.

531
00:26:24,380 --> 00:26:26,320
Start with a binary tree.

532
00:26:26,320 --> 00:26:28,140
And I just do this recursion.

533
00:26:28,140 --> 00:26:30,539
It gives me a linear
order to put the nodes in.

534
00:26:30,539 --> 00:26:32,330
I'm just going to store
them in that order.

535
00:26:32,330 --> 00:26:33,620
It's linear size, all that.

536
00:26:38,630 --> 00:26:43,260
Now in the analysis, again,
I'm allowed to know b.

537
00:26:43,260 --> 00:26:45,735
So let's say b is b.

538
00:26:45,735 --> 00:26:51,180
And let's consider the
level of recursion.

539
00:26:58,070 --> 00:27:02,610
Let's say the first
level of recursion,

540
00:27:02,610 --> 00:27:09,030
where the triangles have less
than or equal to b nodes.

541
00:27:09,030 --> 00:27:11,330
So I'm thinking of this picture.

542
00:27:11,330 --> 00:27:13,290
I cut in the middle.

543
00:27:13,290 --> 00:27:15,755
Then I recursively cut in
the middle of all the pieces.

544
00:27:15,755 --> 00:27:17,940
Then I recursively
cut in the middle.

545
00:27:17,940 --> 00:27:22,320
I started out with a
height of log n and size n.

546
00:27:22,320 --> 00:27:25,170
I keep cutting, basically
square rooting the size.

547
00:27:25,170 --> 00:27:28,030
At some point, when I
cut, I get triangles

548
00:27:28,030 --> 00:27:30,680
that are size, at
most, square root of b.

549
00:27:30,680 --> 00:27:37,832
So the tree now will
look-- actually,

550
00:27:37,832 --> 00:27:39,040
let me draw a bigger picture.

551
00:27:55,670 --> 00:27:57,390
Let's start down here.

552
00:27:57,390 --> 00:28:00,397
So I've got triangle less
than or equal to b, triangle

553
00:28:00,397 --> 00:28:01,430
less than or equal to b.

554
00:28:47,650 --> 00:28:52,230
This is some attempt
to draw a general tree.

555
00:28:52,230 --> 00:28:53,865
And first we cut in
the middle level.

556
00:28:53,865 --> 00:28:55,240
Then we cut in
the middle levels.

557
00:28:55,240 --> 00:28:59,480
And let's say, at that moment,
all of the leftover trees

558
00:28:59,480 --> 00:29:01,960
have, at most, b nodes in them.

559
00:29:01,960 --> 00:29:03,860
It's going to happen
at some point.

560
00:29:03,860 --> 00:29:08,780
It's going to happen
after roughly log n minus

561
00:29:08,780 --> 00:29:12,110
log b levels of recursion.

562
00:29:12,110 --> 00:29:14,580
The heights here will
be roughly log b.

563
00:29:17,759 --> 00:29:19,800
We keep cutting in half,
still with height log b.

564
00:29:19,800 --> 00:29:21,330
Then we know the size of it's b.

565
00:29:23,910 --> 00:29:26,920
OK, so this is a picture
that exists in some sense.

566
00:29:26,920 --> 00:29:29,460
What we know is that
each of these triangles

567
00:29:29,460 --> 00:29:30,900
is stored consecutively.

568
00:29:30,900 --> 00:29:34,440
By this recursive
layout, we guarantee

569
00:29:34,440 --> 00:29:38,310
that, at any level of
recursion, each chunk

570
00:29:38,310 --> 00:29:40,400
is stored consecutively.

571
00:29:40,400 --> 00:29:47,020
So, in particular, this
level-- level b-- is nice.

572
00:29:47,020 --> 00:29:51,900
So what that tells us is that
each triangle with, at most, b

573
00:29:51,900 --> 00:30:00,790
elements is
consecutive, which means

574
00:30:00,790 --> 00:30:07,280
it occupies at most two blocks.

575
00:30:07,280 --> 00:30:08,410
If we're lucky, it's one.

576
00:30:08,410 --> 00:30:11,850
But if we're unlucky in
terms of-- here's memory.

577
00:30:11,850 --> 00:30:13,600
Here's how it's
split into blocks.

578
00:30:13,600 --> 00:30:19,260
Maybe it's consecutive, but
it crosses a block boundary.

579
00:30:19,260 --> 00:30:23,610
But the distance between
these two lines is b and b.

580
00:30:23,610 --> 00:30:26,210
And the length of
the blue thing is b.

581
00:30:26,210 --> 00:30:28,200
So you can only cross one line.

582
00:30:28,200 --> 00:30:29,720
So you fit in two blocks.

583
00:30:29,720 --> 00:30:32,810
Each of these triangles
fits in two blocks.

584
00:30:32,810 --> 00:30:35,480
Now, let's think about
search algorithm.

585
00:30:35,480 --> 00:30:40,100
We're going to do regular binary
search in a binary search tree.

586
00:30:40,100 --> 00:30:40,990
We start at the root.

587
00:30:40,990 --> 00:30:42,140
We compare to x.

588
00:30:42,140 --> 00:30:43,380
We go left to right.

589
00:30:43,380 --> 00:30:45,260
Then we go left to
right, left to right.

590
00:30:45,260 --> 00:30:48,070
Eventually we find the
predecessor or the successor

591
00:30:48,070 --> 00:30:50,590
or, ideally, the element
we're actually searching for.

592
00:30:50,590 --> 00:30:54,110
And so what we're
doing is following

593
00:30:54,110 --> 00:30:59,210
some root-to-node
path in the tree.

594
00:30:59,210 --> 00:31:00,094
Maybe we stop early.

595
00:31:00,094 --> 00:31:01,760
In the worst case,
we go down to a leaf.

596
00:31:01,760 --> 00:31:03,590
But it's a vertical path.

597
00:31:03,590 --> 00:31:06,500
You only go down.

598
00:31:06,500 --> 00:31:09,150
Over here, same thing.

599
00:31:09,150 --> 00:31:11,900
Let's say, because these
are the ones I drew,

600
00:31:11,900 --> 00:31:13,500
you go here somewhere.

601
00:31:17,830 --> 00:31:21,760
But in general, you're following
some root-to-node path.

602
00:31:21,760 --> 00:31:25,990
And you're visiting some
sequence of triangles.

603
00:31:25,990 --> 00:31:28,660
Each triangle fits in,
basically, one block.

604
00:31:28,660 --> 00:31:32,679
Let's assume, as usual,
m over b is at least two.

605
00:31:32,679 --> 00:31:34,220
So you can store at
least two blocks,

606
00:31:34,220 --> 00:31:36,230
which means, once you
start touching a triangle,

607
00:31:36,230 --> 00:31:37,519
all further touches are free.

608
00:31:37,519 --> 00:31:39,310
The first one, you have
to pay the load in,

609
00:31:39,310 --> 00:31:40,740
maybe these two blocks.

610
00:31:40,740 --> 00:31:44,300
Every subsequent touch as you
go down this path is free.

611
00:31:44,300 --> 00:31:45,960
Then you go to a new triangle.

612
00:31:45,960 --> 00:31:47,835
That could be somewhere
completely different.

613
00:31:47,835 --> 00:31:50,080
We don't really now, but
it's some other two blocks.

614
00:31:50,080 --> 00:31:53,250
And as long as you stay within
the triangle, it's free.

615
00:31:53,250 --> 00:31:57,110
So the cost is going to be,
at most, twice the number

616
00:31:57,110 --> 00:31:58,650
of triangles that you visit.

617
00:32:01,360 --> 00:32:07,610
MTN is going to be, at
most, twice the number

618
00:32:07,610 --> 00:32:16,350
of triangles visited
by a root-to-node path,

619
00:32:16,350 --> 00:32:18,530
a downward path in the
binary search tree.

620
00:32:22,840 --> 00:32:25,316
OK, now to figure
that out we need not

621
00:32:25,316 --> 00:32:27,690
only an upper bound on how
big the triangles are but also

622
00:32:27,690 --> 00:32:28,810
a lower bound.

623
00:32:28,810 --> 00:32:31,830
I said the height of
the tree is about log b.

624
00:32:31,830 --> 00:32:33,610
It's close.

625
00:32:33,610 --> 00:32:35,620
Maybe you have
triangles of size b

626
00:32:35,620 --> 00:32:38,900
plus 1, which is a
little bit too big.

627
00:32:38,900 --> 00:32:40,580
So let's think about that case.

628
00:32:40,580 --> 00:32:42,820
You have b plus 1 nodes.

629
00:32:42,820 --> 00:32:46,740
And then you end up cutting
in the middle level.

630
00:32:46,740 --> 00:32:49,400
So before, you had a height of
almost log b-- slightly more

631
00:32:49,400 --> 00:32:50,574
than log b.

632
00:32:50,574 --> 00:32:52,490
Then, when you cut it
in half, the new heights

633
00:32:52,490 --> 00:32:56,360
will be half log b.

634
00:32:56,360 --> 00:32:57,970
And then you'll have
only square root

635
00:32:57,970 --> 00:33:00,360
of b items in the triangle.

636
00:33:00,360 --> 00:33:02,180
So that may seem problematic.

637
00:33:02,180 --> 00:33:03,790
These things are, at most, b.

638
00:33:03,790 --> 00:33:05,560
They're also at
least square root b.

639
00:33:09,180 --> 00:33:15,360
The height of a
triangle at this level

640
00:33:15,360 --> 00:33:21,975
is somewhere between
half log b and log b.

641
00:33:25,055 --> 00:33:26,890
Basically, we're binary
searching on height.

642
00:33:26,890 --> 00:33:28,840
We're stopping when
we divide it by 2.

643
00:33:28,840 --> 00:33:32,360
And we get something less
than log B in height.

644
00:33:32,360 --> 00:33:34,490
Luckily, we only
care about heights.

645
00:33:34,490 --> 00:33:36,660
We don't care that there's
only root b items here.

646
00:33:36,660 --> 00:33:39,720
That may seem inefficient, but
because everything's in a log

647
00:33:39,720 --> 00:33:41,360
here-- because we
only care about

648
00:33:41,360 --> 00:33:45,150
log b in the running time, and
we're basically approximating

649
00:33:45,150 --> 00:33:47,660
log b within a factor
of 2-- everything's

650
00:33:47,660 --> 00:33:50,860
going to work up to
constant factors.

651
00:33:50,860 --> 00:33:52,880
In other words, if
you look at this path,

652
00:33:52,880 --> 00:33:55,080
we know the length
of the path is log n.

653
00:33:55,080 --> 00:33:58,400
We know the height of
each of these triangles

654
00:33:58,400 --> 00:34:00,890
is at least half log b.

655
00:34:00,890 --> 00:34:03,490
That means the number
of triangles you visit

656
00:34:03,490 --> 00:34:12,040
is log n divided by half log b.

657
00:34:26,320 --> 00:34:33,030
And the length of
the path is log n.

658
00:34:33,030 --> 00:34:42,210
So the number of triangles on
the path is, at most, log n

659
00:34:42,210 --> 00:34:45,150
divided by how much progress we
make for each triangle, which

660
00:34:45,150 --> 00:34:50,960
is half log b-- also known
as 2 times log base b of n.

661
00:34:53,750 --> 00:34:56,739
And then we get the
number of memory transfers

662
00:34:56,739 --> 00:34:58,490
is, at most, twice that.

663
00:34:58,490 --> 00:35:02,080
So the number of memory
transfers is going to be,

664
00:35:02,080 --> 00:35:04,600
at most, 4 times
log base b of n.

665
00:35:07,550 --> 00:35:12,520
And that's order log base
b of n, which is optimal.

666
00:35:12,520 --> 00:35:13,890
Now we don't need to know b.

667
00:35:18,354 --> 00:35:20,350
How's that for a cheat?

668
00:35:20,350 --> 00:35:24,507
So we get optimal running time,
except for the constant factor.

669
00:35:24,507 --> 00:35:25,840
Admittedly, this is not perfect.

670
00:35:25,840 --> 00:35:29,670
B trees get basically 1
times log base b of n.

671
00:35:29,670 --> 00:35:33,270
This cache oblivious binary
search gives you 4 times

672
00:35:33,270 --> 00:35:34,770
log base b of n.

673
00:35:34,770 --> 00:35:36,120
But this was a rough analysis.

674
00:35:36,120 --> 00:35:39,880
You can actually get that
down to like 1.4 times

675
00:35:39,880 --> 00:35:40,720
log base b of n.

676
00:35:40,720 --> 00:35:41,644
And that's tight.

677
00:35:41,644 --> 00:35:43,310
So you can't do quite
as well with cache

678
00:35:43,310 --> 00:35:48,850
oblivious as external
memory but close.

679
00:35:48,850 --> 00:35:50,830
And that's sort
of the story here.

680
00:35:50,830 --> 00:35:52,780
If you ignore constant
factors, all is good.

681
00:35:52,780 --> 00:35:54,670
In practice, where
you potentially

682
00:35:54,670 --> 00:35:58,450
win is that, if you designed
a b tree for specific b,

683
00:35:58,450 --> 00:36:00,950
you're going to do really great
for that level of the memory

684
00:36:00,950 --> 00:36:01,410
hierarchy.

685
00:36:01,410 --> 00:36:03,909
But in reality, there's many
levels to the memory hierarchy.

686
00:36:03,909 --> 00:36:05,150
They all matter.

687
00:36:05,150 --> 00:36:07,440
Cache oblivious is
going to win a lot

688
00:36:07,440 --> 00:36:10,140
because it's optimal at
all levels simultaneously.

689
00:36:10,140 --> 00:36:11,940
It's really hard to
build a b tree that's

690
00:36:11,940 --> 00:36:14,220
optimal for many values
of b simultaneously.

691
00:36:17,010 --> 00:36:20,490
OK so that is search.

692
00:36:20,490 --> 00:36:24,320
Any questions before
we go on to source?

693
00:36:24,320 --> 00:36:26,160
[STUDENTS COUGHING]

694
00:36:28,460 --> 00:36:31,190
One obvious question
is, what about dynamic?

695
00:36:31,190 --> 00:36:32,700
Again, I said static.

696
00:36:32,700 --> 00:36:35,650
Obviously the elements
aren't changing here.

697
00:36:35,650 --> 00:36:38,245
Just doing search
in log base b of n,

698
00:36:38,245 --> 00:36:43,140
it turns out you can do insert,
delete, and search in log base

699
00:36:43,140 --> 00:36:49,120
b of n memory transfers
per operation.

700
00:36:49,120 --> 00:36:53,607
This was my first result
in cache oblivious land.

701
00:36:53,607 --> 00:36:55,440
It's when I met Charles
Leiserson, actually.

702
00:37:11,960 --> 00:37:13,210
But I'm not going to cover it.

703
00:37:13,210 --> 00:37:16,510
If you want to know how, you
should take 6851, Advanced Data

704
00:37:16,510 --> 00:37:19,510
Structures, which talks about
all sorts of things like this

705
00:37:19,510 --> 00:37:20,025
but dynamic.

706
00:37:23,930 --> 00:37:30,360
It turns out there's a lot more
to say about this universe.

707
00:37:30,360 --> 00:37:33,090
And I want to go in to sorting
instead of talking about how

708
00:37:33,090 --> 00:37:36,310
to make that dynamic because,
oh, OK, search log base

709
00:37:36,310 --> 00:37:38,366
b of n, that was optimal.

710
00:37:38,366 --> 00:37:40,740
I said, oh, you can also do
insert and delete in log base

711
00:37:40,740 --> 00:37:41,239
b of n.

712
00:37:41,239 --> 00:37:43,440
It turns out that's not optimal.

713
00:37:43,440 --> 00:37:44,750
It's as good as b trees.

714
00:37:44,750 --> 00:37:47,350
But you can do better.

715
00:37:47,350 --> 00:37:49,560
B trees are not good at updates.

716
00:37:49,560 --> 00:37:52,180
And if you've ever worked with
a database, you may know this.

717
00:37:52,180 --> 00:37:54,800
If you have a lot of updates,
b trees are really slow.

718
00:37:54,800 --> 00:37:57,140
They're good for searches,
not good for updates.

719
00:37:57,140 --> 00:37:58,930
You can do a lot better.

720
00:37:58,930 --> 00:38:01,820
And that will be
exhibited by sorting.

721
00:38:01,820 --> 00:38:03,570
So sorting-- I think
you know the problem.

722
00:38:03,570 --> 00:38:07,316
You're given n elements in an
array in some arbitrary order.

723
00:38:07,316 --> 00:38:08,940
You want to put them
into sorted order.

724
00:38:08,940 --> 00:38:11,060
Or, equivalently, you want to
put them into a van Emde Boas

725
00:38:11,060 --> 00:38:11,720
order.

726
00:38:11,720 --> 00:38:14,010
Once their sorted, it's
not too hard to transfer

727
00:38:14,010 --> 00:38:14,810
into this order.

728
00:38:14,810 --> 00:38:17,020
So you can do search
fast or whatever.

729
00:38:17,020 --> 00:38:21,080
Sorting is a very basic
thing we like to do.

730
00:38:21,080 --> 00:38:25,300
And the obvious way
to sort when you have,

731
00:38:25,300 --> 00:38:27,950
basically, a-- let's pretend
we have this b tree data

732
00:38:27,950 --> 00:38:29,549
structure, cache oblivious even.

733
00:38:29,549 --> 00:38:30,840
Or we just use regular b trees.

734
00:38:30,840 --> 00:38:32,022
Let's use regular b trees.

735
00:38:32,022 --> 00:38:33,230
Forget about cache oblivious.

736
00:38:33,230 --> 00:38:35,470
External memory, we
know how to do b trees.

737
00:38:35,470 --> 00:38:37,180
We know how to
insert into a b tree.

738
00:38:37,180 --> 00:38:46,700
So the obvious way to sort is to
do n inserts into, if you want,

739
00:38:46,700 --> 00:38:51,050
a cache oblivious b tree
or just a regular b tree.

740
00:38:51,050 --> 00:38:52,990
How long does that take?

741
00:38:52,990 --> 00:38:54,760
N times log base b of n.

742
00:39:05,290 --> 00:39:06,520
It sounds OK.

743
00:39:06,520 --> 00:39:08,680
But it's not optimal.

744
00:39:08,680 --> 00:39:12,145
It's actually really slow
compared to what you can do.

745
00:39:12,145 --> 00:39:14,195
You can do, roughly,
a factor of b faster.

746
00:39:19,109 --> 00:39:20,900
But it's the best we
know how to do so far.

747
00:39:20,900 --> 00:39:23,310
So the goal is to do better.

748
00:39:23,310 --> 00:39:25,640
And, basically, what's going
on is we can do inserts.

749
00:39:25,640 --> 00:39:27,710
In this universe, we can
do inserts and deletes

750
00:39:27,710 --> 00:39:32,570
faster than we can do searches,
which is a little weird.

751
00:39:32,570 --> 00:39:35,610
It will become clearer
as we go through.

752
00:39:35,610 --> 00:39:39,370
So what's another
natural way to sort?

753
00:39:39,370 --> 00:39:41,300
What means to sorting
algorithm that we've

754
00:39:41,300 --> 00:39:44,137
covered are optimal in
the comparison model?

755
00:39:44,137 --> 00:39:45,152
STUDENT: Merge sort.

756
00:39:45,152 --> 00:39:46,860
PROFESSOR: Merge sort,
that's a good one.

757
00:39:46,860 --> 00:39:49,410
We could do quick
sort, too, I guess.

758
00:39:49,410 --> 00:39:51,490
I'll stick to merge sort.

759
00:39:51,490 --> 00:39:55,010
Merge sort's nice because
A, it's divide and conquer.

760
00:39:55,010 --> 00:39:56,340
And we like divide and conquer.

761
00:39:56,340 --> 00:39:58,570
It seems to work,
if we do it right.

762
00:39:58,570 --> 00:40:00,386
And it's also cache oblivious.

763
00:40:00,386 --> 00:40:01,510
There's no b in merge sort.

764
00:40:01,510 --> 00:40:02,801
We didn't even know what b was.

765
00:40:02,801 --> 00:40:05,645
So great, merge sort
is divide and conquer

766
00:40:05,645 --> 00:40:06,570
and cache oblivious.

767
00:40:06,570 --> 00:40:08,650
So how much does it cost?

768
00:40:08,650 --> 00:40:11,020
Well, let's think
about merge sort.

769
00:40:11,020 --> 00:40:13,630
You take an array.

770
00:40:13,630 --> 00:40:15,640
You divide it in half.

771
00:40:15,640 --> 00:40:16,570
That takes zero time.

772
00:40:16,570 --> 00:40:17,960
That's just a conceptual thing.

773
00:40:17,960 --> 00:40:19,760
You recursively sort this part.

774
00:40:19,760 --> 00:40:21,060
You recursively sort this part.

775
00:40:21,060 --> 00:40:23,560
That looks good because
those items are consecutive.

776
00:40:23,560 --> 00:40:25,810
So that recursion is going
to be an honest to goodness

777
00:40:25,810 --> 00:40:27,920
recursion on an array.

778
00:40:27,920 --> 00:40:30,470
So we can write a recurrence.

779
00:40:30,470 --> 00:40:33,200
And then we have to
merge the two parts.

780
00:40:33,200 --> 00:40:36,710
So in merge, we take the
first element of each guy.

781
00:40:36,710 --> 00:40:38,280
We compare them,
output one of them,

782
00:40:38,280 --> 00:40:41,650
advance that one, compare,
output one of them,

783
00:40:41,650 --> 00:40:43,790
advance that guy.

784
00:40:43,790 --> 00:40:45,640
That's three parallel scans.

785
00:40:45,640 --> 00:40:47,120
We're scanning in this array.

786
00:40:47,120 --> 00:40:48,920
We're scanning in this array.

787
00:40:48,920 --> 00:40:50,960
We're always advancing
forward, which

788
00:40:50,960 --> 00:40:54,505
means as long as we
store the first block

789
00:40:54,505 --> 00:40:57,385
of this guy and the first
block of this guy who

790
00:40:57,385 --> 00:40:59,130
knows how it's aligned--

791
00:40:59,130 --> 00:41:02,470
But we'll read these
items one by one

792
00:41:02,470 --> 00:41:03,870
until we finish that block.

793
00:41:03,870 --> 00:41:07,890
Then we'll just read the next
block, read those one by one.

794
00:41:07,890 --> 00:41:09,870
And similarly for
the output array,

795
00:41:09,870 --> 00:41:12,380
we first start filling a block.

796
00:41:12,380 --> 00:41:14,660
Once it's filled, we
can kick that one out

797
00:41:14,660 --> 00:41:15,700
and read the next one.

798
00:41:15,700 --> 00:41:19,360
As long as m over
b is at least 3,

799
00:41:19,360 --> 00:41:23,199
we can afford this
three-parallel scan.

800
00:41:23,199 --> 00:41:24,240
It's not really parallel.

801
00:41:24,240 --> 00:41:26,540
It's more like inter-leaf scans.

802
00:41:26,540 --> 00:41:28,040
But we're basically
scanning in here

803
00:41:28,040 --> 00:41:29,414
while we're also
scanning in here

804
00:41:29,414 --> 00:41:32,170
and scanning in
the output array.

805
00:41:32,170 --> 00:41:35,400
And we can merge
two sorted arrays

806
00:41:35,400 --> 00:41:42,320
into a new sorted array in
scan time, n over be plus 1.

807
00:41:42,320 --> 00:41:48,710
So that means the number
of memory transfers

808
00:41:48,710 --> 00:41:53,350
is 2 times the number of memory
transfers for half the size,

809
00:41:53,350 --> 00:41:58,960
like regular merge sort,
plus n over b plus 1.

810
00:41:58,960 --> 00:42:00,912
That's our recurrence.

811
00:42:00,912 --> 00:42:02,120
Now we just need to solve it.

812
00:42:02,120 --> 00:42:03,810
Well, before we solve
it, in this case

813
00:42:03,810 --> 00:42:07,390
we always have to be
careful with the base case.

814
00:42:07,390 --> 00:42:12,434
Base case is MT of m.

815
00:42:12,434 --> 00:42:14,100
This is the best base
case we could use.

816
00:42:14,100 --> 00:42:16,260
Let's use it.

817
00:42:16,260 --> 00:42:20,860
When I reach an array of size
m, I read the whole thing.

818
00:42:20,860 --> 00:42:22,400
And then that's all I can pay.

819
00:42:22,400 --> 00:42:25,980
So I won't incur any
more cost as long

820
00:42:25,980 --> 00:42:27,940
as I stay within that
region of size m.

821
00:42:27,940 --> 00:42:30,770
Maybe I should put
some constant times m

822
00:42:30,770 --> 00:42:34,020
because this is not in
place algorithm, so maybe

823
00:42:34,020 --> 00:42:36,460
1/3 m something.

824
00:42:36,460 --> 00:42:39,630
As long as I'm not too
close to the cache size,

825
00:42:39,630 --> 00:42:41,680
I will only pay m over
b memory transfers.

826
00:42:44,360 --> 00:42:45,170
So far so good.

827
00:42:55,450 --> 00:42:57,550
Now we just solve
the recurrence.

828
00:42:57,550 --> 00:43:00,410
This is a nice recurrence,
very similar to

829
00:43:00,410 --> 00:43:01,660
the old merge-sort recurrence.

830
00:43:01,660 --> 00:43:04,270
We just have a different
thing in the additive term.

831
00:43:04,270 --> 00:43:06,120
And we have a
different base case.

832
00:43:06,120 --> 00:43:08,360
The way I like to
solve nice recurrences

833
00:43:08,360 --> 00:43:09,734
is with recursion trees.

834
00:43:09,734 --> 00:43:12,150
This is actually a trick I
learned by teaching this class.

835
00:43:12,150 --> 00:43:14,960
Before this, cache oblivious
was really painful to me

836
00:43:14,960 --> 00:43:17,090
because I could never
solve the currencies.

837
00:43:17,090 --> 00:43:19,746
Then I thought the class and
was like, oh, this is easy.

838
00:43:19,746 --> 00:43:21,840
I hope the same
transformation happens to you.

839
00:43:21,840 --> 00:43:25,660
You'll see how easy it is
once we do this example.

840
00:43:28,400 --> 00:43:30,377
OK, this is merge sort.

841
00:43:30,377 --> 00:43:31,960
Remember recursion
tree, in every node

842
00:43:31,960 --> 00:43:37,220
you put the additive cost
so that, if you added up

843
00:43:37,220 --> 00:43:38,810
the cost of all of
these nodes, you

844
00:43:38,810 --> 00:43:41,060
would get the total
value of this expands

845
00:43:41,060 --> 00:43:42,970
to because we're
basically making

846
00:43:42,970 --> 00:43:45,501
two children of size n over 2.

847
00:43:45,501 --> 00:43:47,000
And then we're
putting, at the root,

848
00:43:47,000 --> 00:43:51,020
this cost, which means, if
you add up all of these nodes,

849
00:43:51,020 --> 00:43:52,540
you're getting all
of these costs.

850
00:43:52,540 --> 00:43:54,830
And that's the total cost.

851
00:43:54,830 --> 00:43:56,440
So it's n over b at the top.

852
00:43:56,440 --> 00:43:59,230
Then it's going to be n over
2 divided by b and so on.

853
00:43:59,230 --> 00:44:02,290
I'm omitting the plus
1 just for cleanliness.

854
00:44:02,290 --> 00:44:05,480
You'd actually have to count.

855
00:44:05,480 --> 00:44:07,660
And this keeps going until
we hit the base case.

856
00:44:07,660 --> 00:44:08,590
This is where
things are a little

857
00:44:08,590 --> 00:44:10,006
different from
regular merge sort,

858
00:44:10,006 --> 00:44:13,360
other than the divided by b.

859
00:44:13,360 --> 00:44:17,065
We stop when we reach
something at size m.

860
00:44:17,065 --> 00:44:18,690
So at the leaf level,
we have something

861
00:44:18,690 --> 00:44:20,250
of size m, which
means we basically

862
00:44:20,250 --> 00:44:24,355
have m over b in each leaf.

863
00:44:24,355 --> 00:44:26,730
And then we should think about
how many leaves there are.

864
00:44:33,020 --> 00:44:36,020
This is just n over
m leaves, I guess.

865
00:44:36,020 --> 00:44:38,650
There's lots of
ways to see that.

866
00:44:38,650 --> 00:44:41,090
One way to think about it
is we're conserving mass.

867
00:44:41,090 --> 00:44:42,610
We started with n items.

868
00:44:42,610 --> 00:44:44,370
Split it in half,
split it in half.

869
00:44:44,370 --> 00:44:46,740
So the number of items
is remaining fixed.

870
00:44:46,740 --> 00:44:49,020
Then at the bottom
we have m items.

871
00:44:49,020 --> 00:44:51,280
And so the number of leaves
has to be exactly n over m

872
00:44:51,280 --> 00:44:54,150
because the total should be n.

873
00:44:54,150 --> 00:44:58,310
You can also think of it as
2 to the power log of that.

874
00:44:58,310 --> 00:45:00,130
We have, usually, log n levels.

875
00:45:00,130 --> 00:45:02,060
But we're cutting off
a log m at the bottom.

876
00:45:02,060 --> 00:45:06,240
So it's log n minus
log m as the height.

877
00:45:06,240 --> 00:45:07,750
I'll actually need that.

878
00:45:07,750 --> 00:45:14,040
The height of this tree is
log n minus log m, also known

879
00:45:14,040 --> 00:45:15,760
as log n/m.

880
00:45:18,930 --> 00:45:21,140
OK, so we've drawn this tree.

881
00:45:21,140 --> 00:45:23,730
Now, what we usually do
is add up level by level.

882
00:45:23,730 --> 00:45:26,290
That usually gives
a very clean answer.

883
00:45:26,290 --> 00:45:27,680
So we add up the top level.

884
00:45:27,680 --> 00:45:29,620
That's n over b.

885
00:45:29,620 --> 00:45:30,810
We add up the second level.

886
00:45:30,810 --> 00:45:33,660
That's n over b, by
conservation of mass

887
00:45:33,660 --> 00:45:37,490
again and because this
was a linear function.

888
00:45:37,490 --> 00:45:41,604
So each level, in fact, is going
to be exactly n over b cost.

889
00:45:41,604 --> 00:45:43,520
We should be a little
careful about the bottom

890
00:45:43,520 --> 00:45:44,850
because the base
case-- I mean, it

891
00:45:44,850 --> 00:45:46,480
happens that the base
case matches this.

892
00:45:46,480 --> 00:45:48,938
But it's always good practice
to think about the leaf level

893
00:45:48,938 --> 00:45:50,260
separately.

894
00:45:50,260 --> 00:45:53,900
But the leaf level is just
m over b times n over m

895
00:45:53,900 --> 00:46:00,720
The m's cancel, so m
over b times n over m.

896
00:46:00,720 --> 00:46:03,100
This is n over b.

897
00:46:03,100 --> 00:46:06,632
So every level is n over b.

898
00:46:06,632 --> 00:46:10,915
The number of levels
is log of n over m.

899
00:46:14,660 --> 00:46:15,830
Cool.

900
00:46:15,830 --> 00:46:18,484
So the number of
memory transfers

901
00:46:18,484 --> 00:46:20,150
is just the product
of those two things.

902
00:46:23,770 --> 00:46:31,720
It's n over b times
that log, log n over m.

903
00:46:31,720 --> 00:46:32,690
Now let's compare.

904
00:46:35,650 --> 00:46:37,560
That's sorting.

905
00:46:37,560 --> 00:46:41,690
Over here, we had a running
time of n times log base b of n.

906
00:46:44,430 --> 00:46:51,620
So this is n log n
divided by log b.

907
00:46:51,620 --> 00:46:54,130
Log base b is the same
as dividing by log b.

908
00:46:54,130 --> 00:46:57,130
So n log n divided by log--
we had regular sorting time.

909
00:46:57,130 --> 00:46:58,820
And then we divided by log b.

910
00:46:58,820 --> 00:47:02,860
Over here, we have basically
regular sorting time.

911
00:47:02,860 --> 00:47:05,610
But now we're dividing by b.

912
00:47:05,610 --> 00:47:10,840
That's a huge improvement-- a
b divided by log b improvement.

913
00:47:10,840 --> 00:47:13,410
I mean, think of the b
being like a million.

914
00:47:13,410 --> 00:47:18,570
So before we were dividing
by 20, which is OK.

915
00:47:18,570 --> 00:47:20,240
But now we're
dividing by a million.

916
00:47:20,240 --> 00:47:22,020
That's better.

917
00:47:22,020 --> 00:47:24,980
So this way of sorting
is so much better

918
00:47:24,980 --> 00:47:28,510
than this way of sorting.

919
00:47:28,510 --> 00:47:31,830
It's still not optimal,
but we're getting better.

920
00:47:45,650 --> 00:47:49,320
We can actually get sort of the
best of both worlds-- divide

921
00:47:49,320 --> 00:47:54,800
by b and divide
by log b, I claim.

922
00:47:54,800 --> 00:47:57,520
But we need a new algorithm.

923
00:47:57,520 --> 00:47:59,230
Any suggestions for
another algorithm?

924
00:48:09,170 --> 00:48:13,650
STUDENT: Divide
into block size b.

925
00:48:13,650 --> 00:48:17,120
PROFESSOR: I want to
divide into block size b.

926
00:48:17,120 --> 00:48:18,955
So, you mean a merge sort?

927
00:48:18,955 --> 00:48:19,920
STUDENT: Yes.

928
00:48:19,920 --> 00:48:21,670
PROFESSOR: So merge
sort, I take my array.

929
00:48:21,670 --> 00:48:24,306
I divide it into
blocks the size b.

930
00:48:24,306 --> 00:48:28,590
I could sort each one
in one memory transfer.

931
00:48:28,590 --> 00:48:31,120
And then I need to merge them.

932
00:48:31,120 --> 00:48:36,680
So then I've got n divided
by b sorted arrays.

933
00:48:36,680 --> 00:48:38,890
I don't know how to merge them.

934
00:48:38,890 --> 00:48:41,640
It's going to be
hard, but very close.

935
00:48:41,640 --> 00:48:44,460
So the answer is
indeed merge sort.

936
00:48:49,220 --> 00:48:51,670
What we covered before
is binary merge sort.

937
00:48:51,670 --> 00:48:53,810
You split into two groups.

938
00:48:53,810 --> 00:48:57,530
What I want to do now is
split into some other number

939
00:48:57,530 --> 00:48:58,050
of groups.

940
00:48:58,050 --> 00:48:59,370
So that was n over b groups.

941
00:48:59,370 --> 00:49:03,920
That's too many because merging
n over b arrays is hard.

942
00:49:03,920 --> 00:49:06,840
Merging two arrays was easy.

943
00:49:06,840 --> 00:49:08,580
Assuming m over
b was at least 3,

944
00:49:08,580 --> 00:49:10,795
I could merge these guys
just by parallel scans.

945
00:49:13,460 --> 00:49:15,097
So you have the right bound?

946
00:49:15,097 --> 00:49:16,440
STUDENT: B way.

947
00:49:16,440 --> 00:49:21,586
PROFESSOR: B way, maybe.

948
00:49:21,586 --> 00:49:23,300
STUDENT: Square root of b.

949
00:49:23,300 --> 00:49:25,000
PROFESSOR: Square root of b?

950
00:49:25,000 --> 00:49:26,680
That's what I like
to call root beer.

951
00:49:26,680 --> 00:49:28,200
[LAUGHTER]

952
00:49:29,880 --> 00:49:31,050
Nope.

953
00:49:31,050 --> 00:49:33,170
I do call it that.

954
00:49:33,170 --> 00:49:34,305
Yeah?

955
00:49:34,305 --> 00:49:35,520
STUDENT: M over b?

956
00:49:35,520 --> 00:49:38,350
PROFESSOR: M over b, that's
what I'm looking for!

957
00:49:38,350 --> 00:49:39,068
Why m over b?

958
00:49:39,068 --> 00:49:41,068
STUDENT: I was just
thinking of the bottom layer

959
00:49:41,068 --> 00:49:44,865
of the [INAUDIBLE]
binary merge sort.

960
00:49:44,865 --> 00:49:46,490
PROFESSOR: Because
m over b is up here?

961
00:49:46,490 --> 00:49:46,990
Nice.

962
00:49:46,990 --> 00:49:48,460
[LAUGHTER]

963
00:49:48,960 --> 00:49:52,558
Not the right reason, but
you get a Frisbee anyway.

964
00:49:52,558 --> 00:49:55,970
All right, let's see
if I can do this.

965
00:49:55,970 --> 00:49:57,172
Would you like another one?

966
00:49:57,172 --> 00:49:58,130
Add to your collection.

967
00:49:58,130 --> 00:50:02,330
All right, so m over b is the
right answer-- wrong reason,

968
00:50:02,330 --> 00:50:05,080
but that's OK.

969
00:50:05,080 --> 00:50:07,220
It all comes down
to this merge step.

970
00:50:07,220 --> 00:50:10,500
So m over b way means I
take my problem of size n.

971
00:50:10,500 --> 00:50:12,690
Let's draw it out.

972
00:50:12,690 --> 00:50:15,390
I divide into chunks.

973
00:50:15,390 --> 00:50:18,370
I want the number of
chunks that I divide into

974
00:50:18,370 --> 00:50:25,830
to be m over b, meaning each of
these has size n over m over b.

975
00:50:25,830 --> 00:50:26,660
That's weird.

976
00:50:26,660 --> 00:50:29,470
This is natural because
this is how many blocks

977
00:50:29,470 --> 00:50:32,310
I can have in cache.

978
00:50:32,310 --> 00:50:34,990
I care about that because, if
I want to do a multi-way merge,

979
00:50:34,990 --> 00:50:36,680
you can mimic the
same binary merge.

980
00:50:36,680 --> 00:50:40,610
You look at the first item
of each of the sorted arrays.

981
00:50:40,610 --> 00:50:41,500
You compare them.

982
00:50:41,500 --> 00:50:43,000
In this model,
comparisons are free.

983
00:50:43,000 --> 00:50:44,250
Let's not even worry about it.

984
00:50:44,250 --> 00:50:46,760
In reality, use a priority
queue, but all right.

985
00:50:46,760 --> 00:50:48,840
So you find the
minimum of these.

986
00:50:48,840 --> 00:50:49,840
Let's say it's this one.

987
00:50:49,840 --> 00:50:52,570
And you output that,
and then you advance.

988
00:50:52,570 --> 00:50:57,090
Same algorithm, that will merge
however many arrays you have.

989
00:50:57,090 --> 00:51:00,130
The issue is, for this to be
efficient like it was here,

990
00:51:00,130 --> 00:51:03,510
we need to be able to store
the first block of each

991
00:51:03,510 --> 00:51:05,100
of these arrays.

992
00:51:05,100 --> 00:51:06,970
How many blocks
we have room for?

993
00:51:06,970 --> 00:51:07,740
M over b.

994
00:51:07,740 --> 00:51:09,300
This is maxing out merge sort.

995
00:51:09,300 --> 00:51:12,620
This is exactly the number
of blocks that we can store.

996
00:51:12,620 --> 00:51:15,110
And so if we do m over
b way merge sort, merge

997
00:51:15,110 --> 00:51:18,080
remains cheap.

998
00:51:18,080 --> 00:51:26,790
An m over b way merge
costs n over b plus 1,

999
00:51:26,790 --> 00:51:28,000
just like before.

1000
00:51:28,000 --> 00:51:31,810
It's m over b parallel scans.

1001
00:51:31,810 --> 00:51:35,270
M over b is exactly the
number of scans we can handle.

1002
00:51:35,270 --> 00:51:37,500
OK, technically we
have, with this picture,

1003
00:51:37,500 --> 00:51:39,070
m over b plus 1 scans.

1004
00:51:39,070 --> 00:51:41,480
So I need to write
m over b minus 1.

1005
00:51:41,480 --> 00:51:44,995
But it won't make a difference.

1006
00:51:44,995 --> 00:51:46,620
OK, so let's write
down the recurrence.

1007
00:51:46,620 --> 00:51:48,680
It's pretty similar.

1008
00:51:48,680 --> 00:51:50,460
Memory transfer's size m.

1009
00:51:50,460 --> 00:51:58,710
We have m over b sub problems
of size n divided by m over b.

1010
00:51:58,710 --> 00:52:01,070
It's Still conservation of mass.

1011
00:52:01,070 --> 00:52:03,350
And then we have plus
the same thing as before,

1012
00:52:03,350 --> 00:52:06,200
n over b plus 1.

1013
00:52:06,200 --> 00:52:09,120
So it's exactly the same
recurrence we had before.

1014
00:52:09,120 --> 00:52:10,640
We're splitting
into more problems.

1015
00:52:10,640 --> 00:52:11,960
But the sums are
going to be the same.

1016
00:52:11,960 --> 00:52:13,720
It's still going to add up
to n over b at each step

1017
00:52:13,720 --> 00:52:15,020
because conservation of mass.

1018
00:52:15,020 --> 00:52:16,700
And we didn't change this.

1019
00:52:16,700 --> 00:52:18,750
So level by level
looks the same.

1020
00:52:18,750 --> 00:52:22,920
The only thing that changes
is the number of levels.

1021
00:52:22,920 --> 00:52:23,890
Now we're taking n.

1022
00:52:23,890 --> 00:52:26,870
We're dividing by m
over b in each step.

1023
00:52:26,870 --> 00:52:28,922
So the height of
the tree, the number

1024
00:52:28,922 --> 00:52:35,170
of levels of the
recursion tree now

1025
00:52:35,170 --> 00:52:40,400
is-- before it was log
base 2 of n over n.

1026
00:52:40,400 --> 00:52:48,500
Now it's going to be log
base m over b of n over m.

1027
00:52:51,394 --> 00:52:52,810
If you're careful,
I guess there's

1028
00:52:52,810 --> 00:52:55,105
a plus 1 for the leaf level.

1029
00:52:55,105 --> 00:52:57,210
I actually want to
mention this plus 1.

1030
00:52:57,210 --> 00:52:58,862
Unlike the other
plus 1's, I've got

1031
00:52:58,862 --> 00:53:02,290
to mention this one because
this is not how I usually

1032
00:53:02,290 --> 00:53:04,520
think of the number of levels.

1033
00:53:04,520 --> 00:53:06,396
I'll show you why.

1034
00:53:06,396 --> 00:53:07,770
If you just change
it by one, you

1035
00:53:07,770 --> 00:53:09,700
get a slightly cleaner formula.

1036
00:53:09,700 --> 00:53:11,230
This has got m's
all over the place.

1037
00:53:14,030 --> 00:53:18,420
So I just want to
rewrite n over m here.

1038
00:53:18,420 --> 00:53:20,280
Then we'll see how good this is.

1039
00:53:20,280 --> 00:53:22,150
This is just pedantics.

1040
00:53:22,150 --> 00:53:27,830
Log base m over b of n--
I really want n over b.

1041
00:53:27,830 --> 00:53:32,690
To make this n over b, I need
to multiply by b, divide by m.

1042
00:53:32,690 --> 00:53:34,010
OK, these are the same thing.

1043
00:53:34,010 --> 00:53:37,460
M over m, b's cancel.

1044
00:53:37,460 --> 00:53:39,140
But I have a log of a product.

1045
00:53:39,140 --> 00:53:42,270
I can separate that out.

1046
00:53:42,270 --> 00:53:44,000
Let's go over here.

1047
00:53:44,000 --> 00:53:52,970
This is log base m over b of n
over b-- this is what I like--

1048
00:53:52,970 --> 00:53:59,342
and then, basically, minus
log base m over b of m over b.

1049
00:53:59,342 --> 00:54:01,400
STUDENT: It's b over m.

1050
00:54:01,400 --> 00:54:03,300
PROFESSOR: I put a
minus, so it's m over b.

1051
00:54:03,300 --> 00:54:05,105
If I put a plus, it
would be b over m.

1052
00:54:05,105 --> 00:54:06,480
But, in fact, m
is bigger than b.

1053
00:54:06,480 --> 00:54:08,690
So I want it this way.

1054
00:54:08,690 --> 00:54:11,400
And now it's obvious this is 1.

1055
00:54:11,400 --> 00:54:13,281
So these cancel.

1056
00:54:13,281 --> 00:54:15,530
So that's why I wanted the
1, just to get rid of that.

1057
00:54:15,530 --> 00:54:17,500
It doesn't really
matter, just a plus 1.

1058
00:54:17,500 --> 00:54:21,080
But it's a cooler way to
see that, in some sense,

1059
00:54:21,080 --> 00:54:24,300
this is the right answer
of the height of the tree.

1060
00:54:24,300 --> 00:54:27,530
Now, we're paying n over
b at each recursive level.

1061
00:54:27,530 --> 00:54:33,280
So the total cost is what's
called the sorting bound.

1062
00:54:33,280 --> 00:54:40,900
This is optimal, n over b times
log base m over b of n over b.

1063
00:54:40,900 --> 00:54:42,100
Oh my gosh, what a mouthful.

1064
00:54:42,100 --> 00:54:44,330
But every person who does
external memory algorithms

1065
00:54:44,330 --> 00:54:47,240
and cache oblivious
algorithms knows this.

1066
00:54:47,240 --> 00:54:49,440
It is the truth, it turns out.

1067
00:54:49,440 --> 00:54:50,800
There's a matching lower bound.

1068
00:54:50,800 --> 00:54:51,620
It's a weird bound.

1069
00:54:51,620 --> 00:54:54,040
But let's compare
it to what we know.

1070
00:54:54,040 --> 00:54:57,780
So we started out with n
log n divided by log b.

1071
00:54:57,780 --> 00:55:01,420
Then we got n log
n divided by b.

1072
00:55:01,420 --> 00:55:03,490
Let's ignore-- I mean,
this has almost no effect,

1073
00:55:03,490 --> 00:55:04,660
the part in here.

1074
00:55:04,660 --> 00:55:11,400
Now we have n log n divided by
b and divided by log m over b.

1075
00:55:11,400 --> 00:55:13,310
It's not quite
dividing by log b.

1076
00:55:13,310 --> 00:55:17,170
But it turns out it's
almost always the same.

1077
00:55:17,170 --> 00:55:19,020
In some sense, this
could be better.

1078
00:55:19,020 --> 00:55:21,460
If you're cache is big, now
you're dividing by log m,

1079
00:55:21,460 --> 00:55:22,270
roughly.

1080
00:55:22,270 --> 00:55:24,960
Before, you were only
dividing by log b.

1081
00:55:24,960 --> 00:55:27,540
And it turns out this
is the best you can do.

1082
00:55:27,540 --> 00:55:30,780
So this is going to be a little
bit better than merge sort.

1083
00:55:30,780 --> 00:55:36,940
If your cache is 16 gigabytes,
like your RAM caching

1084
00:55:36,940 --> 00:55:41,985
your disk, then log
m is pretty big.

1085
00:55:41,985 --> 00:55:48,635
It's going to be 32 or
something, 34 I guess log m.

1086
00:55:48,635 --> 00:55:50,004
OK, I have to divide by b.

1087
00:55:50,004 --> 00:55:50,920
So it's not that good.

1088
00:55:50,920 --> 00:55:53,295
But still, I'm getting an
improvement over regular binary

1089
00:55:53,295 --> 00:55:54,250
merge sort.

1090
00:55:54,250 --> 00:55:55,810
And you would see
that improvement.

1091
00:55:55,810 --> 00:55:57,210
These are big factors.

1092
00:55:57,210 --> 00:55:59,440
The big thing, of course,
is dividing it by b.

1093
00:55:59,440 --> 00:56:03,330
But dividing by log of m over
b is also nice and the best you

1094
00:56:03,330 --> 00:56:04,790
can do.

1095
00:56:04,790 --> 00:56:08,460
OK, obviously I needed to
know what m and b were here.

1096
00:56:08,460 --> 00:56:14,290
So the natural question next
is cache oblivious sorting.

1097
00:56:14,290 --> 00:56:16,550
And that would take
another lecture to cover.

1098
00:56:16,550 --> 00:56:18,570
So I'm not going to do it here.

1099
00:56:18,570 --> 00:56:19,600
But it can be done.

1100
00:56:19,600 --> 00:56:22,720
Cache obliviously, you can
achieve the same thing.

1101
00:56:22,720 --> 00:56:24,760
And I'll give you the intuition.

1102
00:56:24,760 --> 00:56:25,595
There's one catch.

1103
00:56:25,595 --> 00:56:26,860
Let me mention the catch.

1104
00:56:43,930 --> 00:56:53,435
So cache oblivious sorting--
to do optimal cache oblivious

1105
00:56:53,435 --> 00:56:56,940
sorting like that
bound, it turns out you

1106
00:56:56,940 --> 00:57:00,605
need an assumption called
the tall-cache assumption.

1107
00:57:11,840 --> 00:57:13,540
Simple form of the
tall-cache assumption

1108
00:57:13,540 --> 00:57:19,810
is that m is at least b squared.

1109
00:57:19,810 --> 00:57:25,250
What that means is m
over b is at least b.

1110
00:57:25,250 --> 00:57:28,662
In other words, the cache
is taller than it is wide,

1111
00:57:28,662 --> 00:57:29,870
the way I've been drawing it.

1112
00:57:29,870 --> 00:57:32,100
That's why it's called
the tall-cache assumption.

1113
00:57:32,100 --> 00:57:36,006
And if you look at real caches,
this is usually the case.

1114
00:57:36,006 --> 00:57:38,380
I don't know of a great reason
why it should be the case.

1115
00:57:38,380 --> 00:57:41,402
But it usually is,
so all is well.

1116
00:57:41,402 --> 00:57:42,860
You can do cache
oblivious sorting.

1117
00:57:42,860 --> 00:57:44,860
It turns out, if you don't
have this assumption,

1118
00:57:44,860 --> 00:57:47,140
you cannot achieve this bound.

1119
00:57:47,140 --> 00:57:49,080
We don't know what
bound you can achieve.

1120
00:57:49,080 --> 00:57:51,060
But we just know this
one is not possible.

1121
00:57:51,060 --> 00:57:53,200
You can get a contradiction
if you achieve that

1122
00:57:53,200 --> 00:57:54,530
without tall cache.

1123
00:57:54,530 --> 00:57:55,660
So it's a little bit weird.

1124
00:57:55,660 --> 00:57:57,330
You have to make one
bonus assumption.

1125
00:57:57,330 --> 00:57:59,130
You can make a somewhat
weaker form of it,

1126
00:57:59,130 --> 00:58:08,870
which is m is omega
b to the 1.000000001.

1127
00:58:08,870 --> 00:58:09,840
That will do.

1128
00:58:09,840 --> 00:58:11,360
In general, 1 plus epsilon.

1129
00:58:11,360 --> 00:58:13,070
Any epsilon will be fine.

1130
00:58:13,070 --> 00:58:15,540
We just mean that
the number of blocks

1131
00:58:15,540 --> 00:58:17,650
is at least some
b to the epsilon,

1132
00:58:17,650 --> 00:58:21,867
where epsilon's a
constant bigger than zero.

1133
00:58:21,867 --> 00:58:23,700
OK, then you can do
cache oblivious sorting.

1134
00:58:23,700 --> 00:58:25,880
Let me tell you how.

1135
00:58:25,880 --> 00:58:29,110
We want to do m over
b way merge sort.

1136
00:58:29,110 --> 00:58:31,800
But we don't know how to do--
we don't know what m over b is.

1137
00:58:31,800 --> 00:58:35,110
So instead, we're going
to do something like n

1138
00:58:35,110 --> 00:58:38,420
to the epsilon way merge sort.

1139
00:58:43,230 --> 00:58:44,850
That's a so-so interpretation.

1140
00:58:44,850 --> 00:58:47,030
This is back to
your idea roughly.

1141
00:58:47,030 --> 00:58:52,130
We're dividing into
a lot of chunks.

1142
00:58:52,130 --> 00:58:54,130
And then we don't know
how to merge them anymore

1143
00:58:54,130 --> 00:58:58,590
because we can't do regular
merge with n to the epsilon

1144
00:58:58,590 --> 00:59:00,990
chunks it could be n to
the epsilon's too big.

1145
00:59:00,990 --> 00:59:01,980
So how do we do it?

1146
00:59:01,980 --> 00:59:04,770
We do a divide
and conquer merge.

1147
00:59:04,770 --> 00:59:09,200
This is actually
called funnel sort

1148
00:59:09,200 --> 00:59:12,810
because the way you do a
divide and conquer merge looks

1149
00:59:12,810 --> 00:59:13,930
kind of like a funnel.

1150
00:59:13,930 --> 00:59:15,721
Actually, it looks a
lot like the triangles

1151
00:59:15,721 --> 00:59:16,910
we were drawing earlier.

1152
00:59:16,910 --> 00:59:18,510
It's just a lot
messier to analyze.

1153
00:59:18,510 --> 00:59:20,360
So I'm not going to do it here.

1154
00:59:20,360 --> 00:59:23,570
It would take another
40 minutes or so.

1155
00:59:23,570 --> 00:59:27,070
But that's some intuition of
how you do cache oblivious

1156
00:59:27,070 --> 00:59:27,570
merge sort.

1157
00:59:30,270 --> 00:59:32,750
That's what I want to say
cache oblivious stuff.

1158
00:59:32,750 --> 00:59:35,600
Oh, one more thing!

1159
00:59:35,600 --> 00:59:41,580
One more cool thing you can
do-- I'm a data structures guy.

1160
00:59:41,580 --> 00:59:43,720
So sorting is nice.

1161
00:59:43,720 --> 00:59:46,160
But what I really like
are priority queues

1162
00:59:46,160 --> 00:59:49,400
because they're more
general than sorting.

1163
00:59:49,400 --> 00:59:50,940
We started out by
saying, hey, look.

1164
00:59:50,940 --> 00:59:52,870
If you want to sort
and you use a b tree,

1165
00:59:52,870 --> 00:59:55,310
you get a really
bad running time.

1166
00:59:55,310 --> 00:59:58,130
That's weird because
usually BST sort

1167
00:59:58,130 --> 00:59:59,960
is good in a regular
comparison model.

1168
00:59:59,960 --> 01:00:01,804
It's n log n.

1169
01:00:01,804 --> 01:00:03,470
So b trees are clearly
not what we want.

1170
01:00:03,470 --> 01:00:04,886
Is there some other
thing we want?

1171
01:00:04,886 --> 01:00:06,070
And it turns out, yes.

1172
01:00:09,330 --> 01:00:12,020
You can build a
priority queue, which

1173
01:00:12,020 --> 01:00:15,070
supports insert and
delete min and a bunch

1174
01:00:15,070 --> 01:00:16,400
of other operations.

1175
01:00:16,400 --> 01:00:21,945
Each of those operations costs
1 over b log base m over b of n

1176
01:00:21,945 --> 01:00:34,170
over b amortized memory
transfers-- a bit of a mouthful

1177
01:00:34,170 --> 01:00:34,760
again.

1178
01:00:34,760 --> 01:00:39,420
But if you compare this
bound with this bound,

1179
01:00:39,420 --> 01:00:40,390
it's exactly the same.

1180
01:00:40,390 --> 01:00:47,220
But I divided by n, which means
if I insert with this cost n

1181
01:00:47,220 --> 01:00:49,140
times, I pay the sorting bound.

1182
01:00:49,140 --> 01:00:52,200
If I delete min with
this bound n times,

1183
01:00:52,200 --> 01:00:53,570
I get the sorting bound.

1184
01:00:53,570 --> 01:00:56,310
So if I insert n times and
then delete all the items out,

1185
01:00:56,310 --> 01:00:59,790
I've sorted the items
in sorting bound time.

1186
01:00:59,790 --> 01:01:03,040
So this is the data
structure generalization

1187
01:01:03,040 --> 01:01:04,640
of that sorting algorithm.

1188
01:01:04,640 --> 01:01:07,774
Now, this is even harder to do.

1189
01:01:07,774 --> 01:01:09,440
Originally, it was
done external memory.

1190
01:01:09,440 --> 01:01:11,000
It's called buffer trees.

1191
01:01:11,000 --> 01:01:12,967
Then we did it
cache obliviously.

1192
01:01:12,967 --> 01:01:14,800
It's called cache
oblivious priority queues.

1193
01:01:14,800 --> 01:01:16,950
We weren't very creative.

1194
01:01:16,950 --> 01:01:17,960
But it can be done.

1195
01:01:17,960 --> 01:01:19,910
And, again, if you
want to learn more,

1196
01:01:19,910 --> 01:01:25,010
you should take 6851,
Advanced Data Structures,

1197
01:01:25,010 --> 01:01:27,130
which leads us into
the next topic, what

1198
01:01:27,130 --> 01:01:31,029
class you should take
next-- classes, that's

1199
01:01:31,029 --> 01:01:31,820
what I mean to say.

1200
01:01:44,510 --> 01:01:45,610
So a lot of bias here.

1201
01:01:45,610 --> 01:01:48,280
And well I'm just going
to give a lot of classes.

1202
01:01:48,280 --> 01:01:49,850
There's a lot of them.

1203
01:01:49,850 --> 01:01:53,705
I believe this is in roughly
numerical order almost.

1204
01:01:53,705 --> 01:01:57,890
It changed a little
bit-- so many classes.

1205
01:02:00,332 --> 01:02:02,290
Are you OK with numbers,
or do you want titles?

1206
01:02:06,030 --> 01:02:08,300
The obvious follow-on
course to this class

1207
01:02:08,300 --> 01:02:10,810
is 6854, which is
Advanced Algorithms.

1208
01:02:10,810 --> 01:02:13,330
It's the first graduate
algorithms class.

1209
01:02:13,330 --> 01:02:16,500
This is the last undergraduate
class, roughly speaking,

1210
01:02:16,500 --> 01:02:17,830
with the exception of 6047.

1211
01:02:17,830 --> 01:02:20,420
But in terms of straight,
general algorithms,

1212
01:02:20,420 --> 01:02:22,180
this would be the natural class.

1213
01:02:22,180 --> 01:02:24,630
It's only in the fall--
sadly, not next fall.

1214
01:02:24,630 --> 01:02:26,410
But in general,
it's a cool class.

1215
01:02:26,410 --> 01:02:31,310
It's a very broad overview of
algorithms but much more hard

1216
01:02:31,310 --> 01:02:33,300
core, I guess.

1217
01:02:33,300 --> 01:02:36,090
It's an intense class but
covers a lot of fields,

1218
01:02:36,090 --> 01:02:37,412
a lot of areas of algorithms.

1219
01:02:37,412 --> 01:02:39,120
Then all the other
ones I'm going to list

1220
01:02:39,120 --> 01:02:40,330
are more specialized.

1221
01:02:40,330 --> 01:02:42,972
So 6047 is
Computational Biology.

1222
01:02:42,972 --> 01:02:44,430
So if you're
interested in biology,

1223
01:02:44,430 --> 01:02:46,327
you want algorithms
applied to biology.

1224
01:02:46,327 --> 01:02:47,160
That's a cool class.

1225
01:02:47,160 --> 01:02:48,700
It's also an undergrad class.

1226
01:02:48,700 --> 01:02:51,040
Everything else here-- I
mean, you know the story.

1227
01:02:51,040 --> 01:02:52,960
You take grad
classes all the time,

1228
01:02:52,960 --> 01:02:58,020
or you will soon if you
want to do more algorithms.

1229
01:02:58,020 --> 01:03:02,270
So 6850 is
computational geometry.

1230
01:03:02,270 --> 01:03:04,059
I think it's called
Geometric Algorithms.

1231
01:03:04,059 --> 01:03:05,600
So we've seen a
couple examples, like

1232
01:03:05,600 --> 01:03:07,016
the convex hull
divide-and-conquer

1233
01:03:07,016 --> 01:03:10,354
algorithm and the range trees.

1234
01:03:10,354 --> 01:03:12,270
Those are two examples
of geometric algorithms

1235
01:03:12,270 --> 01:03:14,350
where you have points and
lines and stuff-- maybe

1236
01:03:14,350 --> 01:03:16,225
in two dimensions, maybe
in three dimensions,

1237
01:03:16,225 --> 01:03:18,296
maybe log n dimensions.

1238
01:03:18,296 --> 01:03:19,712
If you like that
stuff, you should

1239
01:03:19,712 --> 01:03:21,956
take computational geometry.

1240
01:03:21,956 --> 01:03:23,830
This is the devil that
lad me into algorithms

1241
01:03:23,830 --> 01:03:24,830
in the first place.

1242
01:03:24,830 --> 01:03:26,640
Cool stuff.

1243
01:03:26,640 --> 01:03:30,090
6849 is my class on
folding algorithms.

1244
01:03:30,090 --> 01:03:33,830
This is a special type
of geometric algorithms

1245
01:03:33,830 --> 01:03:37,710
where we think about paper
folding and robotic arm folding

1246
01:03:37,710 --> 01:03:40,346
and protein folding
and things like that.

1247
01:03:40,346 --> 01:03:41,970
So that's a bit of
a specialized class.

1248
01:03:41,970 --> 01:03:46,720
6851, I've mentioned three times
now-- Advanced Data Structures.

1249
01:03:46,720 --> 01:03:49,660
Then we've got
6852, its neighbor.

1250
01:03:49,660 --> 01:03:51,606
This is Nancy's Distributed
Algorithms class.

1251
01:03:51,606 --> 01:03:53,730
So if you liked the week
of distributed algorithms,

1252
01:03:53,730 --> 01:03:55,970
there's a whole class on it.

1253
01:03:55,970 --> 01:03:58,370
She wrote the textbook for it.

1254
01:03:58,370 --> 01:04:01,660
Then there's 6853.

1255
01:04:01,660 --> 01:04:03,570
This is Algorithmic Game Theory.

1256
01:04:03,570 --> 01:04:07,290
If you care about
algorithms involving

1257
01:04:07,290 --> 01:04:09,864
multiple players-- and the
players are each selfish.

1258
01:04:09,864 --> 01:04:11,780
And they have no reason
to tell you the truth.

1259
01:04:11,780 --> 01:04:13,620
And still you want
to compute something

1260
01:04:13,620 --> 01:04:16,114
like minimum spanning tree,
or pick your favorite thing.

1261
01:04:16,114 --> 01:04:17,780
Everyone's lying about
the edge weights.

1262
01:04:17,780 --> 01:04:19,360
And still you want
to figure out how

1263
01:04:19,360 --> 01:04:22,560
to design a mechanism
like an auction

1264
01:04:22,560 --> 01:04:25,170
so that you actually end up
buying a minimum spanning tree.

1265
01:04:25,170 --> 01:04:26,260
You can do that.

1266
01:04:26,260 --> 01:04:29,876
And if you want to know
how, you should take 6853.

1267
01:04:29,876 --> 01:04:32,290
What else do we have?

1268
01:04:32,290 --> 01:04:35,860
6855 is Network Optimization.

1269
01:04:35,860 --> 01:04:38,750
So this is like the natural
follow on of network flows.

1270
01:04:38,750 --> 01:04:40,900
If you like network flows
and things like that,

1271
01:04:40,900 --> 01:04:43,760
there's a whole universe
called network optimization.

1272
01:04:43,760 --> 01:04:46,140
It has lots of fancy,
basically, graph algorithms

1273
01:04:46,140 --> 01:04:48,140
where you're minimizing
or maximizing something.

1274
01:04:51,090 --> 01:04:53,930
OK, this is
fortuitous alignment.

1275
01:04:53,930 --> 01:04:58,080
6856 is kind of
a friend of 6854.

1276
01:04:58,080 --> 01:05:00,660
These are both taught
by David Carter.

1277
01:05:00,660 --> 01:05:02,540
This is Randomized Algorithms.

1278
01:05:02,540 --> 01:05:06,015
So this is a more
specialized approach.

1279
01:05:06,015 --> 01:05:07,890
I don't think you need
one to take the other.

1280
01:05:07,890 --> 01:05:09,920
But this is the
usual starting class.

1281
01:05:09,920 --> 01:05:13,350
And this is specifically
about how randomization makes

1282
01:05:13,350 --> 01:05:15,180
algorithms faster or simpler.

1283
01:05:15,180 --> 01:05:16,680
Usually they're
harder to analyze.

1284
01:05:16,680 --> 01:05:18,680
But you get very simple
algorithms that run just

1285
01:05:18,680 --> 01:05:20,810
as well as their
deterministic versions.

1286
01:05:20,810 --> 01:05:22,790
Sometimes you can
do even better than

1287
01:05:22,790 --> 01:05:24,900
the deterministic versions.

1288
01:05:24,900 --> 01:05:27,750
Then there's the
security universe.

1289
01:05:27,750 --> 01:05:30,606
This is a great
numerical coincidence--

1290
01:05:30,606 --> 01:05:31,730
probably not a coincidence.

1291
01:05:31,730 --> 01:05:34,944
But there's 6857 and 6875.

1292
01:05:34,944 --> 01:05:36,360
I have to remember
which is which.

1293
01:05:36,360 --> 01:05:38,680
6857 is Applied Cryptography.

1294
01:05:38,680 --> 01:05:40,720
6875 is Theoretical
Cryptography,

1295
01:05:40,720 --> 01:05:42,530
at least as I read it.

1296
01:05:42,530 --> 01:05:44,360
So they have similar topics.

1297
01:05:44,360 --> 01:05:46,380
But this is more thinking
about how you really

1298
01:05:46,380 --> 01:05:49,560
achieve security and crypto
systems and things like that.

1299
01:05:49,560 --> 01:05:52,504
And this one is more
algorithm based.

1300
01:05:52,504 --> 01:05:54,170
And what kind of
theoretical assumptions

1301
01:05:54,170 --> 01:05:55,440
do you need to prove
certain things?

1302
01:05:55,440 --> 01:05:56,481
This is more proof based.

1303
01:05:56,481 --> 01:06:03,840
And this is more connecting
to systems, both great topics.

1304
01:06:03,840 --> 01:06:05,687
And I have one
more out of order,

1305
01:06:05,687 --> 01:06:07,520
I guess just because
it's a recent addition.

1306
01:06:07,520 --> 01:06:11,475
6816 is Multicore Programming.

1307
01:06:11,475 --> 01:06:13,950
That has a lot of
algorithms, too.

1308
01:06:13,950 --> 01:06:17,974
And this is all about
parallel computation.

1309
01:06:17,974 --> 01:06:19,890
When you have multiple
cores on your computer,

1310
01:06:19,890 --> 01:06:22,150
how can you compute
things like these things

1311
01:06:22,150 --> 01:06:24,340
faster than
everything we've done?

1312
01:06:24,340 --> 01:06:26,290
It's yet another universe
that we haven't even

1313
01:06:26,290 --> 01:06:27,670
touched on in this class.

1314
01:06:27,670 --> 01:06:31,030
But it's cool stuff, and
you might consider it.

1315
01:06:31,030 --> 01:06:34,890
Then we move on to
other theory classes.

1316
01:06:34,890 --> 01:06:37,100
That was algorithms.

1317
01:06:37,100 --> 01:06:40,080
Some more obvious candidates,
if you like pure theory,

1318
01:06:40,080 --> 01:06:41,980
are 6045 6840.

1319
01:06:41,980 --> 01:06:43,230
This is the undergrad version.

1320
01:06:43,230 --> 01:06:43,800
This is the grad version.

1321
01:06:43,800 --> 01:06:45,841
Although, by now the
classes are quite different.

1322
01:06:45,841 --> 01:06:47,290
So they cover different things.

1323
01:06:47,290 --> 01:06:48,790
Some of you are
already taking 6045.

1324
01:06:48,790 --> 01:06:50,550
It's right before this lecture.

1325
01:06:50,550 --> 01:06:55,190
These are general theory of
computation classes, atomita,

1326
01:06:55,190 --> 01:06:57,240
complexity, things like that.

1327
01:06:57,240 --> 01:07:00,230
If you like the brief
NP completeness lecture,

1328
01:07:00,230 --> 01:07:02,149
then you might like this stuff.

1329
01:07:02,149 --> 01:07:04,690
There's so many more complexity
classes and other cool things

1330
01:07:04,690 --> 01:07:06,060
you can do.

1331
01:07:06,060 --> 01:07:09,190
If you really like it, there's
advanced complexity theory.

1332
01:07:09,190 --> 01:07:12,280
There's, basically,
randomized complexity theory--

1333
01:07:12,280 --> 01:07:14,620
how randomness affects
just the complexity side,

1334
01:07:14,620 --> 01:07:15,950
not algorithms.

1335
01:07:15,950 --> 01:07:19,330
Then there's quantum
complexity theory

1336
01:07:19,330 --> 01:07:21,430
if you care about
quantum computers.

1337
01:07:21,430 --> 01:07:23,800
As Scott says,
it's proving things

1338
01:07:23,800 --> 01:07:26,110
you can't do with
computers we don't have.

1339
01:07:26,110 --> 01:07:27,420
[LAUGHTER]

1340
01:07:29,900 --> 01:07:30,610
It's complexity.

1341
01:07:30,610 --> 01:07:32,790
It's all about lower bounds.

1342
01:07:32,790 --> 01:07:35,970
And then there's coding theory,
which is another universe.

1343
01:07:35,970 --> 01:07:39,190
It's actually closely
related to-- it comes out

1344
01:07:39,190 --> 01:07:42,410
of signals and systems and
electrical engineering.

1345
01:07:42,410 --> 01:07:45,350
But by now it's closely
related to complexity theory.

1346
01:07:45,350 --> 01:07:47,670
You can use bounds on
codes to prove things

1347
01:07:47,670 --> 01:07:49,640
about complexity theory.

1348
01:07:49,640 --> 01:07:54,220
Anyway, choose
your own adventure.

1349
01:07:54,220 --> 01:07:57,179
Now I have one last topic,
which was not on the outline.

1350
01:07:57,179 --> 01:07:58,345
This is a bit of a surprise.

1351
01:08:01,042 --> 01:08:02,000
It's a boring surprise.

1352
01:08:02,000 --> 01:08:05,345
I want to remind you to fill
out student evaluations.

1353
01:08:05,345 --> 01:08:07,200
[LAUGHTER]

1354
01:08:07,606 --> 01:08:09,480
Because, you know, we
want to know how we did

1355
01:08:09,480 --> 01:08:11,700
and how we can continue
to improve the class.

1356
01:08:11,700 --> 01:08:16,200
But really we want to know
who's the better teacher.

1357
01:08:16,200 --> 01:08:20,160
But more importantly than
who is the better teacher,

1358
01:08:20,160 --> 01:08:23,330
I think we all have a dying
question, which is who

1359
01:08:23,330 --> 01:08:25,910
is the better Frisbee thrower?

1360
01:08:25,910 --> 01:08:30,710
So I want to invite Srini
Devadas, our co-lecturer here,

1361
01:08:30,710 --> 01:08:33,146
to a duel.

1362
01:08:33,146 --> 01:08:35,090
[LAUGHTER AND APPLAUSE]

1363
01:08:38,978 --> 01:08:42,170
SRINI DEVADAS: I think
you mean, no contest.

1364
01:08:42,170 --> 01:08:43,149
[LAUGHTER]

1365
01:08:44,090 --> 01:08:45,350
PROFESSOR: Not so sure.

1366
01:08:45,350 --> 01:08:47,720
Maybe-- actually,
I'm pretty sure.

1367
01:08:47,720 --> 01:08:49,460
[LAUGHTER]

1368
01:08:50,330 --> 01:08:52,040
I want to take you on, man.

1369
01:08:52,040 --> 01:08:55,207
Blue or purple?

1370
01:08:55,207 --> 01:08:56,040
SRINI DEVADAS: Blue.

1371
01:08:56,040 --> 01:08:56,460
PROFESSOR: Good choice.

1372
01:08:56,460 --> 01:08:58,084
SRINI DEVADAS: Blue's
better, remember?

1373
01:08:58,084 --> 01:08:58,850
[LAUGHTER]

1374
01:08:58,850 --> 01:09:00,391
PROFESSOR: Purple's
better, remember?

1375
01:09:00,391 --> 01:09:00,914
[LAUGHTER]

1376
01:09:01,840 --> 01:09:04,634
All right, so how are
we going to do this?

1377
01:09:04,634 --> 01:09:08,063
SRINI DEVADAS: So, you
guys get to cheer and bet.

1378
01:09:08,063 --> 01:09:09,880
PROFESSOR: Bet?

1379
01:09:09,880 --> 01:09:12,439
I don't think we can
condone them betting money.

1380
01:09:12,439 --> 01:09:14,792
I think maybe they can
bet their Frisbees.

1381
01:09:14,792 --> 01:09:16,000
Anyone got a Frisbee on them?

1382
01:09:16,000 --> 01:09:17,370
We can bet those.

1383
01:09:17,370 --> 01:09:19,069
SRINI DEVADAS: Yeah, all right.

1384
01:09:19,069 --> 01:09:20,490
PROFESSOR: All right, maybe not.

1385
01:09:20,490 --> 01:09:23,874
SRINI DEVADAS: Put your
Frisbees on me here.

1386
01:09:23,874 --> 01:09:25,189
PROFESSOR: All right.

1387
01:09:25,189 --> 01:09:28,600
SRINI DEVADAS: All right,
so some rules here--

1388
01:09:28,600 --> 01:09:29,850
we actually talked about this.

1389
01:09:32,301 --> 01:09:34,050
So the way this is
going to work-- I mean,

1390
01:09:34,050 --> 01:09:36,542
it's going to be
algorithmic, obviously.

1391
01:09:36,542 --> 01:09:38,250
And we get to choose
our algorithm, maybe

1392
01:09:38,250 --> 01:09:39,729
with a little game theory here.

1393
01:09:39,729 --> 01:09:41,109
We're going to toss the coin.

1394
01:09:41,109 --> 01:09:43,359
And we're going to
decide who goes first.

1395
01:09:43,359 --> 01:09:45,200
So won't spin the Frisbee.

1396
01:09:45,200 --> 01:09:47,330
Remember what
happened with that?

1397
01:09:47,330 --> 01:09:50,452
So you get to call heads or
tails while it's spinning.

1398
01:09:50,452 --> 01:09:51,910
PROFESSOR: Oh,
while it's spinning.

1399
01:09:51,910 --> 01:09:53,368
SRINI DEVADAS:
While it's spinning.

1400
01:09:53,368 --> 01:09:55,610
This is our Super Bowl.

1401
01:09:55,610 --> 01:09:57,300
PROFESSOR: Heads!

1402
01:09:57,300 --> 01:09:59,330
Oh, a trick.

1403
01:09:59,330 --> 01:09:59,890
Tails.

1404
01:09:59,890 --> 01:10:02,500
SRINI DEVADAS: Tails, all right.

1405
01:10:02,500 --> 01:10:03,810
You're going to throw first.

1406
01:10:03,810 --> 01:10:05,000
PROFESSOR: OK,
that's your choice.

1407
01:10:05,000 --> 01:10:06,990
SRINI DEVADAS: And I
don't know if you've heard

1408
01:10:06,990 --> 01:10:09,720
the legend of William Tell.

1409
01:10:09,720 --> 01:10:12,510
How many of you have heard
the legend of William Tell?

1410
01:10:12,510 --> 01:10:13,470
All right.

1411
01:10:13,470 --> 01:10:17,070
So that was a 14th
century Swiss legend

1412
01:10:17,070 --> 01:10:22,680
where there was this archer
who was renowned for his skill.

1413
01:10:22,680 --> 01:10:25,090
And he was forced by
this villainous king

1414
01:10:25,090 --> 01:10:29,662
to shoot an apple off the
top of his son's head.

1415
01:10:29,662 --> 01:10:30,370
PROFESSOR: Yikes.

1416
01:10:30,370 --> 01:10:33,260
SRINI DEVADAS: So we're
going to reenact that.

1417
01:10:33,260 --> 01:10:34,490
[LAUGHTER]

1418
01:10:34,540 --> 01:10:36,165
PROFESSOR: Did you
bring your daughter?

1419
01:10:36,165 --> 01:10:37,859
[LAUGHTER AND APPLAUSE]

1420
01:10:41,019 --> 01:10:42,435
SRINI DEVADAS: I
was thinking TAs.

1421
01:10:44,844 --> 01:10:45,760
PROFESSOR: Our "sons."

1422
01:10:45,760 --> 01:10:47,551
SRINI DEVADAS: But
there's a big difference

1423
01:10:47,551 --> 01:10:50,020
between the 21st century
and the 14th century.

1424
01:10:50,020 --> 01:10:51,239
What is that?

1425
01:10:51,239 --> 01:10:52,155
STUDENT: You get sued.

1426
01:10:52,155 --> 01:10:52,610
[INTERPOSING VOICES]

1427
01:10:52,610 --> 01:10:54,006
PROFESSOR: You get sued, yeah.

1428
01:10:54,006 --> 01:10:55,880
SRINI DEVADAS: Now
there's many more lawsuits

1429
01:10:55,880 --> 01:10:57,562
in the 21st century.

1430
01:10:57,562 --> 01:10:58,770
So we want to avoid lawsuits.

1431
01:10:58,770 --> 01:11:00,579
STUDENT: Genetically
modified apples.

1432
01:11:00,579 --> 01:11:02,620
PROFESSOR: And genetically
modified apples, also.

1433
01:11:02,620 --> 01:11:03,920
SRINI DEVADAS: Electronically
modified Apples,

1434
01:11:03,920 --> 01:11:05,520
yeah that's going to
be another difference.

1435
01:11:05,520 --> 01:11:07,900
So we decided we'd just
throw Frisbees at each other.

1436
01:11:10,406 --> 01:11:12,030
PROFESSOR: So I'm
going to throw to you

1437
01:11:12,030 --> 01:11:13,800
and try to hit an
apple off of your head.

1438
01:11:13,800 --> 01:11:15,000
SRINI DEVADAS:
Yeah, well you might

1439
01:11:15,000 --> 01:11:17,140
want to tell them what we
decided about the apple.

1440
01:11:17,140 --> 01:11:19,860
PROFESSOR: Well, I brought
an easy-to-hit apple,

1441
01:11:19,860 --> 01:11:21,880
a nice big apple,
the cowboy hat.

1442
01:11:21,880 --> 01:11:22,964
SRINI DEVADAS: Cowboy hat.

1443
01:11:22,964 --> 01:11:24,713
PROFESSOR: That should
be a little easier.

1444
01:11:24,713 --> 01:11:27,290
SRINI DEVADAS: So I get to wear
that had first because you're

1445
01:11:27,290 --> 01:11:28,410
going to throw first.

1446
01:11:28,410 --> 01:11:28,993
PROFESSOR: OK.

1447
01:11:28,993 --> 01:11:31,750
SRINI DEVADAS: And this
is really simple, guys.

1448
01:11:31,750 --> 01:11:39,000
Knock the hat off, I guess from
the furthest distance, and win.

1449
01:11:39,000 --> 01:11:40,260
In your case, lose but yeah.

1450
01:11:40,260 --> 01:11:43,510
PROFESSOR: Now for the PETA
people in the audience,

1451
01:11:43,510 --> 01:11:45,170
I want your assure
no humans will

1452
01:11:45,170 --> 01:11:48,125
be harmed during this
performance, only professors.

1453
01:11:48,125 --> 01:11:49,930
[LAUGHTER]

1454
01:11:49,930 --> 01:11:52,150
And maybe egos, pride.

1455
01:11:54,285 --> 01:11:55,160
SRINI DEVADAS: Seven.

1456
01:11:55,160 --> 01:11:57,824
I think seven is a good number.

1457
01:11:57,824 --> 01:11:58,740
PROFESSOR: Seven for--

1458
01:11:58,740 --> 01:11:59,280
SRINI DEVADAS: You get to--

1459
01:11:59,280 --> 01:12:00,780
PROFESSOR: I'm going
to grab purple.

1460
01:12:00,780 --> 01:12:02,679
SRINI DEVADAS: You get to pick.

1461
01:12:02,679 --> 01:12:03,720
You can stand right here.

1462
01:12:03,720 --> 01:12:06,100
That's probably
what's good for you.

1463
01:12:06,100 --> 01:12:07,780
[LAUGHTER]

1464
01:12:08,440 --> 01:12:10,760
Or you can go all
the way up there.

1465
01:12:10,760 --> 01:12:12,810
And after you
knock this hat off,

1466
01:12:12,810 --> 01:12:14,356
I'm going to have to match you.

1467
01:12:14,356 --> 01:12:15,230
PROFESSOR: All right.

1468
01:12:15,230 --> 01:12:16,877
SRINI DEVADAS: So
furthest away wins.

1469
01:12:16,877 --> 01:12:18,710
PROFESSOR: I think I'll
try from about here.

1470
01:12:18,710 --> 01:12:19,585
SRINI DEVADAS: Right.

1471
01:12:19,585 --> 01:12:21,520
Ah!

1472
01:12:21,520 --> 01:12:23,299
I've got to look good here, man.

1473
01:12:23,299 --> 01:12:25,090
PROFESSOR: I have to
do it without looking.

1474
01:12:25,090 --> 01:12:28,290
SRINI DEVADAS: I'm going
to stand right here.

1475
01:12:28,290 --> 01:12:29,920
PROFESSOR: OK.

1476
01:12:29,920 --> 01:12:30,706
SRINI DEVADAS: OK.

1477
01:12:30,706 --> 01:12:31,206
Ah!

1478
01:12:31,206 --> 01:12:32,640
[LAUGHTER]

1479
01:12:33,600 --> 01:12:35,080
I've got to gear up for this.

1480
01:12:35,080 --> 01:12:35,870
PROFESSOR: OK.

1481
01:12:35,870 --> 01:12:37,003
SRINI DEVADAS: Look, I
know how well you throw.

1482
01:12:37,003 --> 01:12:38,086
PROFESSOR: Are you scared?

1483
01:12:41,190 --> 01:12:43,660
That's embarrassing.

1484
01:12:43,660 --> 01:12:45,680
SRINI DEVADAS: Ah!

1485
01:12:45,680 --> 01:12:48,110
I can't deal with this.

1486
01:12:48,110 --> 01:12:51,340
I just have no confidence
in the way you throw.

1487
01:12:51,340 --> 01:12:53,310
So I borrowed this.

1488
01:12:53,310 --> 01:12:55,142
[LAUGHTER]

1489
01:12:59,420 --> 01:13:01,400
Since this is mine,
it's going to cost you

1490
01:13:01,400 --> 01:13:03,060
a throw to wear this.

1491
01:13:03,060 --> 01:13:03,930
PROFESSOR: Hey!

1492
01:13:03,930 --> 01:13:05,138
SRINI DEVADAS: No, all right.

1493
01:13:05,138 --> 01:13:06,420
Whatever, fair.

1494
01:13:06,420 --> 01:13:07,150
All right.

1495
01:13:07,150 --> 01:13:09,880
Now I'm feeling much
better about this.

1496
01:13:09,880 --> 01:13:11,630
I won't claim to
have a pretty face.

1497
01:13:11,630 --> 01:13:15,605
But I like it just the way
it is, just the way it is.

1498
01:13:15,605 --> 01:13:16,480
PROFESSOR: All right.

1499
01:13:16,480 --> 01:13:18,410
Well, since we have a
little more protection,

1500
01:13:18,410 --> 01:13:19,780
maybe I'll start
from farther back.

1501
01:13:19,780 --> 01:13:20,821
SRINI DEVADAS: All right.

1502
01:13:20,821 --> 01:13:22,697
I think I'll hold this
up here like that.

1503
01:13:22,697 --> 01:13:23,280
PROFESSOR: OK.

1504
01:13:23,280 --> 01:13:24,720
So I just have to hit
the hat off, right?

1505
01:13:24,720 --> 01:13:25,015
Easy.

1506
01:13:25,015 --> 01:13:25,806
SRINI DEVADAS: Yup.

1507
01:13:25,806 --> 01:13:27,278
[LAUGHTER]

1508
01:13:35,660 --> 01:13:38,490
PROFESSOR: You can keep that.

1509
01:13:38,490 --> 01:13:40,360
Oh right, so I'm
going to get closer.

1510
01:13:40,360 --> 01:13:41,690
One step closer.

1511
01:13:41,690 --> 01:13:44,990
Luckily, my steps are much
bigger than your steps.

1512
01:13:44,990 --> 01:13:46,438
OK, throw 2.

1513
01:13:46,438 --> 01:13:48,390
[LAUGHTER]

1514
01:13:49,860 --> 01:13:52,209
Throw three.

1515
01:13:52,209 --> 01:13:54,061
[LAUGHTER]

1516
01:13:55,450 --> 01:13:56,890
Throw four.

1517
01:13:56,890 --> 01:13:59,050
SRINI DEVADAS: That
didn't even hurt.

1518
01:13:59,050 --> 01:14:01,340
Come on, man!

1519
01:14:01,340 --> 01:14:04,031
Throw a little harder here!

1520
01:14:04,031 --> 01:14:04,614
PROFESSOR: Oh!

1521
01:14:04,614 --> 01:14:06,558
[CHEERING AND APPLAUSE]

1522
01:14:09,474 --> 01:14:10,950
SRINI DEVADAS: All
right, mark it.

1523
01:14:10,950 --> 01:14:12,680
PROFESSOR: All right,
I marked my spot.

1524
01:14:12,680 --> 01:14:14,000
SRINI DEVADAS: You've got
a couple more throws here

1525
01:14:14,000 --> 01:14:14,600
to do better.

1526
01:14:14,600 --> 01:14:16,360
PROFESSOR: More throws?

1527
01:14:16,360 --> 01:14:16,600
SRINI DEVADAS:
You can do better.

1528
01:14:16,600 --> 01:14:17,550
PROFESSOR: Wait, to go back.

1529
01:14:17,550 --> 01:14:19,466
SRINI DEVADAS: You've
got a couple more, yeah.

1530
01:14:25,172 --> 01:14:26,380
PROFESSOR: You almost hit me.

1531
01:14:26,380 --> 01:14:27,872
[LAUGHTER]

1532
01:14:28,696 --> 01:14:29,980
[GROANING]

1533
01:14:29,980 --> 01:14:32,202
I'm getting better!

1534
01:14:32,202 --> 01:14:33,457
Not much-- can I go back now?

1535
01:14:33,457 --> 01:14:34,540
SRINI DEVADAS: Yeah, sure.

1536
01:14:34,540 --> 01:14:38,160
PROFESSOR: I don't
know the rules.

1537
01:14:38,160 --> 01:14:40,040
[APPLAUSE]

1538
01:14:42,590 --> 01:14:44,932
SRINI DEVADAS: Goodness.

1539
01:14:44,932 --> 01:14:46,140
PROFESSOR: No contest, right?

1540
01:14:46,140 --> 01:14:48,181
SRINI DEVADAS: I'm getting
a little worried here.

1541
01:14:48,181 --> 01:14:49,986
PROFESSOR: One more Frisbee.

1542
01:14:49,986 --> 01:14:53,076
Two more Frisbees.

1543
01:14:53,076 --> 01:14:54,200
Hey, that was your Frisbee.

1544
01:14:54,200 --> 01:14:57,320
SRINI DEVADAS: One more.

1545
01:14:57,320 --> 01:14:57,820
All right.

1546
01:14:57,820 --> 01:15:00,555
PROFESSOR: I think we've
done binary search here.

1547
01:15:00,555 --> 01:15:02,460
[APPLAUSE]

1548
01:15:03,960 --> 01:15:05,960
SRINI DEVADAS: You need this.

1549
01:15:05,960 --> 01:15:10,460
And you don't really need
this, but I'll give it to you.

1550
01:15:10,460 --> 01:15:11,710
PROFESSOR: So much confidence.

1551
01:15:11,710 --> 01:15:13,440
Well, I have so much
confidence in you

1552
01:15:13,440 --> 01:15:16,656
I brought some extra Frisbees.

1553
01:15:16,656 --> 01:15:18,588
[LAUGHTER]

1554
01:15:21,970 --> 01:15:24,800
SRINI DEVADAS: I get to
use all of them, huh?

1555
01:15:24,800 --> 01:15:26,080
PROFESSOR: You need it, man.

1556
01:15:26,080 --> 01:15:26,710
SRINI DEVADAS: All right.

1557
01:15:26,710 --> 01:15:27,835
No, we're going to be fair.

1558
01:15:27,835 --> 01:15:28,519
You threw seven.

1559
01:15:28,519 --> 01:15:29,560
I'm going to throw seven.

1560
01:15:29,560 --> 01:15:37,140
1, 2, 3, 4, 5, 6, 7.

1561
01:15:37,140 --> 01:15:40,389
You've got a bit
of a big head here.

1562
01:15:40,389 --> 01:15:41,180
PROFESSOR: Do this.

1563
01:15:41,180 --> 01:15:43,690
All right, so I put
the hat on the head.

1564
01:15:43,690 --> 01:15:44,800
OK.

1565
01:15:44,800 --> 01:15:47,410
Where were you
standing, by the way?

1566
01:15:47,410 --> 01:15:48,580
Way back here, right?

1567
01:15:48,580 --> 01:15:49,580
SRINI DEVADAS: No, nope.

1568
01:15:49,580 --> 01:15:51,780
I was right there.

1569
01:15:51,780 --> 01:15:52,540
PROFESSOR: OK.

1570
01:15:52,540 --> 01:15:53,664
SRINI DEVADAS: Right there.

1571
01:15:53,664 --> 01:15:55,600
PROFESSOR: All
right, right here.

1572
01:15:55,600 --> 01:15:57,210
Can I hold onto the hat?

1573
01:15:57,210 --> 01:15:58,380
SRINI DEVADAS: No!

1574
01:15:58,380 --> 01:15:59,710
You can hold on to your helmet!

1575
01:15:59,710 --> 01:16:01,470
PROFESSOR: All right.

1576
01:16:01,470 --> 01:16:03,927
SRINI DEVADAS: Wow.

1577
01:16:03,927 --> 01:16:04,510
PROFESSOR: Ah!

1578
01:16:04,510 --> 01:16:06,478
[LAUGHTER]

1579
01:16:08,350 --> 01:16:09,500
How many throws--

1580
01:16:09,500 --> 01:16:12,190
SRINI DEVADAS: Maybe I
should start from right here.

1581
01:16:12,190 --> 01:16:13,284
[GROANING]

1582
01:16:13,284 --> 01:16:13,950
PROFESSOR: Phew.

1583
01:16:13,950 --> 01:16:15,075
That was close.

1584
01:16:18,674 --> 01:16:20,320
SRINI DEVADAS: Oh, I grazed it!

1585
01:16:20,320 --> 01:16:24,040
But it's supposed to fall off.

1586
01:16:24,040 --> 01:16:25,200
PROFESSOR: What, my head?

1587
01:16:25,200 --> 01:16:26,076
[LAUGHTER]

1588
01:16:26,076 --> 01:16:28,076
SRINI DEVADAS: Getting
kind of tight here, guys.

1589
01:16:31,123 --> 01:16:31,623
Wow.

1590
01:16:36,776 --> 01:16:38,752
[YELLING AND GROANING]

1591
01:16:38,752 --> 01:16:40,230
Does that count?

1592
01:16:40,230 --> 01:16:40,730
STUDENT: No!

1593
01:16:40,730 --> 01:16:41,420
SRINI DEVADAS: It does count.

1594
01:16:41,420 --> 01:16:41,987
STUDENT: No!

1595
01:16:41,987 --> 01:16:43,070
SRINI DEVADAS: It's a tie.

1596
01:16:43,070 --> 01:16:44,250
So far, it's a tie.

1597
01:16:44,250 --> 01:16:45,890
So far, it's a tie!

1598
01:16:45,890 --> 01:16:47,690
All right, if I
knock it off, I win.

1599
01:16:47,690 --> 01:16:49,530
[LAUGHTER]

1600
01:16:49,530 --> 01:16:51,370
[GROANING]

1601
01:16:51,370 --> 01:16:52,448
There you you.

1602
01:16:52,448 --> 01:16:53,364
PROFESSOR: Is that it?

1603
01:16:53,364 --> 01:16:55,312
[APPLAUSE]

1604
01:16:58,234 --> 01:17:00,400
SRINI DEVADAS: This
was fair and square.

1605
01:17:00,400 --> 01:17:03,090
We want the world to
know that we did not

1606
01:17:03,090 --> 01:17:04,590
deflate these Frisbees.

1607
01:17:04,590 --> 01:17:06,503
[LAUGHTER]

1608
01:17:07,946 --> 01:17:09,870
[APPLAUSE]

1609
01:17:15,180 --> 01:17:19,210
So not only did we do a bad
job of throwing Frisbee to you

1610
01:17:19,210 --> 01:17:22,420
guys, we didn't throw enough
Frisbees, as you can see,

1611
01:17:22,420 --> 01:17:23,280
through the term.

1612
01:17:23,280 --> 01:17:26,730
So if you want a
Frisbee, pick one up.

1613
01:17:26,730 --> 01:17:29,890
And if you're embarrassed
about throwing Frisbees

1614
01:17:29,890 --> 01:17:32,480
with this lettering on
it, I've got two words

1615
01:17:32,480 --> 01:17:34,500
for you-- paint remover.

1616
01:17:34,500 --> 01:17:36,390
All right, have a good summer.

1617
01:17:36,390 --> 01:17:38,430
And have fun on the final exam.

1618
01:17:38,430 --> 01:17:40,880
[APPLAUSE]