1
00:00:00,090 --> 00:00:02,490
The following content is
provided under a Creative

2
00:00:02,490 --> 00:00:04,030
Commons license.

3
00:00:04,030 --> 00:00:06,360
Your support will help
MIT OpenCourseWare

4
00:00:06,360 --> 00:00:10,720
continue to offer high quality
educational resources for free.

5
00:00:10,720 --> 00:00:13,320
To make a donation or
view additional materials

6
00:00:13,320 --> 00:00:17,280
from hundreds of MIT courses,
visit MIT OpenCourseWare

7
00:00:17,280 --> 00:00:18,450
at ocw.mit.edu.

8
00:00:20,920 --> 00:00:22,670
ERIK DEMAINE: Today,
we continue our theme

9
00:00:22,670 --> 00:00:24,217
on integer data structures.

10
00:00:24,217 --> 00:00:26,300
And we're going to cover
one data structure called

11
00:00:26,300 --> 00:00:31,070
fusion trees, which introduces
a bunch of cool concepts using,

12
00:00:31,070 --> 00:00:35,060
you might call bit tricks,
using the word level parallelism

13
00:00:35,060 --> 00:00:37,329
to do lots of great things.

14
00:00:37,329 --> 00:00:39,620
To do those great things we
need to do something called

15
00:00:39,620 --> 00:00:43,040
sketching, which is essentially
taking w-bit integers

16
00:00:43,040 --> 00:00:44,600
and making them
less than w-bits,

17
00:00:44,600 --> 00:00:47,660
but still having enough
stuff that we care about.

18
00:00:47,660 --> 00:00:50,660
And then we can actually
compute those sketches

19
00:00:50,660 --> 00:00:54,680
using integer multiplication
in a very clever way.

20
00:00:54,680 --> 00:00:58,520
And given a bunch of
these slightly less than w

21
00:00:58,520 --> 00:01:01,610
bit numbers we can
compare several

22
00:01:01,610 --> 00:01:05,700
for the price of one operation
as a parallel comparison.

23
00:01:05,700 --> 00:01:08,930
And then a particularly
nifty thing,

24
00:01:08,930 --> 00:01:11,120
which is new this time I
haven't covered it before,

25
00:01:11,120 --> 00:01:14,330
is how to compute
the leftmost 1 bit

26
00:01:14,330 --> 00:01:19,790
in a w-bit integer in
constant time, all very cool.

27
00:01:19,790 --> 00:01:22,040
And so we're going to
combine all these techniques

28
00:01:22,040 --> 00:01:23,000
and get fusion trees.

29
00:01:23,000 --> 00:01:25,040
What do fusion trees
give us in the end?

30
00:01:30,950 --> 00:01:36,620
Basically, the goal is to get
log base w of n, predecessor

31
00:01:36,620 --> 00:01:39,050
and successor.

32
00:01:39,050 --> 00:01:43,040
So we're again talking about
the predecessor problem.

33
00:01:43,040 --> 00:01:49,430
We did log of w last time with
van Emde Boas and y-fast tries.

34
00:01:49,430 --> 00:01:52,430
So then with the two
of them together, we

35
00:01:52,430 --> 00:01:56,750
get log w and log base w of n.

36
00:01:56,750 --> 00:01:58,400
The min of those
two things is always

37
00:01:58,400 --> 00:02:03,470
at most root log n, but
sometimes much better

38
00:02:03,470 --> 00:02:04,470
than that.

39
00:02:04,470 --> 00:02:07,230
So fusion trees are
great when w is big,

40
00:02:07,230 --> 00:02:10,250
van Emde Boas was good
when w was smallish.

41
00:02:10,250 --> 00:02:14,510
Like poly log n, van
Emde Boas is optimal.

42
00:02:14,510 --> 00:02:20,120
Here, we're thinking
about w being closer to n,

43
00:02:20,120 --> 00:02:22,400
or maybe n to the
epsilon or something.

44
00:02:22,400 --> 00:02:25,950
Then we get constant time,
if it's n to the epsilon.

45
00:02:25,950 --> 00:02:29,765
Anyway, the version we're
going to cover is static.

46
00:02:33,140 --> 00:02:40,850
And it's linear space, and
it runs on the good old word

47
00:02:40,850 --> 00:02:45,200
RAM, which you may
recall is regular C

48
00:02:45,200 --> 00:02:48,504
operations on w-bit words.
w is at least log n,

49
00:02:48,504 --> 00:02:49,670
so you can do random access.

50
00:02:52,310 --> 00:02:55,070
And anything else?

51
00:02:55,070 --> 00:02:56,950
I think that's the
version we will cover.

52
00:02:56,950 --> 00:03:00,230
And that is the original
version of Fredman and Willard,

53
00:03:00,230 --> 00:03:03,410
and it was invented
in 1990, which

54
00:03:03,410 --> 00:03:06,060
was one year after the
cold fusion debacle.

55
00:03:06,060 --> 00:03:08,831
So this may be where
this name came from.

56
00:03:08,831 --> 00:03:11,330
There's other reasons it might
be called fusion trees, which

57
00:03:11,330 --> 00:03:11,910
we'll get to.

58
00:03:14,450 --> 00:03:15,024
Cool.

59
00:03:15,024 --> 00:03:16,940
So now, there's other
versions of fusion trees

60
00:03:16,940 --> 00:03:20,450
which I will not cover, but
just so you know about them.

61
00:03:20,450 --> 00:03:23,510
And in some sense we
will pseudo cover them.

62
00:03:26,210 --> 00:03:30,526
There's a version
for an AC0 RAM.

63
00:03:30,526 --> 00:03:33,150
This is a model we
haven't talked about.

64
00:03:33,150 --> 00:03:36,800
It's another version of
a trans-dichotomous RAM,

65
00:03:36,800 --> 00:03:39,110
somewhere off to the
side next to word RAM.

66
00:03:39,110 --> 00:03:41,390
AC0 is a circuit model.

67
00:03:41,390 --> 00:03:50,090
And it's basically any constant
depth circuit of unbounded fan

68
00:03:50,090 --> 00:03:50,840
in and fan out.

69
00:03:57,950 --> 00:04:00,920
And so in particular
what AC0 forbids

70
00:04:00,920 --> 00:04:03,805
are operations like
multiplication.

71
00:04:08,360 --> 00:04:11,769
I think this used to be a
bigger deal than it is today.

72
00:04:11,769 --> 00:04:14,060
I think multiplication used
to be in order of magnitude

73
00:04:14,060 --> 00:04:16,610
slower than integer addition.

74
00:04:16,610 --> 00:04:18,395
I checked on current
Intel architecture.

75
00:04:18,395 --> 00:04:20,480
It's about 3 times
slower than an addition,

76
00:04:20,480 --> 00:04:23,660
because of pipelining a
lot of things get cheaper.

77
00:04:23,660 --> 00:04:27,080
But in some theoretical
sense, multiplication

78
00:04:27,080 --> 00:04:28,880
is worse than a lot
of other operations,

79
00:04:28,880 --> 00:04:32,030
because you need to have
a log and depth circuit.

80
00:04:32,030 --> 00:04:34,476
So you can't quite get
as much parallelism

81
00:04:34,476 --> 00:04:35,850
as you can with
integer addition.

82
00:04:35,850 --> 00:04:36,830
If you don't know
about circuit models,

83
00:04:36,830 --> 00:04:38,080
don't worry about it too much.

84
00:04:38,080 --> 00:04:42,260
But AC0 means no multiplication,
sort of simpler operations.

85
00:04:42,260 --> 00:04:43,930
The weird thing
about the AC0 RAM

86
00:04:43,930 --> 00:04:49,010
is it lets you do weird things,
which are AC0, but are not

87
00:04:49,010 --> 00:04:52,581
in C. So you could look
at the intersection of AC0

88
00:04:52,581 --> 00:04:54,830
RAM and word RAM, and that
would basically be word RAM

89
00:04:54,830 --> 00:04:56,780
without multiplication.

90
00:04:56,780 --> 00:04:59,090
But AC0 RAM allows
other operations

91
00:04:59,090 --> 00:05:01,876
as long as there's some
circuit that can compute them.

92
00:05:01,876 --> 00:05:03,500
Sometimes they're
reasonable operations

93
00:05:03,500 --> 00:05:06,860
like most significant set
bit is an AC0 operation.

94
00:05:06,860 --> 00:05:10,640
So, you don't have to do any
work to get this operation.

95
00:05:10,640 --> 00:05:12,770
So, in some sense this
makes your life easier.

96
00:05:12,770 --> 00:05:14,060
In other ways it makes
your life harder,

97
00:05:14,060 --> 00:05:15,620
because there is
no multiplication

98
00:05:15,620 --> 00:05:17,450
and we're going to
use multiplication

99
00:05:17,450 --> 00:05:20,660
to compute sketches
and other things.

100
00:05:20,660 --> 00:05:23,390
So it's both better and worse.

101
00:05:23,390 --> 00:05:27,440
This is by Anderson
and others a few years

102
00:05:27,440 --> 00:05:29,780
after the original fusion trees.

103
00:05:29,780 --> 00:05:31,880
More interesting
is that there are--

104
00:05:31,880 --> 00:05:35,690
so this is a version
of fusion trees.

105
00:05:35,690 --> 00:05:38,990
More interesting are the dynamic
versions of fusion trees.

106
00:05:41,990 --> 00:05:47,915
So there's one that
the first version--

107
00:05:47,915 --> 00:05:49,580
it's actually published later.

108
00:06:00,330 --> 00:06:03,430
There's a general trick
for dynamizing static data

109
00:06:03,430 --> 00:06:03,930
structures.

110
00:06:03,930 --> 00:06:04,830
We actually saw one.

111
00:06:04,830 --> 00:06:07,170
You may recall weight balanced
search trees was a way

112
00:06:07,170 --> 00:06:09,870
to dynamize a static
data structure.

113
00:06:09,870 --> 00:06:13,260
It turns out it doesn't work
so great on fusion trees.

114
00:06:13,260 --> 00:06:17,820
Because the time to build a
fusion tree is polynomial.

115
00:06:17,820 --> 00:06:19,364
It's not linear.

116
00:06:19,364 --> 00:06:20,655
And so that's kind of annoying.

117
00:06:20,655 --> 00:06:22,530
You need polynomial time
to build this thing.

118
00:06:22,530 --> 00:06:26,830
So weight balance is not enough
to slow down the rebuilds.

119
00:06:26,830 --> 00:06:28,830
There's another structure,
which we won't cover,

120
00:06:28,830 --> 00:06:30,770
called exponential search trees.

121
00:06:30,770 --> 00:06:33,130
It has this log log n overhead.

122
00:06:33,130 --> 00:06:37,465
But other than that, it gives
you a nice time dynamization.

123
00:06:37,465 --> 00:06:38,540
So these are updates.

124
00:06:42,180 --> 00:06:46,370
There's another version
which uses hashing,

125
00:06:46,370 --> 00:06:54,210
and achieves log base w
expected time for updates.

126
00:06:54,210 --> 00:06:57,600
This is by Raman.

127
00:06:57,600 --> 00:07:04,350
And so this gives you matching
the fusion tree query bound,

128
00:07:04,350 --> 00:07:06,960
you can do inserts and deletes,
the same amount of time

129
00:07:06,960 --> 00:07:08,760
if you allow expected.

130
00:07:08,760 --> 00:07:13,110
And essentially the idea is to
do sketches more like hashing.

131
00:07:13,110 --> 00:07:15,210
I mean, think of sketching
as just hashing down

132
00:07:15,210 --> 00:07:16,720
to a smaller universe.

133
00:07:16,720 --> 00:07:18,990
And in expectation
that will work well,

134
00:07:18,990 --> 00:07:21,030
although it's open,
actually, whether you

135
00:07:21,030 --> 00:07:24,870
can achieve this bound
with high probability.

136
00:07:29,840 --> 00:07:31,800
So it's an interesting
open question.

137
00:07:35,310 --> 00:07:37,269
So that's the various
versions of fusion trees.

138
00:07:37,269 --> 00:07:39,185
But we're going to just
cover the static ones,

139
00:07:39,185 --> 00:07:40,830
because they're
interesting enough.

140
00:07:40,830 --> 00:07:45,880
Dynamic ones are not that
much harder than the regular.

141
00:07:45,880 --> 00:07:48,780
So how do we achieve
log base w of n?

142
00:07:48,780 --> 00:07:52,080
Well we've already seen B-trees
which achieve log base b of n.

143
00:07:52,080 --> 00:07:54,060
So presumably it's
the same idea,

144
00:07:54,060 --> 00:07:59,340
and indeed that's what we'll do.

145
00:07:59,340 --> 00:08:07,520
What we want is a B-tree with
branching factor ideally w.

146
00:08:07,520 --> 00:08:10,130
We can't quite achieve w though.

147
00:08:10,130 --> 00:08:13,780
So it's going to be w to
some small constant power.

148
00:08:13,780 --> 00:08:16,710
And 1/5 is the one that
I'll use in this lecture.

149
00:08:16,710 --> 00:08:21,150
You can improve it or
make it worse, up to you.

150
00:08:21,150 --> 00:08:23,010
But any constant
up here will do.

151
00:08:23,010 --> 00:08:25,200
Because then log
base w to the 1/5

152
00:08:25,200 --> 00:08:30,590
is going to be five
times log base w of n.

153
00:08:30,590 --> 00:08:34,004
So we have a node.

154
00:08:34,004 --> 00:08:36,539
So it has branching
factor w to the 1/5.

155
00:08:40,526 --> 00:08:42,150
Then the height of
the tree, of course,

156
00:08:42,150 --> 00:08:45,750
is theta log base w of n.

157
00:08:45,750 --> 00:08:47,010
So that's all good.

158
00:08:47,010 --> 00:08:49,230
But now what we need to
do is, doing a predecessor

159
00:08:49,230 --> 00:08:53,520
search given a node, we need to
decide in constant time which

160
00:08:53,520 --> 00:08:55,680
branch has our answer.

161
00:08:55,680 --> 00:08:59,580
So normally in a B-tree you
would read in all these keys,

162
00:08:59,580 --> 00:09:02,310
and then compare your
item to all of them,

163
00:09:02,310 --> 00:09:04,350
and then decide which way to go.

164
00:09:04,350 --> 00:09:06,630
Because in a B-tree we can
read all of these items

165
00:09:06,630 --> 00:09:08,400
in one operation.

166
00:09:08,400 --> 00:09:10,236
Now here, is that possible?

167
00:09:10,236 --> 00:09:10,860
Think about it.

168
00:09:10,860 --> 00:09:14,640
You've got each of these
keys is w bits long.

169
00:09:14,640 --> 00:09:18,300
There's w to the 1/5 of them.

170
00:09:18,300 --> 00:09:22,890
So the total number
of bits in the node

171
00:09:22,890 --> 00:09:29,460
to store all those keys is w to
the 1 plus 1/5, which is a lot.

172
00:09:29,460 --> 00:09:32,280
There's no way you can read all
those bits in constant time.

173
00:09:32,280 --> 00:09:34,852
You can only read order
w bits in constant time.

174
00:09:34,852 --> 00:09:36,060
So we can't look at them all.

175
00:09:36,060 --> 00:09:39,060
Somehow, we still have to figure
out which way to go correctly

176
00:09:39,060 --> 00:09:40,450
in constant time.

177
00:09:40,450 --> 00:09:44,810
So this is the idea
of a fusion node.

178
00:09:54,310 --> 00:10:03,200
We need to be able to store k,
which is order w to the 1/5.

179
00:10:03,200 --> 00:10:07,200
Keys, I'm going to
give them some names,

180
00:10:07,200 --> 00:10:11,580
x0 up to xk minus 1.

181
00:10:11,580 --> 00:10:13,220
Assume that they've
been presorted.

182
00:10:13,220 --> 00:10:15,500
We can preprocess and
do all those things.

183
00:10:15,500 --> 00:10:20,490
I'm going to go constant time,
predecessor and successor,

184
00:10:20,490 --> 00:10:23,940
and it's going to
be linear space,

185
00:10:23,940 --> 00:10:27,800
and it's going to require
polynomial preprocessing.

186
00:10:30,510 --> 00:10:32,031
And this is the annoying part.

187
00:10:32,031 --> 00:10:33,405
If this was just
k preprocessing,

188
00:10:33,405 --> 00:10:35,430
it would be easy to make
fusion trees dynamic.

189
00:10:35,430 --> 00:10:38,610
But it's k to squared,
or k cubed or something,

190
00:10:38,610 --> 00:10:40,689
depending on how fancy you are.

191
00:10:40,689 --> 00:10:42,480
It's not known how to
do it in linear time.

192
00:10:45,880 --> 00:10:47,000
So that's really our goal.

193
00:10:47,000 --> 00:10:49,220
If we can implement
fusion tree nodes

194
00:10:49,220 --> 00:10:53,540
and do constant time predecessor
on this small value of n

195
00:10:53,540 --> 00:10:56,361
basically, when n is
only w to the 1/5.

196
00:10:56,361 --> 00:10:57,860
If we can do constant
time for that,

197
00:10:57,860 --> 00:11:02,360
then by plugging in B-trees
we get the log base w of n

198
00:11:02,360 --> 00:11:05,540
for arbitrary values of n.

199
00:11:05,540 --> 00:11:08,340
So it's really all
about a single mode

200
00:11:08,340 --> 00:11:09,620
and doing that fast.

201
00:11:14,060 --> 00:11:18,340
So, the rest of the
lecture will be about that.

202
00:11:18,340 --> 00:11:21,310
So, I want to introduce
this idea of sketching.

203
00:11:21,310 --> 00:11:24,340
And to do that
I'm going to think

204
00:11:24,340 --> 00:11:29,740
about what it takes
to distinguish

205
00:11:29,740 --> 00:11:31,360
this small number of keys.

206
00:11:37,570 --> 00:11:41,770
High level ideas, well
we've got w to the 1/5 keys.

207
00:11:41,770 --> 00:11:43,750
Each of them is w bits.

208
00:11:43,750 --> 00:11:47,120
Do you really need all
w bits for each of them?

209
00:11:47,120 --> 00:11:48,430
It seems a little excessive.

210
00:11:48,430 --> 00:11:51,880
If there's only w to the 1/5,
you should only need about w

211
00:11:51,880 --> 00:11:55,180
to the 1/5 bits of each
to distinguish them all.

212
00:11:55,180 --> 00:11:56,740
So that's the intuition.

213
00:11:56,740 --> 00:12:00,160
And indeed, you can
formalize that intuition

214
00:12:00,160 --> 00:12:04,690
by viewing each of the keys
as a path in a binary tree.

215
00:12:09,770 --> 00:12:17,160
So this represents the
bit string 01011101.

216
00:12:17,160 --> 00:12:18,640
0 means left.

217
00:12:18,640 --> 00:12:20,710
1 means right.

218
00:12:20,710 --> 00:12:22,510
This is a transformation
we'll use a lot.

219
00:12:22,510 --> 00:12:24,220
So maybe that's in your set.

220
00:12:24,220 --> 00:12:30,260
Maybe this other bit
string is in your set.

221
00:12:30,260 --> 00:12:34,438
Maybe this bit string--

222
00:12:34,438 --> 00:12:36,437
oh, I've got to make these
the same height which

223
00:12:36,437 --> 00:12:37,853
is a little challenging.

224
00:12:43,780 --> 00:12:45,250
So maybe those are the three.

225
00:12:45,250 --> 00:12:48,400
Suppose you just have these
three bit strings, w bit

226
00:12:48,400 --> 00:12:49,300
strings in your set.

227
00:12:49,300 --> 00:12:52,900
So this is a tree of height w.

228
00:12:52,900 --> 00:12:58,250
Because each of the keys
has w bits, so maybe k is 3.

229
00:12:58,250 --> 00:12:59,781
And those are your three keys.

230
00:12:59,781 --> 00:13:00,280
OK.

231
00:13:00,280 --> 00:13:03,730
The idea is, look at
the branching nodes.

232
00:13:03,730 --> 00:13:06,960
Where's a color?

233
00:13:06,960 --> 00:13:11,860
So, you've got a branching node
here and a branching node here.

234
00:13:11,860 --> 00:13:14,170
Because there's
three leaves, there's

235
00:13:14,170 --> 00:13:16,730
only going to be
two branching nodes.

236
00:13:16,730 --> 00:13:21,280
So the idea is, well I really
only care about these two bits.

237
00:13:21,280 --> 00:13:23,620
Or it would be enough to
think about these two bits.

238
00:13:26,061 --> 00:13:26,560
OK.

239
00:13:26,560 --> 00:13:29,200
Well, we'll look at this
more formally in a moment.

240
00:13:29,200 --> 00:13:32,800
But by storing this bit,
I know whether the key

241
00:13:32,800 --> 00:13:34,957
is over here on the left
or over here on the right.

242
00:13:34,957 --> 00:13:36,790
And then by storing
this bit, I don't really

243
00:13:36,790 --> 00:13:38,440
care about it for this word.

244
00:13:38,440 --> 00:13:40,930
But it will distinguish
these two words.

245
00:13:40,930 --> 00:13:44,380
So if you just look at the bits
that contain branching nodes

246
00:13:44,380 --> 00:13:51,370
in this tri-view, then it's
enough to distinguish all

247
00:13:51,370 --> 00:13:53,080
of the xi's.

248
00:13:53,080 --> 00:13:57,670
So this is x0, x1, x2.

249
00:14:00,550 --> 00:14:01,220
OK.

250
00:14:01,220 --> 00:14:04,430
Let me formalize
that a little bit.

251
00:14:04,430 --> 00:14:16,280
So we have k minus 1 branching
nodes in this height w

252
00:14:16,280 --> 00:14:27,135
tree of the k keys.

253
00:14:27,135 --> 00:14:28,510
Because there's
k leaves, there's

254
00:14:28,510 --> 00:14:32,600
going to be k minus 1 branching
nodes, because the k leaves are

255
00:14:32,600 --> 00:14:34,310
distinct.

256
00:14:34,310 --> 00:14:41,882
So this means there are at most,
k minus 1 levels containing

257
00:14:41,882 --> 00:14:42,590
a branching node.

258
00:14:42,590 --> 00:14:43,340
It might be fewer.

259
00:14:53,410 --> 00:14:56,080
Maybe it's nice to add
in another key over here

260
00:14:56,080 --> 00:14:57,030
on the left.

261
00:14:57,030 --> 00:14:59,800
I mean if I was lucky, there'd
be another key over here,

262
00:14:59,800 --> 00:15:01,720
and then I'd be using
this bit and getting two

263
00:15:01,720 --> 00:15:03,100
for the price of one.

264
00:15:03,100 --> 00:15:06,731
If I'm less lucky, it
will be more like this.

265
00:15:06,731 --> 00:15:11,230
So here's another x value.

266
00:15:11,230 --> 00:15:13,720
And in this case, I care
about this branching node.

267
00:15:13,720 --> 00:15:17,380
So I care about
another bit here.

268
00:15:17,380 --> 00:15:17,880
OK.

269
00:15:17,880 --> 00:15:22,390
But if I have four keys, it
will be at most three bits

270
00:15:22,390 --> 00:15:24,680
corresponding to these levels.

271
00:15:24,680 --> 00:15:27,640
So call these-- these
levels correspond to bits.

272
00:15:27,640 --> 00:15:30,880
This is the first bit, second
bit, third bit, and so on.

273
00:15:30,880 --> 00:15:33,180
This is the most
significant bit, next,

274
00:15:33,180 --> 00:15:37,310
and then the least
significant is at the bottom.

275
00:15:37,310 --> 00:15:46,720
So these levels correspond
to important bits.

276
00:15:46,720 --> 00:15:47,650
That's the definition.

277
00:15:50,470 --> 00:15:55,500
And we're going to give
these bits a name, b0, b1,

278
00:15:55,500 --> 00:15:57,700
up to br minus 1.

279
00:15:57,700 --> 00:16:02,320
Those are bit indices saying
which bits we care about.

280
00:16:02,320 --> 00:16:09,040
And we know that r is less than
k, and k is order w to the 1/5.

281
00:16:09,040 --> 00:16:13,560
So there are only w to
the 1/5 important bits

282
00:16:13,560 --> 00:16:17,050
overall among these k keys.

283
00:16:17,050 --> 00:16:20,170
So the idea is don't store
all w bits for all the keys.

284
00:16:20,170 --> 00:16:21,590
I mean you have to store them.

285
00:16:21,590 --> 00:16:22,780
But don't look at them.

286
00:16:22,780 --> 00:16:26,680
Just look at these
important bits for the keys.

287
00:16:26,680 --> 00:16:28,180
And then life is good.

288
00:16:28,180 --> 00:16:31,720
Because there's only w
to the 1/5 bits per key.

289
00:16:31,720 --> 00:16:34,250
There's only w to the 1/5 keys.

290
00:16:34,250 --> 00:16:45,280
And so the total number of
important bits among all k keys

291
00:16:45,280 --> 00:16:46,310
is small.

292
00:16:46,310 --> 00:16:49,810
It's only w to the 2/5,
which is less than w.

293
00:16:49,810 --> 00:16:51,520
So it fits in a single
word, and we can

294
00:16:51,520 --> 00:16:52,840
look at this in constant time.

295
00:16:55,490 --> 00:16:57,100
So that seems like a good thing.

296
00:17:00,170 --> 00:17:05,050
Let me tell you what
properties this has.

297
00:17:05,050 --> 00:17:14,910
Let me also define the notion
of a perfect sketch of a word.

298
00:17:14,910 --> 00:17:22,750
x is going to be what
you get when you extract

299
00:17:22,750 --> 00:17:30,130
bits b0 to br minus 1 from x.

300
00:17:30,130 --> 00:17:40,760
So in other words, this is a bit
string, an r-bit string who's

301
00:17:40,760 --> 00:17:49,810
i-th bit equals bit bi of x.

302
00:17:49,810 --> 00:17:52,570
So you've got a bit
string which is x.

303
00:17:52,570 --> 00:17:55,910
You say, oh, the important
ones are this one, this one,

304
00:17:55,910 --> 00:17:57,390
this one, and this one.

305
00:17:57,390 --> 00:18:00,730
Inside here is
either a 0 or a 1.

306
00:18:00,730 --> 00:18:02,811
And there's other bits
which we don't care about,

307
00:18:02,811 --> 00:18:04,060
because they're not important.

308
00:18:06,630 --> 00:18:10,750
And we just compress
this to a 4-bit string.

309
00:18:14,180 --> 00:18:16,280
0110.

310
00:18:16,280 --> 00:18:16,780
OK.

311
00:18:16,780 --> 00:18:18,275
This is sketch of x.

312
00:18:21,790 --> 00:18:24,250
And to be a little bit
more explicit about how I'm

313
00:18:24,250 --> 00:18:26,920
labeling things, this is b0.

314
00:18:26,920 --> 00:18:30,730
This is b1, b2, and b3.

315
00:18:34,240 --> 00:18:36,100
Because you number bits--

316
00:18:36,100 --> 00:18:38,854
I think this is right,
we'll see later--

317
00:18:38,854 --> 00:18:41,020
we're going to number bits
from the right-hand side.

318
00:18:41,020 --> 00:18:45,090
This is 0-th bit, first
bit, second, third, fourth;

319
00:18:45,090 --> 00:18:48,450
which is the opposite of this
picture, unfortunately, sorry.

320
00:18:48,450 --> 00:18:51,670
This is a bit 0, bit 1, anyway.

321
00:18:51,670 --> 00:18:52,805
This will be convenient.

322
00:19:00,490 --> 00:19:03,414
So, that's perfect sketch.

323
00:19:03,414 --> 00:19:05,080
For now, I'm going
to assume that we can

324
00:19:05,080 --> 00:19:07,150
compute this in constant time.

325
00:19:07,150 --> 00:19:12,715
This one answer is
it's an AC0 operation.

326
00:19:12,715 --> 00:19:15,200
That's not so obvious,
but it's true.

327
00:19:15,200 --> 00:19:17,861
So on an AC0 RAM, you can just
say, oh, this is an operation.

328
00:19:17,861 --> 00:19:18,360
Right?

329
00:19:18,360 --> 00:19:20,030
It's given one word.

330
00:19:20,030 --> 00:19:21,440
And, well OK.

331
00:19:21,440 --> 00:19:23,810
It's given these
description of bit numbers,

332
00:19:23,810 --> 00:19:25,700
but those will also
fit in one word.

333
00:19:25,700 --> 00:19:29,256
And then does this
bit extraction.

334
00:19:29,256 --> 00:19:31,130
We're going to see a
reasonable way to do it.

335
00:19:31,130 --> 00:19:33,504
But for now, take that as an
unreasonable way to do this.

336
00:19:36,010 --> 00:19:47,365
So perfect sketch is
good, because it implies

337
00:19:47,365 --> 00:19:49,470
the following nice property.

338
00:19:49,470 --> 00:19:52,380
If you look at the
sketch of x0 that's

339
00:19:52,380 --> 00:19:58,320
going to be less than the
sketch of x1, and so on,

340
00:19:58,320 --> 00:20:07,438
which is going to be less
than the sketch of xk minus 1.

341
00:20:07,438 --> 00:20:09,480
Sketch preserves order.

342
00:20:09,480 --> 00:20:12,630
We assume that--
where do we have it?

343
00:20:12,630 --> 00:20:18,150
Over here, x0 is less than
x1, is less than xk minus 1.

344
00:20:18,150 --> 00:20:22,260
And because we're
keeping all the bits

345
00:20:22,260 --> 00:20:24,660
where are these xi's
get distinguished,

346
00:20:24,660 --> 00:20:26,970
this one it doesn't matter
whether we kept here,

347
00:20:26,970 --> 00:20:28,290
it doesn't matter whether
we kept these guys.

348
00:20:28,290 --> 00:20:30,000
But in particular,
we keep the bits that

349
00:20:30,000 --> 00:20:31,560
have all the branching nodes.

350
00:20:31,560 --> 00:20:34,170
That will preserve
the order of the xi's.

351
00:20:34,170 --> 00:20:36,270
So we know that the
order of the xi's is

352
00:20:36,270 --> 00:20:37,890
preserved under sketching.

353
00:20:37,890 --> 00:20:40,700
The trouble is the following.

354
00:20:40,700 --> 00:20:47,240
Suppose you want to do a
search, a predecessor search.

355
00:20:47,240 --> 00:20:53,550
So, you're given some query q,
and you want to know where does

356
00:20:53,550 --> 00:20:55,380
q fit among the xi's.

357
00:20:55,380 --> 00:20:59,880
Because that will tell you
which child to visit from here.

358
00:20:59,880 --> 00:21:00,510
So, OK.

359
00:21:00,510 --> 00:21:06,540
You compute a sketch
of q, seems reasonable,

360
00:21:06,540 --> 00:21:08,910
and move into sketch world.

361
00:21:08,910 --> 00:21:12,030
And now you try to
find where sketch of q

362
00:21:12,030 --> 00:21:13,729
fits among these guys.

363
00:21:13,729 --> 00:21:14,520
So you can do that.

364
00:21:14,520 --> 00:21:16,490
And I claim you can do
that in constant time.

365
00:21:16,490 --> 00:21:18,690
It's again, an AC0 operation.

366
00:21:18,690 --> 00:21:22,170
But the nice thing is the
sketches all fit in one word.

367
00:21:22,170 --> 00:21:25,890
Also this single sketch fits
in one word, no big surprise

368
00:21:25,890 --> 00:21:27,850
there.

369
00:21:27,850 --> 00:21:29,790
So let's say you can
find where sketch of q

370
00:21:29,790 --> 00:21:32,100
fits among these items
in constant time.

371
00:21:32,100 --> 00:21:35,970
The trouble is were the sketch
of q fits is not necessarily

372
00:21:35,970 --> 00:21:40,860
the same as where q
fits among the xi's.

373
00:21:40,860 --> 00:21:44,520
Because q was not involved
in the definition of sketch.

374
00:21:44,520 --> 00:21:45,730
q is an arbitrary query.

375
00:21:45,730 --> 00:21:49,950
They come online, I mean
any word could be a query,

376
00:21:49,950 --> 00:21:51,180
not just the xi's.

377
00:21:51,180 --> 00:21:54,420
So you've set everything up
to distinguish all the xi's.

378
00:21:54,420 --> 00:21:58,920
But q is going to fall off
this tree at some point.

379
00:21:58,920 --> 00:22:01,200
And that kind of messes you up.

380
00:22:01,200 --> 00:22:04,140
Because if q fell off here,
you don't have that bit.

381
00:22:04,140 --> 00:22:07,740
You won't notice
the q fit there.

382
00:22:07,740 --> 00:22:10,050
So we have to do some work.

383
00:22:10,050 --> 00:22:14,060
And this is what I
call de-sketchifying.

384
00:22:19,020 --> 00:22:20,790
And I like a big board.

385
00:22:45,541 --> 00:22:51,245
OK, let mean draw a some more
methodical and smaller example.

386
00:22:55,376 --> 00:22:57,660
I need to make it the
right number of levels.

387
00:23:23,789 --> 00:23:34,110
A little bigger than my usual
tree, and I'll get my red,

388
00:23:34,110 --> 00:23:35,895
actually maybe use two colors.

389
00:23:59,370 --> 00:24:02,750
So here's a real example.

390
00:24:02,750 --> 00:24:05,210
Now it has four keys.

391
00:24:05,210 --> 00:24:08,830
And here I'm in the lucky case,
where this is an important bit.

392
00:24:11,600 --> 00:24:14,090
And I get two for
the price of one.

393
00:24:14,090 --> 00:24:15,860
I cared about this
branching node,

394
00:24:15,860 --> 00:24:18,770
I cared about this
branching node,

395
00:24:18,770 --> 00:24:21,710
and so I only have to
door two bits in my sketch

396
00:24:21,710 --> 00:24:24,050
for these four nodes.

397
00:24:24,050 --> 00:24:25,580
In general, it
might be three bits.

398
00:24:25,580 --> 00:24:27,070
But this will just
make the point.

399
00:24:27,070 --> 00:24:29,660
So it's actually,
life is in some ways

400
00:24:29,660 --> 00:24:31,590
harder in this situation.

401
00:24:31,590 --> 00:24:32,090
OK.

402
00:24:32,090 --> 00:24:33,423
So what are my bit strings here?

403
00:24:33,423 --> 00:24:38,461
Over here I've got 0000, which
corresponds to always going

404
00:24:38,461 --> 00:24:38,960
left.

405
00:24:38,960 --> 00:24:44,360
And I've got 0010.

406
00:24:44,360 --> 00:24:52,530
Over here I've
got 1100 and 1111.

407
00:24:52,530 --> 00:24:54,725
We drew these pictures
for van Emde Boas, right?

408
00:24:54,725 --> 00:24:57,680
The idea is we're
going to use some

409
00:24:57,680 --> 00:25:00,550
of the similar
perspectives at least.

410
00:25:00,550 --> 00:25:03,770
OK, but the important bits were
the very first, the leftmost

411
00:25:03,770 --> 00:25:05,150
bit I should say.

412
00:25:05,150 --> 00:25:12,810
And then two bits after
that, so these guys.

413
00:25:18,280 --> 00:25:25,197
And so the sketch here
is 11, 10, 01, and 00.

414
00:25:25,197 --> 00:25:27,280
And you can see this is
the minimal number of bits

415
00:25:27,280 --> 00:25:29,300
I need to keep them in order.

416
00:25:29,300 --> 00:25:29,860
But it does.

417
00:25:29,860 --> 00:25:30,717
It works.

418
00:25:30,717 --> 00:25:31,300
You can check.

419
00:25:31,300 --> 00:25:34,430
This works in general.

420
00:25:34,430 --> 00:25:34,930
OK.

421
00:25:34,930 --> 00:25:36,250
Now comes the query.

422
00:25:36,250 --> 00:25:40,420
I have a problematic
query I'd like to draw.

423
00:25:40,420 --> 00:25:45,760
And it is 0101, so 0-1-0-1.

424
00:25:45,760 --> 00:25:48,444
So here's my query queue.

425
00:25:48,444 --> 00:25:50,760
Let me draw these as white.

426
00:25:55,350 --> 00:25:57,120
Query is 0101.

427
00:25:57,120 --> 00:26:04,350
1 If we take the
sketch, we get 00.

428
00:26:04,350 --> 00:26:07,060
Those are the important bits.

429
00:26:07,060 --> 00:26:11,040
So if I search for
the query of 00,

430
00:26:11,040 --> 00:26:13,530
I will find that it
happens to match this key,

431
00:26:13,530 --> 00:26:16,220
or it matches the
sketch of this key.

432
00:26:16,220 --> 00:26:19,740
But that key is neither the
predecessor, nor the successor

433
00:26:19,740 --> 00:26:20,880
of that query.

434
00:26:20,880 --> 00:26:22,140
So this is bad news.

435
00:26:22,140 --> 00:26:25,500
I find the predecessor in sketch
world, which is the red stuff,

436
00:26:25,500 --> 00:26:26,660
I get the wrong answer.

437
00:26:26,660 --> 00:26:29,910
In general, they could be
very far away from each other.

438
00:26:29,910 --> 00:26:32,815
Here, I've got it 1
away, but that's as big

439
00:26:32,815 --> 00:26:34,040
an example as I can draw.

440
00:26:37,162 --> 00:26:39,040
So, how do we deal with this?

441
00:26:52,380 --> 00:26:55,590
This is the de-sketchification.

442
00:26:55,590 --> 00:27:01,180
So when I do this query, I
end up finding this guy, x0.

443
00:27:01,180 --> 00:27:07,080
I claim that I can still use
that for something interesting.

444
00:27:07,080 --> 00:27:15,510
OK, let's say we
have a sketch of xi

445
00:27:15,510 --> 00:27:21,240
as the predecessor
of the sketch of q.

446
00:27:21,240 --> 00:27:23,640
And so sketch of q is
sandwiched between a sketch

447
00:27:23,640 --> 00:27:26,952
of xi and sketch of xi plus 1.

448
00:27:26,952 --> 00:27:28,410
First of all, we're
assuming that I

449
00:27:28,410 --> 00:27:30,000
can compute this
in constant time,

450
00:27:30,000 --> 00:27:32,160
I can find where sketch of
q fits among these guys.

451
00:27:32,160 --> 00:27:34,245
Because it just fits into words.

452
00:27:34,245 --> 00:27:36,120
And for now, let's just
assume all operations

453
00:27:36,120 --> 00:27:37,710
on a constant number of
words are at constant time.

454
00:27:37,710 --> 00:27:38,880
We will see how to do this.

455
00:27:38,880 --> 00:27:41,150
This is parallel comparison.

456
00:27:41,150 --> 00:27:44,700
So you figure out
sketch of q fits here.

457
00:27:44,700 --> 00:27:48,300
I want to learn something about
where q fits among the xi's.

458
00:27:48,300 --> 00:27:51,130
It's obviously, these
may be the wrong answer.

459
00:27:51,130 --> 00:27:54,720
But I claim I can do
something useful by looking

460
00:27:54,720 --> 00:28:00,030
at the longest common
prefix of those words.

461
00:28:08,130 --> 00:28:12,990
So I want to compare q, not
sketch of q but the real value

462
00:28:12,990 --> 00:28:23,340
q, and either xi or xi plus 1.

463
00:28:23,340 --> 00:28:25,570
And what I want is the longest.

464
00:28:25,570 --> 00:28:28,890
So I look at the longest
common prefix of q and xi.

465
00:28:28,890 --> 00:28:32,040
I look at longest common
prefix of q and xi plus 1.

466
00:28:32,040 --> 00:28:34,950
Whichever of those
is the longest that's

467
00:28:34,950 --> 00:28:36,150
my longest common prefix.

468
00:28:36,150 --> 00:28:39,150
In the tree, it's the
longest common ancestor,

469
00:28:39,150 --> 00:28:41,090
or lowest common ancestor.

470
00:28:43,660 --> 00:28:44,160
OK.

471
00:28:44,160 --> 00:28:46,680
So let's do it.

472
00:28:46,680 --> 00:28:49,440
We found that sketch of
q fit between, I guess,

473
00:28:49,440 --> 00:28:51,600
these two guys, the
way I've written it

474
00:28:51,600 --> 00:28:52,560
with the inequalities.

475
00:28:52,560 --> 00:28:56,190
It's between x0 here and x1.

476
00:28:56,190 --> 00:29:00,750
So in this case, the
lowest common ancestor

477
00:29:00,750 --> 00:29:03,810
of this node and q
is going to be here.

478
00:29:03,810 --> 00:29:06,420
Also this node and q
happens to also be here.

479
00:29:06,420 --> 00:29:09,900
So this is the lowest we can go.

480
00:29:09,900 --> 00:29:14,810
And what this means
is that these guys,

481
00:29:14,810 --> 00:29:16,950
they share the bit
string up to here.

482
00:29:16,950 --> 00:29:20,130
We were on the blue
substructure up till here.

483
00:29:20,130 --> 00:29:22,890
This was the node
where q diverged.

484
00:29:22,890 --> 00:29:26,880
We followed a pointer here
along a non-blue edge.

485
00:29:26,880 --> 00:29:28,180
That's where we made a mistake.

486
00:29:28,180 --> 00:29:30,330
So this lets us
find, in some sense,

487
00:29:30,330 --> 00:29:33,360
the first mistake, where
we fell off the tree.

488
00:29:33,360 --> 00:29:35,610
So that's where we
fell off the blue tree.

489
00:29:35,610 --> 00:29:38,040
That's useful information.

490
00:29:38,040 --> 00:29:41,640
Because now we know, well,
we went to the right,

491
00:29:41,640 --> 00:29:44,880
whereas all the actual data is
over here in the left subtree.

492
00:29:44,880 --> 00:29:47,730
There is no blue
stuff in the right.

493
00:29:47,730 --> 00:29:48,810
So that tells us a lot.

494
00:29:48,810 --> 00:29:51,530
If we want to now find
the predecessor of q,

495
00:29:51,530 --> 00:29:55,630
it's going to be whatever
is the max in this subtree.

496
00:29:55,630 --> 00:29:59,080
So, I just need to be able
to find the max over here.

497
00:29:59,080 --> 00:30:00,420
So this is the idea.

498
00:30:00,420 --> 00:30:03,300
Now there's two cases, depending
on whether we were in the right

499
00:30:03,300 --> 00:30:05,220
or in the left from that node.

500
00:30:10,650 --> 00:30:18,090
So let me write
this, find the node y

501
00:30:18,090 --> 00:30:25,680
where q fell off the blue tree.

502
00:30:25,680 --> 00:30:29,070
So this node y, we can
think of as a bit number.

503
00:30:29,070 --> 00:30:31,820
Here the leftmost
bit was still on,

504
00:30:31,820 --> 00:30:34,500
but then the next bit was off.

505
00:30:34,500 --> 00:30:36,180
And so we look at--

506
00:30:36,180 --> 00:30:42,180
I'll call that bit y plus 1,
or maybe size of y plus 1.

507
00:30:42,180 --> 00:30:49,760
If that bit equals 1,
that's the picture we have.

508
00:30:49,760 --> 00:30:56,190
Then what I'm going to do
is set a new quantity e,

509
00:30:56,190 --> 00:30:57,190
which is going to be--

510
00:30:57,190 --> 00:30:58,560
this is a new word.

511
00:30:58,560 --> 00:31:02,280
It's going to be the bit
string y, followed by a 0,

512
00:31:02,280 --> 00:31:03,620
followed by lots of 1's.

513
00:31:07,160 --> 00:31:14,696
Whereas our bit string q had a
1 here, and fell off the tree.

514
00:31:14,696 --> 00:31:16,070
What we're instead
going to do is

515
00:31:16,070 --> 00:31:22,190
identify this node, the
rightmost node in this subtree.

516
00:31:22,190 --> 00:31:24,680
That's not necessarily an xi.

517
00:31:24,680 --> 00:31:26,330
But it's a thing.

518
00:31:26,330 --> 00:31:28,460
And then we're
going to do, again,

519
00:31:28,460 --> 00:31:35,630
this search and sketch space,
but now using e instead of q.

520
00:31:35,630 --> 00:31:38,720
If we do that,
what is this node?

521
00:31:38,720 --> 00:31:42,440
Let's label it, 0011.

522
00:31:42,440 --> 00:31:47,069
If you look at the sketch
bits, this has a sketch of 01.

523
00:31:47,069 --> 00:31:48,860
So if I did a search
here, I would actually

524
00:31:48,860 --> 00:31:51,770
find that this is
the answer, and that

525
00:31:51,770 --> 00:31:53,900
actually is a predecessor of q.

526
00:31:53,900 --> 00:31:56,270
In general, this is
going to work well.

527
00:31:56,270 --> 00:32:00,684
Because essentially
some of these bits

528
00:32:00,684 --> 00:32:01,850
are going to be sketch bits.

529
00:32:01,850 --> 00:32:05,150
This one was not, and
we made a mistake there.

530
00:32:05,150 --> 00:32:05,940
We went right.

531
00:32:05,940 --> 00:32:07,022
We should have got left.

532
00:32:07,022 --> 00:32:09,230
These ones, some of them
are going to be sketch bits.

533
00:32:09,230 --> 00:32:10,460
Some of them are not.

534
00:32:10,460 --> 00:32:12,260
But whichever ones
get underlined,

535
00:32:12,260 --> 00:32:15,050
it's going to be a
1, which means we're

536
00:32:15,050 --> 00:32:16,760
going to do the right thing.

537
00:32:16,760 --> 00:32:20,370
We want the very rightmost
item in this tree.

538
00:32:20,370 --> 00:32:24,050
So if we always go right
whenever there's a sketch bit,

539
00:32:24,050 --> 00:32:26,570
and then do a search
in sketch space,

540
00:32:26,570 --> 00:32:30,240
we will find the rightmost
item in this tree.

541
00:32:30,240 --> 00:32:32,480
So if we then do a
search on e, we're

542
00:32:32,480 --> 00:32:35,100
always going to get
the right answer.

543
00:32:35,100 --> 00:32:38,090
So in the end, we're going do
two searches in sketch space,

544
00:32:38,090 --> 00:32:41,930
once with q to find this
place where we fell off,

545
00:32:41,930 --> 00:32:47,720
then once with e where we
actually find the right answer.

546
00:32:51,470 --> 00:32:54,560
And there's a symmetric case,
which is if we went left and we

547
00:32:54,560 --> 00:32:56,930
should have gone right,
then we go right,

548
00:32:56,930 --> 00:32:58,190
and then we put a lot of 0's.

549
00:32:58,190 --> 00:33:00,420
Because then we want to
find the min in that tree.

550
00:33:10,070 --> 00:33:14,120
So, back to search, we
compute sketch of q.

551
00:33:14,120 --> 00:33:19,785
We find it among the
sketch of the xi's.

552
00:33:24,050 --> 00:33:27,120
This gives us this y.

553
00:33:27,120 --> 00:33:31,690
So we find the
longest common prefix,

554
00:33:31,690 --> 00:33:41,240
y equals longest common prefix
of q and xi or xi plus 1.

555
00:33:41,240 --> 00:33:55,346
Then we compute e, and then we
find sketch of e among sketch

556
00:33:55,346 --> 00:33:55,845
of xi's.

557
00:33:59,480 --> 00:34:05,700
And the claim is that the
predecessor and successor

558
00:34:05,700 --> 00:34:13,250
of sketch of e among sketch
of xi's equals the predecessor

559
00:34:13,250 --> 00:34:26,010
and successor of q, our
actual query, among the xi's.

560
00:34:38,669 --> 00:34:39,460
So this is a claim.

561
00:34:39,460 --> 00:34:40,340
It needs proof.

562
00:34:40,340 --> 00:34:43,790
But it's what I've been
arguing that e gives us

563
00:34:43,790 --> 00:34:44,780
the right structure.

564
00:34:44,780 --> 00:34:47,840
It fixes all the sketch bits
that are potentially wrong.

565
00:34:47,840 --> 00:34:50,480
We found the first sketch
bit that was wrong.

566
00:34:50,480 --> 00:34:51,763
We fixed that one.

567
00:34:51,763 --> 00:34:53,929
And then the remainder, as
long as we go all the way

568
00:34:53,929 --> 00:34:58,070
to the right, we'll find the
max, or in the other case

569
00:34:58,070 --> 00:34:59,840
we want to go all
the way to the left

570
00:34:59,840 --> 00:35:01,970
because we want to find the min.

571
00:35:01,970 --> 00:35:04,060
So that's this claim.

572
00:35:04,060 --> 00:35:06,340
We find the predecessor
of sketch of e.

573
00:35:06,340 --> 00:35:10,530
Run the sketch of the xi's,
which is just this thing again.

574
00:35:10,530 --> 00:35:12,650
So again, we can do
it in constant time.

575
00:35:12,650 --> 00:35:14,390
Then we find--

576
00:35:14,390 --> 00:35:16,939
I mean I have to be a little
bit more precise here.

577
00:35:16,939 --> 00:35:18,980
Of course, we find the
predecessor and successor,

578
00:35:18,980 --> 00:35:20,090
we get a sketch of the xi.

579
00:35:20,090 --> 00:35:22,160
We have to undo that
sketch operation.

580
00:35:22,160 --> 00:35:24,830
Really the way to think of it
is predecessor and successor

581
00:35:24,830 --> 00:35:27,530
are really returning a rank.

582
00:35:27,530 --> 00:35:29,510
I want to know the
i that matters.

583
00:35:29,510 --> 00:35:36,155
So if it fits between sketch
of xi and sketch of xi plus 1,

584
00:35:36,155 --> 00:35:38,180
if sketch of e
fits between those,

585
00:35:38,180 --> 00:35:43,730
then I know that q will fit
between xi and xi plus 1,

586
00:35:43,730 --> 00:35:48,290
in terms of that rank, i,
the index in the array.

587
00:35:51,134 --> 00:35:53,170
So, that makes sense.

588
00:35:53,170 --> 00:35:59,970
This is the end of the--

589
00:35:59,970 --> 00:36:01,410
what's the right way to put it?

590
00:36:01,410 --> 00:36:04,330
This is the big picture
of fusion trees.

591
00:36:04,330 --> 00:36:09,070
At this point you should
believe that everything works.

592
00:36:09,070 --> 00:36:10,590
And overall, what are we doing?

593
00:36:10,590 --> 00:36:13,200
We're building a
w to the 1/5 tree.

594
00:36:13,200 --> 00:36:16,587
It's not yet clear
why w to the 1/5.

595
00:36:16,587 --> 00:36:18,420
And so we have to
implement these nodes that

596
00:36:18,420 --> 00:36:20,114
only have w to the 1/5 keys.

597
00:36:20,114 --> 00:36:21,530
So we're looking
at a single node.

598
00:36:21,530 --> 00:36:23,610
And say, hey look,
there's a bunch of keys.

599
00:36:23,610 --> 00:36:25,380
Let's just look at
the important bits.

600
00:36:25,380 --> 00:36:27,630
That defines the
sketch operation.

601
00:36:27,630 --> 00:36:31,170
Now if we want to do a search,
we do this double search.

602
00:36:31,170 --> 00:36:35,370
We compute the sketch, find
the sketch among the sketches,

603
00:36:35,370 --> 00:36:39,270
find our mistake,
compute our proper query,

604
00:36:39,270 --> 00:36:42,690
compute the sketch of that, find
that sketch among the sketches,

605
00:36:42,690 --> 00:36:46,620
and then that index in the array
of sketches will be the correct

606
00:36:46,620 --> 00:36:51,910
index of our actual
query q among the xi's.

607
00:36:51,910 --> 00:36:53,370
Now, there are
several things left

608
00:36:53,370 --> 00:36:55,200
to be done in a reasonable way.

609
00:36:55,200 --> 00:36:58,530
One is how do we
compute sketches.

610
00:36:58,530 --> 00:37:00,210
How do we do this
kind of operation

611
00:37:00,210 --> 00:37:03,210
of taking the bits we care
about and bringing them

612
00:37:03,210 --> 00:37:06,220
all next to each other?

613
00:37:06,220 --> 00:37:08,250
Second thing is, how
do we do this find?

614
00:37:08,250 --> 00:37:10,260
This parallel comparison.

615
00:37:10,260 --> 00:37:12,990
So it's basically all
the bullets up here.

616
00:37:12,990 --> 00:37:15,960
We have how do we
do a sketch, how

617
00:37:15,960 --> 00:37:19,020
do we do parallel comparison
to find where one sketch fits

618
00:37:19,020 --> 00:37:22,720
among many sketches, and there's
also a most significant set

619
00:37:22,720 --> 00:37:23,220
bit.

620
00:37:23,220 --> 00:37:24,900
Where did we do that?

621
00:37:24,900 --> 00:37:28,267
In computing the
longest common prefix.

622
00:37:28,267 --> 00:37:29,850
So if you have two
bit strings and you

623
00:37:29,850 --> 00:37:32,779
want to know where
did they first differ,

624
00:37:32,779 --> 00:37:34,320
the natural way to
do that is compute

625
00:37:34,320 --> 00:37:37,170
the XOR, which gives you
all the differing bits.

626
00:37:37,170 --> 00:37:40,230
And then find the first
one bit from the left.

627
00:37:40,230 --> 00:37:43,650
So this is really most
significant set bit.

628
00:37:48,630 --> 00:37:50,010
So, we need that operation.

629
00:37:54,760 --> 00:37:56,610
So, we have our
work cut out for us.

630
00:37:56,610 --> 00:37:58,580
But the overall
picture of fusion trees

631
00:37:58,580 --> 00:38:00,740
should now be clear.

632
00:38:00,740 --> 00:38:03,090
It just remains to do
these three things.

633
00:38:03,090 --> 00:38:06,090
And this is where the
engineering comes in,

634
00:38:06,090 --> 00:38:09,010
I would say.

635
00:38:09,010 --> 00:38:11,620
Any questions about
the big picture?

636
00:38:30,980 --> 00:38:34,840
So, the first thing I'm
going to do is sketch.

637
00:38:34,840 --> 00:38:37,115
And as I've hinted at
in the outline here,

638
00:38:37,115 --> 00:38:38,740
we're not going to
do a perfect sketch.

639
00:38:38,740 --> 00:38:40,406
We're going to do an
approximate sketch.

640
00:38:43,810 --> 00:38:45,790
This will probably
be the most work

641
00:38:45,790 --> 00:38:47,059
among any of these operations.

642
00:38:47,059 --> 00:38:48,850
Parallel comparison is
actually quite easy.

643
00:38:51,430 --> 00:38:54,100
Sketching is, I think,
the biggest insight

644
00:38:54,100 --> 00:38:57,340
in Fusion trees.

645
00:38:57,340 --> 00:39:01,450
So, perfect sketch
takes just the bits

646
00:39:01,450 --> 00:39:03,092
you care about that we need.

647
00:39:03,092 --> 00:39:05,050
We only want to look at
the bits we care about.

648
00:39:05,050 --> 00:39:07,049
But it's easy to look at
the bits we care about.

649
00:39:07,049 --> 00:39:10,090
We can apply a mask, and just
AND out the bits we care about.

650
00:39:10,090 --> 00:39:12,040
Everything else we can zero out.

651
00:39:12,040 --> 00:39:12,670
So that's easy.

652
00:39:12,670 --> 00:39:16,480
The hard part is compression,
taking these four bits

653
00:39:16,480 --> 00:39:18,950
and making them four
consecutive bits.

654
00:39:18,950 --> 00:39:21,370
But they don't really
need to be consecutive.

655
00:39:21,370 --> 00:39:28,810
If I added in some 0's here
in a consistent pattern

656
00:39:28,810 --> 00:39:29,770
that would still work.

657
00:39:29,770 --> 00:39:31,769
I'd still preserve the
order among the sketches.

658
00:39:31,769 --> 00:39:33,225
And that's all I care about.

659
00:39:33,225 --> 00:39:35,350
And this is where I'm going
to use the slop I have.

660
00:39:35,350 --> 00:39:39,260
Because right now I
have w to the 1/5 keys.

661
00:39:39,260 --> 00:39:41,410
If I did perfect sketch,
the total number of bits

662
00:39:41,410 --> 00:39:42,970
would only be w to the 2/5.

663
00:39:42,970 --> 00:39:45,010
But I can go up to w.

664
00:39:45,010 --> 00:39:51,010
So what I'm going
to do is basically

665
00:39:51,010 --> 00:40:02,170
spread out the bits,
the important bits,

666
00:40:02,170 --> 00:40:15,480
in a predictable pattern
of length w to the 4/5.

667
00:40:19,690 --> 00:40:22,450
Predictable just means it
doesn't depend on what x is.

668
00:40:22,450 --> 00:40:28,120
So when there are extra 0's
here, you know that's fine.

669
00:40:28,120 --> 00:40:31,220
But there's always going to
be two 0's here, one 0 here,

670
00:40:31,220 --> 00:40:34,300
three 0's here, no
matter what x was.

671
00:40:34,300 --> 00:40:37,100
As long as it's predictable,
I'm going to preserve order.

672
00:40:37,100 --> 00:40:39,960
And as long as it's
length order w to the 4/5,

673
00:40:39,960 --> 00:40:42,176
if I take w to the
1/5 of them, that

674
00:40:42,176 --> 00:40:44,050
will still fit in a
constant number of words.

675
00:40:44,050 --> 00:40:47,390
Because it will be
order w bits total.

676
00:40:47,390 --> 00:40:49,020
So that's what I can afford.

677
00:40:49,020 --> 00:40:52,480
And now I'm going to do it.

678
00:40:52,480 --> 00:40:55,887
So here's how.

679
00:40:55,887 --> 00:40:57,470
First thing, as I
said, is we're going

680
00:40:57,470 --> 00:40:59,680
to mask the important bits.

681
00:41:04,509 --> 00:41:05,800
I just want the important bits.

682
00:41:05,800 --> 00:41:07,660
I should throw away
all the others.

683
00:41:07,660 --> 00:41:16,075
And so this is going to be x
prime equals x bit-wise AND.

684
00:41:25,171 --> 00:41:27,670
And here's where I'm going to
use the notation that the bits

685
00:41:27,670 --> 00:41:29,080
count from the right.

686
00:41:29,080 --> 00:41:32,920
I want the bi-th bit to
correspond to the value 2

687
00:41:32,920 --> 00:41:34,510
to the bi.

688
00:41:34,510 --> 00:41:37,730
This thing is just a bit string.

689
00:41:37,730 --> 00:41:41,920
It has 1's wherever
the important bits are.

690
00:41:41,920 --> 00:41:49,180
So if this is the
b0, b1, b2, and b3;

691
00:41:49,180 --> 00:41:52,000
I just want this bit string.

692
00:41:52,000 --> 00:41:55,360
I mean, you can think of this
as 1 shifted left bi times.

693
00:41:55,360 --> 00:41:57,950
So I get 1's in exactly
the positions I care about.

694
00:41:57,950 --> 00:42:00,010
And if I bit-wise AND
that with x it zeros out

695
00:42:00,010 --> 00:42:01,130
all the other bits.

696
00:42:01,130 --> 00:42:02,350
This is what we call masking.

697
00:42:05,950 --> 00:42:08,480
So that's the
obvious thing to do.

698
00:42:08,480 --> 00:42:14,100
And then the second
idea is multiplication.

699
00:42:14,100 --> 00:42:17,190
And it's just like, well, maybe
we could do it with a multiply,

700
00:42:17,190 --> 00:42:19,060
and then we'll just work it out.

701
00:42:19,060 --> 00:42:21,290
And the answer is yes, you
can do it with a multiply.

702
00:42:21,290 --> 00:42:24,480
So that I imagine
was the big insight

703
00:42:24,480 --> 00:42:29,520
was to see that multiplication
is a very powerful operation.

704
00:42:29,520 --> 00:42:32,975
So we're just going to do x
prime times some number m.

705
00:42:32,975 --> 00:42:35,100
And we're going to prove
that there exists a number

706
00:42:35,100 --> 00:42:37,660
m that does what we need.

707
00:42:37,660 --> 00:42:41,200
So I'm going to write this out
a little bit algebraically.

708
00:42:41,200 --> 00:42:44,890
So we can think about
what m might be.

709
00:42:44,890 --> 00:42:48,280
Now x prime only has
the important bits.

710
00:42:48,280 --> 00:43:00,620
So we can write that as a sum i
equals 0 to r minus 1 of xbi 2

711
00:43:00,620 --> 00:43:02,010
to the bi.

712
00:43:02,010 --> 00:43:04,500
So I am introducing
some notation here. xbi,

713
00:43:04,500 --> 00:43:08,840
is that important bit bi 1 or 0?

714
00:43:08,840 --> 00:43:15,090
This is just a de-reference of
the bit vector or a bit string.

715
00:43:15,090 --> 00:43:17,240
And so you multiply
that by that position.

716
00:43:17,240 --> 00:43:20,691
I mean this the definition
of binary notation, right?

717
00:43:20,691 --> 00:43:22,440
But we only care about
the important bits.

718
00:43:22,440 --> 00:43:23,910
Because only those are set.

719
00:43:23,910 --> 00:43:25,410
So that's x prime.

720
00:43:25,410 --> 00:43:28,440
And then we're
multiplying that by m.

721
00:43:28,440 --> 00:43:31,110
Now m could have any bit set.

722
00:43:31,110 --> 00:43:33,765
So I'm going to--

723
00:43:33,765 --> 00:43:35,265
but I'm going to
write it like this.

724
00:43:54,160 --> 00:43:58,670
I'm going to assume that
m only has r bits set,

725
00:43:58,670 --> 00:44:01,950
same as the number
of important bits,

726
00:44:01,950 --> 00:44:03,380
r is a number of important bits.

727
00:44:10,407 --> 00:44:11,740
But I don't know where they are.

728
00:44:11,740 --> 00:44:14,850
So I'm just going to suppose
they're at positions m0, m1,

729
00:44:14,850 --> 00:44:16,440
up to mr minus 1.

730
00:44:16,440 --> 00:44:20,730
I've got to find what these
mi's should be, or mj's.

731
00:44:20,730 --> 00:44:22,830
And now just taking
this product,

732
00:44:22,830 --> 00:44:25,980
so we can expand out the
product algebraically

733
00:44:25,980 --> 00:44:26,970
and see what we get.

734
00:44:46,960 --> 00:44:51,930
So, what's this product?

735
00:44:51,930 --> 00:44:56,200
Sum i equals 0 to
r minus 1 sum j

736
00:44:56,200 --> 00:45:00,770
equals 0 to r minus 1 of xbi--

737
00:45:04,452 --> 00:45:06,210
I mean just the
product of these.

738
00:45:06,210 --> 00:45:10,320
So 2 to the bi plus mj.

739
00:45:10,320 --> 00:45:12,480
That's the algebraic
product of those two things.

740
00:45:12,480 --> 00:45:13,938
That's why I wrote
it out this way.

741
00:45:13,938 --> 00:45:15,270
So I can see what's going on.

742
00:45:15,270 --> 00:45:17,070
The point is when you
do multiplication,

743
00:45:17,070 --> 00:45:21,140
you're doing these
pairwise products.

744
00:45:21,140 --> 00:45:22,770
Now the guys that
are going to survive

745
00:45:22,770 --> 00:45:25,890
are the ones where the
xbi's are 1, of course.

746
00:45:25,890 --> 00:45:28,590
But they survive
in multiple places.

747
00:45:28,590 --> 00:45:33,090
Essentially the mj's
shift all of those bits

748
00:45:33,090 --> 00:45:35,610
by various amounts.

749
00:45:35,610 --> 00:45:39,510
So it used to be at this
position, 2 to the bi.

750
00:45:39,510 --> 00:45:43,140
But now we're shifting
it by mj for all j.

751
00:45:43,140 --> 00:45:45,960
So some of those bits
might hit each other.

752
00:45:45,960 --> 00:45:46,770
Then they add up.

753
00:45:46,770 --> 00:45:48,150
That's really messy.

754
00:45:48,150 --> 00:45:49,830
We're going to avoid
that, and design

755
00:45:49,830 --> 00:45:53,750
the mj's so that all of
these values are unique.

756
00:45:53,750 --> 00:45:56,250
Therefore, bits
never hit each other.

757
00:45:56,250 --> 00:45:57,990
That's step one.

758
00:45:57,990 --> 00:46:02,130
And then furthermore, what we
care about or what we're trying

759
00:46:02,130 --> 00:46:08,160
to do is to get the xbi's to
appear in a nice little window,

760
00:46:08,160 --> 00:46:14,280
consecutive interval
of w to the 4/5 bits,

761
00:46:14,280 --> 00:46:15,780
somehow by setting the mj's.

762
00:46:15,780 --> 00:46:17,995
So let me tell you
the claim, which

763
00:46:17,995 --> 00:46:20,944
we will prove by induction.

764
00:46:25,790 --> 00:46:28,042
So we're given these bi's
that we can't control.

765
00:46:28,042 --> 00:46:29,250
Those are the important bits.

766
00:46:31,860 --> 00:46:44,790
And the claim is we can
choose the mi's such

767
00:46:44,790 --> 00:46:47,070
that three properties hold.

768
00:46:47,070 --> 00:46:59,790
First one is that bi plus mj
are distinct for all i and j.

769
00:46:59,790 --> 00:47:02,700
So that was that these bits
don't collide with each other.

770
00:47:02,700 --> 00:47:04,200
So there's no actual
summation here.

771
00:47:04,200 --> 00:47:05,730
These sums could
then be replaced

772
00:47:05,730 --> 00:47:08,564
by ORs, which makes
it very easy to keep

773
00:47:08,564 --> 00:47:10,230
track of where the
bits are going, if we

774
00:47:10,230 --> 00:47:12,540
can achieve this property.

775
00:47:12,540 --> 00:47:30,670
Property b is that it
turns out that the bits

776
00:47:30,670 --> 00:47:32,860
I'm going to end up
caring about our b0

777
00:47:32,860 --> 00:47:37,360
plus m0, b1 plus m1,
and general bi plus mi.

778
00:47:37,360 --> 00:47:40,330
In general, we have bi plus
mj for different values

779
00:47:40,330 --> 00:47:41,530
of i and j.

780
00:47:41,530 --> 00:47:43,540
I claim the ones I care
about are the ones where

781
00:47:43,540 --> 00:47:44,662
i and j are equal.

782
00:47:44,662 --> 00:47:46,120
So I'm going to
look at these bits,

783
00:47:46,120 --> 00:47:50,860
and in particular I want
them to appear in order

784
00:47:50,860 --> 00:47:51,976
in the bit string.

785
00:47:54,730 --> 00:47:57,376
And then third property--

786
00:47:57,376 --> 00:47:58,765
I need some more space--

787
00:48:01,570 --> 00:48:05,110
is that if I look at
the span of those bits,

788
00:48:05,110 --> 00:48:10,150
so I look at br minus
1 plus mr minus 1

789
00:48:10,150 --> 00:48:17,680
minus b0 plus m0 that is the
interval that these bits span.

790
00:48:17,680 --> 00:48:23,380
I want that to be order
r to the fourth power.

791
00:48:23,380 --> 00:48:27,130
Because r was w to the 1/5.

792
00:48:27,130 --> 00:48:29,710
So this would be
order w to the 4/5.

793
00:48:29,710 --> 00:48:32,121
That's what I need for
everything to fit in.

794
00:48:35,200 --> 00:48:37,810
So this is guaranteeing
that these bits

795
00:48:37,810 --> 00:48:41,510
are the sketch that I need.

796
00:48:41,510 --> 00:48:44,800
They appear in
order, and they don't

797
00:48:44,800 --> 00:48:48,092
span a very large interval,
just w to the 4/5.

798
00:48:48,092 --> 00:48:49,300
This is what I need to prove.

799
00:48:49,300 --> 00:48:53,695
If I can prove this, I
have approximate sketching.

800
00:48:53,695 --> 00:48:54,730
So let's prove it.

801
00:49:19,600 --> 00:49:21,510
Proof happens in two steps.

802
00:49:21,510 --> 00:49:23,010
First thing I'm
going to worry about

803
00:49:23,010 --> 00:49:25,890
is just getting
these guys distinct.

804
00:49:25,890 --> 00:49:28,840
Then I'll worry about
the order property.

805
00:49:28,840 --> 00:49:31,890
So here's how we
get them distinct.

806
00:49:31,890 --> 00:49:37,530
And these are going
to be the mi primes,

807
00:49:37,530 --> 00:49:39,010
not quite the mi's that we want.

808
00:49:42,450 --> 00:49:47,210
They're all going to be integers
less than r cubed, greater than

809
00:49:47,210 --> 00:49:51,810
or equal to 0, and they're
going to have the property

810
00:49:51,810 --> 00:50:06,030
that the bi's plus mj primes
are distinct mod r cubed.

811
00:50:06,030 --> 00:50:07,740
So this is a stronger
version of a.

812
00:50:07,740 --> 00:50:09,990
We really just need
them to be distinct.

813
00:50:09,990 --> 00:50:12,060
But to make it easier
for the other steps,

814
00:50:12,060 --> 00:50:15,298
we're going to force them
to be distinct mod r cubed.

815
00:50:18,390 --> 00:50:20,400
How do we do this?

816
00:50:20,400 --> 00:50:22,000
By induction.

817
00:50:22,000 --> 00:50:31,716
So let's suppose that we've
picked m0 up to mt minus 1.

818
00:50:31,716 --> 00:50:35,650
So suppose by induction
that we've done that.

819
00:50:35,650 --> 00:50:38,640
And now our goal is
to pick mt prime.

820
00:50:41,970 --> 00:50:46,330
So how do we choose mt prime?

821
00:50:46,330 --> 00:50:49,970
Well, what can't it be?

822
00:50:49,970 --> 00:50:58,070
mt prime has to avoid
basically mi prime,

823
00:50:58,070 --> 00:51:03,611
and believe minus bj plus b--

824
00:51:03,611 --> 00:51:05,050
we're going to call it k?

825
00:51:05,050 --> 00:51:05,690
I guess so.

826
00:51:08,990 --> 00:51:12,550
If it avoids all
expressions like this,

827
00:51:12,550 --> 00:51:19,600
then mt prime plus bj
will be different from mi

828
00:51:19,600 --> 00:51:21,170
prime plus bk.

829
00:51:21,170 --> 00:51:23,980
In other words, all of these
things will be distinct.

830
00:51:23,980 --> 00:51:29,560
So it has to avoid
this modulo r cubed.

831
00:51:29,560 --> 00:51:31,910
If I can avoid all
of these things--

832
00:51:31,910 --> 00:51:34,300
so this is for all ijk--

833
00:51:38,010 --> 00:51:40,600
if I can choose mt prime
to avoid all those,

834
00:51:40,600 --> 00:51:43,260
then I'm happy.

835
00:51:43,260 --> 00:51:45,690
Because then these things
will continue to be distinct,

836
00:51:45,690 --> 00:51:47,700
and then I apply induction.

837
00:51:47,700 --> 00:51:50,320
Well, how many choices
are there for i, j, and k?

838
00:51:50,320 --> 00:51:55,116
For i, there's I
guess t choices.

839
00:51:55,116 --> 00:51:58,950
Because mi can be any
of the previous values.

840
00:51:58,950 --> 00:52:06,970
For j, let's call it r choices
for k, there's r choices.

841
00:52:06,970 --> 00:52:08,740
That's how many
important bits there are.

842
00:52:08,740 --> 00:52:11,815
So total number of
choices is tr squared.

843
00:52:14,410 --> 00:52:16,960
But t here is
always less than r.

844
00:52:16,960 --> 00:52:20,372
So this is going to
be less than r cubed.

845
00:52:20,372 --> 00:52:22,330
So that means there is
less than r cubed things

846
00:52:22,330 --> 00:52:23,560
we have to avoid.

847
00:52:23,560 --> 00:52:25,740
But I have r cubed
allowable choices

848
00:52:25,740 --> 00:52:27,100
on working modulo r cubed.

849
00:52:27,100 --> 00:52:30,020
So I just pick any one
that avoids the collision.

850
00:52:30,020 --> 00:52:33,340
This is basically deterministic
hashing, in a certain sense.

851
00:52:33,340 --> 00:52:35,680
We are choosing these
values deterministically

852
00:52:35,680 --> 00:52:39,231
to avoid collisions in
this simple hash function.

853
00:52:39,231 --> 00:52:39,730
OK.

854
00:52:39,730 --> 00:52:40,805
It takes time.

855
00:52:40,805 --> 00:52:43,180
It's going to take polynomial
time to compute this thing.

856
00:52:43,180 --> 00:52:45,329
And you can imagine if
you just plug in hashing,

857
00:52:45,329 --> 00:52:46,870
this will work with
some probability,

858
00:52:46,870 --> 00:52:47,745
and blah, blah, blah.

859
00:52:47,745 --> 00:52:50,590
But I want to make it always
work deterministically.

860
00:52:50,590 --> 00:52:54,681
Because we know what
the xi's are here.

861
00:52:54,681 --> 00:52:55,180
All right.

862
00:52:55,180 --> 00:52:56,930
So we've avoided collisions.

863
00:52:56,930 --> 00:52:57,880
There's enough space.

864
00:52:57,880 --> 00:52:59,110
That's all.

865
00:52:59,110 --> 00:53:00,540
That was step one.

866
00:53:00,540 --> 00:53:04,540
Step two, and this
will solve property a,

867
00:53:04,540 --> 00:53:06,154
even modulo r cubed.

868
00:53:06,154 --> 00:53:07,570
Now we have a
little bit of space.

869
00:53:07,570 --> 00:53:09,470
We're allowed to go
up to r to the fourth.

870
00:53:09,470 --> 00:53:12,560
And now we just need to
spread out these bits.

871
00:53:12,560 --> 00:53:17,200
So that's step two.

872
00:53:17,200 --> 00:53:21,520
Basically we're going to set
mi to be these values that we

873
00:53:21,520 --> 00:53:32,680
chose plus this weird thing,
w minus bi plus ir cubed

874
00:53:32,680 --> 00:53:42,810
rounded down to a
multiple of r cubed.

875
00:53:45,718 --> 00:53:50,360
So I guess you could put this
in parentheses if you want.

876
00:53:50,360 --> 00:53:55,400
Rough idea is, we want to
take mi prime plus ir cubed.

877
00:53:55,400 --> 00:53:57,780
Because these mi primes,
they're all values between 0,

878
00:53:57,780 --> 00:53:59,450
and r cubed minus 1.

879
00:53:59,450 --> 00:54:03,230
We got everything
working modulo r cubed.

880
00:54:03,230 --> 00:54:06,320
If we could just add ir
cubed to each of these values

881
00:54:06,320 --> 00:54:08,770
that we'll spread them out.

882
00:54:08,770 --> 00:54:10,810
Because each of these
values used to fall just

883
00:54:10,810 --> 00:54:12,250
in this tiny range ir cubed.

884
00:54:12,250 --> 00:54:15,940
So we can move the next
one to the next position,

885
00:54:15,940 --> 00:54:18,130
move the next one to the
next position, and so on.

886
00:54:18,130 --> 00:54:21,280
Spread them out to
the left by adding

887
00:54:21,280 --> 00:54:25,840
on multiples of r cubed, then
that will achieve property b.

888
00:54:28,800 --> 00:54:31,510
The annoying issue here is
we don't want to mess things

889
00:54:31,510 --> 00:54:33,230
up modulo r cubed.

890
00:54:33,230 --> 00:54:35,050
So we need to round
things down to be

891
00:54:35,050 --> 00:54:44,617
a multiple of r cubed so that
this is congruent to mi prime.

892
00:54:44,617 --> 00:54:45,450
That's what we want.

893
00:54:45,450 --> 00:54:47,520
We want it to stay
congruent to mod r cubed.

894
00:54:49,971 --> 00:54:52,470
Well, why do we need to round
down to a multiple of r cubed?

895
00:54:52,470 --> 00:54:54,690
We were adding on ir cubed.

896
00:54:54,690 --> 00:54:56,850
Well, it's not quite
mi that we care about.

897
00:54:56,850 --> 00:54:58,860
It's mi plus bi.

898
00:54:58,860 --> 00:55:02,260
Those are the bits that we
want to be nicely ordered.

899
00:55:02,260 --> 00:55:05,790
And so we kind of
need a minus bi here,

900
00:55:05,790 --> 00:55:10,030
so that when we take mi
plus bi, those cancel.

901
00:55:10,030 --> 00:55:12,990
But then bi is not a
multiple of r cubed.

902
00:55:12,990 --> 00:55:15,150
So you've got to do this
rounding down to r cubed.

903
00:55:15,150 --> 00:55:17,870
Also negative bi is
a negative number.

904
00:55:17,870 --> 00:55:19,920
And we can't really deal
with negative numbers.

905
00:55:19,920 --> 00:55:21,810
Because you can't go left of 0.

906
00:55:21,810 --> 00:55:25,080
So we have to add on this w
just to make things work out.

907
00:55:25,080 --> 00:55:27,720
So it's a little
messy, and I don't

908
00:55:27,720 --> 00:55:31,440
want to spend too much time
on why this formula works.

909
00:55:31,440 --> 00:55:33,750
But I think you have the
essence of what's working.

910
00:55:33,750 --> 00:55:36,150
This is just to avoid
negative numbers.

911
00:55:36,150 --> 00:55:43,200
This negative bi is so that when
you add it to mi that cancels.

912
00:55:43,200 --> 00:55:47,129
And so you get these
r cubes separations.

913
00:55:47,129 --> 00:55:48,920
In the end, let me draw
a picture, perhaps.

914
00:56:09,860 --> 00:56:17,520
In the end, if you look at the
bit space, so this is w bits.

915
00:56:17,520 --> 00:56:21,080
And you divide it up into
multiples of r cubed.

916
00:56:24,710 --> 00:56:30,360
All of the mi primes
are over here.

917
00:56:30,360 --> 00:56:31,670
So these are mi primes.

918
00:56:31,670 --> 00:56:33,420
You don't know in what
order or anything.

919
00:56:33,420 --> 00:56:36,003
They're just kind of randomly
in there, and chosen pretty much

920
00:56:36,003 --> 00:56:38,600
arbitrarily on the low
end of the spectrum,

921
00:56:38,600 --> 00:56:41,550
from 0 to r cubed minus 1.

922
00:56:41,550 --> 00:56:46,550
And then what we want
is for x0 plus m0

923
00:56:46,550 --> 00:56:50,540
to fall somewhere in this
range, and then x1 plus m1

924
00:56:50,540 --> 00:56:54,140
to fall somewhere in this
range, and x2 plus m2

925
00:56:54,140 --> 00:56:57,350
to fall somewhere in this range.

926
00:56:57,350 --> 00:57:00,620
If I do that, and it's
weird because the bits

927
00:57:00,620 --> 00:57:03,140
are numbered from
0 to the left here.

928
00:57:03,140 --> 00:57:06,680
Then I will have this property.

929
00:57:06,680 --> 00:57:10,070
I claim this
assignment does that.

930
00:57:10,070 --> 00:57:15,140
It's an exercise you can check
that indeed xi plus mi will

931
00:57:15,140 --> 00:57:16,250
fall in this range.

932
00:57:21,410 --> 00:57:24,620
So this gives us property b.

933
00:57:24,620 --> 00:57:26,680
It also gives us property c.

934
00:57:26,680 --> 00:57:29,120
Because we've been
fairly tight here.

935
00:57:29,120 --> 00:57:37,520
There's r of these guys, and
r of these intervals of size r

936
00:57:37,520 --> 00:57:38,380
cubed.

937
00:57:38,380 --> 00:57:40,580
And so the total
range of these bits

938
00:57:40,580 --> 00:57:43,040
is going to be r to the fourth.

939
00:57:43,040 --> 00:57:44,810
We started at x0 plus m0.

940
00:57:44,810 --> 00:57:49,790
We end at xr minus
1, plus, mr minus 1.

941
00:57:49,790 --> 00:57:52,530
That's going to
be somewhere here.

942
00:57:52,530 --> 00:57:55,050
But if you look at just
that interval of bits--

943
00:57:55,050 --> 00:58:01,560
so there are more bits actually
over here, in particular,

944
00:58:01,560 --> 00:58:04,040
because of this w bit part.

945
00:58:04,040 --> 00:58:08,650
This whole picture
basically starts at bit w.

946
00:58:08,650 --> 00:58:10,862
Then there's all
this stuff to 0.

947
00:58:10,862 --> 00:58:12,320
So this is a more
accurate picture.

948
00:58:12,320 --> 00:58:14,370
You're doing this
multiplication.

949
00:58:14,370 --> 00:58:15,500
Garbage happens here.

950
00:58:15,500 --> 00:58:16,970
We have no idea.

951
00:58:16,970 --> 00:58:18,194
Garbage happens here.

952
00:58:18,194 --> 00:58:18,860
We have no idea.

953
00:58:18,860 --> 00:58:21,390
Actually, garbage
happens all over here.

954
00:58:21,390 --> 00:58:23,990
But what we know
is that these bits

955
00:58:23,990 --> 00:58:27,350
are the bits we care about.

956
00:58:27,350 --> 00:58:31,010
These are the xi plus mi bits.

957
00:58:31,010 --> 00:58:35,930
If you look at xi plus
mi, they give you xbi.

958
00:58:35,930 --> 00:58:37,160
They exist in other places.

959
00:58:37,160 --> 00:58:41,410
But these bits will
have the important bits.

960
00:58:41,410 --> 00:58:44,240
Now the bits are also
all over everywhere else.

961
00:58:44,240 --> 00:58:45,770
But none of the
bits hit each other.

962
00:58:45,770 --> 00:58:49,490
So these bits remain
correct, because nothing else

963
00:58:49,490 --> 00:58:51,110
collides with it.

964
00:58:51,110 --> 00:58:55,270
And so if I just mask
out those bits, again,

965
00:58:55,270 --> 00:58:57,670
so I have to do another mask.

966
00:58:57,670 --> 00:58:59,830
I did one mask here.

967
00:58:59,830 --> 00:59:03,100
I did a multiplication, and
then I have to do another mask.

968
00:59:06,310 --> 00:59:08,330
So why don't I
write it over here?

969
00:59:08,330 --> 00:59:25,570
So we AND with sum i
equal 0 to r minus 1 of 2

970
00:59:25,570 --> 00:59:29,800
to the bi plus mi.

971
00:59:29,800 --> 00:59:32,330
Those are the circled bits.

972
00:59:32,330 --> 00:59:39,385
So if we grab those things,
and then we shift right by--

973
00:59:42,859 --> 00:59:45,081
why am I writing x?

974
00:59:45,081 --> 00:59:45,580
Sorry.

975
00:59:45,580 --> 00:59:46,480
These are all b's.

976
00:59:50,040 --> 00:59:53,180
Too many letters.

977
00:59:53,180 --> 00:59:56,730
We shift right by b0 plus m0.

978
00:59:56,730 --> 00:59:59,650
Because we don't care about
all those leading bits.

979
00:59:59,650 --> 01:00:01,320
So we shift this
over to the left.

980
01:00:01,320 --> 01:00:05,040
We did the mask, then we will
just have the important bits

981
01:00:05,040 --> 01:00:09,910
and they will occupy over here
an interval of size at most--

982
01:00:09,910 --> 01:00:12,330
I'll say order r to the fourth.

983
01:00:16,440 --> 01:00:18,330
Clear?

984
01:00:18,330 --> 01:00:20,710
So this is
approximate sketching.

985
01:00:20,710 --> 01:00:26,630
This is definitely a bit
complicated, but it works.

986
01:00:26,630 --> 01:00:28,150
Let me review briefly.

987
01:00:28,150 --> 01:00:31,890
So, our algorithm was simple.

988
01:00:31,890 --> 01:00:33,310
We have a bit string, x.

989
01:00:33,310 --> 01:00:34,900
We just want to get
the important bits

990
01:00:34,900 --> 01:00:38,950
and compress them to a thing
of size r to the fourth.

991
01:00:38,950 --> 01:00:41,350
So first of all, we threw away
all the non-important bits

992
01:00:41,350 --> 01:00:42,410
with this mask.

993
01:00:42,410 --> 01:00:43,460
That was easy.

994
01:00:43,460 --> 01:00:46,240
Then we just did an
arbitrary multiplication,

995
01:00:46,240 --> 01:00:49,120
and we proved that there
was a multiplication that

996
01:00:49,120 --> 01:00:50,660
avoided collision.

997
01:00:50,660 --> 01:00:54,760
So the sums basically
turned into ORs or XORs.

998
01:00:54,760 --> 01:00:57,420
I mean you never get two
1 bits hitting each other,

999
01:00:57,420 --> 01:00:59,230
so you don't have
to worry about that.

1000
01:00:59,230 --> 01:01:02,360
And we did that with the
simple inductive argument.

1001
01:01:02,360 --> 01:01:04,840
And then we also wanted
the bi's plus mi's

1002
01:01:04,840 --> 01:01:06,450
to be linearly ordered.

1003
01:01:06,450 --> 01:01:08,950
Because we need to preserve the
order of the important bits.

1004
01:01:08,950 --> 01:01:11,800
We can't just permute them.

1005
01:01:11,800 --> 01:01:14,410
And we needed them to
occupy a small range.

1006
01:01:14,410 --> 01:01:16,249
And we did that
basically by adding

1007
01:01:16,249 --> 01:01:17,290
ir cubed to each of them.

1008
01:01:17,290 --> 01:01:19,206
But it was a little messy
and we had to add w,

1009
01:01:19,206 --> 01:01:22,060
and blah-blah-blah.

1010
01:01:22,060 --> 01:01:24,790
But in the end, we
got our important bits

1011
01:01:24,790 --> 01:01:28,480
to be nicely spaced out
here by pretty much putting

1012
01:01:28,480 --> 01:01:31,390
an r cubed in between each one.

1013
01:01:31,390 --> 01:01:33,670
So those were our
bi plus mi bits.

1014
01:01:33,670 --> 01:01:35,599
They occupied this range
of r to the fourth.

1015
01:01:35,599 --> 01:01:37,390
We'll mask out all the
rest of the garbage.

1016
01:01:37,390 --> 01:01:39,890
Because this multiplication
made a quadratic number of bits.

1017
01:01:39,890 --> 01:01:43,360
We only want these r bits,
the r squared 1 bits in here.

1018
01:01:43,360 --> 01:01:45,280
We'll mask away all the others.

1019
01:01:45,280 --> 01:01:47,210
Take these bits,
shift them over.

1020
01:01:47,210 --> 01:01:51,450
Now they occupy a nice interval
at the beginning size order r

1021
01:01:51,450 --> 01:01:53,560
to the fourth.

1022
01:01:53,560 --> 01:01:56,440
And that's our
approximate sketch.

1023
01:01:56,440 --> 01:02:00,230
So sketch should only take
r, but we're being sloppy.

1024
01:02:00,230 --> 01:02:02,230
With this multiplication
trick, the best we know

1025
01:02:02,230 --> 01:02:03,646
is to get down to
r to the fourth.

1026
01:02:03,646 --> 01:02:07,440
And that's good enough.

1027
01:02:07,440 --> 01:02:10,400
And that's why I set everything
to w to the one fifth.

1028
01:02:10,400 --> 01:02:12,640
Because this is w to the 4/5.

1029
01:02:12,640 --> 01:02:14,960
We're going to have
w to the 1/5 of them.

1030
01:02:14,960 --> 01:02:16,900
And so if you take
these sketches

1031
01:02:16,900 --> 01:02:20,940
and you concatenate them, fuse
them together if you will,

1032
01:02:20,940 --> 01:02:22,700
and that's fusion trees.

1033
01:02:22,700 --> 01:02:28,690
Then the sketches of all of
the keys x0 up to xk minus 1

1034
01:02:28,690 --> 01:02:31,980
will occupy order 1 words.

1035
01:02:31,980 --> 01:02:36,180
Because it's order w bits, w
to the 4/5 times w to the 1/5.

1036
01:02:39,730 --> 01:02:45,250
Which brings us to
parallel comparison.

1037
01:02:45,250 --> 01:02:47,860
I have all of these
approximate sketches.

1038
01:02:47,860 --> 01:02:50,620
So you could start forgetting
approximate sketching.

1039
01:02:50,620 --> 01:02:53,410
Somehow, we get these
w to the 4/5 bits.

1040
01:02:53,410 --> 01:02:56,050
We want to concatenate them
together, and then in parallel

1041
01:02:56,050 --> 01:03:00,690
compare all of them
to the sketch of q.

1042
01:03:00,690 --> 01:03:02,620
The sketch of the xi's
we can preprocess.

1043
01:03:02,620 --> 01:03:04,420
We can actually
spend a lot of time

1044
01:03:04,420 --> 01:03:06,057
finding the sketch function.

1045
01:03:06,057 --> 01:03:07,890
But then we have to fix
the sketch function.

1046
01:03:07,890 --> 01:03:10,610
We have to be able to compute
a sketch of q in constant time.

1047
01:03:10,610 --> 01:03:11,680
That's what we just did.

1048
01:03:11,680 --> 01:03:15,130
Sketch of q is one AND one
multiplication and another AND.

1049
01:03:15,130 --> 01:03:17,530
So computing sketches is fast.

1050
01:03:17,530 --> 01:03:20,110
That's the steps of
computing sketch of q.

1051
01:03:20,110 --> 01:03:23,080
Now, next step is find it
among the sketch of the xi's.

1052
01:03:23,080 --> 01:03:25,525
So this is the next thing
we want to make fast.

1053
01:03:42,050 --> 01:03:43,494
It's actually pretty easy.

1054
01:03:43,494 --> 01:03:45,410
You probably know you
can compare two integers

1055
01:03:45,410 --> 01:03:47,400
by subtracting one
from the other.

1056
01:03:47,400 --> 01:03:50,090
So we're just going to do
that, but in a clever way,

1057
01:03:50,090 --> 01:03:53,390
so we can do k subtractions
for the price of one.

1058
01:04:03,976 --> 01:04:08,600
I'm going to define the sketch
of a node to be 1 bit followed

1059
01:04:08,600 --> 01:04:19,985
by the sketch of x0 dot, dot,
dot, 1 sketch of xk minus one.

1060
01:04:19,985 --> 01:04:26,850
And I'm going to define a
sketch of q to the k-th power,

1061
01:04:26,850 --> 01:04:32,385
so to speak, to be a 0 bit
followed by sketch of q, dot,

1062
01:04:32,385 --> 01:04:36,530
dot dot, zero bit sketch of q.

1063
01:04:39,860 --> 01:04:42,620
This is aligning
things, so that if I

1064
01:04:42,620 --> 01:04:44,990
did this subtraction
and this one,

1065
01:04:44,990 --> 01:04:48,770
I would basically be comparing
q with all the xi's at once.

1066
01:04:52,490 --> 01:04:55,070
The point is these sketches--
this is the thing that

1067
01:04:55,070 --> 01:04:57,110
fits in order 1 words.

1068
01:04:57,110 --> 01:05:00,650
These sketches are
w to the 4/5 bits,

1069
01:05:00,650 --> 01:05:02,690
and there's w to
the 1/5 of them.

1070
01:05:02,690 --> 01:05:04,360
So this whole thing
is order w bits.

1071
01:05:04,360 --> 01:05:06,350
So it fits in one word.

1072
01:05:06,350 --> 01:05:07,850
This thing also.

1073
01:05:07,850 --> 01:05:10,280
It happens to be the same
bits repeated many times

1074
01:05:10,280 --> 01:05:13,310
but also it fits in one word.

1075
01:05:13,310 --> 01:05:15,920
How do I compute this thing?

1076
01:05:15,920 --> 01:05:19,330
I can do it with multiplication.

1077
01:05:19,330 --> 01:05:33,600
It's sketch of q times
0000001, 000000001.

1078
01:05:33,600 --> 01:05:36,990
So, ahead of time, I'll just
pre-compute this bit string

1079
01:05:36,990 --> 01:05:43,010
that has 1's at the rightmost
slot for each of these k

1080
01:05:43,010 --> 01:05:44,665
fields.

1081
01:05:44,665 --> 01:05:46,790
If I just take that and
multiply it by sketch of q,

1082
01:05:46,790 --> 01:05:48,330
then I get this.

1083
01:05:48,330 --> 01:05:52,250
So this is easy to do
in one multiplication.

1084
01:05:52,250 --> 01:05:55,530
Now, I take this thing
minus this thing.

1085
01:05:59,450 --> 01:06:01,800
I take the difference.

1086
01:06:04,500 --> 01:06:08,340
And the key thing is because
I put these 1 bits here, I'm

1087
01:06:08,340 --> 01:06:09,600
taking this minus this.

1088
01:06:09,600 --> 01:06:12,630
The point is either this 1 bit
will get borrowed when I do

1089
01:06:12,630 --> 01:06:14,730
binary subtraction, or it won't.

1090
01:06:14,730 --> 01:06:17,574
It gets borrowed when
this is bigger than this,

1091
01:06:17,574 --> 01:06:18,990
otherwise it doesn't
get borrowed.

1092
01:06:18,990 --> 01:06:22,740
So I'm going to
get either a 0 or 1

1093
01:06:22,740 --> 01:06:24,510
here, and then
some garbage which

1094
01:06:24,510 --> 01:06:30,430
I don't care about, and a 0 or
a 1 here, and then some garbage.

1095
01:06:30,430 --> 01:06:34,110
And I'll just mask that out.

1096
01:06:34,110 --> 01:06:44,760
I'm ANDing with 10000, 100000.

1097
01:06:44,760 --> 01:06:49,650
And so I end up just with
01 bits and the rest 0's.

1098
01:06:57,290 --> 01:07:00,440
And these bits,
if I get it right,

1099
01:07:00,440 --> 01:07:12,410
it's 1 if the sketch
of q is less than

1100
01:07:12,410 --> 01:07:15,250
or equal to the sketch of xi.

1101
01:07:17,870 --> 01:07:26,560
And it's 0 if the sketch of q is
greater than the sketch of xi.

1102
01:07:26,560 --> 01:07:29,150
Because when it's greater
that's when the borrow happens.

1103
01:07:29,150 --> 01:07:30,860
And then the 1 turns into a 0.

1104
01:07:30,860 --> 01:07:37,130
So 1's indicate the query is
too small or they're just right.

1105
01:07:37,130 --> 01:07:40,370
And 0's indicate
that they're greater.

1106
01:07:40,370 --> 01:07:43,220
Now the xi's were in order.

1107
01:07:43,220 --> 01:07:46,500
So probably x0 is too small.

1108
01:07:46,500 --> 01:07:48,530
And so this bit will
end up being a 0.

1109
01:07:48,530 --> 01:07:50,340
Probably xk this
plus 1 is too big.

1110
01:07:50,340 --> 01:07:51,920
So this bit will be a 1.

1111
01:07:51,920 --> 01:07:54,680
In general, it's going to be
a monotone sequence of bits.

1112
01:07:54,680 --> 01:07:58,940
If you look at these
bits, these 01 bits,

1113
01:07:58,940 --> 01:08:01,940
they are going to be monotone.

1114
01:08:01,940 --> 01:08:07,750
They'll be 0 for a while,
and then at some point

1115
01:08:07,750 --> 01:08:10,120
they'll switch to being 1's.

1116
01:08:10,120 --> 01:08:14,740
And that transition from 0 to
1 that's what we want to find.

1117
01:08:14,740 --> 01:08:16,660
These keys are too small.

1118
01:08:16,660 --> 01:08:18,020
These keys are too big.

1119
01:08:18,020 --> 01:08:21,220
This key is just right.

1120
01:08:21,220 --> 01:08:23,529
So we fit between--

1121
01:08:23,529 --> 01:08:26,890
this would be position
i and position i plus 1.

1122
01:08:26,890 --> 01:08:28,830
And we fit between
xi and xi plus 1.

1123
01:08:28,830 --> 01:08:30,899
Well, not actually
xi and xi plus 1.

1124
01:08:30,899 --> 01:08:35,014
We fit between sketch of
xi and sketch of xi plus 1.

1125
01:08:35,014 --> 01:08:36,180
That's what we need to find.

1126
01:08:36,180 --> 01:08:38,170
Now that is again the
problem of finding

1127
01:08:38,170 --> 01:08:40,810
the most significant 1 bit.

1128
01:08:40,810 --> 01:08:44,882
But in this case, I don't
need that operation.

1129
01:08:44,882 --> 01:08:46,090
I can do it in a simpler way.

1130
01:08:50,691 --> 01:08:51,899
But we're almost done, right?

1131
01:08:51,899 --> 01:08:53,690
We've done all of this
parallel comparison.

1132
01:08:53,690 --> 01:08:57,990
We just need to find that
transition between 0's and 1's.

1133
01:08:57,990 --> 01:08:59,594
Turns out there's a
cool way to do it.

1134
01:09:10,462 --> 01:09:21,600
The cool way is multiply that
word times our good friend,

1135
01:09:21,600 --> 01:09:29,612
this thing, 000001, 000001.

1136
01:09:29,612 --> 01:09:31,350
This is a little
harder to think about.

1137
01:09:31,350 --> 01:09:35,399
But take this bit string
and multiply it by this.

1138
01:09:35,399 --> 01:09:38,130
What that does is it
takes this string.

1139
01:09:38,130 --> 01:09:38,760
It includes it.

1140
01:09:38,760 --> 01:09:40,260
Because there's a 1 right there.

1141
01:09:40,260 --> 01:09:44,399
It shifts it over by one field,
and includes it, shifts it over

1142
01:09:44,399 --> 01:09:46,380
by another field, includes it.

1143
01:09:46,380 --> 01:09:48,000
So this repeats this thing.

1144
01:09:48,000 --> 01:09:50,760
And now collision happens,
because they're perfectly

1145
01:09:50,760 --> 01:09:51,720
aligned.

1146
01:09:51,720 --> 01:09:55,490
If these 1 bits ever hit each
other, they'll be summing.

1147
01:09:55,490 --> 01:09:58,440
Now, some of them are
0, some of them are 1.

1148
01:09:58,440 --> 01:10:01,200
Instead of computing
the position of the 0

1149
01:10:01,200 --> 01:10:03,300
to 1 transition, we could
equivalently just count

1150
01:10:03,300 --> 01:10:05,337
how many 1's are there.

1151
01:10:05,337 --> 01:10:06,920
I mean that's counting
from the right,

1152
01:10:06,920 --> 01:10:10,870
whereas this is counting of from
the left, whatever, same thing.

1153
01:10:10,870 --> 01:10:13,020
So if I could count how
many 1's I'd be all set.

1154
01:10:13,020 --> 01:10:21,460
And in this case, if
you look at right here,

1155
01:10:21,460 --> 01:10:25,570
this will be the
number of 1's I claim.

1156
01:10:25,570 --> 01:10:29,070
Because if this one was
there, it will stay there.

1157
01:10:29,070 --> 01:10:31,860
And then all the other bits
get shifted over and fall

1158
01:10:31,860 --> 01:10:34,380
right here on top of this bit.

1159
01:10:34,380 --> 01:10:36,690
So as they get added up,
you'll get some carries

1160
01:10:36,690 --> 01:10:38,460
and things will move over.

1161
01:10:38,460 --> 01:10:39,930
But this is not very big.

1162
01:10:39,930 --> 01:10:43,450
Because we're
talking about k bits.

1163
01:10:43,450 --> 01:10:45,400
So this is only going
to be with log k.

1164
01:10:47,811 --> 01:10:49,560
I mean there's tons
of room here before we

1165
01:10:49,560 --> 01:10:51,622
get to the next shift.

1166
01:10:51,622 --> 01:10:52,830
So I just look at these bits.

1167
01:10:52,830 --> 01:10:53,550
I mask them out.

1168
01:10:53,550 --> 01:10:54,570
I shift them over.

1169
01:10:54,570 --> 01:10:57,340
And that gives me
the number of 1's.

1170
01:10:57,340 --> 01:11:00,600
This is a cute way to count the
number of ones in a bit string

1171
01:11:00,600 --> 01:11:03,330
when the bits are
spread out nicely.

1172
01:11:03,330 --> 01:11:05,590
They have to be at least
log k away from each other.

1173
01:11:05,590 --> 01:11:06,807
Otherwise you get collision.

1174
01:11:06,807 --> 01:11:08,640
It doesn't work for an
arbitrary bit string.

1175
01:11:08,640 --> 01:11:10,590
But for a bit string
like this, we're all set.

1176
01:11:10,590 --> 01:11:12,090
We can count how
many 1's there are.

1177
01:11:12,090 --> 01:11:16,500
Then we figure out where
this transition is.

1178
01:11:16,500 --> 01:11:19,410
That is parallel comparison.

1179
01:11:19,410 --> 01:11:27,422
One more thing to do, which
is most significant set bit.

1180
01:11:33,800 --> 01:11:38,640
The place we needed this, was
we were taking the XOR of q

1181
01:11:38,640 --> 01:11:39,824
with xi.

1182
01:11:39,824 --> 01:11:41,490
And then we wanted
to find the first bit

1183
01:11:41,490 --> 01:11:42,966
where they were differing.

1184
01:11:42,966 --> 01:11:44,340
So after you take
the XOR, you've

1185
01:11:44,340 --> 01:11:47,440
got some bit string
that looks like this.

1186
01:11:47,440 --> 01:11:49,710
And you want to find
this bit, because that's

1187
01:11:49,710 --> 01:11:51,180
the place you diverged.

1188
01:11:51,180 --> 01:11:53,730
Then we would turn that to 0
and change the rest to 1's.

1189
01:11:53,730 --> 01:11:56,852
That's easy to do if we
know where this bit is.

1190
01:11:56,852 --> 01:11:58,560
And this is a generally
useful operation.

1191
01:11:58,560 --> 01:12:03,720
It's used all over computer
science, I would say.

1192
01:12:03,720 --> 01:12:06,750
So much so that most CPUs
have it as an instruction,

1193
01:12:06,750 --> 01:12:10,130
so on Intel it's called CLZ.

1194
01:12:10,130 --> 01:12:11,850
And it has many names.

1195
01:12:11,850 --> 01:12:13,290
They're in the notes.

1196
01:12:13,290 --> 01:12:15,390
Most compilers
provide this to you

1197
01:12:15,390 --> 01:12:18,120
as an operation on
architectures that have it,

1198
01:12:18,120 --> 01:12:20,035
otherwise they simulate it.

1199
01:12:20,035 --> 01:12:21,660
They probably don't
simulate it as well

1200
01:12:21,660 --> 01:12:22,972
as I'm going to tell you.

1201
01:12:22,972 --> 01:12:24,930
Because we're going to
do this in constant time

1202
01:12:24,930 --> 01:12:27,920
on a regular word RAM,
just C operations,

1203
01:12:27,920 --> 01:12:31,620
which does not seem to have
made it into popular culture.

1204
01:12:31,620 --> 01:12:36,600
It's slightly
complicated, which is why.

1205
01:12:36,600 --> 01:12:38,624
But what's cool is
we're going to use--

1206
01:12:38,624 --> 01:12:40,290
I'm going to do this
relatively quickly.

1207
01:12:40,290 --> 01:12:42,000
Because I don't
have a ton of time.

1208
01:12:42,000 --> 01:12:46,310
We're going to use all the
things that we just did again,

1209
01:12:46,310 --> 01:12:47,020
quickly.

1210
01:12:47,020 --> 01:12:49,710
Most of them just
as black boxes.

1211
01:12:49,710 --> 01:12:51,010
All right.

1212
01:12:51,010 --> 01:12:53,700
So, here's what
we're going to do.

1213
01:12:53,700 --> 01:12:56,220
Maybe I should go somewhere new.

1214
01:13:01,430 --> 01:13:03,857
So, I'm going to use sketches,
not approximate sketches,

1215
01:13:03,857 --> 01:13:04,940
but I'm going to sketches.

1216
01:13:04,940 --> 01:13:06,273
I'm going to use multiplication.

1217
01:13:06,273 --> 01:13:08,430
I'm going to use
parallel comparison.

1218
01:13:08,430 --> 01:13:10,930
And in some sense I'm going to
use most significant set bit.

1219
01:13:10,930 --> 01:13:13,010
All of these things
I'm going to use

1220
01:13:13,010 --> 01:13:15,438
to solve the most
significant set bit problem.

1221
01:13:25,410 --> 01:13:26,390
So here's what we do.

1222
01:13:26,390 --> 01:13:41,270
We split the word into root
w clusters of root w bits.

1223
01:13:41,270 --> 01:13:41,990
Sound familiar?

1224
01:13:41,990 --> 01:13:44,370
This is exactly what we
did in van Emde Boas.

1225
01:13:44,370 --> 01:13:46,070
So van Emde Boas did
this recursively.

1226
01:13:46,070 --> 01:13:47,360
We're going to do it once.

1227
01:13:47,360 --> 01:13:49,890
We can only afford
constant time.

1228
01:13:49,890 --> 01:13:51,170
So here's an example.

1229
01:13:51,170 --> 01:14:03,590
x is 0101, 0000, 1000, 1101.

1230
01:14:03,590 --> 01:14:05,660
So each of these is root w bits.

1231
01:14:05,660 --> 01:14:07,712
There's root w of them.

1232
01:14:07,712 --> 01:14:08,420
It's approximate.

1233
01:14:08,420 --> 01:14:09,710
It doesn't it to be exactly.

1234
01:14:09,710 --> 01:14:13,310
But we'll assume x is
a nice power of two,

1235
01:14:13,310 --> 01:14:14,390
so that works cleanly.

1236
01:14:18,140 --> 01:14:21,200
So the first thing, so what
the high level idea is I

1237
01:14:21,200 --> 01:14:23,090
need to find the first
non-empty cluster.

1238
01:14:23,090 --> 01:14:25,100
Here it happens to
be the first cluster.

1239
01:14:25,100 --> 01:14:29,300
And then I need to find the
first 1 bit within the cluster.

1240
01:14:29,300 --> 01:14:32,480
Hard part is finding the
first non-empty cluster.

1241
01:14:32,480 --> 01:14:34,640
Actually, the hard
part or the messy part

1242
01:14:34,640 --> 01:14:36,350
is finding which
clusters are empty

1243
01:14:36,350 --> 01:14:37,580
and which clusters are not.

1244
01:14:37,580 --> 01:14:38,690
This cluster is not empty.

1245
01:14:38,690 --> 01:14:40,130
This cluster is empty.

1246
01:14:40,130 --> 01:14:41,280
These are non-empty.

1247
01:14:41,280 --> 01:14:45,740
So I want the summary
vector which is 1011.

1248
01:14:45,740 --> 01:14:49,140
I claim if I can do that,
everything else is easy.

1249
01:14:49,140 --> 01:14:56,660
So let's spend some time on
identifying non-empty clusters.

1250
01:15:03,380 --> 01:15:11,000
First thing I do is I take x,
ANDed with this thing, which

1251
01:15:11,000 --> 01:15:25,400
I'm going to call F, 1000,
1000, 1000, 1000; F for first.

1252
01:15:25,400 --> 01:15:29,270
So I'm just seeing which
of these first bits

1253
01:15:29,270 --> 01:15:31,280
in each cluster are set.

1254
01:15:31,280 --> 01:15:42,890
So the result is I get
0000, 0000, 1000, and 1000.

1255
01:15:42,890 --> 01:15:46,190
So in particular that tells me
this cluster and this cluster

1256
01:15:46,190 --> 01:15:49,837
are non-empty, because they
have the first bit set.

1257
01:15:49,837 --> 01:15:51,170
What about all those other bits?

1258
01:15:51,170 --> 01:15:53,629
Well, the other bits I'm going
to do in a different way.

1259
01:15:53,629 --> 01:15:55,670
Just the first bits, I
need a little bit of room.

1260
01:16:01,310 --> 01:16:03,290
I need this bit of room.

1261
01:16:03,290 --> 01:16:05,790
I want to put these 1's in.

1262
01:16:05,790 --> 01:16:09,290
So I've got to get rid of some
bits to make room for that.

1263
01:16:09,290 --> 01:16:12,540
So this deals with
the first bits.

1264
01:16:12,540 --> 01:16:13,940
Now I'm going to
clear those out.

1265
01:16:13,940 --> 01:16:19,670
So I'm going to take x XOR this.

1266
01:16:19,670 --> 01:16:23,210
And that will give me everything
with the first bits cleared.

1267
01:16:23,210 --> 01:16:34,250
So I've got 0101,
0000, 0000, and 0101.

1268
01:16:34,250 --> 01:16:36,500
These are the rest of the
bits I've got to figure out.

1269
01:16:36,500 --> 01:16:39,260
This one is non-empty
and this one's non-empty.

1270
01:16:39,260 --> 01:16:40,250
How do you do it?

1271
01:16:40,250 --> 01:16:41,488
With subtraction.

1272
01:16:58,920 --> 01:17:02,550
I take F minus that thing.

1273
01:17:06,340 --> 01:17:09,800
This F has 1's, and they're
going to get borrowed.

1274
01:17:09,800 --> 01:17:12,010
When I take F minus this,
this 1 will get borrowed

1275
01:17:12,010 --> 01:17:13,540
because there's something here.

1276
01:17:13,540 --> 01:17:15,582
This one will not get
borrowed because this is 0.

1277
01:17:15,582 --> 01:17:17,623
This one will not get
borrowed because this is 0.

1278
01:17:17,623 --> 01:17:20,200
This one will get borrowed
because there's something here.

1279
01:17:20,200 --> 01:17:21,190
That's it.

1280
01:17:21,190 --> 01:17:24,520
We're comparing
with 0 everything.

1281
01:17:24,520 --> 01:17:29,050
So we're going to get, in
this case, 0 and some garbage,

1282
01:17:29,050 --> 01:17:37,210
1 and 0's, 1, and 0
with some garbage.

1283
01:17:37,210 --> 01:17:38,540
I just care about these bits.

1284
01:17:38,540 --> 01:17:41,350
These are the bits that tell
me which ones were empty.

1285
01:17:41,350 --> 01:17:42,340
The 0's are empty.

1286
01:17:42,340 --> 01:17:44,230
The 1's are non-empty.

1287
01:17:44,230 --> 01:17:46,020
So I do a mask.

1288
01:17:46,020 --> 01:17:51,415
I get 0, and some 0's, 1 and
some 0's, 1 and some 0's 0

1289
01:17:51,415 --> 01:17:53,930
and some 0's.

1290
01:17:53,930 --> 01:17:54,850
OK.

1291
01:17:54,850 --> 01:18:01,120
Then I do an XOR with F. Because
I really want 1 for these guys,

1292
01:18:01,120 --> 01:18:03,340
and 0 for these guys.

1293
01:18:03,340 --> 01:18:05,350
1 means it's not empty.

1294
01:18:05,350 --> 01:18:06,460
0 means it's empty.

1295
01:18:10,645 --> 01:18:11,650
I got that right.

1296
01:18:11,650 --> 01:18:15,110
So I'm just inverting the 0
bits to 1 bits, and vice versa.

1297
01:18:15,110 --> 01:18:17,112
So 1 means this one's not empty.

1298
01:18:17,112 --> 01:18:18,320
1 means this one's not empty.

1299
01:18:18,320 --> 01:18:20,340
Those are the non-empty guys.

1300
01:18:20,340 --> 01:18:26,760
I take this and I OR
it with this thing.

1301
01:18:29,819 --> 01:18:31,360
This was the thing
that told me which

1302
01:18:31,360 --> 01:18:33,850
ones had that first bit set.

1303
01:18:33,850 --> 01:18:37,909
So if I take the OR of those
two I learn, or any bit set.

1304
01:18:37,909 --> 01:18:40,075
Because this was dealing
with all of the other bits.

1305
01:18:48,150 --> 01:18:50,790
I threw away this bit,
but I had to remember

1306
01:18:50,790 --> 01:18:51,870
that it was non-empty.

1307
01:18:51,870 --> 01:18:52,370
OK.

1308
01:18:52,370 --> 01:18:53,160
So I take that OR.

1309
01:18:53,160 --> 01:18:55,920
Now, this tells me those
three blocks were not empty.

1310
01:18:55,920 --> 01:18:57,550
This one was empty.

1311
01:18:57,550 --> 01:19:01,440
So now here I have the
bits that I care about.

1312
01:19:01,440 --> 01:19:03,420
Sadly they're spread out.

1313
01:19:03,420 --> 01:19:05,190
I'd really like them compressed.

1314
01:19:05,190 --> 01:19:06,360
So I do that with sketch.

1315
01:19:09,130 --> 01:19:13,186
I want to compress them to 1011.

1316
01:19:13,186 --> 01:19:15,150
It would fit in one
little thing here.

1317
01:19:15,150 --> 01:19:16,180
Because this is root w.

1318
01:19:16,180 --> 01:19:17,790
There's root w of them.

1319
01:19:17,790 --> 01:19:19,740
Sadly, I can't use
approximate sketch.

1320
01:19:19,740 --> 01:19:22,170
Because I don't
have enough space.

1321
01:19:22,170 --> 01:19:24,210
This is w to the 1/2.

1322
01:19:24,210 --> 01:19:27,420
If I used approximate
sketch I get w--

1323
01:19:27,420 --> 01:19:31,410
I'd lose this factor of
4 and be bigger than w.

1324
01:19:31,410 --> 01:19:34,490
I really need it to
be perfectly sketched.

1325
01:19:34,490 --> 01:19:38,160
Conveniently, you can do
perfect sketch in this regime.

1326
01:19:38,160 --> 01:19:40,004
Before the bi's were
arbitrary things.

1327
01:19:40,004 --> 01:19:41,670
We had no idea how
they were spread out.

1328
01:19:41,670 --> 01:19:47,700
Here bi is root w minus
1-- that's the first one--

1329
01:19:47,700 --> 01:19:49,830
plus i times root w.

1330
01:19:49,830 --> 01:19:54,750
They're nicely uniformly
spaced by i root w.

1331
01:19:54,750 --> 01:19:55,560
In this case--

1332
01:19:58,308 --> 01:20:00,270
I'm running out of time--

1333
01:20:00,270 --> 01:20:09,792
I claim you can use mj equal to
w minus root w minus 1 minus j

1334
01:20:09,792 --> 01:20:12,884
root w plus j.

1335
01:20:12,884 --> 01:20:14,050
And I won't go to the proof.

1336
01:20:14,050 --> 01:20:15,930
There's a sketch in the notes.

1337
01:20:15,930 --> 01:20:19,620
If you do this, this is
a nice setting of mj.

1338
01:20:19,620 --> 01:20:22,490
It turns out you will get bi--

1339
01:20:22,490 --> 01:20:27,900
if we look at bi plus mi,
this cancels, this cancels,

1340
01:20:27,900 --> 01:20:29,160
because i equals j.

1341
01:20:29,160 --> 01:20:32,290
You're left with w plus j.

1342
01:20:32,290 --> 01:20:35,890
So in other words, if
you look at bi plus mi,

1343
01:20:35,890 --> 01:20:42,350
you get from bit w to bit
w plus root w minus 1.

1344
01:20:42,350 --> 01:20:44,530
These bits will be exactly
the bits you care about.

1345
01:20:44,530 --> 01:20:45,280
So you take those.

1346
01:20:45,280 --> 01:20:46,690
You mask out the others.

1347
01:20:46,690 --> 01:20:49,150
You shift it over to
the right, and you have

1348
01:20:49,150 --> 01:20:50,800
exactly your perfect sketch.

1349
01:20:50,800 --> 01:20:53,810
The thing you need to prove
here is that bi plus mj are all

1350
01:20:53,810 --> 01:20:54,310
distinct.

1351
01:20:54,310 --> 01:20:55,832
So there's no collisions.

1352
01:20:55,832 --> 01:20:57,790
But in this case it's
easy to avoid collisions.

1353
01:20:57,790 --> 01:21:00,070
You've got all your bits
nice and consecutive.

1354
01:21:00,070 --> 01:21:02,640
Now you've got it
down to this thing.

1355
01:21:02,640 --> 01:21:06,226
OK, not quite done though.

1356
01:21:06,226 --> 01:21:08,290
Only one more minute.

1357
01:21:08,290 --> 01:21:11,770
Let's say-- well,
that was step one.

1358
01:21:11,770 --> 01:21:14,440
Identify non-empty clusters.

1359
01:21:14,440 --> 01:21:17,770
Step two was sketch.

1360
01:21:17,770 --> 01:21:24,490
Step three is find the
first non-empty cluster.

1361
01:21:28,120 --> 01:21:31,710
I claim this is easy.

1362
01:21:31,710 --> 01:21:34,150
So I take this sketch vector.

1363
01:21:34,150 --> 01:21:37,040
It only has root w bits.

1364
01:21:37,040 --> 01:21:39,760
So I use parallel comparison.

1365
01:21:44,980 --> 01:21:46,180
What do I compare to?

1366
01:21:46,180 --> 01:21:54,110
I'm going to compare many copies
of this thing to 0001, 0010,

1367
01:21:54,110 --> 01:21:59,020
0100, 1000; the powers of 2.

1368
01:21:59,020 --> 01:22:00,160
So I take this.

1369
01:22:00,160 --> 01:22:04,300
I put them in a vector
like the sketch of a node.

1370
01:22:04,300 --> 01:22:09,760
And I take the k, or I guess
root w copies of the sketch

1371
01:22:09,760 --> 01:22:11,030
of the summary vector.

1372
01:22:11,030 --> 01:22:12,700
That's this 1011.

1373
01:22:12,700 --> 01:22:16,210
So I compare four copies
of this to each of these,

1374
01:22:16,210 --> 01:22:19,150
and I learn which power
of 2 it is greater than.

1375
01:22:19,150 --> 01:22:22,600
In other words, what is the
most significant set bit.

1376
01:22:22,600 --> 01:22:27,252
That's why when I told you
how to do over here, when

1377
01:22:27,252 --> 01:22:28,960
I told you how to do
parallel comparison,

1378
01:22:28,960 --> 01:22:31,084
I didn't want to use
most significant bit

1379
01:22:31,084 --> 01:22:31,750
as a subroutine.

1380
01:22:31,750 --> 01:22:35,110
Because this is a subroutine
to most significant bit.

1381
01:22:35,110 --> 01:22:37,937
Over here, we could just do
this multiplication and boom,

1382
01:22:37,937 --> 01:22:39,520
we found what the
most significant set

1383
01:22:39,520 --> 01:22:41,680
bit was as long as
there was room to fit

1384
01:22:41,680 --> 01:22:43,090
all this stuff in a word.

1385
01:22:43,090 --> 01:22:45,720
And because I've reduced
everything to size root w,

1386
01:22:45,720 --> 01:22:48,314
and then only there's
w of these things

1387
01:22:48,314 --> 01:22:50,230
to compare to, because
that's the width of one

1388
01:22:50,230 --> 01:22:51,490
of these fields.

1389
01:22:51,490 --> 01:22:52,534
This all fits in a word.

1390
01:22:52,534 --> 01:22:53,950
I can do this
parallel comparison.

1391
01:22:53,950 --> 01:22:57,580
Boom, I find the first 1 bit
in this bit string, which

1392
01:22:57,580 --> 01:22:59,710
happens to be the first bit.

1393
01:22:59,710 --> 01:23:04,820
That tells me that this cluster
is a cluster I care about.

1394
01:23:04,820 --> 01:23:06,190
So I take those bits out.

1395
01:23:06,190 --> 01:23:08,410
I mask them out,
shift them over,

1396
01:23:08,410 --> 01:23:15,030
and I find the first
1 bit in that cluster.

1397
01:23:15,030 --> 01:23:16,240
How do I do it?

1398
01:23:16,240 --> 01:23:19,480
In exactly the same way,
clusters again, root w bits.

1399
01:23:19,480 --> 01:23:21,310
I can use parallel
comparison to compare it

1400
01:23:21,310 --> 01:23:23,680
to all these things
in constant time.

1401
01:23:23,680 --> 01:23:26,140
I find where the
first 1 bit is there.

1402
01:23:26,140 --> 01:23:33,700
And then I take this cluster
C, I take this bit D,

1403
01:23:33,700 --> 01:23:40,150
and my answer is
C root w plus D.

1404
01:23:40,150 --> 01:23:43,840
That is the final index of
the most significant 1 bit

1405
01:23:43,840 --> 01:23:47,851
in constant time, using all
those fusion tricks once again.

1406
01:23:47,851 --> 01:23:49,600
And that in the end
gives you fusion trees

1407
01:23:49,600 --> 01:23:51,130
on a word RAM static.

1408
01:23:54,030 --> 01:24:00,070
It's complicated, probably
impractical, but pretty cool.

1409
01:24:00,070 --> 01:24:03,540
And we're going to use
these bit tricks again.