1
00:00:00,090 --> 00:00:02,490
The following content is
provided under a Creative

2
00:00:02,490 --> 00:00:04,030
Commons license.

3
00:00:04,030 --> 00:00:06,360
Your support will help
MIT OpenCourseWare

4
00:00:06,360 --> 00:00:10,720
continue to offer high quality
educational resources for free.

5
00:00:10,720 --> 00:00:13,320
To make a donation or
view additional materials

6
00:00:13,320 --> 00:00:17,280
from hundreds of MIT courses,
visit MIT OpenCourseWare

7
00:00:17,280 --> 00:00:18,450
at ocw.mit.edu.

8
00:00:21,364 --> 00:00:23,530
ERIK DEMAINE: All right,
today we continue our theme

9
00:00:23,530 --> 00:00:25,480
of integer data structures.

10
00:00:25,480 --> 00:00:28,060
And today we're going to look
at priority queues instead

11
00:00:28,060 --> 00:00:30,850
of the predecessor problem, so
something a little different.

12
00:00:30,850 --> 00:00:33,980
Insert, delete, and find min.

13
00:00:33,980 --> 00:00:38,140
It turns out priority queues
are equivalent to sorting

14
00:00:38,140 --> 00:00:54,022
in the sense that if you have,
normally they say f of n,

15
00:00:54,022 --> 00:00:56,230
but here we're going to
depend on both n and the word

16
00:00:56,230 --> 00:00:59,170
size, priority queue.

17
00:01:02,830 --> 00:01:05,530
Well then, of course you
get a sorting algorithm.

18
00:01:05,530 --> 00:01:08,420
Just insert everything
into the priority queue

19
00:01:08,420 --> 00:01:11,560
and then find min delete,
find min delete, and so on.

20
00:01:15,350 --> 00:01:20,080
So that's the trivial
direction of the equivalence.

21
00:01:20,080 --> 00:01:23,550
But it turns out, both
of these are true.

22
00:01:23,550 --> 00:01:28,060
If you have a sorting
algorithm with n times f time,

23
00:01:28,060 --> 00:01:30,235
then you can convert it
into a priority queue

24
00:01:30,235 --> 00:01:37,280
that in the worst case every
operation takes only f of n, w.

25
00:01:37,280 --> 00:01:41,550
So that's the equivalence
as the result of Thorpe

26
00:01:41,550 --> 00:01:44,890
from a few years ago, 2007.

27
00:01:44,890 --> 00:01:46,180
Kind of a cool breakthrough.

28
00:01:46,180 --> 00:01:48,730
There were earlier versions
that weren't quite as efficient

29
00:01:48,730 --> 00:01:51,700
but this gives you perfect
efficiency, turning any sorting

30
00:01:51,700 --> 00:01:53,050
algorithm into priority queue.

31
00:01:53,050 --> 00:01:54,610
Rough sketch of
the data structure

32
00:01:54,610 --> 00:01:57,790
is you have a bunch of sorting
algorithms which you run now

33
00:01:57,790 --> 00:02:00,730
and then to get little
parts of the input

34
00:02:00,730 --> 00:02:02,810
in order to build an
overall priority queue.

35
00:02:02,810 --> 00:02:04,990
But it's very complicated.

36
00:02:04,990 --> 00:02:06,490
There was an entire
class project

37
00:02:06,490 --> 00:02:09,699
on assimilating it
and understanding it.

38
00:02:09,699 --> 00:02:13,010
It's a bit beyond what we
can cover in the class.

39
00:02:13,010 --> 00:02:14,770
So I'm going to
take this as given

40
00:02:14,770 --> 00:02:16,812
and tell you instead
about sorting algorithms

41
00:02:16,812 --> 00:02:19,270
because there's a major open
problem here which is, can you

42
00:02:19,270 --> 00:02:23,470
sort n integers, n w bit
integers in a word ram's,

43
00:02:23,470 --> 00:02:25,750
word size w in linear time.

44
00:02:25,750 --> 00:02:29,524
We still don't know but we
have a bunch of results.

45
00:02:29,524 --> 00:02:30,940
In particular,
we're going to talk

46
00:02:30,940 --> 00:02:33,460
about a couple
interesting cases where

47
00:02:33,460 --> 00:02:36,070
we can get linear time sorting.

48
00:02:36,070 --> 00:02:38,830
If you've seen radix sort you
know another interesting case

49
00:02:38,830 --> 00:02:40,495
where linear time
sorting is possible.

50
00:02:44,410 --> 00:02:48,560
Interesting open problem here.

51
00:02:48,560 --> 00:02:53,620
Can you get constant
time decrease key?

52
00:02:53,620 --> 00:02:56,150
And ideally, also insertion?

53
00:02:56,150 --> 00:02:58,594
But decrease key would be
handy because then this

54
00:02:58,594 --> 00:02:59,260
would be useful.

55
00:03:02,530 --> 00:03:04,630
In this reduction if you
have a sorting algorithm,

56
00:03:04,630 --> 00:03:06,550
like to get a priority queue
with constant time decrease

57
00:03:06,550 --> 00:03:07,750
key, that would
have applications

58
00:03:07,750 --> 00:03:08,750
to Dijkstra's algorithm.

59
00:03:08,750 --> 00:03:10,960
Of course, if you can
sort in linear time

60
00:03:10,960 --> 00:03:13,130
then everything can be
done in constant time.

61
00:03:13,130 --> 00:03:15,430
So this is not the most
interesting open problem

62
00:03:15,430 --> 00:03:19,400
but it would be a step
in that direction.

63
00:03:19,400 --> 00:03:19,900
Cool.

64
00:03:19,900 --> 00:03:26,365
So let me tell you some
bounds about sorting.

65
00:03:32,482 --> 00:03:38,556
So we have n w bit integers.

66
00:03:38,556 --> 00:03:40,180
And you take any of
these bounds divide

67
00:03:40,180 --> 00:03:46,250
by n you get the priority queue
bound, the best that we know.

68
00:03:46,250 --> 00:03:52,690
So of course, there's comparison
sorting, which takes n log n.

69
00:03:52,690 --> 00:03:53,950
So that's an upper bound.

70
00:03:53,950 --> 00:03:55,070
Trying to beat that.

71
00:03:55,070 --> 00:03:57,610
Trying to get rid of the log.

72
00:03:57,610 --> 00:04:01,300
There's a counting sort from
undergraduate algorithms.

73
00:04:01,300 --> 00:04:04,320
That takes n plus u
time because you need

74
00:04:04,320 --> 00:04:06,040
to build a table of size u.

75
00:04:06,040 --> 00:04:08,215
You've got to zero out
all the initial counts

76
00:04:08,215 --> 00:04:11,320
so that requires order u time.

77
00:04:11,320 --> 00:04:20,470
This is interesting when
the word size equals log n.

78
00:04:20,470 --> 00:04:23,120
So that's a case of
potential interest,

79
00:04:23,120 --> 00:04:24,740
because then 2 to
the w is just n.

80
00:04:24,740 --> 00:04:26,760
So this is linear time.

81
00:04:26,760 --> 00:04:30,880
But even if w is 2 log n,
radix or 1.1 times log n,

82
00:04:30,880 --> 00:04:33,250
then this is bigger n log n.

83
00:04:33,250 --> 00:04:34,820
So, not very good.

84
00:04:34,820 --> 00:04:41,110
But we have radix sort,
which is a bit better.

85
00:04:41,110 --> 00:04:49,310
And this is n times
w divided by log n.

86
00:04:49,310 --> 00:04:53,320
If you do it right you
have some word size w,

87
00:04:53,320 --> 00:04:57,190
you're using counting sort
for each digit in the radix.

88
00:04:57,190 --> 00:05:01,780
So you can afford that the word
size of doing counting sort

89
00:05:01,780 --> 00:05:03,010
to be exactly log n.

90
00:05:03,010 --> 00:05:05,260
Which means you have your
overall word you're dividing

91
00:05:05,260 --> 00:05:07,210
into chunks of size log n.

92
00:05:07,210 --> 00:05:09,140
Each of them you can
do in linear time.

93
00:05:09,140 --> 00:05:12,520
So the number of chunks you have
to do is w divided by log n.

94
00:05:12,520 --> 00:05:18,040
And this is linear if
w equals order log n.

95
00:05:22,490 --> 00:05:24,010
Right, but only then.

96
00:05:24,010 --> 00:05:28,882
So this buys you a little
bit, but still not great.

97
00:05:28,882 --> 00:05:30,590
OK, there's some other
sorting algorithms

98
00:05:30,590 --> 00:05:33,860
we've implicitly seen because
we've seen Van Emde Boas.

99
00:05:33,860 --> 00:05:36,010
I mean, any predecessor
data structure

100
00:05:36,010 --> 00:05:37,760
gives you a priority
queue data structure.

101
00:05:37,760 --> 00:05:42,530
Priority queues-- I didn't
write it, but I mentioned--

102
00:05:42,530 --> 00:05:45,710
insert, delete, find min.

103
00:05:45,710 --> 00:05:50,600
Find min is a weaker operation
than predecessor or successor.

104
00:05:50,600 --> 00:05:57,350
So we can take Van Emde Boas's
predecessor data structure

105
00:05:57,350 --> 00:06:01,310
and convert it into a
sorting algorithm and we

106
00:06:01,310 --> 00:06:07,160
get order n times log w.

107
00:06:07,160 --> 00:06:10,130
Now this is a little
bit tricky to compare.

108
00:06:10,130 --> 00:06:12,560
Of course, log w
sounds better than w.

109
00:06:12,560 --> 00:06:15,110
But there's just divided
by log n, which makes

110
00:06:15,110 --> 00:06:18,620
it a little hard to compare.

111
00:06:18,620 --> 00:06:25,160
It turns out there's
another version.

112
00:06:25,160 --> 00:06:28,520
You can take this
sorting algorithm

113
00:06:28,520 --> 00:06:33,520
and achieve a bound
of m log w over log n.

114
00:06:36,680 --> 00:06:39,500
OK, so this is a slight
improvement in here.

115
00:06:39,500 --> 00:06:42,200
It's similar to your
problem set this week.

116
00:06:42,200 --> 00:06:43,850
This specific
problem was a problem

117
00:06:43,850 --> 00:06:47,370
set in 2005 of this class.

118
00:06:47,370 --> 00:06:48,920
So you can check
it out if you want

119
00:06:48,920 --> 00:06:50,900
to see how to do this
slight improvement.

120
00:06:50,900 --> 00:06:52,814
But now this is clearly
better than this

121
00:06:52,814 --> 00:06:54,230
because here we
have w over log n.

122
00:06:54,230 --> 00:06:56,070
Here I've got a log in front.

123
00:06:56,070 --> 00:06:58,540
So this is a win
over radex sort.

124
00:06:58,540 --> 00:07:01,820
So Van Emde Boas, here's another
way in which is interesting.

125
00:07:01,820 --> 00:07:05,830
It gives us a strict
improvement over radex sort.

126
00:07:05,830 --> 00:07:14,020
And in particular,
actually both of these

127
00:07:14,020 --> 00:07:22,310
are order n log log n if w
is not too big, poly log n.

128
00:07:26,090 --> 00:07:28,130
This doesn't actually
matter but it's

129
00:07:28,130 --> 00:07:30,710
clearly better than this one.

130
00:07:30,710 --> 00:07:34,490
Again, if w is order log n
you get this to be linear.

131
00:07:34,490 --> 00:07:38,600
But in particular, it says
overall you get n log log n up

132
00:07:38,600 --> 00:07:41,300
to w being poly log.

133
00:07:41,300 --> 00:07:44,450
OK, why do we care?

134
00:07:44,450 --> 00:07:46,670
Because there's this
other nice algorithm,

135
00:07:46,670 --> 00:07:51,260
signature sort, which
covers all large word sizes.

136
00:07:51,260 --> 00:07:53,570
And this is an algorithm
we will cover today.

137
00:07:58,370 --> 00:08:01,520
It only works when w is
large, and large means

138
00:08:01,520 --> 00:08:04,850
log to the 2 plus epsilon.

139
00:08:04,850 --> 00:08:07,740
And this is no matter
what epsilon is.

140
00:08:07,740 --> 00:08:10,250
In reality, it's probably
log squared times

141
00:08:10,250 --> 00:08:12,140
log log n squared.

142
00:08:12,140 --> 00:08:14,750
But be a little
sloppy and just say,

143
00:08:14,750 --> 00:08:18,410
a little bit bigger
than log squared.

144
00:08:18,410 --> 00:08:20,640
And then it runs in linear time.

145
00:08:20,640 --> 00:08:25,640
OK, so if you were graphing this
in your mind you've got time

146
00:08:25,640 --> 00:08:28,430
and there's a nice bound
here which is order n.

147
00:08:28,430 --> 00:08:32,960
Then we have everything's
above n log n.

148
00:08:32,960 --> 00:08:35,200
Maybe here we have--

149
00:08:35,200 --> 00:08:36,260
is hard to draw--

150
00:08:36,260 --> 00:08:39,590
n log log n.

151
00:08:39,590 --> 00:08:42,630
This is asymptotic space,
which doesn't really exist,

152
00:08:42,630 --> 00:08:44,330
but you know what I mean.

153
00:08:44,330 --> 00:08:49,475
And then this is the
word size versus log n.

154
00:08:49,475 --> 00:08:50,420
Wow, this is hard to--

155
00:08:50,420 --> 00:08:51,800
this is a really weird picture.

156
00:08:51,800 --> 00:08:53,840
I've never drawn this before.

157
00:08:53,840 --> 00:08:58,080
So you know, initially in
a radex sort, OK log n.

158
00:08:58,080 --> 00:09:04,125
We could get linear time
or order log n, great.

159
00:09:04,125 --> 00:09:05,750
What this is saying
is once you get out

160
00:09:05,750 --> 00:09:10,520
to log the 2 plus epsilon
n you can also do linear.

161
00:09:10,520 --> 00:09:13,180
And from here over
we have linear time.

162
00:09:13,180 --> 00:09:15,840
And here over we
have linear time.

163
00:09:15,840 --> 00:09:18,635
So the question is what about
this middle range between log

164
00:09:18,635 --> 00:09:22,190
n and log to the 2 plus epsilon?

165
00:09:22,190 --> 00:09:27,530
Well, one thing we could do
is use Van Emde Boas sort,

166
00:09:27,530 --> 00:09:29,764
get an n log log n bound.

167
00:09:29,764 --> 00:09:32,180
And so that's one way to fill
in this range in the middle.

168
00:09:32,180 --> 00:09:35,570
And that means overall you can
always sort n integers in n log

169
00:09:35,570 --> 00:09:36,920
log n time.

170
00:09:36,920 --> 00:09:39,620
If w is big you
use signature sort,

171
00:09:39,620 --> 00:09:42,170
if w small use Van
Emde Boas sort.

172
00:09:42,170 --> 00:09:49,430
So this implies n
log log n for all w.

173
00:09:49,430 --> 00:09:51,080
So that's something.

174
00:09:51,080 --> 00:09:53,724
It's not the best known.

175
00:09:53,724 --> 00:10:00,680
Let's see, first result in this
direction was by Han in 2001.

176
00:10:00,680 --> 00:10:10,370
And it achieved this bound,
deterministic and in the ac 0

177
00:10:10,370 --> 00:10:14,390
ram, so without
using multiplication.

178
00:10:14,390 --> 00:10:17,310
Which, I guess Van Emde
Boas doesn't either,

179
00:10:17,310 --> 00:10:19,760
unless I'm missing something.

180
00:10:19,760 --> 00:10:21,750
But it got rid of
the randomisation

181
00:10:21,750 --> 00:10:25,800
which is necessary to get
linear space in Van Emde Boas.

182
00:10:25,800 --> 00:10:30,258
The current best
result on sorting

183
00:10:30,258 --> 00:10:34,330
is by Han and Thorpe in
2002, so a while ago.

184
00:10:34,330 --> 00:10:42,150
They achieved n square
root log w over log n,

185
00:10:42,150 --> 00:10:46,560
which is similar
to this bound, just

186
00:10:46,560 --> 00:10:50,200
has an extra square root here.

187
00:10:50,200 --> 00:10:52,680
So it's a little bit better
than Van Emde Boas sort.

188
00:10:52,680 --> 00:10:53,850
It uses some clever tricks.

189
00:10:53,850 --> 00:10:54,910
It is randomized.

190
00:10:58,780 --> 00:11:03,840
So still the best deterministic
bound is n log log n,

191
00:11:03,840 --> 00:11:09,790
And let's see, we can plug
that in for this range here.

192
00:11:09,790 --> 00:11:16,050
And it will imply that
you get n square root log

193
00:11:16,050 --> 00:11:20,130
log n for all w.

194
00:11:20,130 --> 00:11:26,340
Because in particular,
if w is poly log n,

195
00:11:26,340 --> 00:11:28,560
then this is just log log n.

196
00:11:28,560 --> 00:11:31,780
So you get this bound.

197
00:11:31,780 --> 00:11:35,050
Of course, for all w because
large w are already covered.

198
00:11:35,050 --> 00:11:39,176
So this sorting algorithm is a
little bit messy, and possibly

199
00:11:39,176 --> 00:11:40,050
not the right answer.

200
00:11:40,050 --> 00:11:42,540
I would conjecture you
can sort n integers

201
00:11:42,540 --> 00:11:45,450
in linear time for all w.

202
00:11:45,450 --> 00:11:46,815
But that is a big open problem.

203
00:11:52,590 --> 00:11:55,050
This is the current
state of the art.

204
00:11:55,050 --> 00:11:59,300
So this open question is
really about for small w

205
00:11:59,300 --> 00:12:00,870
when w is poly
log n, or actually

206
00:12:00,870 --> 00:12:03,540
when it's between
log and poly log.

207
00:12:03,540 --> 00:12:06,330
So to give you a piece
of the complete picture

208
00:12:06,330 --> 00:12:08,934
I want to tell you
about signature sort.

209
00:12:08,934 --> 00:12:11,100
Because I think you have
to use different techniques

210
00:12:11,100 --> 00:12:12,390
when w is small versus large.

211
00:12:12,390 --> 00:12:15,540
We know that from fusion
trees and Van Emde Boas.

212
00:12:15,540 --> 00:12:18,480
So I'll tell you about the
slightly large case when

213
00:12:18,480 --> 00:12:21,090
w is at least log to
the 2 plus epsilon,

214
00:12:21,090 --> 00:12:22,600
how to get linear time.

215
00:12:22,600 --> 00:12:23,710
And this is fun.

216
00:12:23,710 --> 00:12:25,950
It uses bit tricks, it
uses a lot of cool ideas,

217
00:12:25,950 --> 00:12:28,470
uses hashing, lots of things.

218
00:12:28,470 --> 00:12:32,820
It also uses another sorting
algorithm called packed sort.

219
00:12:32,820 --> 00:12:35,310
This doesn't fit in
the regular table

220
00:12:35,310 --> 00:12:38,700
because it's for a situation
when more than one input

221
00:12:38,700 --> 00:12:40,780
number fits in a word.

222
00:12:40,780 --> 00:12:43,290
So we're going to let b
denote the number of bits

223
00:12:43,290 --> 00:12:44,340
in the inputs.

224
00:12:44,340 --> 00:12:46,560
So we have b bit integers.

225
00:12:46,560 --> 00:12:50,460
Then when w is at least
b times log n log log n,

226
00:12:50,460 --> 00:12:54,540
so you can fit log n log log
n integers in a single word,

227
00:12:54,540 --> 00:12:56,982
then you can sort
them in linear time.

228
00:12:56,982 --> 00:12:59,190
So it's like your machine
is just a little bit bigger

229
00:12:59,190 --> 00:13:01,850
than what you'd need to
represent the integers,

230
00:13:01,850 --> 00:13:04,890
then all sorting can be
done in a linear time.

231
00:13:04,890 --> 00:13:08,580
And as you might imagine,
this log to the 2 plus epsilon

232
00:13:08,580 --> 00:13:12,660
buys you a little bit of
room with some tricks,

233
00:13:12,660 --> 00:13:15,350
namely hashing.

234
00:13:15,350 --> 00:13:19,140
Right, that's where we're going.

235
00:13:19,140 --> 00:13:21,187
And we'll use yet another
sorting algorithm,

236
00:13:21,187 --> 00:13:22,770
bitonic sort, which
you may have seen.

237
00:13:22,770 --> 00:13:25,840
It's a sorting network.

238
00:13:25,840 --> 00:13:27,840
It's in CLRS up
to second edition,

239
00:13:27,840 --> 00:13:29,617
but it got removed
from third edition.

240
00:13:29,617 --> 00:13:31,950
I guess they thought people
didn't use sorting networks,

241
00:13:31,950 --> 00:13:33,360
but we use sorting
networks here.

242
00:13:36,270 --> 00:13:39,690
Because it gives us parallel
sorting and the word ram

243
00:13:39,690 --> 00:13:43,290
is all about using the
parallelism in your word.

244
00:13:43,290 --> 00:13:48,420
So, we'll get to those.

245
00:13:48,420 --> 00:13:53,300
I'm going to start at the top
level, which is signature sort.

246
00:13:53,300 --> 00:13:55,050
And we're going to
leave things to be done

247
00:13:55,050 --> 00:13:57,420
and then pack sorting
will be one of them.

248
00:13:57,420 --> 00:13:59,670
And bitonic sorting will
be used in pack sorting.

249
00:13:59,670 --> 00:14:03,830
So it's a chain like this.

250
00:14:03,830 --> 00:14:05,580
Cool.

251
00:14:05,580 --> 00:14:10,770
So I'm going to
assume my signature

252
00:14:10,770 --> 00:14:15,990
sort that w is at least
log to 2 pulse epsilon n

253
00:14:15,990 --> 00:14:18,640
times log log n.

254
00:14:18,640 --> 00:14:23,407
This is really just a change
in what epsilon means.

255
00:14:23,407 --> 00:14:25,240
If you're given some
particular epsilon make

256
00:14:25,240 --> 00:14:26,906
it a little smaller,
then you can afford

257
00:14:26,906 --> 00:14:28,560
to add that log log factor.

258
00:14:28,560 --> 00:14:32,260
This is just for convenience.

259
00:14:32,260 --> 00:14:37,927
Now here's the idea.

260
00:14:37,927 --> 00:14:39,510
It's a pretty cool
idea I have to say.

261
00:14:39,510 --> 00:14:59,250
It's by Anderson
and others in 1998.

262
00:14:59,250 --> 00:15:01,896
OK, word size is big,
which is both good and bad.

263
00:15:01,896 --> 00:15:03,770
It means the numbers we
have to sort are big.

264
00:15:03,770 --> 00:15:07,600
They have a lot of bits in
them, way more than log n bits.

265
00:15:07,600 --> 00:15:10,090
But also, we have
this powerful machine

266
00:15:10,090 --> 00:15:13,794
that can handle words
of very large size

267
00:15:13,794 --> 00:15:15,460
and do things and
constant time on them.

268
00:15:19,090 --> 00:15:21,160
I'd like to make some slop.

269
00:15:21,160 --> 00:15:23,320
I want my words to be
bigger than my items,

270
00:15:23,320 --> 00:15:26,410
so first few steps
are about doing that.

271
00:15:26,410 --> 00:15:30,340
The first idea is, well,
you've got this giant word

272
00:15:30,340 --> 00:15:32,020
log to the 2 plus epsilon.

273
00:15:32,020 --> 00:15:34,840
So let's split it into chunks.

274
00:15:34,840 --> 00:15:40,180
And the number of chunks
is log to the epsilon.

275
00:15:40,180 --> 00:15:44,200
So that means each chunk is--

276
00:15:44,200 --> 00:15:45,710
well, it could be even bigger.

277
00:15:45,710 --> 00:15:48,020
So it's at least log squared n.

278
00:15:48,020 --> 00:15:51,510
But it could be bigger
than that because w

279
00:15:51,510 --> 00:15:53,572
is at least log to
the 2 plus epsilon.

280
00:15:53,572 --> 00:15:54,280
So we don't know.

281
00:15:54,280 --> 00:15:57,330
These things are of size w
divided by log to the epsilon

282
00:15:57,330 --> 00:15:57,830
n.

283
00:15:57,830 --> 00:16:00,470
So it's at least log squared.

284
00:16:00,470 --> 00:16:04,970
OK, here's the crazy idea.

285
00:16:04,970 --> 00:16:07,480
We're thinking of our
number being divided

286
00:16:07,480 --> 00:16:10,180
into these chunks, you
know, it's being represented

287
00:16:10,180 --> 00:16:15,070
in some base, 2 to the w
divided by log to the epsilon n.

288
00:16:15,070 --> 00:16:17,510
It's a bit of a mouthful.

289
00:16:17,510 --> 00:16:20,840
But think about these digits.

290
00:16:20,840 --> 00:16:27,040
I claim if you look at a digit,
most values don't appear.

291
00:16:27,040 --> 00:16:29,920
Because how many values
are there for a digit?

292
00:16:29,920 --> 00:16:33,310
Well, it's at least 2
to the log squared n.

293
00:16:33,310 --> 00:16:34,220
That's a big number.

294
00:16:34,220 --> 00:16:36,610
That's way bigger
than polynomial and n.

295
00:16:36,610 --> 00:16:39,790
But how many different
digits are there?

296
00:16:39,790 --> 00:16:41,860
Well, there's n
numbers, each of them

297
00:16:41,860 --> 00:16:47,350
has log to the epsilon n
digits in it appearing.

298
00:16:47,350 --> 00:16:52,870
And so there's roughly a linear
number of digit values total.

299
00:16:52,870 --> 00:16:54,850
But the number of
possible digit values

300
00:16:54,850 --> 00:16:57,010
is 2 to the log log
squared n, which

301
00:16:57,010 --> 00:17:00,580
is huge, super polynomial.

302
00:17:00,580 --> 00:17:05,030
So most digit values don't
appear in our numbers.

303
00:17:05,030 --> 00:17:07,359
So what we're going to
do is hash the digit

304
00:17:07,359 --> 00:17:09,177
values to reduce their space.

305
00:17:09,177 --> 00:17:11,760
Because they're currently living
in a universe of 2 to the log

306
00:17:11,760 --> 00:17:14,444
squared n, but there's
only n times log

307
00:17:14,444 --> 00:17:16,089
to the epsilon n of them.

308
00:17:16,089 --> 00:17:23,200
So we should be able to reduce
to have a much smaller universe

309
00:17:23,200 --> 00:17:24,970
using hashing.

310
00:17:24,970 --> 00:17:30,669
Maybe perfect hashing even.

311
00:17:30,669 --> 00:17:32,710
I mean, we don't have to
be too intelligent here.

312
00:17:32,710 --> 00:17:33,585
Tell you what we get.

313
00:17:41,980 --> 00:17:46,120
What I'd like is to reduce
to a polynomial universe.

314
00:17:46,120 --> 00:17:53,380
And another way to say that is
that the hash values are order

315
00:17:53,380 --> 00:17:54,240
log n bits.

316
00:17:54,240 --> 00:17:57,140
So this lets us go to n
to the 10, or whatever,

317
00:17:57,140 --> 00:17:58,360
which is plenty of room.

318
00:17:58,360 --> 00:18:03,050
We take any universal
hash function,

319
00:18:03,050 --> 00:18:06,250
as soon as you get to
roughly n squared size,

320
00:18:06,250 --> 00:18:08,380
I mean the square of
the number of things,

321
00:18:08,380 --> 00:18:10,750
by a birthday paradox, constant
probability none of them

322
00:18:10,750 --> 00:18:12,070
will hit each other.

323
00:18:12,070 --> 00:18:14,320
That was the second
level of perfect hashing.

324
00:18:14,320 --> 00:18:17,140
We don't need two levels
here, just say OK, user order

325
00:18:17,140 --> 00:18:17,760
log n bits.

326
00:18:17,760 --> 00:18:20,320
You can set this
to 100, whatever.

327
00:18:20,320 --> 00:18:23,230
Then with high probability
you will not get collisions,

328
00:18:23,230 --> 00:18:25,300
so none of the chunks
will hit each other.

329
00:18:25,300 --> 00:18:27,880
So I'm omitting a step
here which is how to--

330
00:18:27,880 --> 00:18:31,370
OK, I'm omitting a
couple of things here.

331
00:18:31,370 --> 00:18:36,130
One of them is I want to
do this in linear time.

332
00:18:36,130 --> 00:18:37,967
Everything has to
be in linear time.

333
00:18:37,967 --> 00:18:39,550
So one front part
is I have to compute

334
00:18:39,550 --> 00:18:43,210
the hash value of
all of these digits

335
00:18:43,210 --> 00:18:47,220
in a single word
in constant time.

336
00:18:47,220 --> 00:18:51,550
So a brief aside, how would
I compute a universal hash

337
00:18:51,550 --> 00:18:55,240
function of these digits
individually in constant time?

338
00:18:55,240 --> 00:19:02,230
Well, my personal favorite
universal hashing scheme

339
00:19:02,230 --> 00:19:03,940
is the multiplication method.

340
00:19:03,940 --> 00:19:08,240
You just need to multiply
by a single value m

341
00:19:08,240 --> 00:19:11,210
and then take it
modular something.

342
00:19:11,210 --> 00:19:20,690
So if I do
multiplication, I mean

343
00:19:20,690 --> 00:19:22,490
if you think of there
being something here,

344
00:19:22,490 --> 00:19:25,475
something here, something here,
something here, something here,

345
00:19:25,475 --> 00:19:30,481
m does multiply each
of these individually.

346
00:19:34,170 --> 00:19:36,200
Wow, that's a rough sketch.

347
00:19:36,200 --> 00:19:38,240
I want to hash each of
these guys multiply,

348
00:19:38,240 --> 00:19:40,220
them by the
multiplication method.

349
00:19:40,220 --> 00:19:42,800
This would work fine,
except you get overflows,

350
00:19:42,800 --> 00:19:44,340
which is a bit messy.

351
00:19:44,340 --> 00:19:47,210
So in fact, I'm going to have
to do it for the odd guys

352
00:19:47,210 --> 00:19:50,390
first, multiply them out, mask
out the bits I care about,

353
00:19:50,390 --> 00:19:53,600
then do it for the
even guys separately.

354
00:19:53,600 --> 00:19:54,470
So it's a bit messy.

355
00:19:54,470 --> 00:19:58,700
But hand-wavy proof you can
compute the hash function

356
00:19:58,700 --> 00:20:00,230
of each of these
guys individually,

357
00:20:00,230 --> 00:20:02,100
but in constant time.

358
00:20:02,100 --> 00:20:03,260
So that lets me do this.

359
00:20:03,260 --> 00:20:04,640
You have to then
check for collisions,

360
00:20:04,640 --> 00:20:05,723
which is kind of annoying.

361
00:20:05,723 --> 00:20:06,900
Let's not worry about that.

362
00:20:06,900 --> 00:20:08,750
Set this really high,
very high probability,

363
00:20:08,750 --> 00:20:11,900
there will be no collisions.

364
00:20:11,900 --> 00:20:13,640
Cool.

365
00:20:13,640 --> 00:20:17,319
So at this point we have
mucked up all the digits.

366
00:20:17,319 --> 00:20:18,860
The annoying thing
about this hashing

367
00:20:18,860 --> 00:20:20,450
is we do not preserve order.

368
00:20:24,860 --> 00:20:26,840
It's a little weird for
a sorting algorithm not

369
00:20:26,840 --> 00:20:28,670
preserve order.

370
00:20:28,670 --> 00:20:31,850
The whole point is to
put things in order.

371
00:20:31,850 --> 00:20:34,370
But it's going to be
useful nonetheless.

372
00:20:34,370 --> 00:20:36,740
And so each digit
has been mangled.

373
00:20:36,740 --> 00:20:38,570
Each digit is out of order.

374
00:20:38,570 --> 00:20:40,910
But of course, the
sequence of digits

375
00:20:40,910 --> 00:20:45,800
has been preserved,
in some sense

376
00:20:45,800 --> 00:20:47,730
So, what?

377
00:20:47,730 --> 00:20:50,300
Well, at this point
we can use packed

378
00:20:50,300 --> 00:20:52,640
sorting to sort what remains.

379
00:21:05,836 --> 00:21:08,670
In general packed
sorting can sort n

380
00:21:08,670 --> 00:21:13,680
b bit integers provided
b is small, so w

381
00:21:13,680 --> 00:21:18,500
is at least b log n log log n.

382
00:21:18,500 --> 00:21:21,900
That's the theorem
of packed sorting.

383
00:21:21,900 --> 00:21:25,080
We're going to delay the
coverage of this result

384
00:21:25,080 --> 00:21:26,330
until later.

385
00:21:26,330 --> 00:21:36,830
And I claim that that applies
here because each chunk is now

386
00:21:36,830 --> 00:21:39,290
order log n bits.

387
00:21:39,290 --> 00:21:43,200
And there's log to
the epsilon n bits.

388
00:21:43,200 --> 00:21:51,320
So b is theta log to
the 1 plus epsilon n.

389
00:21:51,320 --> 00:22:01,890
And w is at least log to the 2
plus epsilon times log log n.

390
00:22:01,890 --> 00:22:04,670
OK, so the ratio between
these is as needed.

391
00:22:04,670 --> 00:22:06,844
It's at least log
n times log log n.

392
00:22:09,450 --> 00:22:11,220
OK, so that's why this is good.

393
00:22:11,220 --> 00:22:13,250
There were these things
were huge before.

394
00:22:13,250 --> 00:22:15,330
We've compressed them a lot.

395
00:22:15,330 --> 00:22:18,560
They're at least log square,
now they're only log.

396
00:22:18,560 --> 00:22:21,265
And so now we can
apply packed sorting.

397
00:22:24,080 --> 00:22:28,460
You can fit log times log
log items into one word.

398
00:22:28,460 --> 00:22:30,680
And that's going to make
sorting easy, somehow.

399
00:22:30,680 --> 00:22:32,660
That's to be done.

400
00:22:32,660 --> 00:22:36,800
But now the question is,
suppose we had packed sorting,

401
00:22:36,800 --> 00:22:37,760
how does this help you?

402
00:22:37,760 --> 00:22:40,460
Because this does not sort
the original integers.

403
00:22:40,460 --> 00:22:42,290
It sorts these
integers that have

404
00:22:42,290 --> 00:22:45,140
all of their digits
permuted in some random way.

405
00:22:45,140 --> 00:22:49,070
We have no idea how the hash
function re-orders things.

406
00:22:49,070 --> 00:22:51,977
So we're not done
with signature sort.

407
00:22:51,977 --> 00:22:54,560
It's called signature sort, of
course, because of the hashing.

408
00:22:54,560 --> 00:22:57,390
You're taking signatures
of each of these digits.

409
00:22:57,390 --> 00:22:59,690
Another word for hashes.

410
00:22:59,690 --> 00:23:05,955
All right, next step is to
build a compressed trie.

411
00:23:09,380 --> 00:23:11,060
This is a term we
haven't used yet.

412
00:23:17,005 --> 00:23:18,380
So I'm going to
call these things

413
00:23:18,380 --> 00:23:21,890
that we sorted signatures.

414
00:23:21,890 --> 00:23:26,960
And how many people know
what a compressed trie is?

415
00:23:26,960 --> 00:23:27,820
OK, no one.

416
00:23:27,820 --> 00:23:28,869
Wow, just a few.

417
00:23:28,869 --> 00:23:30,410
So you haven't seen
suffix trees yet.

418
00:23:30,410 --> 00:23:32,930
We're going to cover where
compressed tries come

419
00:23:32,930 --> 00:23:34,490
from in a couple
of lectures but we

420
00:23:34,490 --> 00:23:38,480
get to see them a little early
because they're cool and needed

421
00:23:38,480 --> 00:23:41,640
in this algorithm.

422
00:23:41,640 --> 00:23:44,230
So first notion is
the notion of a trie.

423
00:23:44,230 --> 00:23:49,340
The word trie comes
from retrieval,

424
00:23:49,340 --> 00:23:51,870
from old days when
information retrieval was

425
00:23:51,870 --> 00:23:56,640
what's now called web
searching, or whatever.

426
00:23:56,640 --> 00:24:00,410
So it's now pronounced trie.

427
00:24:00,410 --> 00:24:02,630
It's just another word
for tree, but where

428
00:24:02,630 --> 00:24:06,530
the children of the tree
come from a fixed universe.

429
00:24:10,780 --> 00:24:13,285
We've been working
with tries all the time

430
00:24:13,285 --> 00:24:16,990
in pretty much every lecture
for the last n lectures,

431
00:24:16,990 --> 00:24:19,620
last w lectures, I don't
know, a lot of them.

432
00:24:19,620 --> 00:24:21,790
We said oh, OK, well
let's just consider

433
00:24:21,790 --> 00:24:24,261
the space of all possible keys.

434
00:24:24,261 --> 00:24:26,260
You know, I've drawn this
picture zillion times.

435
00:24:26,260 --> 00:24:28,630
Even when we weren't doing
Van Emde Boas, which actually

436
00:24:28,630 --> 00:24:30,970
stores all of these
things, we were

437
00:24:30,970 --> 00:24:32,740
thinking about fusion
trees like, oh yeah,

438
00:24:32,740 --> 00:24:34,690
so we're storing
this value, and we're

439
00:24:34,690 --> 00:24:37,660
storing this value,
and maybe this value.

440
00:24:37,660 --> 00:24:40,080
And we understand this
corresponds to a 0-bit.

441
00:24:40,080 --> 00:24:41,250
These are 0 and 1 bits.

442
00:24:41,250 --> 00:24:43,560
This is a 1, 0, 1, whatever.

443
00:24:43,560 --> 00:24:47,030
So we understand how to convert
a bit string into this picture.

444
00:24:47,030 --> 00:24:49,750
This is what we
call a binary trie.

445
00:24:49,750 --> 00:24:51,560
I want to do the
same thing over here,

446
00:24:51,560 --> 00:24:53,350
except it's not a binary trie.

447
00:24:53,350 --> 00:24:56,320
Now each digit is giant
so we're going to have

448
00:24:56,320 --> 00:24:58,570
some huge branching factor.

449
00:24:58,570 --> 00:25:00,610
So it's going to be
more like a b-tree.

450
00:25:00,610 --> 00:25:03,280
The first node has a
whole bunch of children.

451
00:25:03,280 --> 00:25:04,540
I'm not going to even--

452
00:25:04,540 --> 00:25:11,060
I mean, it's 2 to the w divided
by log to the epsilon n.

453
00:25:11,060 --> 00:25:13,440
That's the branching factor.

454
00:25:13,440 --> 00:25:15,430
OK, it's big.

455
00:25:15,430 --> 00:25:19,570
That's the number of possible
values in the first digit.

456
00:25:19,570 --> 00:25:20,860
OK.

457
00:25:20,860 --> 00:25:27,190
So then maybe this child
has some node there.

458
00:25:27,190 --> 00:25:29,410
And maybe this one.

459
00:25:34,130 --> 00:25:37,205
Actually, sorry, I didn't
want to draw it that way.

460
00:25:37,205 --> 00:25:40,450
I have a specific trie
in mind, or part of it.

461
00:25:49,200 --> 00:25:52,545
OK, I mean, trie is going
to look something like this.

462
00:25:52,545 --> 00:25:55,780
It's harder to draw, but
it's the larger than binary

463
00:25:55,780 --> 00:25:57,570
equivalent of this picture.

464
00:25:57,570 --> 00:26:01,370
This would be a trie.

465
00:26:01,370 --> 00:26:03,730
So let's think about a
particular situation.

466
00:26:03,730 --> 00:26:08,780
Which is, suppose we just have
two items in this structure.

467
00:26:08,780 --> 00:26:10,310
Then it's easier to draw.

468
00:26:10,310 --> 00:26:14,350
So let's suppose there's
some item over here, x1.

469
00:26:14,350 --> 00:26:16,510
There's some item over here, x2.

470
00:26:16,510 --> 00:26:19,402
Now, generic picture is they
share some common prefix.

471
00:26:19,402 --> 00:26:20,485
This is the common prefix.

472
00:26:24,110 --> 00:26:26,425
It's just to understand some
particular picture here.

473
00:26:26,425 --> 00:26:29,050
Then they diverge, and you know,
then they have their own digit

474
00:26:29,050 --> 00:26:30,034
values.

475
00:26:30,034 --> 00:26:32,200
In some sense, this is the
digit value we care about

476
00:26:32,200 --> 00:26:34,450
because it distinguishes them.

477
00:26:34,450 --> 00:26:39,850
So there's this idea of a
compressed trie, which cleans

478
00:26:39,850 --> 00:26:41,170
up this picture a little bit.

479
00:26:44,772 --> 00:26:46,730
The idea with the compress
trie is very simple.

480
00:26:46,730 --> 00:26:48,050
You draw this trie.

481
00:26:48,050 --> 00:26:53,350
So here, imagine I'm erasing all
the things that do not appear.

482
00:26:53,350 --> 00:26:55,220
Maybe all this is gone.

483
00:26:55,220 --> 00:26:57,530
So I'm just keeping this
part of the structure.

484
00:26:57,530 --> 00:27:00,140
Let's do it for the
binary case first.

485
00:27:00,140 --> 00:27:02,790
The idea in a compress
trie is very simple.

486
00:27:02,790 --> 00:27:10,619
I'm going to contract
every non-branching node

487
00:27:10,619 --> 00:27:11,285
into its parent.

488
00:27:21,930 --> 00:27:23,340
So what that means.

489
00:27:23,340 --> 00:27:28,950
So here I'm storing three
values: 0, 0, 1, 1; 0, 1,

490
00:27:28,950 --> 00:27:34,370
0, 0; and 0, 1, 1, 1.

491
00:27:34,370 --> 00:27:36,680
So some nodes are
non-branching, that's

492
00:27:36,680 --> 00:27:39,605
this node, this
node, and this node.

493
00:27:39,605 --> 00:27:41,480
And the point is well,
why store those nodes?

494
00:27:41,480 --> 00:27:43,938
I mean, there was nothing being
distinguished at that node,

495
00:27:43,938 --> 00:27:45,380
so contract it.

496
00:27:45,380 --> 00:27:52,510
So the new picture if I
redrew this would be 0, 0, 1--

497
00:27:52,510 --> 00:27:55,010
sorry, also this
is non-branching.

498
00:27:55,010 --> 00:27:58,537
So in fact, I'm going
to contract this

499
00:27:58,537 --> 00:28:00,120
into its parent,
this into its parent.

500
00:28:00,120 --> 00:28:01,950
So this whole path,
this non-branching path

501
00:28:01,950 --> 00:28:06,140
will be left with a single edge
which I will label 0, 1, 1.

502
00:28:06,140 --> 00:28:08,570
But it's a single edge
in this structure.

503
00:28:08,570 --> 00:28:14,090
Over here we have an edge 1,
and now we have 0, 0 here,

504
00:28:14,090 --> 00:28:16,640
and 1, 1 here.

505
00:28:16,640 --> 00:28:18,650
That is the compressed
trie representation

506
00:28:18,650 --> 00:28:20,660
of these three values.

507
00:28:20,660 --> 00:28:22,610
The nice thing
about this structure

508
00:28:22,610 --> 00:28:25,580
is every node is branching
except the leaves, which

509
00:28:25,580 --> 00:28:27,050
means the number
of nodes here is

510
00:28:27,050 --> 00:28:29,230
linear in the number of leaves.

511
00:28:29,230 --> 00:28:32,960
That's like twice the number
of leaves minus 1, roughly.

512
00:28:32,960 --> 00:28:35,930
I guess the root might
not be branching.

513
00:28:35,930 --> 00:28:37,510
It has no parent
to contract into.

514
00:28:37,510 --> 00:28:38,660
Whatever.

515
00:28:38,660 --> 00:28:41,612
But more and more
nodes, most 2n nodes.

516
00:28:41,612 --> 00:28:42,570
That's the binary case.

517
00:28:42,570 --> 00:28:47,010
Now over here what we do is in
the corresponding contracted

518
00:28:47,010 --> 00:28:49,180
try a picture of this thing.

519
00:28:49,180 --> 00:28:50,915
Let's see, it's a second edge.

520
00:28:50,915 --> 00:28:53,990
I'm going to put just a label
of 2 on this edge to say, well,

521
00:28:53,990 --> 00:28:54,950
I'm skipping this node.

522
00:28:54,950 --> 00:28:56,210
This guy is non-branching.

523
00:28:56,210 --> 00:28:59,840
So I want to get rid of it and
contract it into the parent.

524
00:28:59,840 --> 00:29:02,510
So now this corresponds
to two edges here.

525
00:29:02,510 --> 00:29:04,820
There are two digits, which
I could write the digits

526
00:29:04,820 --> 00:29:05,690
if I knew what they were.

527
00:29:05,690 --> 00:29:07,731
But particularly I want
to keep track of the fact

528
00:29:07,731 --> 00:29:11,100
that there exactly two of them.

529
00:29:11,100 --> 00:29:12,740
And then there's
a branching node.

530
00:29:12,740 --> 00:29:14,450
That one stays.

531
00:29:14,450 --> 00:29:17,040
But then I actually we'll go
straight to the leaves here.

532
00:29:17,040 --> 00:29:22,360
This will be x1 and x2.

533
00:29:22,360 --> 00:29:24,265
And this will have
a lot of digits.

534
00:29:26,960 --> 00:29:29,000
OK, so some large
number written here.

535
00:29:29,000 --> 00:29:31,810
So that's the idea
with a compress trie.

536
00:29:31,810 --> 00:29:34,030
Now the nice thing
about a compress trie

537
00:29:34,030 --> 00:29:40,000
is they have linear size,
number of edges and vertices

538
00:29:40,000 --> 00:29:42,220
is linear in the
number of leaves,

539
00:29:42,220 --> 00:29:45,760
which is the number of
values we have here.

540
00:29:45,760 --> 00:29:47,440
Another nice thing
about compress tries

541
00:29:47,440 --> 00:29:49,060
is they preserve order.

542
00:29:49,060 --> 00:29:50,830
So if you look at the
order of the leaves

543
00:29:50,830 --> 00:29:54,287
down here it's the same as the
order of the leaves over here.

544
00:29:54,287 --> 00:29:55,870
We're not really
changing order, we're

545
00:29:55,870 --> 00:29:59,620
just making things a little
shorter to write down.

546
00:29:59,620 --> 00:30:00,820
Same thing over here.

547
00:30:00,820 --> 00:30:02,500
I mean, x1 appears
the left of x2.

548
00:30:18,130 --> 00:30:20,992
Say the order of the
leaves is preserved.

549
00:30:20,992 --> 00:30:22,700
The other cool thing
about compress tries

550
00:30:22,700 --> 00:30:25,120
is you can compute
them in linear time.

551
00:30:25,120 --> 00:30:28,360
But let me tell you why
we're drawing this picture.

552
00:30:28,360 --> 00:30:32,320
We've messed up all these
chunks, all these digits.

553
00:30:32,320 --> 00:30:34,730
What does that correspond
to in this picture?

554
00:30:34,730 --> 00:30:37,661
We've permuted all of the
children in this node.

555
00:30:37,661 --> 00:30:39,910
We've permuted all the
children in this node, permuted

556
00:30:39,910 --> 00:30:41,320
all the children in this node.

557
00:30:41,320 --> 00:30:44,020
But otherwise, the structure
of the tree is correct.

558
00:30:44,020 --> 00:30:46,000
What we wanted to
do was compute--

559
00:30:46,000 --> 00:30:48,610
if we could compute the
compress trie representation

560
00:30:48,610 --> 00:30:53,020
of the original values just
having broken the integers

561
00:30:53,020 --> 00:30:55,919
into these pieces, then if we
had that trie we could just

562
00:30:55,919 --> 00:30:58,210
do an in order traversal,
find the order of the leaves,

563
00:30:58,210 --> 00:31:01,720
then we'd have the sorted
order of the integers.

564
00:31:01,720 --> 00:31:05,590
Instead what we've computed
is we didn't compute

565
00:31:05,590 --> 00:31:06,850
the correct compress trie.

566
00:31:06,850 --> 00:31:08,350
But we computed a
different compress

567
00:31:08,350 --> 00:31:12,220
trie where every node has
its children permuted.

568
00:31:12,220 --> 00:31:15,340
So all we need to do is put
the children within each node

569
00:31:15,340 --> 00:31:17,860
in the correct
order, then we would

570
00:31:17,860 --> 00:31:20,220
have the correct compress trie.

571
00:31:20,220 --> 00:31:22,490
OK, that's why we're doing this.

572
00:31:22,490 --> 00:31:27,400
But first, how do we compute a
compress trie in linear time?

573
00:31:32,080 --> 00:31:33,950
This is a simple
amortization argument.

574
00:31:48,887 --> 00:31:49,970
Remember what we're given.

575
00:31:53,240 --> 00:31:57,070
At this point we are given
the leaves in sorted order.

576
00:31:57,070 --> 00:31:59,870
We've just sorted our things
so all I'm trying to do

577
00:31:59,870 --> 00:32:02,970
is compute the trie of the
stuff above the things.

578
00:32:02,970 --> 00:32:06,406
OK, we're given the leaves of
the trie in the correct order.

579
00:32:06,406 --> 00:32:08,030
I just need to build
the compress trie.

580
00:32:08,030 --> 00:32:09,260
The reason I'm building
a compress trie

581
00:32:09,260 --> 00:32:10,635
is so I can do it
in linear time.

582
00:32:10,635 --> 00:32:15,780
Constructing the uncompressed
trie would take too long.

583
00:32:15,780 --> 00:32:17,690
OK, so how do we do this?

584
00:32:26,150 --> 00:32:29,030
We have our items xn in order.

585
00:32:34,340 --> 00:32:41,570
I want to just insert it
into the compress trie.

586
00:32:41,570 --> 00:32:45,080
So I start with an
empty compress trie.

587
00:32:45,080 --> 00:32:47,840
First thing I do is add x1,
so it's just a single leaf.

588
00:32:47,840 --> 00:32:50,210
Then I add x2 to the
trie, x3 to the trie.

589
00:32:50,210 --> 00:32:53,030
In general, I'll have computed--

590
00:32:53,030 --> 00:32:54,760
I'm going to draw a picture.

591
00:32:54,760 --> 00:32:58,060
How to draw a partial
trie, something like this.

592
00:32:58,060 --> 00:33:00,240
I've computed the left
part of trie already.

593
00:33:00,240 --> 00:33:03,080
All these x1's have been done.

594
00:33:03,080 --> 00:33:05,780
And now I need to
add on the next,

595
00:33:05,780 --> 00:33:09,930
say xi over here somewhere.

596
00:33:09,930 --> 00:33:12,860
So think of how that
would look in the trie.

597
00:33:12,860 --> 00:33:15,715
It has to come off
of the right side,

598
00:33:15,715 --> 00:33:17,090
I guess what would
you call this?

599
00:33:17,090 --> 00:33:19,550
The right spine of the tree.

600
00:33:19,550 --> 00:33:22,310
Some right most node over here.

601
00:33:22,310 --> 00:33:27,140
We want xi to be a new
to the right child.

602
00:33:27,140 --> 00:33:30,890
So one possible
picture is there was

603
00:33:30,890 --> 00:33:33,830
some node which had
some other children,

604
00:33:33,830 --> 00:33:37,100
maybe this one had xi minus 1.

605
00:33:37,100 --> 00:33:39,160
Maybe xi belongs over here.

606
00:33:39,160 --> 00:33:43,700
Maybe this leads to xi
in the uncompressed trie.

607
00:33:43,700 --> 00:33:46,110
This could be a
potential picture.

608
00:33:46,110 --> 00:33:49,310
In the compress trie
this node either exists

609
00:33:49,310 --> 00:33:50,960
or got contracted.

610
00:33:50,960 --> 00:33:55,250
If it exists maybe there
was a previous xi minus 2

611
00:33:55,250 --> 00:33:56,464
belongs over here.

612
00:33:56,464 --> 00:33:58,880
So we already had this node
because it distinguished these

613
00:33:58,880 --> 00:34:01,410
two xj's.

614
00:34:01,410 --> 00:34:03,000
To add the xi is really easy.

615
00:34:03,000 --> 00:34:07,820
I just add a node here and then
which points to a leaf of xi.

616
00:34:07,820 --> 00:34:10,760
So that could be one situation.

617
00:34:10,760 --> 00:34:13,480
The other situation is only
slightly more complicated.

618
00:34:21,980 --> 00:34:25,550
So the other
possibility is this node

619
00:34:25,550 --> 00:34:28,730
didn't exist in the compressed
trie, which means we jumped

620
00:34:28,730 --> 00:34:32,909
over it in the compress trie.

621
00:34:32,909 --> 00:34:35,210
Some number bigger
than 1 here, maybe you

622
00:34:35,210 --> 00:34:39,120
skipped three nodes, who knows.

623
00:34:39,120 --> 00:34:40,451
So now this node matters.

624
00:34:40,451 --> 00:34:41,450
It didn't matter before.

625
00:34:41,450 --> 00:34:44,570
Before it was non-branching, but
now we add xi, it's branching.

626
00:34:44,570 --> 00:34:48,909
All we do is update,
add this new guy,

627
00:34:48,909 --> 00:34:52,389
add the new pointer to xi.

628
00:34:52,389 --> 00:34:54,949
OK, so constant work.

629
00:34:54,949 --> 00:34:59,780
Great, constant work per xi
inserted, linear time overall.

630
00:34:59,780 --> 00:35:00,920
Not really.

631
00:35:00,920 --> 00:35:02,510
I've cheated a little
bit here, which

632
00:35:02,510 --> 00:35:05,780
is how do we know what we're
going to do this operation.

633
00:35:05,780 --> 00:35:08,930
So the changes we make
is constant, that's fine.

634
00:35:08,930 --> 00:35:11,120
But how did I find that
this was the edge where

635
00:35:11,120 --> 00:35:13,100
I need to put xi in?

636
00:35:13,100 --> 00:35:14,990
You can't do it
in constant time,

637
00:35:14,990 --> 00:35:17,840
but we can do it in
constant amortized time.

638
00:35:17,840 --> 00:35:22,700
So what we're going to do
is start at the right most--

639
00:35:22,700 --> 00:35:26,690
that is not very red.

640
00:35:26,690 --> 00:35:29,965
Not a red chalk.

641
00:35:29,965 --> 00:35:32,210
Red chalk.

642
00:35:32,210 --> 00:35:35,630
We start at this leaf, the
rightmost leaf which you go

643
00:35:35,630 --> 00:35:37,380
going right most as possible.

644
00:35:37,380 --> 00:35:42,090
We're going to walk up the tree
until we find the right point.

645
00:35:42,090 --> 00:35:45,340
We're going to spend
linear time this path walk.

646
00:35:45,340 --> 00:35:47,532
OK, how could this
possibly be OK?

647
00:35:47,532 --> 00:35:49,990
I think you believe that I can
do it in linear time, right.

648
00:35:49,990 --> 00:35:51,870
Each point I see--

649
00:35:51,870 --> 00:35:53,660
what is my common prefix?

650
00:35:53,660 --> 00:35:55,470
Actually an easy way
is right up front.

651
00:35:55,470 --> 00:35:59,510
This thing right
here is xi minus 1.

652
00:35:59,510 --> 00:36:00,830
These are single integers.

653
00:36:00,830 --> 00:36:04,070
So I can just compute
their x or, find

654
00:36:04,070 --> 00:36:06,560
the most significant one bit,
which we did with the fusion

655
00:36:06,560 --> 00:36:08,180
tree lecture.

656
00:36:08,180 --> 00:36:12,890
And boom, I know where they
differ in bit position.

657
00:36:12,890 --> 00:36:14,630
By some rounding
I can figure out

658
00:36:14,630 --> 00:36:19,070
which chunk they differ
in, just by dividing,

659
00:36:19,070 --> 00:36:19,880
taking the ceiling.

660
00:36:22,950 --> 00:36:26,180
So I know the correct depth.

661
00:36:26,180 --> 00:36:30,170
And then these numbers tell
me how much depth I'm losing.

662
00:36:30,170 --> 00:36:32,480
As I walk up at some
point I'll discover,

663
00:36:32,480 --> 00:36:34,160
oops, I just jumped
over the depth

664
00:36:34,160 --> 00:36:36,120
I care about where they differ.

665
00:36:36,120 --> 00:36:38,480
And so at that point I
know to do that operation.

666
00:36:38,480 --> 00:36:43,280
So if there are k edges
here on the compress trie,

667
00:36:43,280 --> 00:36:48,390
I can do this in order k
time, k plus 1, let's say.

668
00:36:48,390 --> 00:36:50,090
Why is that OK?

669
00:36:50,090 --> 00:36:55,190
Because my potential
function, amortization

670
00:36:55,190 --> 00:37:03,045
is going to be the length
of the rightmost path.

671
00:37:06,030 --> 00:37:10,020
OK, before my rightmost
path was this thing.

672
00:37:10,020 --> 00:37:13,350
Afterwards, my rightmost path
is going to be this thing.

673
00:37:13,350 --> 00:37:17,270
So it got roughly k
shorter, k minus 1 shorter.

674
00:37:17,270 --> 00:37:19,710
OK, so I charged
this k plus 1 cost

675
00:37:19,710 --> 00:37:24,850
to a potential
decrease of k minus 1.

676
00:37:24,850 --> 00:37:28,250
And so the amortize
cost is only constant.

677
00:37:28,250 --> 00:37:29,670
Easy amortization.

678
00:37:29,670 --> 00:37:31,620
Another way to see
this is, overall

679
00:37:31,620 --> 00:37:34,050
we're basically doing it in
order traversal of the tree.

680
00:37:34,050 --> 00:37:37,080
So whenever we walk up we'll
never visit these nodes again,

681
00:37:37,080 --> 00:37:39,112
because then we go down
at a different place.

682
00:37:39,112 --> 00:37:41,070
OK, yeah, this guy we're
going to walk up again

683
00:37:41,070 --> 00:37:43,195
because in an in order
traversal we would have gone

684
00:37:43,195 --> 00:37:45,270
left and then come back
to it and then gone down.

685
00:37:45,270 --> 00:37:46,686
So overall we're
doing an in order

686
00:37:46,686 --> 00:37:51,140
traversal of the compress trie,
so it's linear time total.

687
00:37:56,271 --> 00:37:56,770
OK.

688
00:38:00,510 --> 00:38:02,084
Still not done though.

689
00:38:02,084 --> 00:38:03,750
What we have so far--
what have we done?

690
00:38:03,750 --> 00:38:06,450
We've taken our numbers,
we've split them into digits.

691
00:38:06,450 --> 00:38:09,870
We randomly permuted the
digits via hash function.

692
00:38:09,870 --> 00:38:12,660
That magically let us
sort them in linear time.

693
00:38:12,660 --> 00:38:15,900
Now we've built a compress
trie of that representation.

694
00:38:15,900 --> 00:38:19,290
The last thing to do is to
fix the order of the children

695
00:38:19,290 --> 00:38:20,760
from each node.

696
00:38:20,760 --> 00:38:23,130
The tree is correct
except for the order

697
00:38:23,130 --> 00:38:25,170
of the children in each node.

698
00:38:25,170 --> 00:38:30,060
So we've kind of sorted a little
bit, but haven't fully sorted.

699
00:38:30,060 --> 00:38:34,760
So that's the last step.

700
00:38:34,760 --> 00:38:39,320
And we're going to do
that with recursion.

701
00:38:43,430 --> 00:38:45,725
We're only going to recurse
a constant number of times.

702
00:38:48,410 --> 00:38:49,550
Recursively sort.

703
00:39:20,970 --> 00:39:24,890
OK, I'm going to walk
through this compress

704
00:39:24,890 --> 00:39:29,450
try trie we've built. And I
guess it looks more like this.

705
00:39:29,450 --> 00:39:33,877
And for each edge I'm going
to write down this key.

706
00:39:33,877 --> 00:39:35,460
First thing you're
going to write down

707
00:39:35,460 --> 00:39:36,720
is the ID of the node.

708
00:39:36,720 --> 00:39:42,260
Let's say the ID is the in order
traversal index of the node.

709
00:39:42,260 --> 00:39:43,380
OK, so boom.

710
00:39:43,380 --> 00:39:47,910
As we traverse the tree we can
just increment this counter.

711
00:39:47,910 --> 00:39:51,390
Then we also want--

712
00:39:51,390 --> 00:39:57,960
what I'm going to put in here
is the actual chunk value

713
00:39:57,960 --> 00:39:59,940
that's at the top of the node.

714
00:39:59,940 --> 00:40:02,700
So something like this,
or I guess this one.

715
00:40:02,700 --> 00:40:05,400
This one consists
of two chunk values.

716
00:40:05,400 --> 00:40:09,352
So it's called c1, for
chunk values, c1 and c2.

717
00:40:09,352 --> 00:40:11,310
It's the first chunk
value and the second chunk

718
00:40:11,310 --> 00:40:13,451
value, first digit
and the second digit.

719
00:40:13,451 --> 00:40:14,700
Just write down the first one.

720
00:40:18,062 --> 00:40:19,770
That's really the only
one we care about.

721
00:40:19,770 --> 00:40:21,936
The second one wasn't
branching so it's no big deal.

722
00:40:24,440 --> 00:40:26,240
OK, so there's some chunk value.

723
00:40:26,240 --> 00:40:30,380
Now, I don't want to write
down the signature chunk value.

724
00:40:30,380 --> 00:40:32,180
I don't want to write
down the hash value.

725
00:40:32,180 --> 00:40:37,040
I want the actual chunk,
original chunk not the hash.

726
00:40:37,040 --> 00:40:39,590
That's why I write
actual over here

727
00:40:39,590 --> 00:40:41,480
and then also, I'm
going to write down

728
00:40:41,480 --> 00:40:42,780
the index of the edge.

729
00:40:42,780 --> 00:40:46,240
So this is the second
edge, for example.

730
00:40:46,240 --> 00:40:49,560
OK, now this is in some
sense the wrong index.

731
00:40:49,560 --> 00:40:52,190
And what I want to compute
is where it's supposed to be.

732
00:40:52,190 --> 00:40:54,050
The edges have been permuted.

733
00:40:54,050 --> 00:40:56,090
So if I sort this
thing, I claim that I'll

734
00:40:56,090 --> 00:41:00,390
find that all the
desired permutations.

735
00:41:00,390 --> 00:41:02,060
OK, why?

736
00:41:02,060 --> 00:41:07,280
Because if I sort by
this triple, first thing

737
00:41:07,280 --> 00:41:10,760
I sort by is by node ID, then
I'm sorting by actual chunk

738
00:41:10,760 --> 00:41:12,119
value, then edge index.

739
00:41:12,119 --> 00:41:13,910
I don't care that I'm
sorting by it, that's

740
00:41:13,910 --> 00:41:15,870
just coming along for the ride.

741
00:41:15,870 --> 00:41:20,270
So what this tells
me is for each node

742
00:41:20,270 --> 00:41:23,150
it gives me the sorted value
of the actual chunks, not

743
00:41:23,150 --> 00:41:24,560
the hash chunks.

744
00:41:24,560 --> 00:41:28,160
And then it tells me the
order they used to be in.

745
00:41:28,160 --> 00:41:33,200
So this is a way to figure
out the sorted permutation

746
00:41:33,200 --> 00:41:34,250
of these things.

747
00:41:34,250 --> 00:41:35,954
This gives you the old index.

748
00:41:35,954 --> 00:41:37,370
And so once you've
sorted them you

749
00:41:37,370 --> 00:41:41,510
know that new correct
order is in order by chunk.

750
00:41:41,510 --> 00:41:43,040
And so this gives
you the inverse

751
00:41:43,040 --> 00:41:47,420
of the permutation over here,
of where each edge should be.

752
00:41:47,420 --> 00:41:52,670
So once I've done
this the last step is

753
00:41:52,670 --> 00:42:04,340
for each node permute
the edges as given

754
00:42:04,340 --> 00:42:07,190
by this inverse permutation.

755
00:42:07,190 --> 00:42:08,118
Questions?

756
00:42:08,118 --> 00:42:09,659
AUDIENCE: Can you
explain again that?

757
00:42:09,659 --> 00:42:11,330
You were basically
sorting three times.

758
00:42:11,330 --> 00:42:14,756
ERIK DEMAINE: No, I'm sorting
these single key values.

759
00:42:14,756 --> 00:42:16,880
The key value, the most
significant part of the key

760
00:42:16,880 --> 00:42:17,780
is this.

761
00:42:17,780 --> 00:42:19,340
The next most significant
part of the key is this.

762
00:42:19,340 --> 00:42:21,360
And the least significant
part of the key is this.

763
00:42:21,360 --> 00:42:23,276
So imagine concatenating
these things together

764
00:42:23,276 --> 00:42:24,400
into a single key value.

765
00:42:24,400 --> 00:42:26,540
Yeah, good question.

766
00:42:26,540 --> 00:42:29,270
What I'll talk about in a
moment, how big this key is.

767
00:42:29,270 --> 00:42:32,390
But if I sort by the
concatenation of those keys

768
00:42:32,390 --> 00:42:34,460
then what happens is
I'm sorting by this

769
00:42:34,460 --> 00:42:35,632
and then sorting by this.

770
00:42:35,632 --> 00:42:37,340
But really, I'm doing
one sort operation.

771
00:42:37,340 --> 00:42:39,620
This is one recursion.

772
00:42:39,620 --> 00:42:42,530
Other questions?

773
00:42:42,530 --> 00:42:44,200
OK.

774
00:42:44,200 --> 00:42:46,520
Do you see why this
gives us what we need?

775
00:42:46,520 --> 00:42:49,500
For every node we want
to sort all those chunks.

776
00:42:49,500 --> 00:42:51,806
And so this information
is just so we

777
00:42:51,806 --> 00:42:53,180
can keep track of
the permutation

778
00:42:53,180 --> 00:42:54,920
when we're just sorting
by value and not

779
00:42:54,920 --> 00:42:56,500
keeping the permutation.

780
00:42:56,500 --> 00:42:58,650
This let's us keep track of it.

781
00:42:58,650 --> 00:43:01,760
But our real goal is to
sort all the chunk values.

782
00:43:01,760 --> 00:43:04,430
And we add in the node
ID so that we just

783
00:43:04,430 --> 00:43:06,740
learn for every node what
we're supposed to do instead

784
00:43:06,740 --> 00:43:08,712
of globally what we need to do.

785
00:43:08,712 --> 00:43:10,670
Maybe you don't have to
do this but it's easier

786
00:43:10,670 --> 00:43:13,010
to keep the node ID in there.

787
00:43:13,010 --> 00:43:17,004
OK, how many bits
are in this key?

788
00:43:17,004 --> 00:43:18,920
So our whole point was
to make chunks small so

789
00:43:18,920 --> 00:43:20,169
that the keys were small.

790
00:43:20,169 --> 00:43:22,460
But I claim this will be a
little bit smaller than what

791
00:43:22,460 --> 00:43:23,730
we started with.

792
00:43:23,730 --> 00:43:26,300
So the node ID, well,
there's only n nodes,

793
00:43:26,300 --> 00:43:29,510
so this is log n bits.

794
00:43:29,510 --> 00:43:35,090
Chunk value is, I
guess, w divided by log

795
00:43:35,090 --> 00:43:38,630
to the epsilon n bits.

796
00:43:38,630 --> 00:43:42,070
Because there are a log
to the epsilon n chunks,

797
00:43:42,070 --> 00:43:44,240
so it's w divided by
log to the epsilon bits.

798
00:43:44,240 --> 00:43:50,390
The edge index, how many
different things were there?

799
00:43:54,822 --> 00:43:57,090
Is it also w divided
by log to the epsilon?

800
00:43:57,090 --> 00:43:59,524
I guess so.

801
00:43:59,524 --> 00:44:00,440
Depends how you count.

802
00:44:04,390 --> 00:44:06,800
Let me think for a second
how these edge indices should

803
00:44:06,800 --> 00:44:07,300
be stored.

804
00:44:12,140 --> 00:44:14,540
I think I'd like to
store the edge indices--

805
00:44:14,540 --> 00:44:20,150
I don't want to store
the absent down pointers.

806
00:44:20,150 --> 00:44:23,680
The way that I built this
tree, this compress trie,

807
00:44:23,680 --> 00:44:25,920
I was always adding
things to the right.

808
00:44:25,920 --> 00:44:28,554
So it's like OK,
I'm storing here.

809
00:44:28,554 --> 00:44:30,470
Here's the first child,
here's the next child,

810
00:44:30,470 --> 00:44:32,344
here's the next child,
here's the next child.

811
00:44:32,344 --> 00:44:33,450
I just keep appending on.

812
00:44:33,450 --> 00:44:35,570
It's like an array that you
can grow to the right, which

813
00:44:35,570 --> 00:44:37,460
you should know how to
do in constant time,

814
00:44:37,460 --> 00:44:39,168
even though linked
list is probably fine.

815
00:44:41,342 --> 00:44:42,800
If there are a
bunch of values here

816
00:44:42,800 --> 00:44:44,258
that are just null
pointers I don't

817
00:44:44,258 --> 00:44:46,470
want to store those
items in the array.

818
00:44:46,470 --> 00:44:50,360
So in fact, the maximum number
of children of a node is n.

819
00:44:50,360 --> 00:44:53,190
I mean, at the worst case you
have branching everywhere.

820
00:44:53,190 --> 00:45:03,050
So this is only log n bits to
store the index in that array.

821
00:45:03,050 --> 00:45:05,240
So it's an extra
level of compression.

822
00:45:05,240 --> 00:45:13,150
I guess I didn't mention
don't store null pointers,

823
00:45:13,150 --> 00:45:16,040
just skip them.

824
00:45:16,040 --> 00:45:19,510
And that's easy to do because
you're always just appending.

825
00:45:19,510 --> 00:45:24,300
And in the end what
do I need to do here?

826
00:45:24,300 --> 00:45:33,010
Step seven is in order
traversal of the compress trie.

827
00:45:37,180 --> 00:45:43,654
And I output all the
leaves I get in order.

828
00:45:43,654 --> 00:45:45,820
And the point is we computed
the compress trie which

829
00:45:45,820 --> 00:45:48,910
has the rough correct
topological structure.

830
00:45:48,910 --> 00:45:50,710
Then we fix the order
within each node,

831
00:45:50,710 --> 00:45:53,320
now we have the correctly
sorted compress trie.

832
00:45:53,320 --> 00:45:55,980
And so you're just doing
in order traversal, output

833
00:45:55,980 --> 00:45:57,610
the leave you get in order.

834
00:45:57,610 --> 00:46:00,690
Boom, we've sorted
all the items.

835
00:46:00,690 --> 00:46:01,315
A little crazy.

836
00:46:03,887 --> 00:46:06,220
So if all we need to do in
the end is in order traversal

837
00:46:06,220 --> 00:46:08,553
then it's fine we don't need
to store the null pointers.

838
00:46:08,553 --> 00:46:10,920
So we don't need to be able
to search in this trie,

839
00:46:10,920 --> 00:46:11,710
so this is enough.

840
00:46:14,590 --> 00:46:16,090
Clear?

841
00:46:16,090 --> 00:46:17,950
OK, the remaining
question is how

842
00:46:17,950 --> 00:46:19,970
expensive is this recursion?

843
00:46:19,970 --> 00:46:23,170
So I computed the number
of bits basically w

844
00:46:23,170 --> 00:46:27,020
divided by log to the
epsilon n, plus order log n.

845
00:46:27,020 --> 00:46:29,050
That order log n won't hurt us.

846
00:46:29,050 --> 00:46:32,660
Now, we started with
integers of size w,

847
00:46:32,660 --> 00:46:35,740
the total size here was w.

848
00:46:35,740 --> 00:46:39,850
Now we have integers that
their size is w divided

849
00:46:39,850 --> 00:46:40,910
by log to the epsilon n.

850
00:46:40,910 --> 00:46:41,868
So we've made progress.

851
00:46:45,010 --> 00:46:46,520
Cool.

852
00:46:46,520 --> 00:46:47,645
How much progress?

853
00:46:53,380 --> 00:46:54,210
Over here maybe.

854
00:47:01,710 --> 00:47:05,280
After, let's say, 1
over epsilon plus 1

855
00:47:05,280 --> 00:47:17,230
recursions we will
have reached b will

856
00:47:17,230 --> 00:47:27,110
have decreased to order log n.

857
00:47:27,110 --> 00:47:36,350
I guess, like 1 over epsilon
log n plus w over log

858
00:47:36,350 --> 00:47:39,800
to the 1 plus epsilon n.

859
00:47:47,480 --> 00:47:48,940
OK.

860
00:47:48,940 --> 00:47:50,850
Right, if we take
w and every time we

861
00:47:50,850 --> 00:47:54,630
divide by log to the epsilon n,
then after 1 over epsilon times

862
00:47:54,630 --> 00:47:57,700
we have divided by log
n, get rid of epsilon.

863
00:47:57,700 --> 00:47:59,910
Do it one more time,
we get w divided

864
00:47:59,910 --> 00:48:03,480
by log to the 1 plus epsilon n.

865
00:48:03,480 --> 00:48:05,720
And at this point
we're in good shape.

866
00:48:09,600 --> 00:48:11,700
If we get the size
of our integers

867
00:48:11,700 --> 00:48:15,360
to be only w divided by log
to the 1 plus epsilon n,

868
00:48:15,360 --> 00:48:18,450
then we can use packed sorting.

869
00:48:18,450 --> 00:48:20,310
Because packed sorting,
conveniently it's

870
00:48:20,310 --> 00:48:22,410
the black box at the
moment, says well,

871
00:48:22,410 --> 00:48:25,710
as long as w divided by b is
a little bigger than log n.

872
00:48:25,710 --> 00:48:28,260
So here we've made it quite
a bit bigger than log n.

873
00:48:28,260 --> 00:48:31,560
I mean, log to the 1
plus epsilon n bigger,

874
00:48:31,560 --> 00:48:33,270
then we can use packed sorting.

875
00:48:37,680 --> 00:48:44,550
Reach base case, and
then use packed sorting.

876
00:48:50,740 --> 00:48:53,210
OK, the only thing is
there's this log n.

877
00:48:53,210 --> 00:48:56,890
Now, epsilon is a constant here,
so 1 over epsilon is constant.

878
00:48:56,890 --> 00:48:58,740
So this is just order log n.

879
00:48:58,740 --> 00:49:00,280
Order log n doesn't hurt you.

880
00:49:00,280 --> 00:49:03,030
I mean, if this happened
to dominate this

881
00:49:03,030 --> 00:49:06,300
then you're saying your values
are order log n bits long,

882
00:49:06,300 --> 00:49:08,740
and so you use radex sort.

883
00:49:08,740 --> 00:49:10,612
So that's not going to dominate.

884
00:49:10,612 --> 00:49:12,070
In fact, you can
prove in this case

885
00:49:12,070 --> 00:49:14,528
it's not going to dominate. w
is at least log to the 2 plus

886
00:49:14,528 --> 00:49:15,500
epsilon.

887
00:49:15,500 --> 00:49:17,190
So you take w
divided by log to 1

888
00:49:17,190 --> 00:49:20,010
plus epsilon, that's
still at least log n.

889
00:49:20,010 --> 00:49:21,600
So this will always dominate.

890
00:49:27,360 --> 00:49:31,500
OK, so we did all
this work and all

891
00:49:31,500 --> 00:49:35,142
we did was reduce our word size
by a log to the epsilon factor.

892
00:49:35,142 --> 00:49:36,600
But that's enough,
because after we

893
00:49:36,600 --> 00:49:40,310
do that 1 over
epsilon plus 1 times

894
00:49:40,310 --> 00:49:42,990
the words are small enough,
or the items we're sorting

895
00:49:42,990 --> 00:49:45,750
are small enough, that we
can use packed sorting.

896
00:49:45,750 --> 00:49:47,700
And boom, we're done.

897
00:49:47,700 --> 00:49:52,140
OK, that is the beauty
of signature sort.

898
00:49:52,140 --> 00:49:54,780
Any questions about that?

899
00:49:54,780 --> 00:49:56,070
Definitely not easy.

900
00:49:59,020 --> 00:50:01,640
But, it works.

901
00:50:01,640 --> 00:50:04,594
The crazy idea is, well,
decompose into these digits.

902
00:50:04,594 --> 00:50:06,760
And when we decompose into
log to the epsilon digits

903
00:50:06,760 --> 00:50:08,270
because that would be enough.

904
00:50:08,270 --> 00:50:10,270
If we can get things down
to a single digit life

905
00:50:10,270 --> 00:50:12,460
would be easy.

906
00:50:12,460 --> 00:50:16,550
Which I guess is kind of
like Van Emde Boas, right.

907
00:50:16,550 --> 00:50:18,790
It's like our lower
bounds from last time.

908
00:50:18,790 --> 00:50:19,900
I don't know.

909
00:50:19,900 --> 00:50:22,960
If we can make it to only
one digit mattered then

910
00:50:22,960 --> 00:50:25,750
we could afford to do it because
each digit is quite a bit

911
00:50:25,750 --> 00:50:26,290
smaller.

912
00:50:26,290 --> 00:50:30,790
It's smaller by a factor
of log to the epsilon n.

913
00:50:30,790 --> 00:50:32,700
So we did that
splitting and then there

914
00:50:32,700 --> 00:50:36,200
was this observation that
most digit values aren't used.

915
00:50:36,200 --> 00:50:39,520
So if we hash then
things got a lot smaller.

916
00:50:39,520 --> 00:50:41,260
Then we can afford to pack sort.

917
00:50:41,260 --> 00:50:43,455
Except that hash didn't
preserve the order.

918
00:50:43,455 --> 00:50:44,830
How didn't it
preserve the order?

919
00:50:44,830 --> 00:50:46,660
Well, when you look
at the compress trie

920
00:50:46,660 --> 00:50:48,430
you see how it got messed up.

921
00:50:48,430 --> 00:50:50,210
Each node got messed
up individually,

922
00:50:50,210 --> 00:50:52,535
but overall the structure
of the trie was correct.

923
00:50:52,535 --> 00:50:54,910
So we did all this work to
get the structure of the trie.

924
00:50:54,910 --> 00:50:56,950
Then we had to fix
the node order.

925
00:50:56,950 --> 00:50:59,080
But that's really just
sorting a single digit.

926
00:50:59,080 --> 00:51:01,900
And yeah, we have to do it for
all nodes but you add them up.

927
00:51:01,900 --> 00:51:05,200
It's still only n items
that we need to sort.

928
00:51:05,200 --> 00:51:08,500
So I wrote that b went down.

929
00:51:08,500 --> 00:51:10,515
n stays the same.

930
00:51:10,515 --> 00:51:12,140
The number of things
we have to sort of

931
00:51:12,140 --> 00:51:15,375
stays the same, which is fine.

932
00:51:15,375 --> 00:51:17,190
We're not trying
to reduce n, we're

933
00:51:17,190 --> 00:51:19,030
just trying to reduce
the size of our items

934
00:51:19,030 --> 00:51:20,700
to be a little
bit smaller than w

935
00:51:20,700 --> 00:51:22,770
so then we can use
packed sorting.

936
00:51:22,770 --> 00:51:26,800
OK, I think I've gone over
this algorithm enough times.

937
00:51:26,800 --> 00:51:29,320
The next step is packed sorting.

938
00:51:32,290 --> 00:51:36,940
If we have integers that are
much smaller than our word,

939
00:51:36,940 --> 00:51:39,360
how do we sort in linear time?

940
00:51:39,360 --> 00:51:43,140
And this is going to be
essentially a fancy merge sort.

941
00:52:09,284 --> 00:52:10,575
This is done in the same paper.

942
00:52:26,560 --> 00:52:30,910
Before we said omega, I'm
going to essentially make omega

943
00:52:30,910 --> 00:52:32,057
a little bit bigger than 2.

944
00:52:32,057 --> 00:52:33,640
So I want to assume
that w is at least

945
00:52:33,640 --> 00:52:36,910
twice b plus 1 log n log log n.

946
00:52:36,910 --> 00:52:38,860
That's just to make
this convenient,

947
00:52:38,860 --> 00:52:42,042
but of course it doesn't really
matter what the constants are.

948
00:52:42,042 --> 00:52:43,750
This will make everything
fit in one word

949
00:52:43,750 --> 00:52:46,410
though, instead of a
constant number of words.

950
00:52:46,410 --> 00:53:03,000
OK, step zero of this
algorithm is the packing part.

951
00:53:07,260 --> 00:53:09,780
We can fit more than one
element into each word.

952
00:53:09,780 --> 00:53:12,060
So put them in.

953
00:53:12,060 --> 00:53:13,860
Now, we have linear
time overall.

954
00:53:13,860 --> 00:53:16,550
This is easy to do.

955
00:53:16,550 --> 00:53:17,940
You have some word already.

956
00:53:20,580 --> 00:53:23,940
Take those items, shift them
left or in the next item,

957
00:53:23,940 --> 00:53:25,530
shift them left or
in the next item,

958
00:53:25,530 --> 00:53:27,440
shift them left or
in the next item.

959
00:53:27,440 --> 00:53:30,460
OK, linear time.

960
00:53:30,460 --> 00:53:33,610
So in the end what it's
going to look like--

961
00:53:33,610 --> 00:53:36,810
oh, I also want to leave a 0
bit in between each of them.

962
00:53:36,810 --> 00:53:39,390
So we're going to have b.

963
00:53:39,390 --> 00:53:53,150
This is going to
be x1 here, x2, xn.

964
00:53:53,150 --> 00:53:55,171
Each of these is b bits long.

965
00:53:55,171 --> 00:53:56,420
And then we have one more bit.

966
00:53:58,940 --> 00:54:00,710
And in fact, the way
we set things up this

967
00:54:00,710 --> 00:54:02,074
is only half the word.

968
00:54:02,074 --> 00:54:03,740
So there's another
half over here, which

969
00:54:03,740 --> 00:54:06,470
is going to be all 0 for now.

970
00:54:06,470 --> 00:54:08,890
So we have one word contains--

971
00:54:08,890 --> 00:54:10,680
sorry, this is not n items.

972
00:54:10,680 --> 00:54:14,929
This is going to
be log n log log n.

973
00:54:14,929 --> 00:54:16,220
Can't fit them all in one word.

974
00:54:16,220 --> 00:54:18,194
That would make
life really easy.

975
00:54:18,194 --> 00:54:20,360
You can fit log n log log
n items in the first word.

976
00:54:20,360 --> 00:54:22,940
Then the next word
is the next log n

977
00:54:22,940 --> 00:54:25,260
log log n items and so on.

978
00:54:25,260 --> 00:54:26,480
Linear time to do that.

979
00:54:30,170 --> 00:54:33,472
Next step, this is an algorithm.

980
00:54:33,472 --> 00:54:34,430
It's not really a step.

981
00:54:34,430 --> 00:54:38,320
I should say these are
things that we're able to do,

982
00:54:38,320 --> 00:54:41,144
and we're going to combine
them in different ways.

983
00:54:41,144 --> 00:54:42,560
So the next thing
to observe, this

984
00:54:42,560 --> 00:54:46,070
is sort of a bottom
up perspective.

985
00:54:46,070 --> 00:54:48,320
So first thing we can do
is pack things into word.

986
00:54:48,320 --> 00:54:52,000
Next thing we can do is
merge two sorted words.

987
00:54:52,000 --> 00:54:55,670
Let's say they each have k items
and k is at most log n log log

988
00:54:55,670 --> 00:54:56,170
n.

989
00:55:04,480 --> 00:55:15,575
And I want to merge them into
one sorted word with two k

990
00:55:15,575 --> 00:55:16,075
elements.

991
00:55:18,970 --> 00:55:22,060
So I have two of these
things and I'll merge them

992
00:55:22,060 --> 00:55:25,244
in sorted order.

993
00:55:25,244 --> 00:55:27,160
And when I did this
packing, nothing's sorted.

994
00:55:27,160 --> 00:55:29,170
But don't worry about
that at the moment.

995
00:55:29,170 --> 00:55:31,690
Just if I had two
these words and suppose

996
00:55:31,690 --> 00:55:34,990
they were already
sorted, then I want

997
00:55:34,990 --> 00:55:39,610
to merge them and make
them still sorted.

998
00:55:39,610 --> 00:55:41,860
OK, I want x1 to be
less than x2 and so on.

999
00:55:44,470 --> 00:55:45,895
How much time?

1000
00:55:45,895 --> 00:55:48,340
It'd be cool if you could
do this in constant time

1001
00:55:48,340 --> 00:55:49,591
on a transdichotomous rem.

1002
00:55:49,591 --> 00:55:51,340
You could do anything
on a constant number

1003
00:55:51,340 --> 00:55:52,532
of words in constant time.

1004
00:55:52,532 --> 00:55:54,240
We're not going to
achieve constant time,

1005
00:55:54,240 --> 00:55:55,880
but we don't need to.

1006
00:55:55,880 --> 00:55:57,960
We're going to do
order log k time.

1007
00:56:01,235 --> 00:56:03,610
ERIK DEMAINE: This is the
hardest step of packed sorting,

1008
00:56:03,610 --> 00:56:06,790
and I'm not going to tell
you how to do it until later.

1009
00:56:06,790 --> 00:56:08,890
I will tell you, don't worry.

1010
00:56:08,890 --> 00:56:13,210
But this is going to
require bitonic sorting.

1011
00:56:13,210 --> 00:56:15,650
So I have to delay
it a little bit.

1012
00:56:15,650 --> 00:56:17,320
Suppose you had that for now.

1013
00:56:17,320 --> 00:56:19,030
Let me tell you how
the rest is easy.

1014
00:56:25,900 --> 00:56:28,860
So many black boxes, but they're
all filled in in this lecture.

1015
00:56:31,660 --> 00:56:37,060
OK, next thing you
might ask is, well,

1016
00:56:37,060 --> 00:56:38,860
how do I get a sorted word?

1017
00:56:38,860 --> 00:56:42,190
I see how to get a single word
that has log n log log n items.

1018
00:56:42,190 --> 00:56:44,640
But how would I
sort those items?

1019
00:56:44,640 --> 00:56:46,360
Use merge sort.

1020
00:56:46,360 --> 00:56:47,410
Here's a merger.

1021
00:56:47,410 --> 00:56:49,599
You have a way to
merge two sorted lists,

1022
00:56:49,599 --> 00:56:51,140
so use merge sort,
that will give you

1023
00:56:51,140 --> 00:56:53,270
a way to sort the whole thing.

1024
00:56:53,270 --> 00:56:58,060
So I'm going to merge sort
k equals log n log log n

1025
00:56:58,060 --> 00:57:06,300
items into one sorted word.

1026
00:57:13,050 --> 00:57:15,690
So I start out with a
word that's unsorted.

1027
00:57:15,690 --> 00:57:17,490
I do merge sort.

1028
00:57:17,490 --> 00:57:19,884
And I sort them.

1029
00:57:19,884 --> 00:57:21,550
So this is not the
whole sorting problem

1030
00:57:21,550 --> 00:57:25,300
but it's the one
word sorting problem.

1031
00:57:25,300 --> 00:57:27,060
So how long does this take?

1032
00:57:27,060 --> 00:57:30,020
Well, it's usual
merge sort recurrence,

1033
00:57:30,020 --> 00:57:34,095
but then it's plus how long
it takes me to split the word.

1034
00:57:34,095 --> 00:57:35,470
Well, splitting
the word is easy,

1035
00:57:35,470 --> 00:57:38,310
you just mask out the
low end or the high end

1036
00:57:38,310 --> 00:57:40,140
and then maybe shift over.

1037
00:57:40,140 --> 00:57:42,900
So in constant time I
can get the left half

1038
00:57:42,900 --> 00:57:46,230
and the right half of the
array, which is the word.

1039
00:57:46,230 --> 00:57:47,850
Then the hard part
is merging them.

1040
00:57:47,850 --> 00:57:54,468
And merging them is 1,
and that takes log k.

1041
00:57:54,468 --> 00:57:57,710
So this is step one.

1042
00:57:57,710 --> 00:57:59,010
That's our merger.

1043
00:57:59,010 --> 00:58:00,560
And so what does this solve to?

1044
00:58:00,560 --> 00:58:03,600
Well, probably easiest to
draw the recursion tree.

1045
00:58:03,600 --> 00:58:05,550
So at the root we pay log k.

1046
00:58:05,550 --> 00:58:13,890
Then we have 2 times log
k over 2, log k over 2.

1047
00:58:16,770 --> 00:58:20,160
So at this level
we're paying log k.

1048
00:58:20,160 --> 00:58:26,280
At this level we're paying
2 log k minus 1, I guess.

1049
00:58:28,880 --> 00:58:30,720
I'm just going to
call this maybe--

1050
00:58:30,720 --> 00:58:33,480
yeah, I do need to do that.

1051
00:58:33,480 --> 00:58:36,030
I mean everything is constant
size so you pay a constant.

1052
00:58:36,030 --> 00:58:39,200
How many leaves are there
at the bottom level?

1053
00:58:39,200 --> 00:58:42,260
k of them.

1054
00:58:42,260 --> 00:58:44,090
OK, so at the root
we're paying log k,

1055
00:58:44,090 --> 00:58:45,750
at the leaves we're paying k.

1056
00:58:45,750 --> 00:58:47,310
This is roughly geometric.

1057
00:58:47,310 --> 00:58:50,540
There's this minus 1,
but it's geometric enough

1058
00:58:50,540 --> 00:58:54,210
and when you add up all these
levels it's dominated by k.

1059
00:58:54,210 --> 00:58:56,872
So this ends up
being order k time.

1060
00:58:59,450 --> 00:59:03,860
The minus 1 here turns
out not to matter much.

1061
00:59:03,860 --> 00:59:04,710
Cool.

1062
00:59:04,710 --> 00:59:08,110
So this linear time sorting
on log n log log n items.

1063
00:59:08,110 --> 00:59:10,050
Everything it fits in a word.

1064
00:59:10,050 --> 00:59:13,060
Cool, but we're not done yet.

1065
00:59:13,060 --> 00:59:16,320
We need another level.

1066
00:59:16,320 --> 00:59:19,455
The rest is pretty easy.

1067
00:59:19,455 --> 00:59:21,010
It's just scaling up.

1068
00:59:54,660 --> 00:59:56,540
OK, let's see.

1069
00:59:56,540 --> 01:00:00,530
At this point we're going to
assume that all of our words

1070
01:00:00,530 --> 01:00:02,150
are completely full.

1071
01:00:02,150 --> 01:00:05,270
So this merging and stuff
was in order to fill up words

1072
01:00:05,270 --> 01:00:07,310
and to make those words sorted.

1073
01:00:07,310 --> 01:00:10,640
But once we've done that
for each cluster of log n

1074
01:00:10,640 --> 01:00:13,130
log log n items, now
we can assume each word

1075
01:00:13,130 --> 01:00:14,650
is sorted and completely full.

1076
01:00:14,650 --> 01:00:18,150
It has exactly k elements in it.

1077
01:00:18,150 --> 01:00:25,540
So each of these r sorted
words has k elements.

1078
01:00:25,540 --> 01:00:28,100
OK, so now the issue is
things don't fit in a word.

1079
01:00:28,100 --> 01:00:30,410
And so now suppose they
fit in r sorted words.

1080
01:00:30,410 --> 01:00:35,120
So supposed we've already made
a sorted list of r times k items

1081
01:00:35,120 --> 01:00:38,720
which fit in r words,
each with k items in it.

1082
01:00:38,720 --> 01:00:40,760
Then I want to take
two of these lists

1083
01:00:40,760 --> 01:00:42,590
and merge them
together and get one

1084
01:00:42,590 --> 01:00:44,620
sorted list of sorted words.

1085
01:00:44,620 --> 01:00:50,570
So it's 2 rk items that are
distributed into 2r words,

1086
01:00:50,570 --> 01:00:53,960
each of size k.

1087
01:00:53,960 --> 01:00:57,750
How do I do that
merge in linear time?

1088
01:00:57,750 --> 01:01:03,700
And I only need to do it
in r log k time, only.

1089
01:01:03,700 --> 01:01:04,550
Yeah, question?

1090
01:01:04,550 --> 01:01:08,860
AUDIENCE: I'm assuming k is
the number of items per word?

1091
01:01:08,860 --> 01:01:12,230
ERIK DEMAINE: Yeah, k
is this from now on.

1092
01:01:12,230 --> 01:01:16,129
And there are exactly
k elements per word.

1093
01:01:16,129 --> 01:01:18,170
We've already filled up
the word so at this point

1094
01:01:18,170 --> 01:01:19,320
we can assume that.

1095
01:01:19,320 --> 01:01:21,427
AUDIENCE: But it's exactly
equal, not less than.

1096
01:01:21,427 --> 01:01:22,260
ERIK DEMAINE: Right.

1097
01:01:22,260 --> 01:01:23,132
AUDIENCE: Because in one--

1098
01:01:23,132 --> 01:01:24,830
ERIK DEMAINE: Yeah, over here
it's less than or equal to.

1099
01:01:24,830 --> 01:01:26,121
I know it's a little confusing.

1100
01:01:26,121 --> 01:01:29,280
I use k for different--
call this k prime.

1101
01:01:29,280 --> 01:01:33,080
OK, and at this point we've
filled up to this value of k.

1102
01:01:33,080 --> 01:01:35,570
Now everything has exactly
log n log log n items.

1103
01:01:35,570 --> 01:01:38,120
Thanks.

1104
01:01:38,120 --> 01:01:40,070
OK, let's see.

1105
01:01:42,650 --> 01:01:45,590
A regular merging algorithm
would take r times k time

1106
01:01:45,590 --> 01:01:47,970
because they're r times k items.

1107
01:01:47,970 --> 01:01:51,980
But we have this merger
over here, which will

1108
01:01:51,980 --> 01:01:55,040
merge two words in log k time.

1109
01:01:55,040 --> 01:01:58,250
So that's why it's
going to be r log k.

1110
01:01:58,250 --> 01:01:59,300
What's the picture?

1111
01:01:59,300 --> 01:02:00,450
Well, let me draw it.

1112
01:02:16,900 --> 01:02:21,080
We have a list of words.

1113
01:02:21,080 --> 01:02:24,040
A list of words.

1114
01:02:24,040 --> 01:02:24,880
How do we merge?

1115
01:02:24,880 --> 01:02:27,880
Well, we take the first
two words, merge them.

1116
01:02:27,880 --> 01:02:30,620
When we merge them and we know
how to do this n log k time,

1117
01:02:30,620 --> 01:02:33,220
that's step 1.

1118
01:02:33,220 --> 01:02:34,954
When we merge them
we get two words,

1119
01:02:34,954 --> 01:02:35,995
they just are now sorted.

1120
01:02:39,100 --> 01:02:41,470
So we've got these
two guys sorted

1121
01:02:41,470 --> 01:02:43,510
and then there's the rest.

1122
01:02:43,510 --> 01:02:45,610
I'd like to say output
these and repeat,

1123
01:02:45,610 --> 01:02:48,640
but that wouldn't
be quite correct.

1124
01:02:48,640 --> 01:02:54,979
Well, these guys I can output
because I've got k items here,

1125
01:02:54,979 --> 01:02:55,520
k items here.

1126
01:02:55,520 --> 01:02:57,590
These are the k smallest
among all of them.

1127
01:02:57,590 --> 01:03:00,340
Those must be the
overall smallest k items.

1128
01:03:00,340 --> 01:03:02,050
Because I compared k with k.

1129
01:03:02,050 --> 01:03:04,930
These higher k
items, I don't know.

1130
01:03:04,930 --> 01:03:06,560
They might be good or bad.

1131
01:03:06,560 --> 01:03:11,971
So I have to put this high part
back into one of these lists.

1132
01:03:11,971 --> 01:03:13,720
You have to think a
little bit about which

1133
01:03:13,720 --> 01:03:14,590
list to put it into.

1134
01:03:14,590 --> 01:03:16,690
Turns out if you
look at the max item,

1135
01:03:16,690 --> 01:03:19,357
say where did this come
from, list one or list two?

1136
01:03:19,357 --> 01:03:20,440
You put it into that list.

1137
01:03:20,440 --> 01:03:22,815
Let's say this
came from list two.

1138
01:03:22,815 --> 01:03:25,970
And you put it back into
list two and then repeat.

1139
01:03:25,970 --> 01:03:32,710
OK, so in log k time here
we have output k items.

1140
01:03:32,710 --> 01:03:36,790
And so we end up with an r
log k running time overall.

1141
01:03:36,790 --> 01:03:39,490
OK, so it's like
regular merge algorithm,

1142
01:03:39,490 --> 01:03:43,420
except we use step
one to merge words,

1143
01:03:43,420 --> 01:03:45,690
and then there's a
little bit of work.

1144
01:03:45,690 --> 01:03:47,470
This is like old merging
algorithm, right?

1145
01:03:47,470 --> 01:03:50,770
You take the min and
you put the max back in.

1146
01:03:50,770 --> 01:03:55,040
Except now the max
is a whole word.

1147
01:03:55,040 --> 01:03:58,030
OK that's how we merge.

1148
01:03:58,030 --> 01:03:59,260
Now, why did we do a merger?

1149
01:03:59,260 --> 01:04:01,770
Because next thing
we do is merge sort.

1150
01:04:20,250 --> 01:04:22,160
And this is the
overall algorithm.

1151
01:04:22,160 --> 01:04:23,070
It's one merge sort.

1152
01:04:23,070 --> 01:04:24,750
We're going to use
this is the merger

1153
01:04:24,750 --> 01:04:26,650
and we're going to use
this as the base case.

1154
01:04:26,650 --> 01:04:30,110
So it's actually two recursive
levels of merge sort.

1155
01:04:30,110 --> 01:04:33,390
Merge sort is recursive
within step four for a while,

1156
01:04:33,390 --> 01:04:35,751
until we get down to the
level of a single word.

1157
01:04:35,751 --> 01:04:38,250
Then we use this thing to deal
with things in a single word.

1158
01:04:42,580 --> 01:04:49,230
So we end up with t of n
is 2 times t of n over 2

1159
01:04:49,230 --> 01:04:55,890
plus n over k log k.

1160
01:04:55,890 --> 01:05:00,000
And base case of
t of k is order k.

1161
01:05:07,350 --> 01:05:08,670
Why is it all this?

1162
01:05:08,670 --> 01:05:11,750
This thing is r.

1163
01:05:15,450 --> 01:05:20,430
So we have r log k to do a
merge. r here is n over k.

1164
01:05:20,430 --> 01:05:21,470
Why is it n over k?

1165
01:05:24,250 --> 01:05:26,190
Because n is the
total number of items.

1166
01:05:26,190 --> 01:05:29,190
We had r times k
items, so n equals rk.

1167
01:05:29,190 --> 01:05:31,040
So r equals n over k.

1168
01:05:31,040 --> 01:05:32,177
That's that.

1169
01:05:32,177 --> 01:05:33,260
So this is our merge cost.

1170
01:05:33,260 --> 01:05:38,160
This is the cost of 3,
this is the cost of 2.

1171
01:05:38,160 --> 01:05:41,920
If we have only log
n log log n items,

1172
01:05:41,920 --> 01:05:48,150
so k items, then we can
sort in linear time.

1173
01:05:48,150 --> 01:05:50,332
So we just need to
solve this recurrence.

1174
01:05:52,990 --> 01:05:56,820
Well, how's it going to work?

1175
01:05:56,820 --> 01:06:00,770
It's going to be kind of like
merge sort, but not quite.

1176
01:06:00,770 --> 01:06:01,270
Question?

1177
01:06:03,920 --> 01:06:06,170
AUDIENCE: When you say-- so
you have-- it's definitely

1178
01:06:06,170 --> 01:06:08,620
merging two lists
of sorted words

1179
01:06:08,620 --> 01:06:11,070
into one sorted
list of two words.

1180
01:06:11,070 --> 01:06:12,050
[INAUDIBLE]?

1181
01:06:16,950 --> 01:06:18,910
You may have to do that, right?

1182
01:06:21,232 --> 01:06:22,440
ERIK DEMAINE: Say that again.

1183
01:06:22,440 --> 01:06:24,316
AUDIENCE: So you
have two sorted--

1184
01:06:24,316 --> 01:06:27,425
two sorted lists
of R sorted words.

1185
01:06:27,425 --> 01:06:28,800
ERIK DEMAINE: The
words are going

1186
01:06:28,800 --> 01:06:31,530
to change, which elements are
in which words are changing.

1187
01:06:31,530 --> 01:06:36,300
But that's thanks to
step one, for example.

1188
01:06:36,300 --> 01:06:38,640
I mean, step one can
merge any two sorted words

1189
01:06:38,640 --> 01:06:41,070
and make one sorted word.

1190
01:06:41,070 --> 01:06:45,180
Or actually, one sorted word.

1191
01:06:45,180 --> 01:06:48,360
In our case, we're going
to get two sorted words.

1192
01:06:48,360 --> 01:06:49,840
That was the picture over here.

1193
01:06:49,840 --> 01:06:51,510
I merge the two guys.

1194
01:06:51,510 --> 01:06:54,720
I could represent it as one
word that's kind of double full,

1195
01:06:54,720 --> 01:06:57,420
and then I can split
that into two words.

1196
01:06:57,420 --> 01:06:59,190
So we again need
the split operation,

1197
01:06:59,190 --> 01:07:04,620
but split is just a mask
these guys out and you

1198
01:07:04,620 --> 01:07:05,760
get the high part.

1199
01:07:05,760 --> 01:07:07,090
This is all 0.

1200
01:07:07,090 --> 01:07:09,480
Then you shift it
over to the left,

1201
01:07:09,480 --> 01:07:14,130
now you've got the high
word over the single word.

1202
01:07:14,130 --> 01:07:17,230
So they're shuffling
within those two words,

1203
01:07:17,230 --> 01:07:20,980
or within one word,
essentially, via this procedure,

1204
01:07:20,980 --> 01:07:22,859
which we haven't covered.

1205
01:07:22,859 --> 01:07:24,400
So everything is
going to mix around.

1206
01:07:24,400 --> 01:07:27,064
But then there's the low
items and the high items.

1207
01:07:27,064 --> 01:07:28,980
And so all the items in
here are less than all

1208
01:07:28,980 --> 01:07:32,080
of the items in here.

1209
01:07:32,080 --> 01:07:33,630
Other questions?

1210
01:07:33,630 --> 01:07:34,980
Cool.

1211
01:07:34,980 --> 01:07:37,150
So I need to solve
this recurrence.

1212
01:07:37,150 --> 01:07:40,040
So let's draw another
recursion tree.

1213
01:07:40,040 --> 01:07:42,250
That's the east way
to solve recurrences.

1214
01:07:42,250 --> 01:07:44,970
Root is n over k log k.

1215
01:07:44,970 --> 01:07:48,750
Next level down, see
n got divided by 2.

1216
01:07:48,750 --> 01:07:57,690
So it's going to be 1/2 n over
k log k, 1/2 n over k log k,

1217
01:07:57,690 --> 01:07:59,460
because the n gets divided by 2.

1218
01:07:59,460 --> 01:08:01,410
So if I add up
everything on this level,

1219
01:08:01,410 --> 01:08:04,080
I get n over k log k.

1220
01:08:04,080 --> 01:08:06,870
And in general as I
go down, all levels

1221
01:08:06,870 --> 01:08:10,080
are going to sum to this
same value, n over k log k.

1222
01:08:10,080 --> 01:08:13,335
So up here we are going
to have a cost of n

1223
01:08:13,335 --> 01:08:18,120
over k log k times
the number of levels.

1224
01:08:18,120 --> 01:08:20,069
Now, this is where
it's a little tricky.

1225
01:08:20,069 --> 01:08:22,080
It's not log n levels
like regular merge sort.

1226
01:08:22,080 --> 01:08:25,010
We stop at level k.

1227
01:08:25,010 --> 01:08:28,050
So it's going to
be log of n over k.

1228
01:08:33,050 --> 01:08:35,370
That's the number
of levels we get to.

1229
01:08:35,370 --> 01:08:40,939
Now at the leaf level finally
we have things of size order k,

1230
01:08:40,939 --> 01:08:44,020
and we only pay order k.

1231
01:08:44,020 --> 01:08:46,470
Great.

1232
01:08:46,470 --> 01:08:48,915
And then how many
leaves are there?

1233
01:08:48,915 --> 01:08:50,460
There's n over k leaves.

1234
01:08:56,230 --> 01:08:56,729
OK.

1235
01:08:56,729 --> 01:09:00,050
So, that's an order n cost.

1236
01:09:00,050 --> 01:09:05,640
This is order k, that's
n over k, order n.

1237
01:09:05,640 --> 01:09:08,715
What's this thing?

1238
01:09:08,715 --> 01:09:09,779
Well, what's k?

1239
01:09:09,779 --> 01:09:12,840
k was log n log log n.

1240
01:09:12,840 --> 01:09:16,680
So this is n divided
by log n log log n.

1241
01:09:16,680 --> 01:09:20,130
This, therefore, is
approximately log log n.

1242
01:09:20,130 --> 01:09:22,330
So that log log n
cancels this one.

1243
01:09:22,330 --> 01:09:26,065
This is approximately log n,
so that cancels with this one.

1244
01:09:26,065 --> 01:09:29,340
So this is order n.

1245
01:09:29,340 --> 01:09:32,104
That's why we
chose these values.

1246
01:09:32,104 --> 01:09:34,229
So this n over k was not
actually that significant.

1247
01:09:34,229 --> 01:09:35,700
We could have called that n.

1248
01:09:35,700 --> 01:09:40,140
What is important is that this
recursion doesn't work when

1249
01:09:40,140 --> 01:09:42,240
you get down to a single word.

1250
01:09:42,240 --> 01:09:43,800
Can't do the same strategy.

1251
01:09:43,800 --> 01:09:47,700
You need to use a different
merger for single words

1252
01:09:47,700 --> 01:09:49,800
because here we have
to do bit tricks.

1253
01:09:49,800 --> 01:09:51,180
For this merge
strategy we didn't

1254
01:09:51,180 --> 01:09:53,305
have to do big tricks
because everything was bigger

1255
01:09:53,305 --> 01:09:55,856
than a word.

1256
01:09:55,856 --> 01:09:57,480
The reason why we
have these two levels

1257
01:09:57,480 --> 01:10:01,050
is not to make the
time fast, it's just

1258
01:10:01,050 --> 01:10:02,640
to make the algorithm work.

1259
01:10:02,640 --> 01:10:05,880
I mean, to define the algorithm
there's within a word merging

1260
01:10:05,880 --> 01:10:08,880
and over many words merging.

1261
01:10:08,880 --> 01:10:11,910
The hard part is this one,
within a word merging.

1262
01:10:11,910 --> 01:10:14,650
So that's our next goal.

1263
01:10:14,650 --> 01:10:19,560
But if we could do that we
get linear time sorting for

1264
01:10:19,560 --> 01:10:22,530
packed sorting when you can fit
log log log n things in a word.

1265
01:10:22,530 --> 01:10:24,280
Any questions about
packed sorting?

1266
01:10:27,180 --> 01:10:28,160
Good.

1267
01:10:28,160 --> 01:10:33,800
So, last thing is how do
we merge two sorted words

1268
01:10:33,800 --> 01:10:34,790
into one sorted word.

1269
01:10:37,562 --> 01:10:39,770
And this is where we're
going to use bitonic sorting.

1270
01:10:39,770 --> 01:10:42,394
See, it says right here, bitonic
sort for merging sorted words.

1271
01:10:42,394 --> 01:10:45,500
Here we know what to do.

1272
01:10:45,500 --> 01:10:47,048
Much easier in hindsight.

1273
01:10:51,660 --> 01:10:53,660
So I'm going to take a
little diversion tell you

1274
01:10:53,660 --> 01:10:56,405
about bitonic sorting, briefly.

1275
01:11:01,240 --> 01:11:04,820
Because it's NCLRS I don't want
to spend too much time on it,

1276
01:11:04,820 --> 01:11:07,740
but it is something we need.

1277
01:11:07,740 --> 01:11:09,380
So you need to
know how it works.

1278
01:11:14,310 --> 01:11:17,810
So first thing is the idea
of a bitonic sequence.

1279
01:11:17,810 --> 01:11:26,150
This is going to be a cyclic
shift of a uni-modal sequence.

1280
01:11:26,150 --> 01:11:32,900
So a non-decreasing plus
a non-increasing sequence.

1281
01:11:36,110 --> 01:11:39,650
So I want non-decreasing
and then non-increasing.

1282
01:11:39,650 --> 01:11:41,530
So this is a bitonic sequence.

1283
01:11:41,530 --> 01:11:44,210
I think you know
what this means.

1284
01:11:44,210 --> 01:11:46,920
Or it could be a
cyclic shift thereof.

1285
01:11:46,920 --> 01:11:49,280
So I could, for example,
take this part here

1286
01:11:49,280 --> 01:11:51,270
and shift it over to this side.

1287
01:11:51,270 --> 01:11:55,220
So then I'd end up with
this, this, and this.

1288
01:11:55,220 --> 01:11:56,606
This is also bitonic.

1289
01:11:56,606 --> 01:11:57,980
or I could shift
this whole thing

1290
01:11:57,980 --> 01:12:01,710
and put it over there
so I end up with that.

1291
01:12:01,710 --> 01:12:03,470
That's also bitonic.

1292
01:12:03,470 --> 01:12:06,890
So it basically has one
max, one min if you're

1293
01:12:06,890 --> 01:12:08,630
reading it cyclically.

1294
01:12:08,630 --> 01:12:10,882
One local max, one local min.

1295
01:12:10,882 --> 01:12:13,090
Except, it's non-decreasing
so it's a little messier,

1296
01:12:13,090 --> 01:12:16,230
but same thing.

1297
01:12:16,230 --> 01:12:20,860
OK, let's go over here.

1298
01:12:37,900 --> 01:12:39,870
So if you have a
bitonic sequence

1299
01:12:39,870 --> 01:12:43,780
there's something called a
bitonic sorting network, which

1300
01:12:43,780 --> 01:12:46,410
you can think of as a
parallel sorting algorithm,

1301
01:12:46,410 --> 01:12:47,560
to sort such a thing.

1302
01:12:47,560 --> 01:12:50,700
So it's almost
sorted, but not quite.

1303
01:12:50,700 --> 01:12:52,620
And there's this
way of sorting them.

1304
01:12:52,620 --> 01:12:57,900
So I'm going to draw a picture
to make it easier for me.

1305
01:13:00,692 --> 01:13:02,400
It's a little more
annoying to write down

1306
01:13:02,400 --> 01:13:03,450
the general algorithm.

1307
01:13:03,450 --> 01:13:06,260
But so what I want to do,
suppose the number of elements

1308
01:13:06,260 --> 01:13:07,014
is a power of 2.

1309
01:13:07,014 --> 01:13:08,180
Here's what I'm going to do.

1310
01:13:08,180 --> 01:13:12,059
First I compare the first
and the midway element.

1311
01:13:12,059 --> 01:13:14,100
And then the next one with
the next one, and then

1312
01:13:14,100 --> 01:13:16,800
the next one with the next one,
the next one with the next one.

1313
01:13:16,800 --> 01:13:18,758
For each of these
comparisons I put those items

1314
01:13:18,758 --> 01:13:20,080
in the correct order.

1315
01:13:20,080 --> 01:13:22,250
So if I do that 3 versus
6, 3 is less than 6,

1316
01:13:22,250 --> 01:13:24,800
so 3 stays here, 6 stays there.

1317
01:13:24,800 --> 01:13:27,840
5 is bigger than 4 though,
so 4 comes over here,

1318
01:13:27,840 --> 01:13:29,370
5 goes over here.

1319
01:13:29,370 --> 01:13:35,430
Next is 7 versus 2, so 2 comes
over here, 7 comes over here.

1320
01:13:35,430 --> 01:13:39,700
Nine versus 0, so 0 is
over here, 9 is over there.

1321
01:13:39,700 --> 01:13:42,480
OK, that's an easy
set comparisons to do.

1322
01:13:42,480 --> 01:13:45,710
The nice thing is you can
do all those in parallel.

1323
01:13:45,710 --> 01:13:47,400
OK, just to check here.

1324
01:13:47,400 --> 01:13:50,460
This was a monotone
increasing sub-sequence

1325
01:13:50,460 --> 01:13:53,590
and then a monotone decreasing.

1326
01:13:53,590 --> 01:13:57,270
Now, we have increasing
then decreasing.

1327
01:13:57,270 --> 01:14:02,250
And then we have
decreasing then increasing.

1328
01:14:02,250 --> 01:14:05,310
But if you look at the left
half or the right half,

1329
01:14:05,310 --> 01:14:06,860
they are still bitonic.

1330
01:14:06,860 --> 01:14:08,610
And that's an invariant.

1331
01:14:08,610 --> 01:14:10,560
Bitonic will be preserved here.

1332
01:14:10,560 --> 01:14:12,870
Also notice, all the
elements in the left

1333
01:14:12,870 --> 01:14:14,870
are smaller than all the
elements in the right.

1334
01:14:14,870 --> 01:14:16,620
That's not true for
an arbitrary sequence,

1335
01:14:16,620 --> 01:14:18,328
but it will be true
for bitonic sequence.

1336
01:14:18,328 --> 01:14:22,260
Essentially the max
is somewhere here

1337
01:14:22,260 --> 01:14:25,050
and so it's going to
be spanned by some

1338
01:14:25,050 --> 01:14:27,260
of these intervals, that max.

1339
01:14:27,260 --> 01:14:29,760
And by spanning those intervals,
and in particular comparing

1340
01:14:29,760 --> 01:14:31,504
that guy with
somebody on the right,

1341
01:14:31,504 --> 01:14:32,920
the max will get
over to the right

1342
01:14:32,920 --> 01:14:36,870
and then the rest of the
increasing and decreasing

1343
01:14:36,870 --> 01:14:38,530
sub-sequences will follow.

1344
01:14:38,530 --> 01:14:40,440
That's a very
hand-wavy argument.

1345
01:14:40,440 --> 01:14:45,000
You can see CLRS for a
way to prove that OK, just

1346
01:14:45,000 --> 01:14:46,740
take it for granted now.

1347
01:14:46,740 --> 01:14:48,420
Now, we recurse
on the two sides.

1348
01:14:48,420 --> 01:14:52,320
So I compare 3 with 2, 4
with 0, I compare 6 with 7,

1349
01:14:52,320 --> 01:14:53,510
and 6 with 9.

1350
01:14:53,510 --> 01:14:54,990
So what do I get?

1351
01:14:54,990 --> 01:14:56,760
2 is less than 3.

1352
01:14:56,760 --> 01:14:58,570
0 is less than 4.

1353
01:14:58,570 --> 01:15:02,450
Then I compare 6 with
7, so 6 less than 7.

1354
01:15:02,450 --> 01:15:05,910
5 is less than 9.

1355
01:15:05,910 --> 01:15:11,010
So now I've got four chunks,
this one, this one, this one,

1356
01:15:11,010 --> 01:15:11,670
and this one.

1357
01:15:11,670 --> 01:15:13,320
I recurse on each of them.

1358
01:15:13,320 --> 01:15:18,110
So 0 is less than 2, 3 is
less than 4, 5 is less than 6,

1359
01:15:18,110 --> 01:15:19,660
7 is less than 9.

1360
01:15:19,660 --> 01:15:23,929
Now I have all my items in
sorted order, magically.

1361
01:15:23,929 --> 01:15:25,470
OK, but if you
believe that invariant

1362
01:15:25,470 --> 01:15:28,680
that the smallest
n over two items

1363
01:15:28,680 --> 01:15:30,900
get to the left and the
larger n over two items

1364
01:15:30,900 --> 01:15:34,740
get to the right, then by
induction this will sort them.

1365
01:15:34,740 --> 01:15:37,431
Kind of like quick
sort in that sense.

1366
01:15:37,431 --> 01:15:37,930
OK.

1367
01:15:37,930 --> 01:15:39,600
The cool thing is I can
do all these in parallel,

1368
01:15:39,600 --> 01:15:41,474
all these in parallel,
all these in parallel.

1369
01:15:41,474 --> 01:15:46,140
So the total time is log k,
if I'm a parallel sorting

1370
01:15:46,140 --> 01:15:48,030
algorithm or a sorting network.

1371
01:15:48,030 --> 01:15:51,270
And log k is exactly
what I can afford.

1372
01:15:51,270 --> 01:15:54,480
So the last thing to do is to
implement an operation like

1373
01:15:54,480 --> 01:15:57,600
this, or like this-- this is
kind of the generic picture

1374
01:15:57,600 --> 01:16:00,030
here--

1375
01:16:00,030 --> 01:16:01,500
in constant time on a word ram.

1376
01:16:13,380 --> 01:16:15,540
One other thing,
because I don't want

1377
01:16:15,540 --> 01:16:17,010
to sort a bitonic sequence.

1378
01:16:17,010 --> 01:16:17,890
That wasn't my goal.

1379
01:16:17,890 --> 01:16:20,910
My goal was to merge
two sorted sequences.

1380
01:16:20,910 --> 01:16:24,646
I have two sorted sequences,
I want to merge them.

1381
01:16:24,646 --> 01:16:25,770
How do I make that bitonic?

1382
01:16:28,460 --> 01:16:32,130
I just flip, like that.

1383
01:16:32,130 --> 01:16:34,642
I take this sequence
and I reverse it.

1384
01:16:34,642 --> 01:16:36,600
If I take a sorted sequence
and reverse it then

1385
01:16:36,600 --> 01:16:37,391
it looks like this.

1386
01:16:37,391 --> 01:16:39,600
And so then the
result is bitonic.

1387
01:16:39,600 --> 01:16:41,850
Then I can use bitonic
sorting to sort it.

1388
01:16:41,850 --> 01:16:43,950
So this is the merge
problem and I've just

1389
01:16:43,950 --> 01:16:47,220
reduced it to bitonic sorting.

1390
01:16:47,220 --> 01:16:48,060
Cool.

1391
01:16:48,060 --> 01:16:52,610
Except, how do I
reverse a sequence

1392
01:16:52,610 --> 01:16:56,770
in log k time on a word ram?

1393
01:16:56,770 --> 01:16:57,480
Cute trick.

1394
01:17:00,735 --> 01:17:04,950
If I have a word and it
has many items in it--

1395
01:17:04,950 --> 01:17:07,042
that's our picture,
that's our set up here--

1396
01:17:07,042 --> 01:17:09,000
and I want to reverse it
over all, what I'll do

1397
01:17:09,000 --> 01:17:12,990
is cut it in half, take all
these guys put them over here,

1398
01:17:12,990 --> 01:17:15,870
take all these guys
put them over here.

1399
01:17:15,870 --> 01:17:18,315
So I've got the right side--

1400
01:17:18,315 --> 01:17:21,060
sorry, it's only four--

1401
01:17:21,060 --> 01:17:22,480
cells over here.

1402
01:17:22,480 --> 01:17:24,930
I've got the left
side over here.

1403
01:17:24,930 --> 01:17:30,420
Then I recursively
reverse the sequence

1404
01:17:30,420 --> 01:17:34,300
and recursively
reverse this sequence.

1405
01:17:34,300 --> 01:17:37,220
And then I have the
reverse sequence.

1406
01:17:37,220 --> 01:17:39,040
Right, this is a standard.

1407
01:17:39,040 --> 01:17:42,220
One way to reverse items
is to do the big part

1408
01:17:42,220 --> 01:17:43,830
and then recursively reverse.

1409
01:17:43,830 --> 01:17:48,430
Then I will get, I mean
this is like ab reverse is

1410
01:17:48,430 --> 01:17:51,180
equal to b reverse a reverse.

1411
01:17:51,180 --> 01:17:52,570
Just a fun fact.

1412
01:17:52,570 --> 01:17:55,180
So we're implementing that.

1413
01:17:55,180 --> 01:17:56,810
Now how do I do this?

1414
01:17:56,810 --> 01:17:59,860
Well, this is the thing I've
been talking about over here.

1415
01:17:59,860 --> 01:18:02,764
You want to take the high part
you just mask that part out,

1416
01:18:02,764 --> 01:18:03,680
shift it to the right.

1417
01:18:03,680 --> 01:18:06,600
You want to take the low
part, just mask that part out.

1418
01:18:06,600 --> 01:18:08,440
And so I can take
each of these parts

1419
01:18:08,440 --> 01:18:10,330
individually, shift
this one to the left,

1420
01:18:10,330 --> 01:18:12,610
shift this one to the
right, order them together,

1421
01:18:12,610 --> 01:18:13,690
and I've reversed.

1422
01:18:13,690 --> 01:18:16,000
So this takes constant time.

1423
01:18:16,000 --> 01:18:18,340
This is recursion but
the total number of steps

1424
01:18:18,340 --> 01:18:23,770
will be order log k, which
is exactly what I can afford.

1425
01:18:23,770 --> 01:18:26,260
So that's how I do
the reverse part.

1426
01:18:26,260 --> 01:18:29,620
So if I want to merge, now
I have a bitonic sequence.

1427
01:18:29,620 --> 01:18:35,080
Last thing to do is, how do
I do this bitonic comparison

1428
01:18:35,080 --> 01:18:36,296
in constant time.

1429
01:18:45,970 --> 01:18:47,665
So, that's next page.

1430
01:18:52,734 --> 01:18:53,650
Here's a quick sketch.

1431
01:19:01,740 --> 01:19:03,690
I'm going to try to draw
the generic picture.

1432
01:19:03,690 --> 01:19:05,900
So the generic picture is
I have this thing which

1433
01:19:05,900 --> 01:19:08,660
I call a, this thing
which I call b,

1434
01:19:08,660 --> 01:19:11,960
and we basically want to compare
corresponding items in a and b.

1435
01:19:11,960 --> 01:19:14,330
Though in fact, there's
another a over here and another

1436
01:19:14,330 --> 01:19:15,320
b over here.

1437
01:19:15,320 --> 01:19:18,110
In general, there are many
a's, many b's, but they appear

1438
01:19:18,110 --> 01:19:19,580
in this nice periodic pattern.

1439
01:19:19,580 --> 01:19:21,650
And so I'm just going
to look at one ab pair,

1440
01:19:21,650 --> 01:19:24,500
but it's going to simultaneously
work for all of them

1441
01:19:24,500 --> 01:19:26,186
with the same operations.

1442
01:19:26,186 --> 01:19:27,560
So that's the dot,
dot, dot here.

1443
01:19:27,560 --> 01:19:29,101
There's another a
and then another b,

1444
01:19:29,101 --> 01:19:30,860
and another a and another b.

1445
01:19:30,860 --> 01:19:32,810
So what do I do?

1446
01:19:32,810 --> 01:19:40,760
First thing I do is
make 1aa, 1aa, 1aa.

1447
01:19:40,760 --> 01:19:42,510
In other words, I
add in these 1 bits.

1448
01:19:42,510 --> 01:19:46,555
Remember, we had zeros
there hanging out.

1449
01:19:46,555 --> 01:19:48,230
That was when we were
going to use them.

1450
01:19:48,230 --> 01:19:51,030
So I set them all to
ones in the a list.

1451
01:19:51,030 --> 01:19:54,095
I take the b list, I just
shift it over and leave them

1452
01:19:54,095 --> 01:19:54,595
as zeros.

1453
01:20:00,300 --> 01:20:01,870
Time for bit tricks.

1454
01:20:01,870 --> 01:20:05,160
Then I take this number, I
subtract it from this number.

1455
01:20:05,160 --> 01:20:06,610
What do I get?

1456
01:20:06,610 --> 01:20:10,140
I get zeros or ones for
whether this bit got

1457
01:20:10,140 --> 01:20:12,030
borrowed, which tells
me whether little a is

1458
01:20:12,030 --> 01:20:13,350
bigger than little b.

1459
01:20:13,350 --> 01:20:14,440
Then I get some junk.

1460
01:20:14,440 --> 01:20:15,300
And then I get 0, 1.

1461
01:20:15,300 --> 01:20:16,320
Junk.

1462
01:20:16,320 --> 01:20:17,310
So on.

1463
01:20:17,310 --> 01:20:20,730
0, 1, junk.

1464
01:20:20,730 --> 01:20:23,880
So 0 corresponds
to a being smaller

1465
01:20:23,880 --> 01:20:26,720
than b, and 1 corresponds
to the reverse.

1466
01:20:26,720 --> 01:20:27,800
OK, whatever.

1467
01:20:27,800 --> 01:20:31,820
Then I do my usual masking,
just the 0, 1 bits,

1468
01:20:31,820 --> 01:20:32,820
then there's some zeros.

1469
01:20:36,640 --> 01:20:38,760
OK, here's a new trick
which we haven't seen yet.

1470
01:20:38,760 --> 01:20:42,450
I'm going to shift this to the
right by two, or in general

1471
01:20:42,450 --> 01:20:44,470
by the size of those items.

1472
01:20:44,470 --> 01:20:48,810
So now I have 0, 1 here,
0, 1 here, 0, 1 here.

1473
01:20:48,810 --> 01:20:50,880
Each of these bits
might be different.

1474
01:20:50,880 --> 01:20:53,140
So I take this,
shift to the right.

1475
01:20:53,140 --> 01:20:55,560
Now I subtract.

1476
01:20:55,560 --> 01:20:57,400
What this gives me--

1477
01:20:57,400 --> 01:21:00,690
so I'm taking 1,
0, 0, 0, minus 1.

1478
01:21:00,690 --> 01:21:03,750
What I get is 0, 1, 1, 1.

1479
01:21:03,750 --> 01:21:06,160
OK, in this case,
there's just two bits.

1480
01:21:06,160 --> 01:21:07,230
Except it's not 1, 1.

1481
01:21:07,230 --> 01:21:14,760
It's either 0, 0 or 1, 1.

1482
01:21:14,760 --> 01:21:22,120
And then we have here 0, and
then 0, 0 or 1, 1, and so on.

1483
01:21:22,120 --> 01:21:24,630
So now these are masks.

1484
01:21:24,630 --> 01:21:28,410
The zeros correspond to
when the a's were smaller,

1485
01:21:28,410 --> 01:21:31,020
the ones correspond to
when the b's are smaller.

1486
01:21:31,020 --> 01:21:38,430
So if I take this [? added ?]
with this, I get the large b's.

1487
01:21:38,430 --> 01:21:42,480
If I take this [? added ?]
with this, I get the small a's.

1488
01:21:42,480 --> 01:21:45,630
Or one of some of
those combinations.

1489
01:21:45,630 --> 01:21:48,240
The point is I take--

1490
01:21:48,240 --> 01:21:51,895
I think here I
get the small a's.

1491
01:21:55,080 --> 01:22:05,265
And then if I take this thing
and negate it and then mask it

1492
01:22:05,265 --> 01:22:07,740
with this I get the small b's.

1493
01:22:11,750 --> 01:22:13,120
And then I take this thing--

1494
01:22:13,120 --> 01:22:14,950
I ran out of room.

1495
01:22:14,950 --> 01:22:19,210
So I've got small
a's, small b's.

1496
01:22:22,240 --> 01:22:25,450
I take this thing I shift it
over so I have small b's here.

1497
01:22:29,020 --> 01:22:33,620
And then I OR these together,
I get all the smalls.

1498
01:22:33,620 --> 01:22:36,940
OK, these are designed to
never conflict because I

1499
01:22:36,940 --> 01:22:38,604
took the inversion here.

1500
01:22:38,604 --> 01:22:40,770
So I get the smaller of the
a's, smaller of the b's.

1501
01:22:40,770 --> 01:22:44,590
So now I put the smaller
things in the right order.

1502
01:22:44,590 --> 01:22:48,170
I do exactly the same procedure
negated, and get the larges.

1503
01:22:51,210 --> 01:22:54,530
And then I OR these
together and I get sorted.

1504
01:22:57,430 --> 01:22:58,330
Well, not sorted.

1505
01:22:58,330 --> 01:23:01,600
I get whatever bitonic
sort would have done.

1506
01:23:01,600 --> 01:23:05,140
It's doing these pairwise
comparisons by doing this shift

1507
01:23:05,140 --> 01:23:06,970
and it's putting things
in the right order.

1508
01:23:06,970 --> 01:23:08,890
The small ones always
end up on the left,

1509
01:23:08,890 --> 01:23:10,640
the large ones always
end up on the right.

1510
01:23:10,640 --> 01:23:13,790
That's the definition of
the operation we want to do.

1511
01:23:13,790 --> 01:23:16,769
Bitonic operated.

1512
01:23:16,769 --> 01:23:19,060
And because we are just doing
shifts, and subtractions,

1513
01:23:19,060 --> 01:23:21,610
and all these things, it works
even though there's not just

1514
01:23:21,610 --> 01:23:23,830
one ab pair, but there's
a whole bunch of ab pairs.

1515
01:23:23,830 --> 01:23:28,460
They will all be shifted, and
computed, and sorted together.

1516
01:23:28,460 --> 01:23:31,210
It's a little hard
to visualize but this

1517
01:23:31,210 --> 01:23:34,090
will do one bitonic operation
and a constant number

1518
01:23:34,090 --> 01:23:35,890
of word ram operations.

1519
01:23:35,890 --> 01:23:38,350
And so you pop out
all these stacks

1520
01:23:38,350 --> 01:23:41,110
this gives us a merger, that
gives us packed sorting,

1521
01:23:41,110 --> 01:23:45,130
and with packed sorting
we get signature sorting.

1522
01:23:45,130 --> 01:23:47,351
Easy, right?

1523
01:23:47,351 --> 01:23:49,600
All right, that's probably
one of the most complicated

1524
01:23:49,600 --> 01:23:51,058
algorithms we'll
see in this class.

1525
01:23:51,058 --> 01:23:53,230
But from that you get
a priority queue that

1526
01:23:53,230 --> 01:23:56,110
runs in constant time provided
w is a little bigger than log

1527
01:23:56,110 --> 01:23:57,010
squared n.

1528
01:23:57,010 --> 01:23:58,560
Boom.