1
00:00:08,000 --> 00:00:11,000
OK.
Today we are going to talk

2
00:00:11,000 --> 00:00:18,000
about a very interesting
algorithm called Quicksort --

3
00:00:23,000 --> 00:00:30,000
-- which was invented by Tony
Hoare in 1962.

4
00:00:30,000 --> 00:00:35,000
And it has ended up being a
really interesting algorithm

5
00:00:35,000 --> 00:00:40,000
from many points of view.
And because of that,

6
00:00:40,000 --> 00:00:47,000
it turns out today's lecture is
going to be both hard and fast.

7
00:00:47,000 --> 00:00:52,000
If you see the person next to
you sleeping,

8
00:00:52,000 --> 00:00:56,000
you will want to say let's get
going.

9
00:00:56,000 --> 00:01:01,000
It's a divide-and-conquer
algorithm.

10
00:01:06,000 --> 00:01:08,000
And it sorts,
as they say,

11
00:01:08,000 --> 00:01:14,000
in place, meaning that it just
rearranged the elements where

12
00:01:14,000 --> 00:01:17,000
they are.
That is like insertion sort

13
00:01:17,000 --> 00:01:20,000
rearranges elements where they
are.

14
00:01:20,000 --> 00:01:24,000
Mergesort does not.
Mergesort requires extra

15
00:01:24,000 --> 00:01:30,000
storage in order to do the merge
operation.

16
00:01:30,000 --> 00:01:34,000
To merge in linear time and
place, it doesn't merge in place

17
00:01:34,000 --> 00:01:37,000
in linear time.
It doesn't do it just by

18
00:01:37,000 --> 00:01:41,000
rearranging.
It is nice because it is in

19
00:01:41,000 --> 00:01:45,000
place, so that means that it is
fairly efficient in its use of

20
00:01:45,000 --> 00:01:48,000
storage.
And it also happens to be very

21
00:01:48,000 --> 00:01:53,000
practical if you tune it a bit.
The basic algorithm turns out,

22
00:01:53,000 --> 00:01:58,000
if you just implement that,
it's not necessarily that

23
00:01:58,000 --> 00:02:02,000
efficient.
But if what you do was then do

24
00:02:02,000 --> 00:02:07,000
the standard kinds of things you
do to goose up the runtime of

25
00:02:07,000 --> 00:02:12,000
something, and we'll talk a
little about what those things

26
00:02:12,000 --> 00:02:16,000
are, then it can be very,
very practical.

27
00:02:16,000 --> 00:02:20,000
So, it uses divide-and-conquer
paradigm.

28
00:02:33,000 --> 00:02:40,000
First step is divide.
And to do this basically it

29
00:02:40,000 --> 00:02:48,000
does it by partitioning.
So, it partitions the input

30
00:02:48,000 --> 00:02:59,000
array into two subarrays around
an element we call the pivot --

31
00:03:04,000 --> 00:03:14,000
-- such that elements in the
lower subarray are less than or

32
00:03:14,000 --> 00:03:25,000
equal to x, are less than or
equal to elements in the upper

33
00:03:25,000 --> 00:03:31,000
subarray.
If we draw a picture of the

34
00:03:31,000 --> 00:03:38,000
input array, this partition step
basically takes some element x

35
00:03:38,000 --> 00:03:45,000
and everything over here is less
than or equal to x after the

36
00:03:45,000 --> 00:03:52,000
partition step and everything
over here is greater than or

37
00:03:52,000 --> 00:03:57,000
equal to x.
And so now the conquer step is

38
00:03:57,000 --> 00:04:04,000
pretty easy.
You just recursively sort the

39
00:04:04,000 --> 00:04:10,000
two subarrays.
So, I recursively sort the

40
00:04:10,000 --> 00:04:19,000
elements less than or equal to
x, I recursively sort the

41
00:04:19,000 --> 00:04:24,000
elements greater than or equal
to x.

42
00:04:24,000 --> 00:04:32,000
And then combine is then just
trivial.

43
00:04:32,000 --> 00:04:36,000
Because once I have sorted the
things less than or equal to x,

44
00:04:36,000 --> 00:04:40,000
then sorted the things greater
than or equal to x,

45
00:04:40,000 --> 00:04:44,000
the whole thing is sorted.
So, there is nothing to do

46
00:04:44,000 --> 00:04:48,000
really for the combine.
The key step in quicksort is

47
00:04:48,000 --> 00:04:52,000
this partition step.
That is the thing that does all

48
00:04:52,000 --> 00:04:55,000
of the work.
And so you can view quicksort

49
00:04:55,000 --> 00:04:57,000
of just as recursive
partitioning.

50
00:04:57,000 --> 00:05:06,000
That's all it is.
Just as mergesort was recursive

51
00:05:06,000 --> 00:05:18,000
merging, quicksort sort of goes
the other way around and does

52
00:05:18,000 --> 00:05:28,000
recursive partitioning.
The key is the linear time,

53
00:05:28,000 --> 00:05:40,000
by which I mean theta n,
partitioning subroutine.

54
00:05:40,000 --> 00:05:43,000
And here are some pseudocode
for it.

55
00:05:43,000 --> 00:05:47,000
This is actually slightly
different from the book.

56
00:05:47,000 --> 00:05:51,000
The book has one.
In fact, there is a nice

57
00:05:51,000 --> 00:05:55,000
problem in the book that has
even a different one,

58
00:05:55,000 --> 00:06:00,000
but they are all basically the
same idea.

59
00:06:00,000 --> 00:06:01,000
Partition (A,
p, q).

60
00:06:01,000 --> 00:06:06,000
And what we are looking at,
at this step of the recursion,

61
00:06:06,000 --> 00:06:11,000
is the subarray A from p to q.
And basically we pick a pivot,

62
00:06:11,000 --> 00:06:17,000
which is we are going to just
pick as the first element of the

63
00:06:17,000 --> 00:06:19,000
array A of p.

64
00:06:26,000 --> 00:06:29,000
And the book,
just for your information,

65
00:06:29,000 --> 00:06:32,000
uses A of q.
I use A of p.

66
00:06:32,000 --> 00:06:37,000
It doesn't really matter.
And then we set an index to p

67
00:06:37,000 --> 00:06:40,000
and then we have a loop.

68
00:07:35,000 --> 00:07:40,000
This is the code.
Basically the structure of it

69
00:07:40,000 --> 00:07:47,000
is a for loop with an "if"
statement in the middle.

70
00:07:47,000 --> 00:07:54,000
And so the structure of the
algorithm of this partitioning

71
00:07:54,000 --> 00:08:00,000
step looks as follows.
We set the pivot to be the

72
00:08:00,000 --> 00:08:05,000
first element.
Here is p and here is q.

73
00:08:05,000 --> 00:08:10,000
This is going to be our
invariant for the loop.

74
00:08:10,000 --> 00:08:15,000
And, at any time during the
execution of a loop,

75
00:08:15,000 --> 00:08:22,000
I essentially have some values
up to i which are already less

76
00:08:22,000 --> 00:08:28,000
than or equal to x and then some
values that end at j minus 1

77
00:08:28,000 --> 00:08:34,000
that are greater than or equal
to x.

78
00:08:34,000 --> 00:08:38,000
And then I don't know about the
rest.

79
00:08:38,000 --> 00:08:44,000
And so we start out with i
equal to p and j equal to p plus

80
00:00:01,000 --> 00:08:47,000
It starts out at p plus 1 so
This is called,
So, once again,

81
00:08:47,000 --> 00:08:52,000
that everything is unknown
except for x here.

82
00:08:52,000 --> 00:08:58,000
And then the idea is that it is
going to preserve this

83
00:08:58,000 --> 00:09:02,000
invariant.
And the way it does it is,

84
00:09:02,000 --> 00:09:06,000
as we go through the loop,
it looks at a of j and says is

85
00:09:06,000 --> 00:09:10,000
it greater than or equal to x?
Sorry, is it less than or equal

86
00:09:10,000 --> 00:09:12,000
to x?
If it is greater than or equal

87
00:09:12,000 --> 00:09:15,000
to x it does nothing,
because what can happen?

88
00:09:15,000 --> 00:09:19,000
If this is greater than or
equal to x, essentially it just

89
00:09:19,000 --> 00:09:23,000
goes to the next iterational
loop which moves this boundary

90
00:09:23,000 --> 00:09:27,000
and the invariant is satisfied.
Does everybody see that?

91
00:09:27,000 --> 00:09:31,000
Yeah, OK.
But if it is less than or equal

92
00:09:31,000 --> 00:09:35,000
to x, I have got a problem if I
want to maintain the invariant

93
00:09:35,000 --> 00:09:38,000
if this next element is less
than or equal to x.

94
00:09:38,000 --> 00:09:43,000
And so what it does then is it
says oh, let me just move this

95
00:09:43,000 --> 00:09:47,000
boundary and swap this element
here, which is greater than or

96
00:09:47,000 --> 00:09:51,000
equal to x, with this one here
that is less than or equal to x,

97
00:09:51,000 --> 00:09:55,000
thereby increasing the size of
this subarray and then the

98
00:09:55,000 --> 00:09:59,000
invariant is satisfied again.
It is a fairly simple

99
00:09:59,000 --> 00:10:03,000
algorithm.
And it is actually a very tight

100
00:10:03,000 --> 00:10:08,000
and easy algorithm.
That is one reason that this is

101
00:10:08,000 --> 00:10:13,000
such a great piece of code
because it is very efficient.

102
00:10:13,000 --> 00:10:18,000
Now, in principle,
the running time for this on n

103
00:10:18,000 --> 00:10:21,000
elements is order n.

104
00:10:28,000 --> 00:10:31,000
Because I am basically just
going through the n elements and

105
00:10:31,000 --> 00:10:35,000
just doing a constant amount of
work and then just a constant

106
00:10:35,000 --> 00:10:39,000
amount of work outside.
This is a clever piece of code.

107
00:10:39,000 --> 00:10:41,000
In fact, in principle partition
is easy, right?

108
00:10:41,000 --> 00:10:44,000
If I weren't worrying about
doing it in place,

109
00:10:44,000 --> 00:10:47,000
it is really a pretty easy
thing to do.

110
00:10:47,000 --> 00:10:50,000
I take an element and just
compare every other element with

111
00:10:50,000 --> 00:10:52,000
it.
I throw one into one bin and

112
00:10:52,000 --> 00:10:57,000
one into the other.
That is clearly linear time.

113
00:10:57,000 --> 00:11:01,000
But often what you find is that
just because you can do it that

114
00:11:01,000 --> 00:11:04,000
way theoretically doesn't mean
that that is going to end up

115
00:11:04,000 --> 00:11:08,000
giving you good code.
And this is a nice piece of

116
00:11:08,000 --> 00:11:10,000
code that allows you to do it in
place.

117
00:11:10,000 --> 00:11:14,000
And that is one reason why this
is a particularly good

118
00:11:14,000 --> 00:11:16,000
algorithm, because the constants
are good.

119
00:11:16,000 --> 00:11:20,000
So, yes, when we do asymptotic
analysis we tend to ignore the

120
00:11:20,000 --> 00:11:24,000
constants, but when you're
actually building code you care

121
00:11:24,000 --> 00:11:30,000
about the constants.
But first you care much more

122
00:11:30,000 --> 00:11:37,000
than just about the constants,
is whether overall it is going

123
00:11:37,000 --> 00:11:43,000
to be a fast algorithm.
Let's go through an example of

124
00:11:43,000 --> 00:11:50,000
this, I guess I will do it over
here, just so we get the gist.

125
00:11:50,000 --> 00:11:57,000
Here is a sample array that I
have created out of hallcloth.

126
00:11:57,000 --> 00:12:05,000
And here we are going to set x,
the pivot, to be 6.

127
00:12:05,000 --> 00:12:08,000
Let's look to see how this
algorithm works.

128
00:12:08,000 --> 00:12:12,000
So, i starts out here and j
starts out here if we

129
00:12:12,000 --> 00:12:15,000
initialize.
And what we do is start

130
00:12:15,000 --> 00:12:18,000
scanning right,
essentially that code is

131
00:12:18,000 --> 00:12:23,000
scanning right until it gets
something which is less than or

132
00:12:23,000 --> 00:12:27,000
equal to the pivot.
It keeps going here until it

133
00:12:27,000 --> 00:12:32,000
finds, j keeps incrementing
until it finds something that is

134
00:12:32,000 --> 00:12:38,000
less than or equal to the pivot.
And, in that case,

135
00:12:38,000 --> 00:12:44,000
it is the number 5.
Then it says we will swap these

136
00:12:44,000 --> 00:12:49,000
two things.
And it does that and we get 6,

137
00:12:49,000 --> 00:12:52,000
5, 13, 10, 8,
3, 2, 11.

138
00:12:52,000 --> 00:12:58,000
And meanwhile now i gets
incremented and j continues

139
00:12:58,000 --> 00:13:05,000
where it left off.
And so now we keep scanning

140
00:13:05,000 --> 00:13:12,000
right until we get to something
that is less than or equal to

141
00:13:12,000 --> 00:13:16,000
the pivot.
In this case it is 3.

142
00:13:16,000 --> 00:13:20,000
We swap 3 and 5 and get 6,
3, etc.

143
00:13:20,000 --> 00:13:27,000
And now, at this step we
increment i, we start j out

144
00:13:27,000 --> 00:13:30,000
here.
And in this case,

145
00:13:30,000 --> 00:13:35,000
right off the bat,
we have something which is less

146
00:13:35,000 --> 00:13:39,000
than or equal to x,
so we swap these two.

147
00:13:39,000 --> 00:13:42,000
I blew it, didn't I?
Oops.

148
00:13:42,000 --> 00:13:46,000
What did I do?
I swapped the wrong thing,

149
00:13:46,000 --> 00:13:48,000
didn't I, here?
Ah-ha.

150
00:13:48,000 --> 00:13:51,000
That is why I am not a
computer.

151
00:13:51,000 --> 00:13:54,000
Good.
We should have swapped this

152
00:13:54,000 --> 00:13:57,000
guy, right?
Swapped i plus 1,

153
00:13:57,000 --> 00:14:01,000
right?
This was i.

154
00:14:01,000 --> 00:14:03,000
We swap i plus 1,
good.

155
00:14:03,000 --> 00:14:09,000
So, that's all wrong.
Let's swap the right things.

156
00:14:09,000 --> 00:14:12,000
Now we have 6,
5, 3, 10, 8,

157
00:14:12,000 --> 00:14:16,000
13, 2, 11.
That even corresponds to my

158
00:14:16,000 --> 00:14:22,000
notes for some strange reason.
This is i and now this is j.

159
00:14:22,000 --> 00:14:28,000
And now when I look,
I immediately have something

160
00:14:28,000 --> 00:14:34,000
that is less than or equal to
the pivot.

161
00:14:34,000 --> 00:14:41,000
We swap this and i plus 1,
so now we have 6,

162
00:14:41,000 --> 00:14:45,000
5, 3, 2, 8, 13,
10, 11.

163
00:14:45,000 --> 00:14:52,000
And we, at that point,
increment i to here.

164
00:14:52,000 --> 00:15:03,000
And we have j now going here
and j runs to the end.

165
00:15:03,000 --> 00:15:08,000
And the loop terminates.
When the loop terminates there

166
00:15:08,000 --> 00:15:14,000
is one less swap that we do,
which is to put our pivot

167
00:15:14,000 --> 00:15:18,000
element in the middle between
the two subarrays.

168
00:15:18,000 --> 00:15:25,000
Here we swap this one and this
one, and so that gives us then

169
00:15:25,000 --> 00:15:27,000
2, 5, 3, 6, 8,
13, 10, 11.

170
00:15:27,000 --> 00:15:33,000
And this is the pivot.
And everything over here is

171
00:15:33,000 --> 00:15:40,000
less than or equal to the pivot.
And everything over here is

172
00:15:40,000 --> 00:15:44,000
greater than or equal to the
pivot.

173
00:15:56,000 --> 00:15:59,000
OK, so the quicksort routine.
Once we have this partition

174
00:15:59,000 --> 00:16:03,000
routine, quicksort is a pretty
easy piece of code to write.

175
00:16:18,000 --> 00:16:21,000
I should have said return here
i, right?

176
00:16:21,000 --> 00:16:24,000
You have got to return with the
pivot.

177
00:16:24,000 --> 00:16:29,000
Here I have got to return i
because we want to know where

178
00:16:29,000 --> 00:16:33,000
the pivot element is.
Sorry.

179
00:16:33,000 --> 00:16:46,000
I will plug in my code.
r gets partition of (A,

180
00:16:46,000 --> 00:17:03,000
p, q) and then we quicksort (A,
p, r-1) and quicksort of (A,

181
00:17:03,000 --> 00:17:09,000
r+1, q).
And that is it.

182
00:17:09,000 --> 00:17:17,000
That's the code.
The initial call is quicksort

183
00:17:17,000 --> 00:17:24,000
of (A, 1, n).
Because once we partitioned,

184
00:17:24,000 --> 00:17:31,000
we just have to quicksort the
two portions,

185
00:17:31,000 --> 00:17:39,000
the left and right portions.
Just the boundary case is

186
00:17:39,000 --> 00:17:41,000
probably worth mentioning for a
second.

187
00:17:41,000 --> 00:17:44,000
If there are zero or one
elements, that is basically what

188
00:17:44,000 --> 00:17:47,000
can possibly happen here,
is that I get zero or one

189
00:17:47,000 --> 00:17:49,000
elements here.
Then the point is there is

190
00:17:49,000 --> 00:17:52,000
nothing to do because the array
is sorted, either because it is

191
00:17:52,000 --> 00:17:56,000
an empty array or because it
only has one element.

192
00:17:56,000 --> 00:17:59,000
One of the tricks to making
quicksort go fast,

193
00:17:59,000 --> 00:18:01,000
as one tunes this,
is to, in fact,

194
00:18:01,000 --> 00:18:04,000
look at having a special
purpose sorting routine for

195
00:18:04,000 --> 00:18:07,000
small numbers of elements.
For example,

196
00:18:07,000 --> 00:18:10,000
if you get down to five
elements having some straight

197
00:18:10,000 --> 00:18:14,000
line piece of code that knows
how to sort five elements

198
00:18:14,000 --> 00:18:17,000
sufficiently as opposed to
continuing to go through

199
00:18:17,000 --> 00:18:20,000
recursion in order to accomplish
that.

200
00:18:20,000 --> 00:18:22,000
And there are a variety of
other things.

201
00:18:22,000 --> 00:18:27,000
This is a tail recursive code,
and so you can use certain tail

202
00:18:27,000 --> 00:18:31,000
recursion optimizations.
And there are a variety of

203
00:18:31,000 --> 00:18:35,000
other kinds of optimizations
that you can use to make this

204
00:18:35,000 --> 00:18:37,000
code go fast.
So, yeah, you can tune it up a

205
00:18:37,000 --> 00:18:40,000
bit beyond what is there,
but the core of it is this

206
00:18:40,000 --> 00:18:43,000
efficient partitioning routine.

207
00:18:53,000 --> 00:18:58,000
That is the algorithm.
It turns out that looking at

208
00:18:58,000 --> 00:19:05,000
how fast it runs is actually a
little bit challenging.

209
00:19:05,000 --> 00:19:09,000
In the analysis,
we are going to assume that all

210
00:19:09,000 --> 00:19:13,000
elements are distinct.
It turns out that this

211
00:19:13,000 --> 00:19:19,000
particular code does not work
very well when you have repeated

212
00:19:19,000 --> 00:19:25,000
elements, but Hoare's original
partitioning routine is actually

213
00:19:25,000 --> 00:19:31,000
more efficient in that case if
there are duplicates in what you

214
00:19:31,000 --> 00:19:36,000
are sorting.
And I encourage you to look at

215
00:19:36,000 --> 00:19:38,000
that.
It has a much more complicated

216
00:19:38,000 --> 00:19:42,000
invariant for partitioning
routine, but it does a similar

217
00:19:42,000 --> 00:19:45,000
kind of thing.
It's just a bit more

218
00:19:45,000 --> 00:19:48,000
complicated.
If they weren't all distinct,

219
00:19:48,000 --> 00:19:52,000
there are things you can do to
make them distinct or you can

220
00:19:52,000 --> 00:19:56,000
just use this code.
The easiest thing to do is just

221
00:19:56,000 --> 00:20:00,000
use Hoare's original code
because that works pretty well

222
00:20:00,000 --> 00:20:07,000
when they are nondistinct.
But this is a little bit easier

223
00:20:07,000 --> 00:20:12,000
to understand.
Let's let T(n) be the

224
00:20:12,000 --> 00:20:18,000
worst-case running time on n
elements.

225
00:20:18,000 --> 00:20:27,000
And so what is the worse-case?
What is the worse-case going to

226
00:20:27,000 --> 00:20:31,000
be for quicksort?

227
00:20:40,000 --> 00:20:43,000
That's right.
If you always pick the pivot

228
00:20:43,000 --> 00:20:47,000
and everything is greater than
or everything is less than,

229
00:20:47,000 --> 00:20:50,000
you are not going to partition
the array very well.

230
00:20:50,000 --> 00:20:54,000
And when does that happen?
What does the original input

231
00:20:54,000 --> 00:20:57,000
look like that makes that
happen?

232
00:20:57,000 --> 00:21:01,000
If it is already sorted or
reverse sorted.

233
00:21:01,000 --> 00:21:05,000
So, if the input is sorted or
reverse sorted.

234
00:21:05,000 --> 00:21:09,000
That is actually kind of
important to understand,

235
00:21:09,000 --> 00:21:14,000
because it turns out the most
common thing to sort is

236
00:21:14,000 --> 00:21:19,000
something that is already
sorted, surprisingly,

237
00:21:19,000 --> 00:21:22,000
or things that are nearly
sorted.

238
00:21:22,000 --> 00:21:27,000
But often it is just sorted and
somebody wants to make sure it

239
00:21:27,000 --> 00:21:33,000
is sorted.
Well, let's just sort it again

240
00:21:33,000 --> 00:21:38,000
rather than checking to see if
it is sorted.

241
00:21:38,000 --> 00:21:43,000
And, in those cases,
one side of the partition of

242
00:21:43,000 --> 00:21:50,000
each partition has no elements.
Then we can write out what the

243
00:21:50,000 --> 00:21:54,000
recursion is for that.
We have T(n).

244
00:21:54,000 --> 00:22:00,000
If one side has no elements,
we are going to have T(0) on

245
00:22:00,000 --> 00:22:06,000
that side.
And on the other side we are

246
00:22:06,000 --> 00:22:11,000
going to have T(n-1).
We are just writing out the

247
00:22:11,000 --> 00:22:16,000
recursion for this.
One side has no elements.

248
00:22:16,000 --> 00:22:19,000
The other side has n-1
elements.

249
00:22:19,000 --> 00:22:26,000
And then partitioning and all
the bookkeeping and so forth is

250
00:22:26,000 --> 00:22:30,000
order n.
What is T(0)?

251
00:22:30,000 --> 00:22:35,000
What is T(0)?
What is that asymptotically?

252
00:22:35,000 --> 00:22:38,000
It's a constant,
order 1.

253
00:22:38,000 --> 00:22:44,000
That is just order 1 + T(n-1) +
order n.

254
00:22:44,000 --> 00:22:50,000
Well, the order 1 can be
absorbed into the order n,

255
00:22:50,000 --> 00:22:59,000
so this is really just saying
it is T(n-1) + order n.

256
00:22:59,000 --> 00:23:08,000
And what is that equal to?
That is order n^2.

257
00:23:08,000 --> 00:23:19,000
Why is that order n^2?
It is an arithmetic series.

258
00:23:19,000 --> 00:23:30,000
Actually, just like we got for
insertion sort.

259
00:23:30,000 --> 00:23:35,000
Just like for insertion sort it
is an arithmetic series.

260
00:23:35,000 --> 00:23:42,000
Going through all that work and
we have an algorithm called

261
00:23:42,000 --> 00:23:47,000
quicksort, and it is no faster
than insertion sort.

262
00:23:47,000 --> 00:23:51,000
Nevertheless,
I said it was a good algorithm.

263
00:23:51,000 --> 00:23:57,000
The reason it is a good
algorithm is because its average

264
00:23:57,000 --> 00:24:04,000
case time, as we are going to
see, is very good.

265
00:24:04,000 --> 00:24:09,000
But let's try to understand
this a little bit more just so

266
00:24:09,000 --> 00:24:15,000
that we understand the
difference between what is going

267
00:24:15,000 --> 00:24:20,000
to happen in the average case
and what is going to happen in

268
00:24:20,000 --> 00:24:25,000
the worse-case.
Let's draw a recursion tree for

269
00:24:25,000 --> 00:24:31,000
this for T(n) = T(0) + T(n-1) +
and I will make the constant

270
00:24:31,000 --> 00:24:36,000
explicit for cn.
So, we get an intuition of what

271
00:24:36,000 --> 00:24:39,000
is going on.
Some constant times n.

272
00:24:39,000 --> 00:24:44,000
And then we have T(n) is equal
to, and we write it with the

273
00:24:44,000 --> 00:24:47,000
constant part here,
cn, and then T(0) here,

274
00:24:47,000 --> 00:24:51,000
and then T(n-1) here.
Now, I know that all you folks

275
00:24:51,000 --> 00:24:56,000
are really fast and want to jump
immediately to the full-blown

276
00:24:56,000 --> 00:24:59,000
tree.
But, let me tell you,

277
00:24:59,000 --> 00:25:04,000
my advice is that you spend
just a couple of minutes writing

278
00:25:04,000 --> 00:25:06,000
it out.
Since the tree grows

279
00:25:06,000 --> 00:25:10,000
exponentially,
it only costs you a constant

280
00:25:10,000 --> 00:25:14,000
overhead to write out the small
cases and make sure that you

281
00:25:14,000 --> 00:25:18,000
have got the pattern that you
are developing.

282
00:25:18,000 --> 00:25:21,000
So, I am going to go one more
step.

283
00:25:21,000 --> 00:25:26,000
Here we have T(0) and now this
becomes c(n-1) and now we have

284
00:25:26,000 --> 00:25:30,000
another T(0) over here and
T(n-2).

285
00:25:30,000 --> 00:25:35,000
And we continue that,
dot, dot, dot.

286
00:25:35,000 --> 00:25:45,000
That is all equal to cn with a
T(0) here, c(n-1) with a T(0),

287
00:25:45,000 --> 00:25:53,000
c(n-2), T(0) here,
and that goes all the way down

288
00:25:53,000 --> 00:26:01,000
until we end up with T(1) down
here.

289
00:26:01,000 --> 00:26:07,000
What is the height of this tree?

290
00:26:17,000 --> 00:26:20,000
What is the height of the tree
here?

291
00:26:20,000 --> 00:26:22,000
Yeah, n.
Good.

292
00:26:22,000 --> 00:26:30,000
Because every step we are just
decrementing the argument by 1.

293
00:26:30,000 --> 00:26:35,000
So, the height is n.
To analyze this,

294
00:26:35,000 --> 00:26:40,000
let's first add up everything
that is here.

295
00:26:40,000 --> 00:26:48,000
Just so we understand where
these things are coming from,

296
00:26:48,000 --> 00:26:56,000
this is just theta of the
summation of k equals 1 to n of

297
00:26:56,000 --> 00:27:02,000
k, actually of ck.
That is what is in there.

298
00:27:02,000 --> 00:27:08,000
And that is equal to order n^2.
That is where our algorithmatic

299
00:27:08,000 --> 00:27:12,000
series is coming from.
So, that is Theta(n^2).

300
00:27:12,000 --> 00:27:18,000
And then all of these things
here are all Theta(1).

301
00:27:23,000 --> 00:27:33,000
And how many of them are there?
There are n Theta(1)'s.

302
00:27:33,000 --> 00:27:54,000
So, the total amount is T(n) =
Theta(n) + Theta(n^2) =

303
00:27:54,000 --> 00:28:03,000
Theta(n^2).
Just to see what the structure

304
00:28:03,000 --> 00:28:09,000
is in terms of the recursion
tree, it is a highly unbalanced

305
00:28:09,000 --> 00:28:14,000
recursion tree.
Now I am going to do something

306
00:28:14,000 --> 00:28:20,000
that I told you should never do,
which is we are going to be do

307
00:28:20,000 --> 00:28:25,000
a best-case analysis.
This is for intuition only.

308
00:28:25,000 --> 00:28:32,000
And, in general,
we don't do best-case analyses.

309
00:28:32,000 --> 00:28:35,000
It doesn't mean anything,
unless we get some intuition

310
00:28:35,000 --> 00:28:38,000
for it maybe.
But basically it means nothing

311
00:28:38,000 --> 00:28:43,000
mathematically because it's
providing no guarantee.

312
00:28:55,000 --> 00:29:00,000
And so this is intuition only.

313
00:29:05,000 --> 00:29:13,000
If we are really lucky what
happens for partition?

314
00:29:13,000 --> 00:29:20,000
What is going to be the lucky
case?

315
00:29:26,000 --> 00:29:28,000
Yeah, it splits right in the
middle.

316
00:29:28,000 --> 00:29:31,000
Which is essentially --

317
00:29:43,000 --> 00:29:46,000
-- n/2 : n/2.
It is really (n-1)/2 :

318
00:29:46,000 --> 00:29:51,000
(n-1)/2, but we're not going to
worry about the details because

319
00:29:51,000 --> 00:29:56,000
we're only doing intuition for
the best-case because best-case

320
00:29:56,000 --> 00:30:01,000
is not what we want.
If that happened,

321
00:30:01,000 --> 00:30:11,000
what is the recurrence I get?
Imagine it split it exactly in

322
00:30:11,000 --> 00:30:18,000
the middle every time,
then what happens?

323
00:30:18,000 --> 00:30:26,000
You get T(n) = 2T(n/2) + order
n for partitioning and

324
00:30:26,000 --> 00:30:33,000
bookkeeping.
And what is the solution of

325
00:30:33,000 --> 00:30:36,000
that recurrence?
That is n log n.

326
00:30:36,000 --> 00:30:42,000
That is the same as the merge
sort recurrence.

327
00:30:42,000 --> 00:30:46,000
It is which case of the master
theorem?

328
00:30:46,000 --> 00:30:51,000
Case 2, right?
Because n to the log base 2 of

329
00:30:51,000 --> 00:30:55,000
2 is n to the 1,
it is the same,

330
00:30:55,000 --> 00:31:05,000
so we tack on the extra log n.
Case 2 of the master theorem.

331
00:31:05,000 --> 00:31:14,000
That is pretty good.
That says that in the best-case

332
00:31:14,000 --> 00:31:24,000
quicksort is going to do well.
How about let's suppose the

333
00:31:24,000 --> 00:31:32,000
split is always let's say 1/10 :
9/10, 1/10n :

334
00:31:32,000 --> 00:31:35,000
9/10n.
In that case,

335
00:31:35,000 --> 00:31:43,000
are we lucky or are we unlucky?

336
00:31:52,000 --> 00:31:57,000
I mean, if the split is really
skewed, we clearly are going to

337
00:31:57,000 --> 00:32:00,000
be unlucky, right,
because then it's,

338
00:32:00,000 --> 00:32:04,000
say, 1 to n.
If it is really in the middle

339
00:32:04,000 --> 00:32:08,000
it is n log n.
What do you suppose it is if it

340
00:32:08,000 --> 00:32:11,000
is 1/10 : 9/10?
Is that lucky or unlucky?

341
00:32:11,000 --> 00:32:14,000
We will have a little democracy
here.

342
00:32:14,000 --> 00:32:17,000
Who thinks that that is a lucky
case?

343
00:32:17,000 --> 00:32:20,000
It is going to be fast running
time.

344
00:32:20,000 --> 00:32:23,000
And who thinks it is an unlucky
case?

345
00:32:23,000 --> 00:32:26,000
OK, so we have some brave
souls.

346
00:32:26,000 --> 00:32:30,000
And who didn't vote?
Oh, come on.

347
00:32:30,000 --> 00:32:32,000
Come on.
It is always better,

348
00:32:32,000 --> 00:32:35,000
by the way, to say yes or no
and be right or wrong,

349
00:32:35,000 --> 00:32:40,000
because then you have some
emotional commitment to it and

350
00:32:40,000 --> 00:32:43,000
we will remember better,
rather than just sitting and

351
00:32:43,000 --> 00:32:47,000
being quiet.
You don't manipulate your own

352
00:32:47,000 --> 00:32:50,000
emotions well enough to remember
things well.

353
00:32:50,000 --> 00:32:54,000
Those people who voted win over
the people who don't vote,

354
00:32:54,000 --> 00:32:56,000
whether they are right or
wrong.

355
00:32:56,000 --> 00:33:02,000
Well, let's take a look.
Here is the recurrence.

356
00:33:02,000 --> 00:33:07,000
T(n) = T(1/10n) + T(9/10n) +
Theta(n).

357
00:33:07,000 --> 00:33:15,000
And we will assume that this
part here is less than or equal

358
00:33:15,000 --> 00:33:19,000
to some cn in order to analyze
it.

359
00:33:19,000 --> 00:33:25,000
We will just do a recursion
tree for this and see.

360
00:33:25,000 --> 00:33:30,000
Here is a recursion tree.

361
00:33:35,000 --> 00:33:41,000
We have T(n) = cn,
T(1/10n), T(9/10n).

362
00:33:41,000 --> 00:33:46,000
Now we have again cn at the
top.

363
00:33:46,000 --> 00:33:51,000
This gets complicated,
right?

364
00:33:51,000 --> 00:34:00,000
This is 1/10cn.
Now, over here we have 1/10.

365
00:34:00,000 --> 00:34:09,000
And then we are plugging it
into the recursion again,

366
00:34:09,000 --> 00:34:20,000
so we now get T(1/100n) and
over here we get T(9/100n).

367
00:34:20,000 --> 00:34:26,000
And over here we have now
9/10cn.

368
00:34:26,000 --> 00:34:34,000
And that gives us T(9/100n)
again.

369
00:34:34,000 --> 00:34:48,000
And here we get T(81/100n).
And we keep going on.

370
00:34:48,000 --> 00:34:59,000
That is equal to cn,
1/10cn here.

371
00:34:59,000 --> 00:35:08,000
Down this way we have 1/100cn.
And that keeps going down until

372
00:35:08,000 --> 00:35:16,000
we get to order 1 down here.
And over here we have 9/10cn.

373
00:35:16,000 --> 00:35:24,000
And here, let's see,
this is 9/100cn and this is now

374
00:35:24,000 --> 00:35:32,000
9/100cn and this is 81/100cn.
And these things keep going

375
00:35:32,000 --> 00:35:36,000
down until they get down to
order 1.

376
00:35:36,000 --> 00:35:41,000
But the leaves are not all at
uniform depth here,

377
00:35:41,000 --> 00:35:45,000
right?
This side is way further up

378
00:35:45,000 --> 00:35:47,000
than this side,
right?

379
00:35:47,000 --> 00:35:53,000
Because here we are only going
down by 9/10 each time.

380
00:35:53,000 --> 00:35:58,000
So, in fact,
what is the length of this path

381
00:35:58,000 --> 00:36:02,000
here?
What is the length of this path

382
00:36:02,000 --> 00:36:06,000
down to this,
if I take the left most spine?

383
00:36:15,000 --> 00:36:23,000
Somebody raise there hand.
Yeah?

384
00:36:23,000 --> 00:36:30,000
Log base 10 of n.
Because I am basically cutting

385
00:36:30,000 --> 00:36:33,000
down by a factor of 10 each
time.

386
00:36:33,000 --> 00:36:35,000
And how long does it take me to
reduce it to 1?

387
00:36:35,000 --> 00:36:38,000
That is the definition,
if you will,

388
00:36:38,000 --> 00:36:40,000
of what a log is,
log base 10.

389
00:36:40,000 --> 00:36:43,000
What is this one?
What is this path going that

390
00:36:43,000 --> 00:36:44,000
way?

391
00:36:53,000 --> 00:37:02,000
Log of n. Log base 10/9 of n.
Because we're going down by

392
00:37:02,000 --> 00:37:05,000
9/10 each time.
Once again, essentially the

393
00:37:05,000 --> 00:37:09,000
definition of n.
And everything in between there

394
00:37:09,000 --> 00:37:14,000
is somewhere between log base 10
of n and log base 10/9 of n.

395
00:37:14,000 --> 00:37:17,000
So, everything is in between
there.

396
00:37:17,000 --> 00:37:22,000
Now what I can do is do the
trick that we did for mergesort

397
00:37:22,000 --> 00:37:26,000
in looking at what the
evaluation of this is by adding

398
00:37:26,000 --> 00:37:31,000
up what is the cost of the total
level.

399
00:37:31,000 --> 00:37:36,000
That is just cn.
What is the cost of the next

400
00:37:36,000 --> 00:37:38,000
level?
cn.

401
00:37:38,000 --> 00:37:43,000
And what is the cost of the
next level?

402
00:37:43,000 --> 00:37:47,000
cn.
Every level we are still doing

403
00:37:47,000 --> 00:37:54,000
the same amount of work.
And we take that all the way

404
00:37:54,000 --> 00:38:00,000
down.
And the last levels --

405
00:38:00,000 --> 00:38:04,000
Eventually we hit some point
where it is not equal to cn

406
00:38:04,000 --> 00:38:09,000
where we start getting things
that are less than or equal to

407
00:38:09,000 --> 00:38:14,000
cn because some of the leaves
start dropping out starting at

408
00:38:14,000 --> 00:38:17,000
this level.
Basically this part is going to

409
00:38:17,000 --> 00:38:21,000
be log base 10 of n,
and then we start getting

410
00:38:21,000 --> 00:38:25,000
things that are less than or
equal to cn, and so forth,

411
00:38:25,000 --> 00:38:30,000
until finally we get to add it
all up.

412
00:38:30,000 --> 00:38:36,000
T(n) is going to be less than
or equal to cn times,

413
00:38:36,000 --> 00:38:43,000
well, what is the longest that
this could possibly be?

414
00:38:43,000 --> 00:38:49,000
Log base 10/9 of n.
Plus we have all of the leaves

415
00:38:49,000 --> 00:38:56,000
that we have to add in,
but all the leaves together add

416
00:38:56,000 --> 00:39:03,000
up to just order n.
All the leaves add up to order

417
00:39:03,000 --> 00:39:08,000
n, so we have + Theta(n).
And so this is how much?

418
00:39:08,000 --> 00:39:14,000
If I add all of this together,
what is this asymptotically?

419
00:39:14,000 --> 00:39:18,000
That is n log n.
So, T(n) is actually bounded by

420
00:39:18,000 --> 00:39:21,000
n log n.
We are lucky.

421
00:39:21,000 --> 00:39:25,000
Those people who guessed lucky
were right.

422
00:39:25,000 --> 00:39:30,000
A 1/10 : 9/10 split is
asymptotically as good as a 50 :

423
00:39:30,000 --> 00:39:34,000
50 split.
And, in fact,

424
00:39:34,000 --> 00:39:41,000
we can lower bound this by just
looking at these things here and

425
00:39:41,000 --> 00:39:45,000
discover that,
in fact, T(n) is lower bounded

426
00:39:45,000 --> 00:39:49,000
by cn log base 10 of n + order
n.

427
00:39:49,000 --> 00:39:55,000
And so T(n) is lower bounded by
also asymptotically n log n.

428
00:39:55,000 --> 00:40:00,000
So, T(n) is actually Theta(n lg
n).

429
00:40:00,000 --> 00:40:04,000
Now, this is not really proof.
I generally recommend that you

430
00:40:04,000 --> 00:40:07,000
don't do this kind of thing to
do a proof.

431
00:40:07,000 --> 00:40:10,000
This is a good intuition of a
recursion tree.

432
00:40:10,000 --> 00:40:14,000
The way you prove this is what?
Substitution method.

433
00:40:14,000 --> 00:40:17,000
Good.
What you do is use this to get

434
00:40:17,000 --> 00:40:20,000
your guess and then use
substitution method to prove

435
00:40:20,000 --> 00:40:25,000
that your guess is right.
It is too easy to make mistakes

436
00:40:25,000 --> 00:40:28,000
with this method.
It is very easy to make

437
00:40:28,000 --> 00:40:32,000
mistakes.
With the substitution method it

438
00:40:32,000 --> 00:40:37,000
is harder to make mistakes
because there is just algebra

439
00:40:37,000 --> 00:40:40,000
there that you are cranking
through.

440
00:40:40,000 --> 00:40:43,000
It is easier to verify rather
than dot, dot,

441
00:40:43,000 --> 00:40:48,000
dots and trees that you drew
improperly and wrote in wrong

442
00:40:48,000 --> 00:40:50,000
amounts and so forth.
OK?

443
00:40:50,000 --> 00:40:53,000
So, this is n log n.
That's pretty good.

444
00:40:53,000 --> 00:40:56,000
It is order n log n.
And we are lucky.

445
00:40:56,000 --> 00:41:01,000
Now let's try another one.
This is all for intuition

446
00:41:01,000 --> 00:41:05,000
because, I will tell you,
by the time we get to the end

447
00:41:05,000 --> 00:41:10,000
of this class you folks are
going to bolting for the door

448
00:41:10,000 --> 00:41:13,000
because we are going to do some
good math today,

449
00:41:13,000 --> 00:41:16,000
actually.
It is actually fun math,

450
00:41:16,000 --> 00:41:20,000
I think, but it is challenging.
If you are not awake,

451
00:41:20,000 --> 00:41:23,000
you can still sleep now,
but I will tell you when to

452
00:41:23,000 --> 00:41:26,000
wake up.
One more bit of intuition.

453
00:41:26,000 --> 00:41:30,000
Suppose that we alternate
steps.

454
00:41:30,000 --> 00:41:33,000
Suppose we do the partitioning
thing.

455
00:41:33,000 --> 00:41:38,000
And it happens that we start
out lucky and then we have a

456
00:41:38,000 --> 00:41:43,000
partitioning step that is
unlucky and then we have a step

457
00:41:43,000 --> 00:41:49,000
that is lucky and a step that is
unlucky and we do that all the

458
00:41:49,000 --> 00:41:54,000
way down the tree.
Suppose we alternate.

459
00:42:07,000 --> 00:42:09,000
Are we lucky or unlucky if we
do that?

460
00:42:09,000 --> 00:42:11,000
This time I want everybody
voting.

461
00:42:11,000 --> 00:42:13,000
It doesn't matter what your
answer is.

462
00:42:13,000 --> 00:42:16,000
Everybody has to have a stake
in the game.

463
00:42:16,000 --> 00:42:19,000
It is sort of like horseracing.
If ever you have watched

464
00:42:19,000 --> 00:42:23,000
horseracing, it is really
boring, but if you put a little

465
00:42:23,000 --> 00:42:26,000
bit of money down,
a little skin in the game

466
00:42:26,000 --> 00:42:30,000
suddenly it is interesting.
The same thing here.

467
00:42:30,000 --> 00:42:33,000
I want everybody to put some
skin in the game.

468
00:42:33,000 --> 00:42:37,000
Who thinks that this is going
to be lucky?

469
00:42:37,000 --> 00:42:40,000
Who thinks it is going to be
unlucky?

470
00:42:40,000 --> 00:42:42,000
OK.
Who didn't vote?

471
00:42:42,000 --> 00:42:46,000
[LAUGHTER] You guys.
No skin in the game,

472
00:42:46,000 --> 00:42:48,000
ha?
Let's analyze this so we can

473
00:42:48,000 --> 00:42:53,000
once again write a recurrence.
On the lucky step,

474
00:42:53,000 --> 00:42:57,000
we will have L(n) be the
running time on a lucky step of

475
00:42:57,000 --> 00:43:02,000
size n.
And that is going to be twice.

476
00:43:02,000 --> 00:43:07,000
While the next step is going to
be unlucky.

477
00:43:07,000 --> 00:43:10,000
It is two unluckies over 2 plus
order n.

478
00:43:10,000 --> 00:43:15,000
That is our lucky step.
And then for the unlucky step

479
00:43:15,000 --> 00:43:20,000
it is essentially going to be L
of n minus 1,

480
00:43:20,000 --> 00:43:25,000
it is going to be lucky on the
next step, plus order n.

481
00:43:25,000 --> 00:43:30,000
That is unlucky.
See how I have described this

482
00:43:30,000 --> 00:43:36,000
behavior with a system now of
recurrences that are dependent

483
00:43:36,000 --> 00:43:41,000
where the boundary cases,
once again which are unstated,

484
00:43:41,000 --> 00:43:47,000
is that the recurrences have a
constant solution with constant

485
00:43:47,000 --> 00:43:50,000
input.
Now we just do a little bit of

486
00:43:50,000 --> 00:43:54,000
algebra using substitution.
L(n) is then equal to,

487
00:43:54,000 --> 00:44:00,000
well, I can just plug in,
for U(n/2) plug in the value of

488
00:44:00,000 --> 00:44:06,000
U(n/2).
And that gives me 2[L(n/2-1) +

489
00:44:06,000 --> 00:44:11,000
Theta(n) + Theta(n)].
See what I did here?

490
00:44:11,000 --> 00:44:18,000
I simply plugged in,
for U(n/2), this recurrence.

491
00:44:18,000 --> 00:44:27,000
In fact, technically I guess I
should have said Theta(n/2) just

492
00:44:27,000 --> 00:44:35,000
to make this substitution more
straightforward.

493
00:44:35,000 --> 00:44:41,000
It is the same thing,
but just to not skip a step.

494
00:44:41,000 --> 00:44:49,000
That we can now crank through.
And that is 2L(n/2 - 1) +,

495
00:44:49,000 --> 00:44:57,000
and now I have two T(n/2) plus
another one, so all of that is

496
00:44:57,000 --> 00:45:04,000
just order n.
And what is the solution to

497
00:45:04,000 --> 00:45:07,000
that recurrence?
n log n.

498
00:45:07,000 --> 00:45:13,000
Theta(n lg n).
Does everybody see that?

499
00:45:13,000 --> 00:45:15,000
OK?
Theta(n lg n).

500
00:45:15,000 --> 00:45:24,000
This is basically just,
once again, master theorem with

501
00:45:24,000 --> 00:45:33,000
a little bit of jiggering here.
That minus one is only going to

502
00:45:33,000 --> 00:45:38,000
help us, actually,
in the solution of the master

503
00:45:38,000 --> 00:45:41,000
theorem.
So, it is order n lg n.

504
00:45:41,000 --> 00:45:45,000
We are lucky.
If we alternate lucky and

505
00:45:45,000 --> 00:45:50,000
unlucky, we are lucky.
How can we insure that we are

506
00:45:50,000 --> 00:45:55,000
usually lucky?
If I have the input already

507
00:45:55,000 --> 00:46:00,000
sorted, I am going to be
unlucky.

508
00:46:00,000 --> 00:46:03,000
Excuse me?
You could randomly arrange the

509
00:46:03,000 --> 00:46:07,000
elements, that is one way.
What is another way?

510
00:46:07,000 --> 00:46:10,000
That is a perfectly good way,
actually.

511
00:46:10,000 --> 00:46:13,000
In fact, it is a common thing
to do.

512
00:46:13,000 --> 00:46:16,000
Randomly choose the pivot,
OK.

513
00:46:16,000 --> 00:46:20,000
It turns out those are
effectively equivalent,

514
00:46:20,000 --> 00:46:24,000
but we are going to do the
randomly choose the pivot

515
00:46:24,000 --> 00:46:30,000
because it is a little bit
easier to analyze.

516
00:46:30,000 --> 00:46:33,000
But they are effectively
equivalent.

517
00:46:33,000 --> 00:46:39,000
That gives us the algorithm
called randomized quicksort.

518
00:46:46,000 --> 00:46:53,000
And the nice thing about
randomized quicksort is that the

519
00:46:53,000 --> 00:47:00,000
running time is independent of
the input ordering.

520
00:47:00,000 --> 00:47:04,000
Very much for the same reason
that if I just scramble the

521
00:47:04,000 --> 00:47:08,000
input, it would be independent
of the input ordering.

522
00:47:08,000 --> 00:47:12,000
If I randomly scramble the
input then it doesn't matter

523
00:47:12,000 --> 00:47:17,000
what the order of the input was.
Whereas, original quicksort has

524
00:47:17,000 --> 00:47:21,000
some slow cases,
input sorted or reverse sorted,

525
00:47:21,000 --> 00:47:24,000
and some fast cases.
In particular,

526
00:47:24,000 --> 00:47:28,000
it turns out that if it is
random it is going to be pretty

527
00:47:28,000 --> 00:47:31,000
fast.
If I actually randomly scramble

528
00:47:31,000 --> 00:47:35,000
the input or pivot on a random
element, it doesn't matter what

529
00:47:35,000 --> 00:47:38,000
the input was.
One way of thinking about this

530
00:47:38,000 --> 00:47:40,000
is with an adversary.
Imagine your adversary,

531
00:47:40,000 --> 00:47:44,000
you are saying I have a good
sorting algorithm and he says I

532
00:47:44,000 --> 00:47:47,000
have a good sorting algorithm
and you're trying to sell to a

533
00:47:47,000 --> 00:47:50,000
single customer.
And the customer says OK,

534
00:47:50,000 --> 00:47:53,000
you guys come up with
benchmarks for each of your

535
00:47:53,000 --> 00:47:55,000
algorithms.
And you get to look at his

536
00:47:55,000 --> 00:47:59,000
algorithm.
Well, you look and you say oh,

537
00:47:59,000 --> 00:48:02,000
he is using quicksort.
I will just give him something

538
00:48:02,000 --> 00:48:06,000
that is already sorted.
That is what you could do to

539
00:48:06,000 --> 00:48:08,000
him.
If you had quicksort,

540
00:48:08,000 --> 00:48:10,000
he would do the same thing to
you.

541
00:48:10,000 --> 00:48:14,000
So, how can you defeat him?
Well, one way is with

542
00:48:14,000 --> 00:48:17,000
randomization.
Big idea in computer science,

543
00:48:17,000 --> 00:48:20,000
use random numbers.
The idea here is if I permute

544
00:48:20,000 --> 00:48:23,000
the ordering at random,
as one suggestion,

545
00:48:23,000 --> 00:48:28,000
or I pivot at random places,
then the input ordering didn't

546
00:48:28,000 --> 00:48:32,000
matter.
And so there is no bad ordering

547
00:48:32,000 --> 00:48:36,000
that he can provide that is
going to make my code run

548
00:48:36,000 --> 00:48:39,000
slowly.
Now, I might get unlucky.

549
00:48:39,000 --> 00:48:43,000
But that is just unlucky in my
use of my random-number

550
00:48:43,000 --> 00:48:46,000
generator.
It is not unlucky with respect

551
00:48:46,000 --> 00:48:50,000
to what the input was.
What the input was doesn't

552
00:48:50,000 --> 00:48:52,000
matter.
Everybody follow that?

553
00:48:52,000 --> 00:48:55,000
OK.
The nice thing about randomized

554
00:48:55,000 --> 00:48:59,000
quicksort is that it makes no
assumptions about the input

555
00:48:59,000 --> 00:49:05,000
distribution.
You don't have to assume that

556
00:49:05,000 --> 00:49:13,000
all inputs are equally likely
because either you can make it

557
00:49:13,000 --> 00:49:20,000
that way or you pivot in a way
that makes that effectively

558
00:49:20,000 --> 00:49:23,000
whole.
And, in particular,

559
00:49:23,000 --> 00:49:30,000
there is no specific input that
can elicit the worst-case

560
00:49:30,000 --> 00:49:45,000
behavior.
The worst-case is determined

561
00:49:45,000 --> 00:50:00,000
only by a random-number
generator.

562
00:50:00,000 --> 00:50:03,000
And, therefore,
since it is only determined by

563
00:50:03,000 --> 00:50:08,000
a random-number generator,
we can essentially bound the

564
00:50:08,000 --> 00:50:13,000
unluckiness mathematically.
We can say what are the odds?

565
00:50:13,000 --> 00:50:16,000
So, we are going to analyze
this.

566
00:50:16,000 --> 00:50:21,000
And this is where you know if
you belong in this course or

567
00:50:21,000 --> 00:50:24,000
not.
If you have skipped 6.042 or

568
00:50:24,000 --> 00:50:30,000
whatever, this is a good place
to do the comparison.

569
00:50:30,000 --> 00:50:34,000
Since it is going to be a
little bit, why don't people

570
00:50:34,000 --> 00:50:37,000
just stand up for a moment and
take a stretch break.

571
00:50:37,000 --> 00:50:42,000
Since this is going to be a
nice piece of mathematics we are

572
00:50:42,000 --> 00:50:47,000
going to do, you are going to
want to feel fresh for it.

573
00:51:02,000 --> 00:51:04,000
Stretch break is over.

574
00:51:10,000 --> 00:51:11,000
Analysis.
Good.

575
00:51:11,000 --> 00:51:16,000
I think we are going to make
this.

576
00:51:16,000 --> 00:51:22,000
I am sort of racing.
There is a lot of stuff to

577
00:51:22,000 --> 00:51:24,000
cover today.
Good.

578
00:51:24,000 --> 00:51:32,000
Let's let T(n) now be the
random variable for the running

579
00:51:32,000 --> 00:51:36,000
time assuming --

580
00:51:46,000 --> 00:51:47,000
Wow.
I didn't even write here what

581
00:51:47,000 --> 00:51:49,000
we did here.
So, we are going to pivot on a

582
00:51:49,000 --> 00:51:51,000
random element.

583
00:51:59,000 --> 00:52:01,000
That is the basic scheme we are
going to do.

584
00:52:01,000 --> 00:52:05,000
And the way I do that,
by the way, is just in the code

585
00:52:05,000 --> 00:52:07,000
for partition,
rather than partitioning on the

586
00:52:07,000 --> 00:52:10,000
first element,
before I do the partition,

587
00:52:10,000 --> 00:52:14,000
I just swap the first element
with some other element in the

588
00:52:14,000 --> 00:52:16,000
array chosen at random,
perhaps itself.

589
00:52:16,000 --> 00:52:19,000
So, they are all equally likely
to be pivoted on.

590
00:52:19,000 --> 00:52:23,000
And then just run the ordinary
partition.

591
00:52:23,000 --> 00:52:27,000
This is a random variable for
running in time assuming,

592
00:52:27,000 --> 00:52:32,000
we have to make an assumption
for doing probability,

593
00:52:32,000 --> 00:52:36,000
the random numbers are
independent.

594
00:52:41,000 --> 00:52:44,000
So that when I pivot in one
place, it is independent of how

595
00:52:44,000 --> 00:52:48,000
I pivoted in some other place as
I am running this algorithm.

596
00:52:48,000 --> 00:52:52,000
Then, to analyze this,
what I am going to do is I want

597
00:52:52,000 --> 00:52:57,000
to know where we pivoted.
For k = 0, 1,

598
00:52:57,000 --> 00:53:08,000
..., n-1, let's let,
for a particular partition,

599
00:53:08,000 --> 00:53:21,000
the random variable X_k = 1 if
partition generates a k :

600
00:53:21,000 --> 00:53:30,000
n-k-1 split,
and 0 otherwise.

601
00:53:35,000 --> 00:53:39,000
In the partition routine,
I am picking a random element

602
00:53:39,000 --> 00:53:42,000
to pivot on.
And X_k is going to be my

603
00:53:42,000 --> 00:53:47,000
random variable that is 1 if it
generates a split that has k

604
00:53:47,000 --> 00:53:52,000
elements on the left side and
n-k-1 elements on the right side

605
00:53:52,000 --> 00:53:54,000
of the pivot.
Some of those,

606
00:53:54,000 --> 00:54:00,000
too, of course are n-1 because
I also have the pivot.

607
00:54:00,000 --> 00:54:03,000
And 0 otherwise.
So, I now have n random

608
00:54:03,000 --> 00:54:09,000
variables that I have defined
associated with a single

609
00:54:09,000 --> 00:54:15,000
partition where all of them are
going to be zero except one of

610
00:54:15,000 --> 00:54:21,000
them, whichever one happens to
occur is going to have the value

611
00:54:22,000 --> 00:54:26,000
by the way.
What is the name of this type

612
00:54:26,000 --> 00:54:30,000
of random variable?

613
00:54:37,000 --> 00:54:40,000
Bernoulli.
Well, Bernoulli has other

614
00:54:40,000 --> 00:54:43,000
assumptions.
It is an indicator random

615
00:54:43,000 --> 00:54:46,000
variable.
It turns out it is Bernoulli,

616
00:54:46,000 --> 00:54:50,000
but that's OK.
It is an indicator random

617
00:54:50,000 --> 00:54:53,000
variable.
It just takes on the value of

618
00:54:53,000 --> 00:54:56,000
0, 1.
And Bernoulli random variables

619
00:54:56,000 --> 00:55:02,000
are a particular type of
indicator random variable.

620
00:55:02,000 --> 00:55:06,000
Which it turns out these are.
That is an indicator random

621
00:55:06,000 --> 00:55:09,000
variable.
Indicator random variables are

622
00:55:09,000 --> 00:55:14,000
a good way when you are trying
to understand what the sum of a

623
00:55:14,000 --> 00:55:18,000
bunch of things is.
It is a good way to break apart

624
00:55:18,000 --> 00:55:22,000
your big random variables into
smaller ones that can be

625
00:55:22,000 --> 00:55:25,000
analyzed.
Let's just take a look at this

626
00:55:25,000 --> 00:55:30,000
indicator random variable.
What is the expectation of X_k

627
00:55:30,000 --> 00:55:32,000
equal to?

628
00:55:42,000 --> 00:55:45,000
In other words,
what is the probability that I

629
00:55:45,000 --> 00:55:48,000
generate a k :
n-k-1 split?

630
00:56:00,000 --> 00:56:05,000
X_k is, let's just write out
what that means,

631
00:56:05,000 --> 00:56:08,000
just to refresh people's
memory.

632
00:56:08,000 --> 00:56:15,000
That is 0 times the probability
that X_k equals 0 plus 1 times

633
00:56:15,000 --> 00:56:21,000
the probability that X_k equals
1, which is equal,

634
00:56:21,000 --> 00:56:27,000
well, that is all zero.
That is just equal to the

635
00:56:27,000 --> 00:56:34,000
probability that X_k equals 1.
And that is a general property

636
00:56:34,000 --> 00:56:39,000
of indicator random variables,
is that their expectation is

637
00:56:39,000 --> 00:56:45,000
the probability that they are 1.
The nice thing about indicator

638
00:56:45,000 --> 00:56:51,000
random variables is it directly
connects the probability to the

639
00:56:51,000 --> 00:56:55,000
expectation without any other
terms going on.

640
00:56:55,000 --> 00:56:58,000
What is the probability of X_k
equals 1?

641
00:56:58,000 --> 00:57:02,000
1/n.
So, all splits are equally

642
00:57:02,000 --> 00:57:04,000
likely.
And I have n elements,

643
00:57:04,000 --> 00:57:10,000
so each ones has a 1/n chance
of being picked as the pivot.

644
00:57:10,000 --> 00:57:15,000
And, once you pick the pivot,
that determines what is on the

645
00:57:15,000 --> 00:57:19,000
left and the right and so forth.
So, it is 1/n.

646
00:57:19,000 --> 00:57:23,000
Everybody with me so far?
More or less?

647
00:57:23,000 --> 00:57:26,000
OK.
As I say, this is going to test

648
00:57:26,000 --> 00:57:32,000
whether you're in the class.
If you go home and you study

649
00:57:32,000 --> 00:57:37,000
this and you cannot get it,
and you have a deficiency in

650
00:57:37,000 --> 00:57:41,000
your math background in trying
to take the course,

651
00:57:41,000 --> 00:57:46,000
this is a good indication that
probably you have taken

652
00:57:46,000 --> 00:57:49,000
something a little over your
head.

653
00:57:49,000 --> 00:57:54,000
Let's write out what T(n) is
equal to here.

654
00:58:00,000 --> 00:58:11,000
T(n) is going to be equal to
T(0) + T(n-1) + Theta(n) if we

655
00:58:11,000 --> 00:58:24,000
get a 0 : n-1 split and is equal
to T(1) + T(n-2) + order n if we

656
00:58:24,000 --> 00:58:30,000
have a 1 : n-2 split.

657
00:58:35,000 --> 00:58:46,000
And now down here it is going
to be T(n-1) + T(0) + Theta(n)

658
00:58:46,000 --> 00:58:52,000
if we end up with an n-1 :
0 split.

659
00:58:52,000 --> 00:59:00,000
So, this is our recurrence for
T(n).

660
00:59:00,000 --> 00:59:03,000
And, unfortunately,
the recurrence is kind of hairy

661
00:59:03,000 --> 00:59:06,000
because it has got n cases.
And this is,

662
00:59:06,000 --> 00:59:11,000
once again, where the
brilliance of being able to use

663
00:59:11,000 --> 00:59:13,000
indicator random variables comes
in.

664
00:59:13,000 --> 00:59:18,000
Because we will be able to take
this case analysis and reduce it

665
00:59:18,000 --> 00:59:23,000
to mathematics so we don't have
cases using indicator random

666
00:59:23,000 --> 00:59:29,000
variables.
And the way we do that is using

667
00:59:29,000 --> 00:59:37,000
the following trick of
converting the cases into a

668
00:59:37,000 --> 00:59:39,000
summation.

669
00:59:55,000 --> 01:00:00,666
Let's just take a look at why
these two things are the same.

670
01:00:00,666 --> 01:00:06,333
The indicator random variable
is zero, except if you get the

671
01:00:06,333 --> 01:00:10,655
particular split.
Therefore, this summation is

672
01:00:10,655 --> 01:00:14,497
going to be zero,
except for that k which

673
01:00:14,497 --> 01:00:20,355
actually appeared in which case
it is the value that we say it

674
01:00:20,355 --> 01:00:22,468
is.
See the trick using

675
01:00:22,468 --> 01:00:27,079
multiplication by 0,
1 variable to handle all the

676
01:00:27,079 --> 01:00:31,404
cases?
I think that is damn clever.

677
01:00:31,404 --> 01:00:36,213
I think that is damn clever.
And this is like the classic

678
01:00:36,213 --> 01:00:40,421
thing that you do with indicator
random variables.

679
01:00:40,421 --> 01:00:45,144
It's one of the reasons they
are a very powerful method.

680
01:00:45,144 --> 01:00:49,781
Because now we actually have a
mathematical expression,

681
01:00:49,781 --> 01:00:53,559
hairy although it may be,
for our recurrence.

682
01:00:53,559 --> 01:00:58,454
Now, what we are going to
analyze is the expected value of

683
01:00:58,454 --> 01:01:02,977
T(n).
That is what we want to do.

684
01:01:02,977 --> 01:01:06,727
What is the expected value of
T(n)?

685
01:01:06,727 --> 01:01:13,235
To do that, I just write the
expected value of T(n) is equal

686
01:01:13,235 --> 01:01:17,977
to the expected value of this
big summation.

687
01:01:17,977 --> 01:01:24,264
And now we can go ahead and
start to evaluate the expected

688
01:01:24,264 --> 01:01:30,000
value of that summation.
Everybody with me?

689
01:01:30,000 --> 01:01:32,486
Yes?
Any questions at this point?

690
01:01:32,486 --> 01:01:35,360
I see a thumbs up.
That's nice to see.

691
01:01:35,360 --> 01:01:39,944
But I generally believe that
what I want to see is no thumbs

692
01:01:39,944 --> 01:01:42,508
down.
It is good to see the thumbs

693
01:01:42,508 --> 01:01:46,704
up, but that means one person
understands, or thinks he

694
01:01:46,704 --> 01:01:48,724
understands.
[LAUGHTER] So,

695
01:01:48,724 --> 01:01:51,832
this is, I claim,
equal to the following.

696
01:01:51,832 --> 01:01:56,571
Actually, I am going to need a
little space here so I am going

697
01:01:56,571 --> 01:02:01,000
to move the equal sign over a
little bit.

698
01:02:25,000 --> 01:02:27,702
I claim that summation is equal
to that.

699
01:02:27,702 --> 01:02:32,000
This expectation is equal to
that summation of expectations.

700
01:02:32,000 --> 01:02:36,042
Why is that?
What are the magic words that

701
01:02:36,042 --> 01:02:40,281
justify this step?
Linearity of expectation.

702
01:02:40,281 --> 01:02:45,704
The expectation of a sum is the
sum of the expectations.

703
01:02:45,704 --> 01:02:49,253
So, that is linearity of
expectation.

704
01:02:49,253 --> 01:02:52,605
I don't need independence for
that.

705
01:02:52,605 --> 01:02:57,830
That is just always true for
expectation of any random

706
01:02:57,830 --> 01:03:04,591
variables.
The sum of the expectations is

707
01:03:04,591 --> 01:03:10,867
the expectation of the sum and
vice versa.

708
01:03:10,867 --> 01:03:19,744
Here we did the vice versa.
That is equal to now the sum of

709
01:03:19,744 --> 01:03:30,000
k=0 to n-1 of expectation of X_k
[T(k) + T(n-k-1) + Theta(n)].

710
01:03:36,000 --> 01:03:42,272
Why is that true?
What I have done is I've said

711
01:03:42,272 --> 01:03:49,227
the expectation of the product
is the product of the

712
01:03:49,227 --> 01:03:53,454
expectations.
That is because of

713
01:03:53,454 --> 01:04:00,000
independence.
What is independent of what?

714
01:04:00,000 --> 01:04:02,492
The X_k here,
random variable,

715
01:04:02,492 --> 01:04:06,962
are independent of any of the
other partitionings in,

716
01:04:06,962 --> 01:04:10,401
if you will,
the X_k that would exist for

717
01:04:10,401 --> 01:04:13,151
any of the other recursive
calls.

718
01:04:13,151 --> 01:04:18,223
So, whatever happens in here is
independent of what happened

719
01:04:18,223 --> 01:04:20,716
there.
We are actually hiding.

720
01:04:20,716 --> 01:04:25,358
Since we have a recurrence,
we are not partitioning the

721
01:04:25,358 --> 01:04:30,000
same wage time.
We have a different one.

722
01:04:30,000 --> 01:04:33,168
We actually have something
going on underneath the

723
01:04:33,168 --> 01:04:36,271
mathematics you have to pay
attention to that the

724
01:04:36,271 --> 01:04:40,150
mathematics alone isn't really
showing, which is that in T(k)

725
01:04:40,150 --> 01:04:43,706
there is actually a set of
random choices that are being

726
01:04:43,706 --> 01:04:46,745
made, if you will.
And so you have to understand

727
01:04:46,745 --> 01:04:50,366
that those are independent of
those, in which case we can

728
01:04:50,366 --> 01:04:53,469
multiple the probabilities of
their expectations.

729
01:04:53,469 --> 01:04:55,991
Is everybody with me?
That is a big one,

730
01:04:55,991 --> 01:05:00,000
independence of X_k from other
random choices.

731
01:05:00,000 --> 01:05:04,624
That is equal to now,
well, first of all,

732
01:05:04,624 --> 01:05:09,710
this is nice.
What is the expectation of X_k?

733
01:05:09,710 --> 01:05:13,179
1/n.
That actually doesn't even

734
01:05:13,179 --> 01:05:20,000
belong in the summation.
We will just pop it outside.

735
01:05:20,000 --> 01:05:35,541
I get 1/n times the sum of k=0
to n-1 of expectation of T(k) +

736
01:05:35,541 --> 01:05:49,808
1/n summation k=0 to n-1 of
expectation of T(n-k-1) + 1/n

737
01:05:49,808 --> 01:06:00,000
summation k=0 to n-1 up to
Theta(n).

738
01:06:00,000 --> 01:06:04,838
That is, again,
using linearity of expectation

739
01:06:04,838 --> 01:06:11,075
there this time to split up
these pieces and just factoring

740
01:06:11,075 --> 01:06:15,053
out the expectation of k as
being 1/n.

741
01:06:15,053 --> 01:06:20,430
Everybody with me still?
All of this is elementary.

742
01:06:20,430 --> 01:06:26,989
It is just one of these things
that is hard just because there

743
01:06:26,989 --> 01:06:33,174
are so many steps.
And it takes that you have seen

744
01:06:33,174 --> 01:06:37,986
some of this before.
Now the next observation is

745
01:06:37,986 --> 01:06:43,003
that these two summations are,
in fact, identical.

746
01:06:43,003 --> 01:06:48,532
They are the same summation,
just in a different order.

747
01:06:48,532 --> 01:06:53,549
This is going T(0),
T(1), T(2), T(3) up to T(n-1).

748
01:06:53,549 --> 01:07:00,000
This one is going T(n-1),
T(n-2), T(n-3) down to T(0).

749
01:07:00,000 --> 01:07:02,049
These are, in fact,
equal.

750
01:07:02,049 --> 01:07:05,000
So, therefore,
I have two of them.

751
01:07:19,000 --> 01:07:22,000
And then what is this term
equal to?

752
01:07:37,000 --> 01:07:40,991
What is that one equal to?
Theta(n).

753
01:07:40,991 --> 01:07:45,438
Let's just see why.
The summation of 0 :

754
01:07:45,438 --> 01:07:50,000
n of Theta(n) is Theta(n^2)
divided by n.

755
01:07:50,000 --> 01:07:54,447
Or, if I want to bring the
Theta(n) out,

756
01:07:54,447 --> 01:08:01,631
I have 1 times the summation of
k equals1 to n of Theta(1) or of

757
01:08:04,527 --> 01:08:07,888
you get n, either way of doing
it.

758
01:08:07,888 --> 01:08:12,981
This is, in some sense,
because the summations have

759
01:08:12,981 --> 01:08:17,157
identical terms,
and this is just algebra.

760
01:08:17,157 --> 01:08:22,861
Now what we are going to do is
do something for technical

761
01:08:22,861 --> 01:08:30,216
convenience.
Because we are going to absorb

762
01:08:30,216 --> 01:08:38,390
the k=0, 1 terms into the
Theta(n) for technical

763
01:08:38,390 --> 01:08:41,000
convenience.

764
01:08:46,000 --> 01:08:50,432
We have a recurrence here where
I have an order n.

765
01:08:50,432 --> 01:08:54,412
And, if I look at the cases
where k=0 or k=1,

766
01:08:54,412 --> 01:08:59,929
I know what the expectation is.
For 0, 1, the expected cost is

767
01:08:59,929 --> 01:09:04,000
the worst case cost,
which is constant.

768
01:09:04,000 --> 01:09:07,920
Because I am only solving the
problem for a constant size.

769
01:09:07,920 --> 01:09:12,252
And we know that for any of the
boundary cases that our solution

770
01:09:12,252 --> 01:09:15,142
of recurrence,
our assumption is that it is

771
01:09:15,142 --> 01:09:18,100
constant time.
So, I basically can just take

772
01:09:18,100 --> 01:09:21,608
those two terms out.
And all that does it accumulate

773
01:09:21,608 --> 01:09:24,291
some more constant here in the
Theta(n).

774
01:09:24,291 --> 01:09:27,798
It is going to make the
solution of the recurrence a

775
01:09:27,798 --> 01:09:33,395
little bit easier.
And, if I do that,

776
01:09:33,395 --> 01:09:43,584
I get expectation of T(n) = 2/n
summation k=2 to n-1 of

777
01:09:43,584 --> 01:09:50,000
expectation of T(k) + Theta(n).

778
01:10:00,000 --> 01:10:06,864
So, all of that work was to
derive the recurrence.

779
01:10:06,864 --> 01:10:14,570
And now we have to solve it.
Just to review what we did,

780
01:10:14,570 --> 01:10:21,434
we started out with a
recurrence which was for the

781
01:10:21,434 --> 01:10:29,000
random variable which involved a
case statement.

782
01:10:29,000 --> 01:10:33,128
We converted that into some
mathematics without the case

783
01:10:33,128 --> 01:10:37,031
statement, just with a product,
and then we derived a

784
01:10:37,031 --> 01:10:41,535
recurrence for the expectation.
And now we are in the process

785
01:10:41,535 --> 01:10:44,087
of trying to solve that
recurrence.

786
01:10:44,087 --> 01:10:47,765
We have done some
simplification of the recurrence

787
01:10:47,765 --> 01:10:52,568
so that we understand what it is
that we are going to solve here.

788
01:10:52,568 --> 01:10:56,472
By the way, I don't give things
like this on quizzes.

789
01:10:56,472 --> 01:11:00,000
I do expect you to understand
it.

790
01:11:00,000 --> 01:11:03,120
The elements of this you will
find on a quiz.

791
01:11:03,120 --> 01:11:05,602
This is a lot of work to figure
out.

792
01:11:05,602 --> 01:11:09,148
This took smart people to do.
Even though it is all

793
01:11:09,148 --> 01:11:12,907
elementary, but working out
something like this at the

794
01:11:12,907 --> 01:11:17,375
elementary level is still a bit
of work even for somebody who is

795
01:11:17,375 --> 01:11:23,465
knowledgeable in this area.
Now we are going to solve that

796
01:11:23,465 --> 01:11:30,395
last recurrence over there and
we are going to prove that the

797
01:11:30,395 --> 01:11:37,556
expectation of T(n) is less than
or equal to (an lg n) for some

798
01:11:37,556 --> 01:11:44,139
constant a greater than 0.
That is going to be what we are

799
01:11:44,139 --> 01:11:48,759
going to do.
And so what technique do you

800
01:11:48,759 --> 01:11:52,571
think we should use to prove
this?

801
01:11:52,571 --> 01:11:58,000
Does this look like a master
method?

802
01:12:03,000 --> 01:12:08,600
It is nothing like the master
method.

803
01:12:08,600 --> 01:12:13,733
So, when in doubt do
substitution.

804
01:12:13,733 --> 01:12:19,333
It is the grand-daddy of all
methods.

805
01:12:19,333 --> 01:12:28,355
What we will do is solve the
base case by simply choosing a

806
01:12:28,355 --> 01:12:37,844
big enough so that (an lg n) is
bigger than the expectation of

807
01:12:37,844 --> 01:12:45,000
T(n) for sufficiently large
small n.

808
01:12:45,000 --> 01:12:49,212
So, I just pick a to be big
enough.

809
01:12:49,212 --> 01:12:54,044
And this is,
by the way, why I wanted to

810
01:12:54,044 --> 01:12:59,000
exclude 0 and 1 from the
recurrence.

811
01:12:59,000 --> 01:13:03,338
Because, for example,
when n=0, log of 0 is,

812
01:13:03,338 --> 01:13:08,181
it's like dividing by 0,
right, you cannot do it.

813
01:13:08,181 --> 01:13:12,519
Log of 1 is 0.
So here, even if I restricted

814
01:13:12,519 --> 01:13:18,370
it to 1, here I would have a 0,
and I can't ever pick a big

815
01:13:18,370 --> 01:13:24,424
enough to dominate those cases.
What I do is I just say look,

816
01:13:24,424 --> 01:13:30,577
I just absorb whatever the cost
is into the T(n) for technical

817
01:13:30,577 --> 01:13:37,169
convenience.
And that lets me address it as

818
01:13:37,169 --> 01:13:44,119
(an lg n) to be big enough to
handle the base case.

819
01:13:44,119 --> 01:13:50,930
So, that is why we made that
technical assumption.

820
01:13:50,930 --> 01:13:58,714
We are going to use a fact
which is that the summation of

821
01:13:58,714 --> 01:14:06,776
k=2 to n-1 of k lg k is less
than or equal to 1/2n^2 lg n -

822
01:14:06,776 --> 01:14:11,242
1/8n^2.
I am going to leave that as an

823
01:14:11,242 --> 01:14:15,429
exercise for you to workout.
I think it is an exercise in

824
01:14:15,429 --> 01:14:18,644
the book, too.
I want you to go and evaluate

825
01:14:18,644 --> 01:14:21,261
this.
There are two ways to evaluate

826
01:14:21,261 --> 01:14:23,130
it.
One is by using purely

827
01:14:23,130 --> 01:14:27,093
summations and facts about
summations by splitting the

828
01:14:27,093 --> 01:14:31,579
summation into two pieces and
reconstituting it to prove this

829
01:14:31,579 --> 01:14:35,338
bound.
The other way is to use the

830
01:14:35,338 --> 01:14:38,629
integral method for solving
summations.

831
01:14:38,629 --> 01:14:43,220
Either way you can prove.
The integral method actually

832
01:14:43,220 --> 01:14:46,165
gets you a tighter bound than
this.

833
01:14:46,165 --> 01:14:50,582
This is a basic fact,
and you should go off and know

834
01:14:50,582 --> 01:14:55,000
how to do that.
Now let's do substitution.

835
01:15:06,000 --> 01:15:18,821
The expectation of T(n) is less
than or equal to 2/n times the

836
01:15:18,821 --> 01:15:29,331
summation k=2 to n-1,
now we do the substitution of

837
01:15:29,331 --> 01:15:39,000
ak lg k, the smaller values plus
Theta(n).

838
01:15:39,000 --> 01:15:42,416
I might mentioned,
by the way, that the hard part

839
01:15:42,416 --> 01:15:45,334
of doing this,
it is easy to get the bound

840
01:15:45,334 --> 01:15:48,679
without this term,
it is easy to get this bound,

841
01:15:48,679 --> 01:15:51,669
1/2n^2 lg n,
it is harder to get the second

842
01:15:51,669 --> 01:15:54,231
order term.
It turns out you need the

843
01:15:54,231 --> 01:15:59,000
second order term in order to do
what we are going to do.

844
01:15:59,000 --> 01:16:05,621
You have to be able to subtract
a quadratic amount of the n^2 lg

845
01:16:05,621 --> 01:16:09,195
n in order to make this proof
work.

846
01:16:09,195 --> 01:16:15,291
And that is the trickier part
in evaluating that summation.

847
01:16:15,291 --> 01:16:20,126
So, we get this.
That is less than or equal to?

848
01:16:20,126 --> 01:16:26,537
Well, I happen to know how much
this is by using that formula.

849
01:16:26,537 --> 01:16:31,792
I use my fact and get 2a/n
(1/2n^2 lg n - 1/8n^2) +

850
01:16:31,792 --> 01:16:43,558
Theta(n).
Did I do something wrong?

851
01:16:43,558 --> 01:16:51,970
There we go.
Very good.

852
01:16:51,970 --> 01:17:05,272
That is equal to -
If I multiply this first part

853
01:17:05,272 --> 01:17:14,363
through that is an lg n.
And now, so I don't make a

854
01:17:14,363 --> 01:17:22,545
mistake, I want to express this
as my desired,

855
01:17:22,545 --> 01:17:32,000
this is what I want it to be,
minus a residual.

856
01:17:32,000 --> 01:17:38,702
I am going to write the
residual as this part.

857
01:17:38,702 --> 01:17:47,489
And so, the way to write that
is, that is going to be minus.

858
01:17:47,489 --> 01:17:56,723
And then it is going to be this
term here, which is going to be

859
01:17:56,723 --> 01:18:00,000
an/4 - Theta(n).

860
01:18:05,000 --> 01:18:12,309
And that is going to be less
than or equal to an lg n if this

861
01:18:12,309 --> 01:18:17,304
part is positive.
And I can make that part

862
01:18:17,304 --> 01:18:24,614
positive by picking a big enough
such that an/4 dominates the

863
01:18:24,614 --> 01:18:34,590
constant in the Theta(n) here.
Whatever the constant is here,

864
01:18:34,590 --> 01:18:45,211
I can find an a that is big
enough so that this term makes

865
01:18:45,211 --> 01:18:54,527
this part positive.
If a is big enough so that an/4

866
01:18:54,527 --> 01:19:01,217
dominates Theta(n).
And so the running time of

867
01:19:01,217 --> 01:19:04,288
randomized quicksort is order n
lg n.

868
01:19:04,288 --> 01:19:09,064
That is what we just proved,
the expected running time is

869
01:19:09,064 --> 01:19:11,623
order n lg n.
Now, in practice,

870
01:19:11,623 --> 01:19:16,741
quicksort is a great algorithm.
It is typically three or more

871
01:19:16,741 --> 01:19:21,688
times faster than mergesort.
It doesn't give you the strong

872
01:19:21,688 --> 01:19:26,464
guarantee necessarily of
mergesort and being worst-case n

873
01:19:26,464 --> 01:19:29,245
lg n.
But in practice,

874
01:19:29,245 --> 01:19:33,136
if you use randomized
quicksort, it is generally as

875
01:19:33,136 --> 01:19:37,573
much as three times faster.
It does require code tuning in

876
01:19:37,573 --> 01:19:40,219
order to get it up to be that
fast.

877
01:19:40,219 --> 01:19:44,966
You do have to go and coarsen
the base cases and do some other

878
01:19:44,966 --> 01:19:47,613
tricks there,
but most good sorting

879
01:19:47,613 --> 01:19:51,660
algorithms that you will find
are based on quicksort.

880
01:19:51,660 --> 01:19:56,018
Also one of the other reasons
it works well is because it

881
01:19:56,018 --> 01:20:01,000
tends to work well with caches
in virtual memory.

882
01:20:01,000 --> 01:20:05,431
We are not really talking much
about caching models and so

883
01:20:05,431 --> 01:20:09,941
forth, big topic these days in
algorithms, but it does work

884
01:20:09,941 --> 01:20:12,973
very well with caches in virtual
memory.

885
01:20:12,973 --> 01:20:17,405
It is another reason that this
is a good algorithm to use.

886
01:20:17,405 --> 01:20:20,359
Good recitation,
by the way, on Friday.

887
01:20:20,359 --> 01:20:24,247
We are going to see another n
log n time algorithm,

888
01:20:24,247 --> 01:20:27,000
a very important one in
recitation on Friday.