1
00:00:06,000 --> 00:00:10,000
And finally we look at the 6
And at that point we are done.

2
00:00:10,000 --> 00:00:11,665
We're going to get started.

3
00:00:11,665 --> 00:00:16,000
Handouts are the by the door
if anybody didn't pick one up.

4
00:00:16,000 --> 00:00:18,000
My name is Charles Leiserson.

5
00:00:18,000 --> 00:00:22,500
I will be lecturing this
course this term, Introduction

6
00:00:22,500 --> 00:00:25,000
to Algorithms,
with Erik Demaine.

7
00:00:25,000 --> 00:00:29,200
In addition, this is an
SMA course, a Singapore MIT

8
00:00:29,200 --> 00:00:35,000
Alliance course which will be
run in Singapore by David Hsu.

9
00:00:35,000 --> 00:00:39,160
And so all the lectures
will be videotaped and made

10
00:00:39,160 --> 00:00:42,724
available on the Web for
the Singapore students,

11
00:00:42,724 --> 00:00:47,664
as well as for MIT students
who choose to watch them

12
00:00:47,664 --> 00:00:49,000
on the Web.

13
00:00:49,000 --> 00:00:55,000
If you have an issue of not
wanting to be on the videotape,

14
00:00:55,000 --> 00:00:57,500
you should sit in the back row.

15
00:00:57,500 --> 00:00:58,000
OK?

16
00:00:58,000 --> 00:01:00,724
Otherwise, you will be on it.

17
00:01:00,724 --> 00:01:03,375
There is a video
recording policy,

18
00:01:03,375 --> 00:01:06,000
but it seems like they ran out.

19
00:01:06,000 --> 00:01:08,331
If anybody wants
to see it, people,

20
00:01:08,331 --> 00:01:11,089
if they could just
sort of pass them

21
00:01:11,089 --> 00:01:14,666
around maybe a little bit,
once you're done reading it,

22
00:01:14,666 --> 00:01:16,331
or you can come up.

23
00:01:16,331 --> 00:01:18,000
I did secure one copy.

24
00:01:18,000 --> 00:01:21,000
Before we get into the
content of the course,

25
00:01:21,000 --> 00:01:24,108
let's briefly go over
the course information

26
00:01:24,108 --> 00:01:27,724
because there are some
administrative things that we

27
00:01:27,724 --> 00:01:30,000
sort of have to do.

28
00:01:30,000 --> 00:01:33,000
As you can see, this
term we have a big staff.

29
00:01:33,000 --> 00:01:35,000
Take a look at the handout here.

30
00:01:35,000 --> 00:01:37,496
Including this
term six TAs, which

31
00:01:37,496 --> 00:01:42,500
is two more TAs than we
normally get for this course.

32
00:01:42,500 --> 00:01:45,666
That means recitations
will be particularly small.

33
00:01:45,666 --> 00:01:50,500
There is a World Wide Web page,
and you should bookmark that

34
00:01:50,500 --> 00:01:55,142
and go there regularly
because that is where

35
00:01:55,142 --> 00:01:57,834
everything will be distributed.

36
00:01:57,834 --> 00:01:58,333
Email.

37
00:01:58,333 --> 00:02:00,800
You should not be
emailing directly to,

38
00:02:00,800 --> 00:02:04,000
even though we give you
our email addresses,

39
00:02:04,000 --> 00:02:06,000
to the individual
members of the staff.

40
00:02:06,000 --> 00:02:07,665
You should email us generally.

41
00:02:07,665 --> 00:02:11,284
And the reason is you will
get much faster response.

42
00:02:11,284 --> 00:02:13,500
And also, for any
communications,

43
00:02:13,500 --> 00:02:17,600
generally we like to monitor
what the communications are

44
00:02:17,600 --> 00:02:21,284
so it's helpful to have
emails coming to everybody

45
00:02:21,284 --> 00:02:23,000
on the course staff.

46
00:02:23,000 --> 00:02:26,999
As I mentioned, we will be doing
distance learning this term.

47
00:02:26,999 --> 00:02:29,500
And so you can watch
lectures online

48
00:02:29,500 --> 00:02:32,500
if you choose to do that.

49
00:02:32,500 --> 00:02:37,000
I would recommend, for people
who have the opportunity

50
00:02:37,000 --> 00:02:38,999
to watch, to come live.

51
00:02:38,999 --> 00:02:40,000
It's better live.

52
00:02:40,000 --> 00:02:42,000
You get to interact.

53
00:02:42,000 --> 00:02:45,815
There's an intangible that
comes with having it live.

54
00:02:45,815 --> 00:02:48,454
In fact, in addition
to the videos,

55
00:02:48,454 --> 00:02:51,632
I meet weekly with
the Singapore students

56
00:02:51,632 --> 00:02:57,284
so that they have a
live session as well.

57
00:02:57,284 --> 00:02:58,000
Prerequisites.

58
00:02:58,000 --> 00:03:00,140
The prerequisites
for this course

59
00:03:00,140 --> 00:03:05,000
are 6.042, which is Math for
Computer Science, and 6.001.

60
00:03:05,000 --> 00:03:09,000
You basically need discrete
mathematics and probability,

61
00:03:09,000 --> 00:03:11,220
as well as
programming experience

62
00:03:11,220 --> 00:03:13,500
to take this course
successfully.

63
00:03:13,500 --> 00:03:17,284
People do not have that
background should not

64
00:03:17,284 --> 00:03:19,000
be in the class.

65
00:03:19,000 --> 00:03:22,000
We will be checking
prerequisites.

66
00:03:22,000 --> 00:03:24,496
If you have any
questions, please

67
00:03:24,496 --> 00:03:27,666
come to talk to us after class.

68
00:03:27,666 --> 00:03:29,000
Let's see.

69
00:03:29,000 --> 00:03:30,500
Lectures are here.

70
00:03:30,500 --> 00:03:33,452
For SMA students, they
have the videotapes

71
00:03:33,452 --> 00:03:36,428
and they will also
have a weekly meeting.

72
00:03:36,428 --> 00:03:41,220
Students must attend a one hour
recitation session each week.

73
00:03:41,220 --> 00:03:45,712
There will be new material
presented in the recitation.

74
00:03:45,712 --> 00:03:49,500
Unlike the lectures,
they will not be online.

75
00:03:49,500 --> 00:03:52,200
Unlike the lectures,
there will not

76
00:03:52,200 --> 00:03:56,142
be lecture notes distributed
for the recitations in general.

77
00:03:56,142 --> 00:04:00,000
And, yet, there will
be material there

78
00:04:00,000 --> 00:04:03,000
that is directly on the exams.

79
00:04:03,000 --> 00:04:07,000
And so every term we say
oh, when did you cover that?

80
00:04:07,000 --> 00:04:08,500
That was in recitation.

81
00:04:08,500 --> 00:04:10,000
You missed that one.

82
00:04:10,000 --> 00:04:12,284
So, recitations are mandatory.

83
00:04:12,284 --> 00:04:15,452
And, in particular,
also let me just

84
00:04:15,452 --> 00:04:18,416
mention your recitation
instructor is the one who

85
00:04:18,416 --> 00:04:20,080
assigns your final grade.

86
00:04:20,080 --> 00:04:24,500
So we have a grade meeting
and keep everybody normal,

87
00:04:24,500 --> 00:04:29,500
but your recitation has the
final say on your grade.

88
00:04:29,500 --> 00:04:30,000
Handouts.

89
00:04:30,000 --> 00:04:34,000
Handouts are available
on the course Web page.

90
00:04:34,000 --> 00:04:39,142
We will not generally, except
for this one, first handout,

91
00:04:39,142 --> 00:04:42,000
be bringing handouts to class.

92
00:04:42,000 --> 00:04:46,000
Textbook is this book,
Introduction to Algorithms.

93
00:04:46,000 --> 00:04:50,500
MIT students can get it any of
the local bookstores, including

94
00:04:50,500 --> 00:04:52,000
the MIT Coop.

95
00:04:52,000 --> 00:04:55,545
There is also a
new online service

96
00:04:55,545 --> 00:04:57,180
that provides textbooks.

97
00:04:57,180 --> 00:05:02,125
You can also get a
discount if you buy it

98
00:05:02,125 --> 00:05:04,000
at the MIT Press Bookstore.

99
00:05:04,000 --> 00:05:09,450
There is a coupon in the MIT
Student Telephone Directory

100
00:05:09,450 --> 00:05:11,998
for a discount on
MIT Press books.

101
00:05:11,998 --> 00:05:17,000
And you can use that to purchase
this book at a discount.

102
00:05:17,000 --> 00:05:18,142
Course website.

103
00:05:18,142 --> 00:05:21,000
This is the course website.

104
00:05:21,000 --> 00:05:24,108
It links to the
Stellar website, which

105
00:05:24,108 --> 00:05:30,000
is where, actually,
everything will be kept.

106
00:05:30,000 --> 00:05:33,000
And SMA students have
their own website.

107
00:05:33,000 --> 00:05:36,108
Some students find this
course particularly challenges

108
00:05:36,108 --> 00:05:38,776
so we will have extra help.

109
00:05:38,776 --> 00:05:42,500
We will post weekly
office hours on the course

110
00:05:42,500 --> 00:05:44,000
website for the TAs.

111
00:05:44,000 --> 00:05:46,912
And then as an
experiment this term,

112
00:05:46,912 --> 00:05:51,000
we are going to offer
homework labs for this class.

113
00:05:51,000 --> 00:05:55,331
What a homework lab is,
is it's a place and a time

114
00:05:55,331 --> 00:05:58,500
you can go where other
people in the course

115
00:05:58,500 --> 00:06:01,000
will go to do homework.

116
00:06:01,000 --> 00:06:05,360
And there will be typically
two TAs who staff the lab.

117
00:06:05,360 --> 00:06:07,666
And so, as you're
working on your homework,

118
00:06:07,666 --> 00:06:11,332
you can get help from
the TAs if you need it.

119
00:06:11,332 --> 00:06:14,665
And it's generally a place,
we're going to schedule those,

120
00:06:14,665 --> 00:06:18,535
and they will be on the course
calendar for where it is

121
00:06:18,535 --> 00:06:22,152
and when it is that they
will be held, but usually

122
00:06:22,152 --> 00:06:26,444
Sundays 2:00 to 4:00 pm, or
else it will be some evening.

123
00:06:26,444 --> 00:06:29,250
I think the first one
is an evening, right?

124
00:06:29,250 --> 00:06:33,333
Near to when the
homework is due.

125
00:06:33,333 --> 00:06:36,428
Your best bet is try
to do the homework

126
00:06:36,428 --> 00:06:39,000
in advance of the homework lab.

127
00:06:39,000 --> 00:06:41,149
But then, if you
want extra help,

128
00:06:41,149 --> 00:06:45,000
if you want to talk over
your solutions with people

129
00:06:45,000 --> 00:06:49,000
because as we will
talk about problem sets

130
00:06:49,000 --> 00:06:54,200
you can solve in collaboration
with other people in the class.

131
00:06:54,200 --> 00:07:00,000
In addition, there are several
peer assistance programs.

132
00:07:00,000 --> 00:07:02,664
Also the office of
Minority Education

133
00:07:02,664 --> 00:07:05,776
has an assistance
program, and those usually

134
00:07:05,776 --> 00:07:08,000
get booked up pretty quickly.

135
00:07:08,000 --> 00:07:10,800
If you're interested
in those, good idea

136
00:07:10,800 --> 00:07:15,000
to make an appointment to
get there and get help soon.

137
00:07:15,000 --> 00:07:19,666
The homework labs, I hope a lot
of people will try that out.

138
00:07:19,666 --> 00:07:21,000
We've never done this.

139
00:07:21,000 --> 00:07:24,000
I don't know of
any other course.

140
00:07:24,000 --> 00:07:28,000
Do other people know of courses
at MIT that have done this?

141
00:07:28,000 --> 00:07:30,928
6.011 did it, OK.

142
00:07:30,928 --> 00:07:31,428
Good.

143
00:07:31,428 --> 00:07:34,500
And was it successful
in that class?

144
00:07:34,500 --> 00:07:35,962
It never went,

145
00:07:35,962 --> 00:07:36,461
OK.

146
00:07:36,461 --> 00:07:38,766
Good. [LAUGHTER] We will see.

147
00:07:38,766 --> 00:07:43,000
If it's not paying
off then we will just

148
00:07:43,000 --> 00:07:47,000
return to ordinary office
hours for those TAs,

149
00:07:47,000 --> 00:07:52,000
but I think for some students
that is a good opportunity.

150
00:07:52,000 --> 00:07:55,852
If you wish to be
registered in this course,

151
00:07:55,852 --> 00:08:00,664
you must sign up on
the course Web page.

152
00:08:00,664 --> 00:08:04,000
So, that is requirement one.

153
00:08:04,000 --> 00:08:06,270
It must be done today.

154
00:08:06,270 --> 00:08:11,800
You will find it difficult to
pass the course if you are not

155
00:08:11,800 --> 00:08:13,000
in the class.

156
00:08:13,000 --> 00:08:16,424
And you should
notify your TA if you

157
00:08:16,424 --> 00:08:20,452
decide to drop so that
we can get you off

158
00:08:20,452 --> 00:08:23,000
and stop the mailings,
stop the spam.

159
00:08:23,000 --> 00:08:29,000
And you should register
today before 7:00 PM.

160
00:08:29,000 --> 00:08:32,600
And then we're going to email
your recitation assignment

161
00:08:32,600 --> 00:08:34,200
to you before Noon tomorrow.

162
00:08:34,200 --> 00:08:37,816
And if you don't receive this
information by Thursday Noon,

163
00:08:37,816 --> 00:08:41,856
please send us an email
to the course staff

164
00:08:41,856 --> 00:08:44,000
generally, not to
me individually,

165
00:08:44,000 --> 00:08:48,000
saying that you didn't receive
your recitation assignment.

166
00:08:48,000 --> 00:08:51,070
And so if you haven't
received it by Thursday Noon

167
00:08:51,070 --> 00:08:52,000
you want to.

168
00:08:52,000 --> 00:08:55,267
I think generally they
are going to send them

169
00:08:55,267 --> 00:08:59,426
out tonight or at least
by tomorrow morning.

170
00:08:59,426 --> 00:09:00,000
Yeah.

171
00:09:00,000 --> 00:09:00,499
OK.

172
00:09:00,499 --> 00:09:02,459
SMA students don't have
to worry about this.

173
00:09:02,459 --> 00:09:03,000
Problem sets.

174
00:09:03,000 --> 00:09:06,630
We have nine problem sets
that we project will be

175
00:09:06,630 --> 00:09:08,125
assigned during the semester.

176
00:09:08,125 --> 00:09:10,400
A couple things
about problem sets.

177
00:09:10,400 --> 00:09:12,428
Homeworks won't
generally be accepted,

178
00:09:12,428 --> 00:09:15,000
if you have extenuating
circumstances you

179
00:09:15,000 --> 00:09:17,500
should make prior arrangements
with your recitation

180
00:09:17,500 --> 00:09:18,000
instructor.

181
00:09:18,000 --> 00:09:21,000
In fact, almost all of
the administrative stuff,

182
00:09:21,000 --> 00:09:23,565
you shouldn't come
to me to ask and say

183
00:09:23,565 --> 00:09:25,285
can I hand in something late?

184
00:09:25,285 --> 00:09:28,250
You should be talking to
your recitation instructor.

185
00:09:28,250 --> 00:09:32,999
You can read the other
things about the form,

186
00:09:32,999 --> 00:09:36,664
but let me just mention that
there are exercises that

187
00:09:36,664 --> 00:09:41,875
should be solved but not handed
in as well to give you drill

188
00:09:41,875 --> 00:09:43,000
on the material.

189
00:09:43,000 --> 00:09:46,000
I highly recommend you
doing the exercises.

190
00:09:46,000 --> 00:09:50,000
They both test your
understanding of the material,

191
00:09:50,000 --> 00:09:55,000
and exercises have this way of
finding themselves on quizzes.

192
00:09:55,000 --> 00:10:00,000
You're often asked to
describe algorithms.

193
00:10:00,000 --> 00:10:04,280
And here is a little
outline of what you can

194
00:10:04,280 --> 00:10:06,500
use to describe an algorithm.

195
00:10:06,500 --> 00:10:11,400
The grading policy is
something that somehow I cover.

196
00:10:11,400 --> 00:10:15,832
And always every term
there are at least

197
00:10:15,832 --> 00:10:20,888
a couple of students who pretend
like I never showed them this.

198
00:10:20,888 --> 00:10:28,285
If you skip problems it has a
nonlinear effect on your grade.

199
00:10:28,285 --> 00:10:30,000
Nonlinear, OK?

200
00:10:30,000 --> 00:10:34,000
If you don't skip any problems,
no effect on your grade.

201
00:10:34,000 --> 00:10:38,000
If you skip one problem, a
hundredth of a letter grade,

202
00:10:38,000 --> 00:10:39,600
we can handle that.

203
00:10:39,600 --> 00:10:42,000
But two problems it's a tenth.

204
00:10:42,000 --> 00:10:47,332
And, as you see, by the time you
have skipped like five letter

205
00:10:47,332 --> 00:10:50,000
grades, it is already
five problems.

206
00:10:50,000 --> 00:10:53,000
This is not problem
sets, by the way.

207
00:10:53,000 --> 00:10:54,000
This is problems, OK?

208
00:10:54,000 --> 00:10:59,000
You're down a third
of a letter grade.

209
00:10:59,000 --> 00:11:01,664
And if you don't
do nine or more,

210
00:11:01,664 --> 00:11:05,000
so that's typically about
three to four problem sets,

211
00:11:05,000 --> 00:11:07,000
you don't pass the class.

212
00:11:07,000 --> 00:11:11,000
I always have some students
coming at the end of the year

213
00:11:11,000 --> 00:11:14,000
saying oh, I didn't
do any of my problems.

214
00:11:14,000 --> 00:11:18,000
Can you just pass me because
I did OK on the exams?

215
00:11:18,000 --> 00:11:23,000
Answer no, a very simple answer
because we've said it upfront.

216
00:11:23,000 --> 00:11:27,000
So, the problem sets are an
integral part of the course.

217
00:11:27,000 --> 00:11:28,428
Collaboration policy.

218
00:11:28,428 --> 00:11:32,900
This is extremely important
so everybody pay attention.

219
00:11:32,900 --> 00:11:35,000
If you are asleep now wake up.

220
00:11:35,000 --> 00:11:39,000
Like that's going to
wake anybody up, right?

221
00:11:39,000 --> 00:11:41,000
[LAUGHTER] The goal of homework.

222
00:11:41,000 --> 00:11:43,000
Professor Demaine
and my philosophy

223
00:11:43,000 --> 00:11:48,000
is that the goal of homework is
to help you learn the material.

224
00:11:48,000 --> 00:11:50,565
And one way of helping
to learn is not

225
00:11:50,565 --> 00:11:53,600
to just be stuck and
unable to solve something

226
00:11:53,600 --> 00:11:56,400
because then you're
in no better shape

227
00:11:56,400 --> 00:12:00,571
when the exam roles around,
which is where we're actually

228
00:12:00,571 --> 00:12:01,713
evaluating you.

229
00:12:01,713 --> 00:12:04,500
So, you're encouraged
to collaborate.

230
00:12:04,500 --> 00:12:08,332
But there are some commonsense
things about collaboration.

231
00:12:08,332 --> 00:12:12,089
If you go and you
collaborate to the extent

232
00:12:12,089 --> 00:12:15,856
that all you're doing is getting
the information from somebody

233
00:12:15,856 --> 00:12:18,363
else, you're not
learning the material

234
00:12:18,363 --> 00:12:22,000
and you're not going to
do well on the exams.

235
00:12:22,000 --> 00:12:25,625
In our experience, students
who collaborate generally

236
00:12:25,625 --> 00:12:30,000
do better than students
who work alone.

237
00:12:30,000 --> 00:12:31,840
But you owe it to
yourself, if you're

238
00:12:31,840 --> 00:12:36,200
going to work in a study group,
to be prepared for your study

239
00:12:36,200 --> 00:12:37,000
group meeting.

240
00:12:37,000 --> 00:12:39,448
And specifically you
should spend a half an hour

241
00:12:39,448 --> 00:12:41,842
to 45 minutes on each
problem before you

242
00:12:41,842 --> 00:12:44,428
go to group so
you're up to speed

243
00:12:44,428 --> 00:12:47,000
and you've tried out your ideas.

244
00:12:47,000 --> 00:12:48,610
And you may have
solutions to some,

245
00:12:48,610 --> 00:12:50,800
you may be stuck
on some other ones,

246
00:12:50,800 --> 00:12:54,000
but at least you
applied yourself to it.

247
00:12:54,000 --> 00:12:57,000
After 30 to 45 minutes, if
you cannot get the problem,

248
00:12:57,000 --> 00:13:00,928
just sitting there and banging
your head against it makes no

249
00:13:00,928 --> 00:13:01,428
sense.

250
00:13:01,428 --> 00:13:04,666
It's not a productive
use of your time.

251
00:13:04,666 --> 00:13:08,000
And I know most of you have
issues with having time

252
00:13:08,000 --> 00:13:09,000
on your hands, right?

253
00:13:09,000 --> 00:13:10,600
Like it's not there.

254
00:13:10,600 --> 00:13:13,600
So, don't go banging your
head against problems

255
00:13:13,600 --> 00:13:16,333
that are too hard or
where you don't understand

256
00:13:16,333 --> 00:13:18,000
what's going on or whatever.

257
00:13:18,000 --> 00:13:21,000
That's when the study
group can help out.

258
00:13:21,000 --> 00:13:23,664
And, as I mentioned,
we'll have homework labs

259
00:13:23,664 --> 00:13:25,665
which will give
you an opportunity

260
00:13:25,665 --> 00:13:29,200
to go and do that and
coordinate with other students

261
00:13:29,200 --> 00:13:32,750
rather than necessarily
having to form your own group.

262
00:13:32,750 --> 00:13:35,000
And the TAs will be there.

263
00:13:35,000 --> 00:13:39,160
If your group is unable
to solve the problem then

264
00:13:39,160 --> 00:13:43,000
talk to other groups or ask
your recitation instruction.

265
00:13:43,000 --> 00:13:46,000
And, that's how you
go about solving them.

266
00:13:46,000 --> 00:13:49,330
Writing up the
problem sets, however,

267
00:13:49,330 --> 00:13:51,500
is your individual
responsibility

268
00:13:51,500 --> 00:13:54,000
and should be done alone.

269
00:13:54,000 --> 00:13:58,000
You don't write up your problem
solutions with other students,

270
00:13:58,000 --> 00:14:01,227
you write them up on your own.

271
00:14:01,227 --> 00:14:04,428
And you should on
your problem sets,

272
00:14:04,428 --> 00:14:07,000
because this is
an academic place,

273
00:14:07,000 --> 00:14:11,000
we understand that the source
of academic information

274
00:14:11,000 --> 00:14:15,000
is very important, if you
collaborated on solutions

275
00:14:15,000 --> 00:14:18,664
you should write a list
of the collaborators.

276
00:14:18,664 --> 00:14:22,750
Say I worked with so
and so on this solution.

277
00:14:22,750 --> 00:14:25,000
It does not affect your grade.

278
00:14:25,000 --> 00:14:30,000
It's just a question
of being scholarly.

279
00:14:30,000 --> 00:14:34,363
It is a violation of this policy
to submit a problem solution

280
00:14:34,363 --> 00:14:38,500
that you cannot orally explain
to a member of the course

281
00:14:38,500 --> 00:14:39,000
staff.

282
00:14:39,000 --> 00:14:44,000
You say oh, well, my write up is
similar to that other person's.

283
00:14:44,000 --> 00:14:45,600
I didn't copy them.

284
00:14:45,600 --> 00:14:49,284
We may ask you to orally
explain your solution.

285
00:14:49,284 --> 00:14:52,776
If you are unable,
according to this policy,

286
00:14:52,776 --> 00:14:55,375
the presumption is
that you cheated.

287
00:14:55,375 --> 00:14:59,800
So, do not write up stuff
that you don't understand.

288
00:14:59,800 --> 00:15:06,220
You should be able to write up
the stuff that you understand.

289
00:15:06,220 --> 00:15:10,220
Understand why you're putting
down what you're putting down.

290
00:15:10,220 --> 00:15:13,284
If it isn't obvious, no
collaboration whatsoever

291
00:15:13,284 --> 00:15:15,000
is permitted on exams.

292
00:15:15,000 --> 00:15:17,496
Exams is when we evaluate you.

293
00:15:17,496 --> 00:15:21,284
And now we're not interested
in evaluating other people,

294
00:15:21,284 --> 00:15:23,500
we're interested
in evaluating you.

295
00:15:23,500 --> 00:15:26,000
So, no collaboration on exams.

296
00:15:26,000 --> 00:15:31,000
We will have a take home
exam for the second quiz.

297
00:15:31,000 --> 00:15:33,000
You should look at the schedule.

298
00:15:33,000 --> 00:15:36,000
If there are problems
with the schedule of that,

299
00:15:36,000 --> 00:15:37,360
we want to know early.

300
00:15:37,360 --> 00:15:39,444
And we will give
you more details

301
00:15:39,444 --> 00:15:43,500
about the collaboration in the
lecture on Monday, November

302
00:15:43,500 --> 00:15:44,000
28th.

303
00:15:44,000 --> 00:15:47,500
Now, generally, the lectures
here, they're mandatory

304
00:15:47,500 --> 00:15:52,272
and you have to know them, but
I know that some people say gee,

305
00:15:52,272 --> 00:15:53,904
9:30 is kind of
early, especially

306
00:15:53,904 --> 00:15:55,333
on a Monday or whatever.

307
00:15:55,333 --> 00:15:58,750
It can be kind of
early to get up.

308
00:15:58,750 --> 00:16:01,800
However, on Monday,
November 28th,

309
00:16:01,800 --> 00:16:07,304
you fail the exam if you do
not show up to lecture on time.

310
00:16:07,304 --> 00:16:10,000
That one day you must show up.

311
00:16:10,000 --> 00:16:11,452
Any questions about that?

312
00:16:11,452 --> 00:16:14,400
That one day you
must show up here,

313
00:16:14,400 --> 00:16:18,000
even if you've been
watching them on the Web.

314
00:16:18,000 --> 00:16:21,333
And generally, if you think
you have transgressed,

315
00:16:21,333 --> 00:16:25,500
the best is to come to
us to talk about it.

316
00:16:25,500 --> 00:16:28,571
We can usually
work something out.

317
00:16:28,571 --> 00:16:34,270
It's when we find somebody has
transgressed from a third party

318
00:16:34,270 --> 00:16:38,500
or from obvious analyses
that we do with homeworks,

319
00:16:38,500 --> 00:16:41,000
that's when things get messy.

320
00:16:41,000 --> 00:16:45,000
So, if you think, for
some reason or other,

321
00:16:45,000 --> 00:16:47,331
oh, I may have done
something wrong,

322
00:16:47,331 --> 00:16:49,500
please come and talk to us.

323
00:16:49,500 --> 00:16:54,855
We actually were students once,
too, albeit many years ago.

324
00:16:54,855 --> 00:16:56,000
Any questions?

325
00:16:56,000 --> 00:17:00,000
So, this class has
great material.

326
00:17:00,000 --> 00:17:02,000
Fabulous material.

327
00:17:02,000 --> 00:17:13,000
And it's really fun, but
you do have to work hard.

328
00:17:13,000 --> 00:17:16,000
Let's talk content.

329
00:17:29,000 --> 00:17:32,000
This is the topic of the
first part of the course.

330
00:17:32,000 --> 00:17:35,000
The first part of the course
is focused on analysis.

331
00:17:35,000 --> 00:17:39,000
The second part of the
course is focused on design.

332
00:17:39,000 --> 00:17:41,331
Before you can do
design, you have

333
00:17:41,331 --> 00:17:45,000
to master a bunch of techniques
for analyzing algorithms.

334
00:17:45,000 --> 00:17:49,000
And then you'll be in a
position to design algorithms

335
00:17:49,000 --> 00:17:52,000
that you can analyze and
that which are efficient.

336
00:17:52,000 --> 00:18:06,333
The analysis of algorithm is
the theoretical study -- --

337
00:18:06,333 --> 00:18:21,220
of computer program performance
-- -- and resource usage.

338
00:18:21,220 --> 00:18:24,666
And a particular
focus on performance.

339
00:18:24,666 --> 00:18:29,500
We're studying how
to make things fast.

340
00:18:29,500 --> 00:18:32,250
In particular,
computer programs.

341
00:18:32,250 --> 00:18:37,998
We also will discover and
talk about other resources

342
00:18:37,998 --> 00:18:43,000
such as communication, such
as memory, whether RAM memory

343
00:18:43,000 --> 00:18:44,666
or disk memory.

344
00:18:44,666 --> 00:18:49,776
There are other resources
that we may care about,

345
00:18:49,776 --> 00:18:52,714
but predominantly we
focus on performance.

346
00:18:52,714 --> 00:18:57,500
Because this is a course
about performance,

347
00:18:57,500 --> 00:19:02,625
I like to put things in
perspective a little bit

348
00:19:02,625 --> 00:19:10,000
by starting out and asking,
in programming, what is more

349
00:19:10,000 --> 00:19:13,000
important than performance?

350
00:19:13,000 --> 00:19:18,000
If you're in an engineering
situation and writing code,

351
00:19:18,000 --> 00:19:24,500
writing software, what's more
important than performance?

352
00:19:24,500 --> 00:19:26,000
Correctness.

353
00:19:26,000 --> 00:19:26,714
Good.

354
00:19:26,714 --> 00:19:27,428
OK.

355
00:19:27,428 --> 00:19:28,856
What else?

356
00:19:28,856 --> 00:19:31,000
Simplicity can be.

357
00:19:31,000 --> 00:19:31,501
Very good.

358
00:19:31,501 --> 00:19:32,000
Yeah.

359
00:19:32,000 --> 00:19:40,000
Maintainability often much more
important than performance.

360
00:19:40,000 --> 00:19:40,500
Cost.

361
00:19:40,500 --> 00:19:44,714
And what type of cost
are you thinking?

362
00:19:44,714 --> 00:19:49,000
No, I mean cost of what?

363
00:19:49,000 --> 00:19:53,000
We're talking
software here, right?

364
00:19:53,000 --> 00:20:00,000
What type of cost
do you have in mind?

365
00:20:00,000 --> 00:20:04,500
There are some costs that
are involved when programming

366
00:20:04,500 --> 00:20:05,856
like programmer time.

367
00:20:05,856 --> 00:20:10,500
So, programmer time is another
thing also that might be.

368
00:20:10,500 --> 00:20:11,000
Stability.

369
00:20:11,000 --> 00:20:13,000
Robustness of the software.

370
00:20:13,000 --> 00:20:16,000
Does it break all the time?

371
00:20:16,000 --> 00:20:18,000
What else?

372
00:20:25,000 --> 00:20:25,750
Come on.

373
00:20:25,750 --> 00:20:28,400
We've got a bunch
of engineers here.

374
00:20:28,400 --> 00:20:30,000
A lot of things.

375
00:20:30,000 --> 00:20:31,125
How about features?

376
00:20:31,125 --> 00:20:33,000
Features can be more important.

377
00:20:33,000 --> 00:20:37,000
Having a wider collection of
features than your competitors.

378
00:20:37,000 --> 00:20:38,000
Functionality.

379
00:20:38,000 --> 00:20:39,000
Modularity.

380
00:20:39,000 --> 00:20:42,927
Is it designed in a way
where you can make changes

381
00:20:42,927 --> 00:20:47,377
in a local part of the code and
you don't have to make changes

382
00:20:47,377 --> 00:20:50,904
across the code in order
to affect a simple change

383
00:20:50,904 --> 00:20:52,000
in the functionality?

384
00:20:52,000 --> 00:20:54,800
There is one big one
which definitely,

385
00:20:54,800 --> 00:21:01,000
especially in the `90s, was
like the big thing in computers.

386
00:21:01,000 --> 00:21:01,999
The big thing.

387
00:21:01,999 --> 00:21:03,000
Well, security actually.

388
00:21:03,000 --> 00:21:03,500
Good.

389
00:21:03,500 --> 00:21:06,500
I don't even have that one down.

390
00:21:06,500 --> 00:21:08,000
Security is excellent.

391
00:21:08,000 --> 00:21:11,000
That's actually been
more in the 2000.

392
00:21:11,000 --> 00:21:13,250
Security has been
far more important

393
00:21:13,250 --> 00:21:14,600
often than performance.

394
00:21:14,600 --> 00:21:18,200
Scalability has been important,
although scalability,

395
00:21:18,200 --> 00:21:21,500
in some sense, is
performance related.

396
00:21:21,500 --> 00:21:24,000
But, yes, scalability is good.

397
00:21:24,000 --> 00:21:29,000
What was the big breakthrough
and why do people use Macintosh

398
00:21:29,000 --> 00:21:30,998
rather than Windows,
those people who

399
00:21:30,998 --> 00:21:34,000
are of that religion?

400
00:21:34,000 --> 00:21:35,750
User-friendliness.

401
00:21:35,750 --> 00:21:36,250
Wow.

402
00:21:36,250 --> 00:21:40,500
If you look at the number of
cycles of computers that went

403
00:21:40,500 --> 00:21:43,363
into user friendliness
in the `90s,

404
00:21:43,363 --> 00:21:48,110
it grew from almost nothing to
where it's now the vast part

405
00:21:48,110 --> 00:21:52,000
of the computation goes
into user friendly.

406
00:21:52,000 --> 00:21:56,000
So, all those things are more
important than performance.

407
00:21:56,000 --> 00:22:00,000
This is a course on performance.

408
00:22:00,000 --> 00:22:04,160
Then you can say OK,
well, why do we bother

409
00:22:04,160 --> 00:22:08,816
and why study algorithms
and performance if it's

410
00:22:08,816 --> 00:22:12,600
at the bottom of the heap?

411
00:22:12,600 --> 00:22:15,714
Almost always
people would rather

412
00:22:15,714 --> 00:22:20,000
have these other things
than performance.

413
00:22:20,000 --> 00:22:24,000
You go off and you
say to somebody,

414
00:22:24,000 --> 00:22:32,000
would I rather have performance
or more user friendliness?

415
00:22:32,000 --> 00:22:36,000
It's almost always more
important than performance.

416
00:22:36,000 --> 00:22:38,500
Why do we care then?

417
00:22:38,500 --> 00:22:39,000
Yeah?

418
00:22:44,000 --> 00:22:45,712
That wasn't user friendly.

419
00:22:45,712 --> 00:22:49,250
Sometimes performance
is correlated with user

420
00:22:49,250 --> 00:22:50,400
friendliness, absolutely.

421
00:22:50,400 --> 00:22:55,000
Nothing is more frustrating than
sitting there waiting, right?

422
00:22:55,000 --> 00:22:56,500
So, that's a good reason.

423
00:22:56,500 --> 00:22:58,666
What are some other reasons why?

424
00:22:58,666 --> 00:23:02,625
Sometimes they have
real time constraints

425
00:23:02,625 --> 00:23:09,000
so they don't actually work
unless they perform adequately.

426
00:23:09,000 --> 00:23:10,000
Yeah?

427
00:23:10,000 --> 00:23:13,500
Hard to get, well,
we don't usually

428
00:23:13,500 --> 00:23:17,000
quantify user friendliness
so I'm not sure,

429
00:23:17,000 --> 00:23:20,142
but I understand
what you're saying.

430
00:23:20,142 --> 00:23:25,250
He said we don't get exponential
performance improvements

431
00:23:25,250 --> 00:23:27,500
in user friendliness.

432
00:23:27,500 --> 00:23:34,000
We often don't get that in
performance either, by the way.

433
00:23:34,000 --> 00:23:38,000
[LAUGHTER] Sometimes
we do, but that's good.

434
00:23:38,000 --> 00:23:42,000
There are several reasons
that I think are important.

435
00:23:42,000 --> 00:23:45,000
Once is that often
performance measures

436
00:23:45,000 --> 00:23:48,600
the line between the
feasible and the infeasible.

437
00:23:48,600 --> 00:23:51,666
We have heard some
of these things.

438
00:23:51,666 --> 00:23:56,000
For example, when there
are real time requirements,

439
00:23:56,000 --> 00:24:02,000
if it's not fast enough
it's simply not functional.

440
00:24:02,000 --> 00:24:04,300
Or, if it uses too much
memory it's simply not

441
00:24:04,300 --> 00:24:05,666
going to work for you.

442
00:24:05,666 --> 00:24:08,360
And, as a consequence,
what you find is algorithms

443
00:24:08,360 --> 00:24:10,500
are on the cutting edge
of entrepreneurship.

444
00:24:10,500 --> 00:24:13,999
If you're talking about
just re implementing stuff

445
00:24:13,999 --> 00:24:16,428
that people did ten
years ago, performance

446
00:24:16,428 --> 00:24:19,000
isn't that important
at some level.

447
00:24:19,000 --> 00:24:21,541
But, if you're talking
about doing stuff

448
00:24:21,541 --> 00:24:24,665
that nobody has done
before, one of the reasons

449
00:24:24,665 --> 00:24:28,200
often that they haven't
done it is because it's too

450
00:24:28,200 --> 00:24:29,000
time consuming.

451
00:24:29,000 --> 00:24:31,600
Things don't scale and so forth.

452
00:24:31,600 --> 00:24:36,000
So, that's one reason, is the
feasible versus infeasible.

453
00:24:36,000 --> 00:24:39,108
Another thing is that
algorithms give you

454
00:24:39,108 --> 00:24:42,000
a language for talking
about program behavior,

455
00:24:42,000 --> 00:24:45,776
and that turns out
to be a language that

456
00:24:45,776 --> 00:24:48,454
has been pervasive
through computer science,

457
00:24:48,454 --> 00:24:53,400
is that the theoretical language
is what gets adopted by all

458
00:24:53,400 --> 00:24:57,000
the practitioners because
it's a clean way of thinking

459
00:24:57,000 --> 00:24:58,250
about things.

460
00:24:58,250 --> 00:25:02,454
A good way I think
about performance,

461
00:25:02,454 --> 00:25:07,000
and the reason it's on
the bottom of the heap,

462
00:25:07,000 --> 00:25:13,000
is sort of like performance is
like money, it's like currency.

463
00:25:13,000 --> 00:25:17,224
You say what good does a
stack of hundred dollar bills

464
00:25:17,224 --> 00:25:18,428
do for you?

465
00:25:18,428 --> 00:25:23,725
Would you rather have food or
water or shelter or whatever?

466
00:25:23,725 --> 00:25:28,000
And you're willing to
pay those hundred dollar

467
00:25:28,000 --> 00:25:31,600
bills, if you have
hundred dollar bills,

468
00:25:31,600 --> 00:25:33,400
for that commodity.

469
00:25:33,400 --> 00:25:39,000
Even though water is far more
important to your living.

470
00:25:39,000 --> 00:25:42,000
Well, similarly,
performance is what you use

471
00:25:42,000 --> 00:25:44,000
to pay for user friendliness.

472
00:25:44,000 --> 00:25:46,178
It's what you pay for security.

473
00:25:46,178 --> 00:25:48,666
And you hear people
say, for example,

474
00:25:48,666 --> 00:25:50,428
that I want greater
functionality,

475
00:25:50,428 --> 00:25:54,362
so people will program
in Java, even though it's

476
00:25:54,362 --> 00:25:57,086
much slower than
C, because they'll

477
00:25:57,086 --> 00:26:00,331
say it costs me maybe
a factor of three

478
00:26:00,331 --> 00:26:03,332
or something in performance
to program in Java.

479
00:26:03,332 --> 00:26:06,444
But Java is worth
it because it's

480
00:26:06,444 --> 00:26:10,333
got all these object oriented
features and so forth,

481
00:26:10,333 --> 00:26:12,000
exception mechanisms and so on.

482
00:26:12,000 --> 00:26:15,663
And so people are willing
to pay a factor of three

483
00:26:15,663 --> 00:26:16,333
in performance.

484
00:26:16,333 --> 00:26:18,333
So, that's why you
want performance

485
00:26:18,333 --> 00:26:22,000
because you can use it to
pay for these other things

486
00:26:22,000 --> 00:26:22,999
that you want.

487
00:26:22,999 --> 00:26:27,000
And that's why, in some sense,
it's on the bottom of the heap,

488
00:26:27,000 --> 00:26:32,000
because it's the universal
thing that you quantify.

489
00:26:32,000 --> 00:26:35,663
Do you want to spend a
factor of two on this

490
00:26:35,663 --> 00:26:39,000
or spend a factor of three
on security, et cetera?

491
00:26:39,000 --> 00:26:42,000
And, in addition, the
lessons generalize

492
00:26:42,000 --> 00:26:46,000
to other resource measures
like communication,

493
00:26:46,000 --> 00:26:47,815
like memory and so forth.

494
00:26:47,815 --> 00:26:51,142
And the last reason we
study algorithm performance

495
00:26:51,142 --> 00:26:54,000
is it's tons of fun.

496
00:26:54,000 --> 00:26:56,000
Speed is always fun, right?

497
00:26:56,000 --> 00:27:00,000
Why do people drive fast
cars, race horses, whatever?

498
00:27:00,000 --> 00:27:04,665
Rockets, et cetera,
why do we do that?

499
00:27:04,665 --> 00:27:05,700
Because speed is fun.

500
00:27:05,700 --> 00:27:06,200
Ski.

501
00:27:06,200 --> 00:27:07,000
Who likes to ski?

502
00:27:07,000 --> 00:27:08,200
I love to ski.

503
00:27:08,200 --> 00:27:10,333
I like going fast on those skis.

504
00:27:10,333 --> 00:27:11,000
It's fun.

505
00:27:11,000 --> 00:27:13,000
Hockey, fast sports, right?

506
00:27:13,000 --> 00:27:14,800
We all like the fast sports.

507
00:27:14,800 --> 00:27:16,570
Not all of us, I mean.

508
00:27:16,570 --> 00:27:18,855
Some people say he's
not talking to me.

509
00:27:18,855 --> 00:27:20,000
OK, let's move on.

510
00:27:20,000 --> 00:27:24,375
That's sort of a little bit of a
notion as to why we study this,

511
00:27:24,375 --> 00:27:27,272
is that it does,
in some sense, form

512
00:27:27,272 --> 00:27:30,714
a common basis for all these
other things we care about.

513
00:27:30,714 --> 00:27:37,775
And so we want to understand
how can we generate money

514
00:27:37,775 --> 00:27:40,000
for ourselves in computation?

515
00:27:40,000 --> 00:27:44,000
We're going to start out
with a very simple problem.

516
00:27:44,000 --> 00:27:47,178
It's one of the
oldest problems that

517
00:27:47,178 --> 00:27:52,000
has been studied in algorithms,
is the problem of sorting.

518
00:27:52,000 --> 00:27:57,000
We're going to actually study
this for several lectures

519
00:27:57,000 --> 00:28:03,000
because sorting contains
many algorithmic techniques.

520
00:28:03,000 --> 00:28:10,000
The sorting problem
is the following.

521
00:28:10,000 --> 00:28:21,142
We have a sequence a 1, a 2
up to a n of numbers as input.

522
00:28:21,142 --> 00:28:33,000
And our output is a
permutation of those numbers.

523
00:28:42,000 --> 00:28:47,000
A permutation is a
rearrangement of the numbers.

524
00:28:47,000 --> 00:28:53,000
Every number appears exactly
once in the rearrangement such

525
00:28:53,000 --> 00:28:57,708
that, I sometimes use a dollar
sign to mean "such that,"

526
00:28:57,708 --> 00:29:03,664
a 1 is less than or
equal to a 2 prime.

527
00:29:03,664 --> 00:29:11,000
Such that they are monotonically
increasing in size.

528
00:29:11,000 --> 00:29:17,000
Take a bunch of numbers,
put them in order.

529
00:29:17,000 --> 00:29:23,000
Here's an algorithm to do it.

530
00:29:23,000 --> 00:29:27,000
It's called insertion sort.

531
00:29:40,000 --> 00:29:44,500
And we will write this algorithm
in what we call pseudocode.

532
00:29:44,500 --> 00:29:47,500
It's sort of a
programming language,

533
00:29:47,500 --> 00:29:51,000
except it's got
English in there often.

534
00:29:51,000 --> 00:29:57,000
And it's just a shorthand for
writing for being precise.

535
00:29:57,000 --> 00:30:01,800
So this sorts A from 1 to n.

536
00:30:01,800 --> 00:30:06,000
And here is the code for it.

537
00:30:59,000 --> 00:31:01,000
This is what we call pseudocode.

538
00:31:01,000 --> 00:31:04,632
And if you don't understand
the pseudocode then

539
00:31:04,632 --> 00:31:09,000
you should ask questions
about any of the notations.

540
00:31:09,000 --> 00:31:13,000
You will start to get
used to it as we go on.

541
00:31:13,000 --> 00:31:16,500
One thing is that
in the pseudocode

542
00:31:16,500 --> 00:31:19,816
we use indentation,
where in most languages

543
00:31:19,816 --> 00:31:24,600
they have some kind of begin
end delimiters like curly braces

544
00:31:24,600 --> 00:31:28,332
or something in Java
or C, for example.

545
00:31:28,332 --> 00:31:31,000
We just use indentation.

546
00:31:31,000 --> 00:31:33,331
The whole idea of
the pseudocode is

547
00:31:33,331 --> 00:31:37,220
to try to get the algorithms
as short as possible

548
00:31:37,220 --> 00:31:41,000
while still understanding
what the individual steps are.

549
00:31:41,000 --> 00:31:44,000
In practice, there
actually have been

550
00:31:44,000 --> 00:31:47,665
languages that use indentation
as a means of showing

551
00:31:47,665 --> 00:31:49,000
the nesting of things.

552
00:31:49,000 --> 00:31:52,000
It's generally a bad idea,
because if things go over one

553
00:31:52,000 --> 00:31:56,775
page to another, for example,
you cannot tell what level

554
00:31:56,775 --> 00:31:59,000
of nesting it is.

555
00:31:59,000 --> 00:32:03,000
Whereas, with explicit braces
it's much easier to tell.

556
00:32:03,000 --> 00:32:09,000
So, there are reasons why this
is a bad notation if you were

557
00:32:09,000 --> 00:32:10,500
doing software engineering.

558
00:32:10,500 --> 00:32:15,362
But it's a good one
for us because it just

559
00:32:15,362 --> 00:32:20,142
keeps things short and makes
fewer things to write down.

560
00:32:20,142 --> 00:32:23,000
So, this is insertion sort.

561
00:32:23,000 --> 00:32:29,000
Let's try to figure out a
little bit what this does.

562
00:32:29,000 --> 00:32:34,380
It basically takes an
array A and at any point

563
00:32:34,380 --> 00:32:42,000
the thing to understand is,
we're setting basically,

564
00:32:42,000 --> 00:32:48,000
we're running the outer
loop from j is 2 to n,

565
00:32:48,000 --> 00:32:53,710
and the inner loop that
starts at j minus 1

566
00:32:53,710 --> 00:32:57,998
and then goes down
until it's zero.

567
00:32:57,998 --> 00:33:03,776
Basically, if we look at
any point in the algorithm,

568
00:33:03,776 --> 00:33:07,600
we essentially are looking
at some element here j.

569
00:33:07,600 --> 00:33:10,000
A of j, the jth element.

570
00:33:10,000 --> 00:33:14,708
And what we do essentially
is we pull a value out

571
00:33:14,708 --> 00:33:16,999
here that we call the key.

572
00:33:16,999 --> 00:33:20,665
And at this point the
important thing to understand,

573
00:33:20,665 --> 00:33:25,600
and we'll talk more about
this in recitation on Friday,

574
00:33:25,600 --> 00:33:30,800
is that there is an invariant
that is being maintained

575
00:33:30,800 --> 00:33:35,000
by this loop each time through.

576
00:33:35,000 --> 00:33:40,000
And the invariant is that this
part of the array is sorted.

577
00:33:40,000 --> 00:33:45,000
And the goal each time through
the loop is to increase,

578
00:33:45,000 --> 00:33:51,000
is to add one to the length
of the things that are sorted.

579
00:33:51,000 --> 00:33:54,996
And the way we do that
is we pull out the key

580
00:33:54,996 --> 00:33:58,725
and we just copy
values up like this.

581
00:33:58,725 --> 00:34:02,384
And keep copying
up until we find

582
00:34:02,384 --> 00:34:05,840
the place where this
key goes, and then we

583
00:34:05,840 --> 00:34:07,856
insert it in that place.

584
00:34:07,856 --> 00:34:11,000
And that's why it's
called insertion sort.

585
00:34:11,000 --> 00:34:16,270
We just sort of move the
things, copy the things up

586
00:34:16,270 --> 00:34:21,331
until we find where it goes,
and then we put it into place.

587
00:34:21,331 --> 00:34:25,535
And now we have it from A
from one to j is sorted,

588
00:34:25,535 --> 00:34:28,714
and now we can
work on j plus one.

589
00:34:28,714 --> 00:34:33,000
Let's give an example of that.

590
00:34:33,000 --> 00:34:38,000
Imagine we are doing
8, 2, 4, 9, 3, 6.

591
00:34:38,000 --> 00:34:41,500
We start out with j equals 2.

592
00:34:41,500 --> 00:34:47,000
And we figure out that we
want to insert it there.

593
00:34:47,000 --> 00:34:51,080
Now we have 2, 8, 4, 9, 3, 6.

594
00:34:51,080 --> 00:35:00,000
Then we look at the four and say
oh, well, that goes over here.

595
00:35:00,000 --> 00:35:03,500
We get 2, 4, 8, 9, 3, 6
after the second iteration

596
00:35:03,500 --> 00:35:05,500
of the outer loop.

597
00:35:05,500 --> 00:35:10,500
Then we look at 9 and
discover immediately it just

598
00:35:10,500 --> 00:35:12,000
goes right there.

599
00:35:12,000 --> 00:35:15,000
Very little work
to do on that step.

600
00:35:15,000 --> 00:35:20,000
So, we have exactly the same
output after that iteration.

601
00:35:20,000 --> 00:35:24,149
Then we look at the
3 and that's going

602
00:35:24,149 --> 00:35:26,666
to be inserted over there.

603
00:35:26,666 --> 00:35:36,500
2, 3, 4, 8, 9, and
that goes in there.

604
00:35:36,500 --> 00:35:44,000
2, 3, 4, 6, 8,

605
00:35:44,000 --> 00:35:47,000
Question?

606
00:35:58,000 --> 00:36:01,000
The array initially
starts at one, yes.

607
00:36:01,000 --> 00:36:02,000
A[1...n], OK?

608
00:36:02,000 --> 00:36:05,666
So, this is the
insertion sort algorithm.

609
00:36:05,666 --> 00:36:11,725
And it's the first algorithm
that we're going to analyze.

610
00:36:11,725 --> 00:36:15,832
And we're going to
pull out some tools

611
00:36:15,832 --> 00:36:18,744
that we have from
our math background

612
00:36:18,744 --> 00:36:21,200
to help to analyze it.

613
00:36:21,200 --> 00:36:29,000
First of all, let's take a look
at the issue of running time.

614
00:36:29,000 --> 00:36:33,900
The running time depends,
of this algorithm

615
00:36:33,900 --> 00:36:37,500
depends on a lot of things.

616
00:36:37,500 --> 00:36:42,500
One thing it depends
on is the input itself.

617
00:36:42,500 --> 00:36:55,500
For example, if the input
is already sorted -- --

618
00:36:55,500 --> 00:37:00,000
then insertion sort has
very little work to do.

619
00:37:00,000 --> 00:37:04,000
Because every time through it's
going to be like this case.

620
00:37:04,000 --> 00:37:07,267
It doesn't have to
shuffle too many guys over

621
00:37:07,267 --> 00:37:09,284
because they're
already in place.

622
00:37:09,284 --> 00:37:12,712
Whereas, in some sense,
what's the worst case

623
00:37:12,712 --> 00:37:14,000
for insertion sort?

624
00:37:14,000 --> 00:37:16,499
If it is reverse
sorted then it's

625
00:37:16,499 --> 00:37:19,726
going to have to
do a lot of work

626
00:37:19,726 --> 00:37:23,000
because it's going to have
to shuffle everything over

627
00:37:23,000 --> 00:37:26,227
on each step of the outer loop.

628
00:37:26,227 --> 00:37:30,500
In addition to the actual
input it depends, of course,

629
00:37:30,500 --> 00:37:32,000
on the input size.

630
00:37:38,000 --> 00:37:41,000
Here, for example,
we did six elements.

631
00:37:41,000 --> 00:37:47,500
It's going to take longer if
we, for example, do six times

632
00:37:47,500 --> 00:37:50,000
ten to the ninth elements.

633
00:37:50,000 --> 00:37:53,424
If we were sorting
a lot more stuff,

634
00:37:53,424 --> 00:37:57,142
it's going to take
us a lot longer.

635
00:37:57,142 --> 00:38:01,635
Typically, the way
we handle that is we

636
00:38:01,635 --> 00:38:06,500
are going to parameterize
things in the input size.

637
00:38:06,500 --> 00:38:11,284
We are going to talk
about time as a function

638
00:38:11,284 --> 00:38:14,708
of the size of
things that we are

639
00:38:14,708 --> 00:38:19,000
sorting so we can look at
what is the behavior of that.

640
00:38:19,000 --> 00:38:23,576
And the last thing I want
to say about running time

641
00:38:23,576 --> 00:38:30,000
is generally we want upper
bonds on the running time.

642
00:38:30,000 --> 00:38:34,428
We want to know that the time is
no more than a certain amount.

643
00:38:34,428 --> 00:38:38,500
And the reason is because
that represents a guarantee

644
00:38:38,500 --> 00:38:40,000
to the user.

645
00:38:40,000 --> 00:38:44,071
If I say it's not going to
run, for example, if I tell

646
00:38:44,071 --> 00:38:46,927
you here's a program
and it won't run

647
00:38:46,927 --> 00:38:50,000
more than three
seconds, that gives you

648
00:38:50,000 --> 00:38:55,220
real information about how
you could use it, for example,

649
00:38:55,220 --> 00:38:58,000
in a real time setting.

650
00:38:58,000 --> 00:39:00,331
Whereas, if I said
here's a program

651
00:39:00,331 --> 00:39:02,888
and it goes at
least three seconds,

652
00:39:02,888 --> 00:39:07,600
you don't know if it's
going to go for three years.

653
00:39:07,600 --> 00:39:10,375
It doesn't give you
that much guarantee

654
00:39:10,375 --> 00:39:13,000
if you are a user of it.

655
00:39:13,000 --> 00:39:16,552
Generally we want upper
bonds because it represents

656
00:39:16,552 --> 00:39:20,000
a guarantee to the user.

657
00:39:30,000 --> 00:39:33,000
There are different kinds
of analyses that people do.

658
00:39:44,000 --> 00:39:50,544
The one we're mostly
going to focus on

659
00:39:50,544 --> 00:39:55,400
is what's called
worst case analysis.

660
00:39:55,400 --> 00:40:04,996
And this is what we do
usually where we define T of n

661
00:40:04,996 --> 00:40:12,142
to be the maximum time
on any input of size n.

662
00:40:12,142 --> 00:40:16,535
So, it's the maximum input,
the maximum it could possibly

663
00:40:16,535 --> 00:40:19,000
cost us on an input of size n.

664
00:40:19,000 --> 00:40:21,720
What that does is, if
you look at the fact

665
00:40:21,720 --> 00:40:24,220
that sometimes the
inputs are better

666
00:40:24,220 --> 00:40:26,333
and sometimes
they're worse, we're

667
00:40:26,333 --> 00:40:28,664
looking at the
worst case of those

668
00:40:28,664 --> 00:40:30,888
because that's the
way we're going

669
00:40:30,888 --> 00:40:34,000
to be able to make a guarantee.

670
00:40:34,000 --> 00:40:36,664
It always does something
rather than just sometimes

671
00:40:36,664 --> 00:40:37,500
does something.

672
00:40:37,500 --> 00:40:40,285
So, we're looking
at the maximum.

673
00:40:40,285 --> 00:40:44,664
Notice that if I didn't have
maximum then T(n) in some sense

674
00:40:44,664 --> 00:40:47,800
is a relation, not a
function, because the time

675
00:40:47,800 --> 00:40:52,000
on an input of size n depends
on which input of size n.

676
00:40:52,000 --> 00:40:54,400
I could have many
different times,

677
00:40:54,400 --> 00:40:56,999
but by putting
the maximum at it,

678
00:40:56,999 --> 00:40:59,400
it turns that relation
into a function

679
00:40:59,400 --> 00:41:04,333
because there's only one
maximum time that it will take.

680
00:41:04,333 --> 00:41:13,856
Sometimes we will talk
about average case.

681
00:41:13,856 --> 00:41:21,000
Sometimes we will do this.

682
00:41:21,000 --> 00:41:36,332
Here T of n is then the expected
time over all inputs of size n.

683
00:41:36,332 --> 00:41:39,000
It's the expected time.

684
00:41:39,000 --> 00:41:43,280
Now, if I talk about
expected time, what else do

685
00:41:43,280 --> 00:41:45,400
I need to say here?

686
00:41:45,400 --> 00:41:48,000
What does that
mean, expected time?

687
00:41:48,000 --> 00:41:49,000
I'm sorry.

688
00:41:49,000 --> 00:41:50,800
Raise your hand.

689
00:41:50,800 --> 00:41:52,000
Expected inputs.

690
00:41:52,000 --> 00:41:56,000
What does that mean,
expected inputs?

691
00:42:05,000 --> 00:42:06,816
I need more math.

692
00:42:06,816 --> 00:42:10,888
What do I need by
expected time here, math?

693
00:42:10,888 --> 00:42:15,142
You have to take the
time of every input

694
00:42:15,142 --> 00:42:18,000
and then average them, OK.

695
00:42:18,000 --> 00:42:22,000
That's kind of what we
mean by expected time.

696
00:42:22,000 --> 00:42:22,666
Good.

697
00:42:22,666 --> 00:42:24,000
Not quite.

698
00:42:24,000 --> 00:42:28,000
I mean, what you say
is completely correct,

699
00:42:28,000 --> 00:42:32,165
except is not quite enough.

700
00:42:32,165 --> 00:42:33,000
Yeah?

701
00:42:33,000 --> 00:42:37,905
It's the time of every
input times the probability

702
00:42:37,905 --> 00:42:40,816
that it will be that input.

703
00:42:40,816 --> 00:42:45,665
It's a way of taking a weighted
average, exactly right.

704
00:42:45,665 --> 00:42:51,270
How do I know what the
probability of every input is?

705
00:42:51,270 --> 00:42:57,200
How do I know what the
probability a particular input

706
00:42:57,200 --> 00:43:02,000
occurs is in a given situation?

707
00:43:02,000 --> 00:43:03,000
I don't.

708
00:43:03,000 --> 00:43:06,000
I have to make an assumption.

709
00:43:06,000 --> 00:43:08,400
What's that assumption called?

710
00:43:08,400 --> 00:43:13,875
What kind of assumption
do I have to meet?

711
00:43:13,875 --> 00:43:26,855
I need an assumption -- -- of
the statistical distribution

712
00:43:26,855 --> 00:43:28,000
of inputs.

713
00:43:28,000 --> 00:43:31,750
Otherwise, expected time
doesn't mean anything

714
00:43:31,750 --> 00:43:38,000
because I don't know what the
probability of something is.

715
00:43:38,000 --> 00:43:42,500
In order to do probability,
you need some assumptions

716
00:43:42,500 --> 00:43:48,000
and you've got to state
those assumptions clearly.

717
00:43:48,000 --> 00:43:51,000
One of the most
common assumptions

718
00:43:51,000 --> 00:43:54,713
is that all inputs
are equally likely.

719
00:43:54,713 --> 00:43:57,571
That's called the
uniform distribution.

720
00:43:57,571 --> 00:44:04,000
Every input of size n is equally
likely, that kind of thing.

721
00:44:04,000 --> 00:44:09,000
But there are other ways that
you could make that assumption,

722
00:44:09,000 --> 00:44:11,912
and they may not all be true.

723
00:44:11,912 --> 00:44:15,600
This is much more
complicated, as you can see.

724
00:44:15,600 --> 00:44:20,000
Fortunately, all of you have a
strong probability background.

725
00:44:20,000 --> 00:44:23,600
And so we will not have
any trouble addressing

726
00:44:23,600 --> 00:44:27,000
these probabilistic
issues of dealing

727
00:44:27,000 --> 00:44:30,000
with expectations and such.

728
00:44:30,000 --> 00:44:34,000
If you don't, time
to go and say gee,

729
00:44:34,000 --> 00:44:38,200
maybe I should take
that Probability class

730
00:44:38,200 --> 00:44:42,180
that is a prerequisite
for this class.

731
00:44:42,180 --> 00:44:49,180
The last one I am going to
mention is best case analysis.

732
00:44:49,180 --> 00:44:53,000
And this I claim is bogus.

733
00:44:53,000 --> 00:44:53,666
Bogus.

734
00:44:53,666 --> 00:44:55,000
No good.

735
00:44:55,000 --> 00:44:59,834
Why is best-case analysis bogus?

736
00:44:59,834 --> 00:45:00,333
Yeah?

737
00:45:00,333 --> 00:45:03,200
The best case probably
doesn't ever happen.

738
00:45:03,200 --> 00:45:06,816
Actually, it's interesting
because for the sorting

739
00:45:06,816 --> 00:45:10,625
problem, the most common
things that get sorted

740
00:45:10,625 --> 00:45:15,000
are things that are already
sorted interestingly,

741
00:45:15,000 --> 00:45:17,140
or at least almost sorted.

742
00:45:17,140 --> 00:45:21,744
For example, one of the most
common things that are sorted

743
00:45:21,744 --> 00:45:23,570
is check numbers by banks.

744
00:45:23,570 --> 00:45:28,750
They tend to come in, in
the same order that they

745
00:45:28,750 --> 00:45:30,000
are written.

746
00:45:30,000 --> 00:45:36,000
They're sorting things that
are almost always sorted.

747
00:45:36,000 --> 00:45:38,856
I mean, it's good.

748
00:45:38,856 --> 00:45:42,665
When upper bond,
not lower bound?

749
00:45:42,665 --> 00:45:46,500
Yeah, you want to
make a guarantee.

750
00:45:46,500 --> 00:45:51,000
And so why is this
not a guarantee?

751
00:45:51,000 --> 00:46:01,125
You're onto something there, but
we need a little more precision

752
00:46:01,125 --> 00:46:02,000
here.

753
00:46:02,000 --> 00:46:03,501
How can I cheat?

754
00:46:03,501 --> 00:46:04,000
Yeah?

755
00:46:04,000 --> 00:46:06,000
Yeah, you can cheat.

756
00:46:06,000 --> 00:46:07,000
You cheat.

757
00:46:07,000 --> 00:46:11,664
You take any slow
algorithm that you want

758
00:46:11,664 --> 00:46:15,875
and just check for
some particular input,

759
00:46:15,875 --> 00:46:23,200
and if it's that input, then you
say immediately yeah, OK, here

760
00:46:23,200 --> 00:46:25,000
is the answer.

761
00:46:25,000 --> 00:46:30,000
And then it's got
a good best case.

762
00:46:30,000 --> 00:46:35,830
But I didn't tell you anything
about the vast majority

763
00:46:35,830 --> 00:46:38,500
of what is going on.

764
00:46:38,500 --> 00:46:42,571
So, you can cheat
with a slow algorithm

765
00:46:42,571 --> 00:46:46,000
that works fast on some input.

766
00:46:46,000 --> 00:46:50,081
It doesn't really
do much for you

767
00:46:50,081 --> 00:46:54,500
so we normally don't
worry about that.

768
00:46:54,500 --> 00:46:56,000
Let's see.

769
00:46:56,000 --> 00:47:02,000
What is insertion
sorts worst case time?

770
00:47:02,000 --> 00:47:07,000
Now we get into some
sort of funny issues.

771
00:47:07,000 --> 00:47:12,833
First of all, it sort of
depends on the computer

772
00:47:12,833 --> 00:47:15,332
you're running on.

773
00:47:15,332 --> 00:47:17,625
Whose computer, right?

774
00:47:17,625 --> 00:47:24,499
Is it a big supercomputer
or is it your wristwatch?

775
00:47:24,499 --> 00:47:29,000
They have different
computational abilities.

776
00:47:29,000 --> 00:47:34,428
And when we compare
algorithms, we

777
00:47:34,428 --> 00:47:37,000
compare them typically
for relative speed.

778
00:47:37,000 --> 00:47:42,000
This is if you compared two
algorithms on the same machine.

779
00:47:42,000 --> 00:47:44,625
You could argue, well,
it doesn't really

780
00:47:44,625 --> 00:47:48,328
matter what the machine
is because I will just

781
00:47:48,328 --> 00:47:50,500
look at their relative speed.

782
00:47:50,500 --> 00:47:55,000
But, of course, I may also be
interested in absolute speed.

783
00:47:55,000 --> 00:47:58,108
Is one algorithm
actually better no matter

784
00:47:58,108 --> 00:48:02,000
what machine it's run on?

785
00:48:08,000 --> 00:48:11,213
And so this kind of
gets sort of confusing

786
00:48:11,213 --> 00:48:14,920
as to how I can talk
about the worst case

787
00:48:14,920 --> 00:48:18,500
time of an algorithm
of a piece of software

788
00:48:18,500 --> 00:48:23,000
when I am not talking
about the hardware because,

789
00:48:23,000 --> 00:48:27,000
clearly, if I had run
on a faster machine,

790
00:48:27,000 --> 00:48:30,000
my algorithms are
going to go faster.

791
00:48:30,000 --> 00:48:36,000
So, this is where you get
the big idea of algorithms.

792
00:48:36,000 --> 00:48:39,000
Which is why algorithm
is such a huge field,

793
00:48:39,000 --> 00:48:43,000
why it spawns companies
like Google, like Akamai,

794
00:48:43,000 --> 00:48:44,200
like Amazon.

795
00:48:44,200 --> 00:48:47,200
Why algorithmic analysis,
throughout the history

796
00:48:47,200 --> 00:48:50,416
of computing, has been
such a huge success,

797
00:48:50,416 --> 00:48:53,744
is our ability to
master and to be

798
00:48:53,744 --> 00:48:57,775
able to take what is
apparently a really

799
00:48:57,775 --> 00:49:02,220
messy, complicated situation
and reduce it to being

800
00:49:02,220 --> 00:49:05,000
able to do some mathematics.

801
00:49:05,000 --> 00:49:09,000
And that idea is called
asymptotic analysis.

802
00:49:17,000 --> 00:49:21,000
And the basic idea of
asymptotic analysis is to ignore

803
00:49:21,000 --> 00:49:34,500
machine-dependent
constants -- --

804
00:49:34,500 --> 00:49:38,000
and, instead of the
actual running time,

805
00:49:38,000 --> 00:49:43,000
look at the growth
of the running time.

806
00:49:59,000 --> 00:50:02,000
So, we don't look at
the actual running time.

807
00:50:02,000 --> 00:50:04,080
We look at the growth.

808
00:50:04,080 --> 00:50:07,000
Let's see what we mean by that.

809
00:50:07,000 --> 00:50:08,500
This is a huge idea.

810
00:50:08,500 --> 00:50:11,000
It's not a hard
idea, otherwise I

811
00:50:11,000 --> 00:50:16,000
wouldn't be able to teach
it in the first lecture,

812
00:50:16,000 --> 00:50:17,665
but it's a huge idea.

813
00:50:17,665 --> 00:50:21,110
We are going to spend
a couple of lectures

814
00:50:21,110 --> 00:50:23,885
understanding the
implications of that

815
00:50:23,885 --> 00:50:30,000
and will basically be doing
it throughout the term.

816
00:50:30,000 --> 00:50:33,000
And if you go on to be
practicing engineers,

817
00:50:33,000 --> 00:50:36,000
you will be doing
it all the time.

818
00:50:36,000 --> 00:50:39,600
In order to do that,
we adopt some notations

819
00:50:39,600 --> 00:50:42,140
that are going to help us.

820
00:50:42,140 --> 00:50:46,000
In particular, we will
adopt asymptotic notation.

821
00:50:46,000 --> 00:50:51,000
Most of you have seen some
kind of asymptotic notation.

822
00:50:51,000 --> 00:50:53,997
Maybe a few of you
haven't, but mostly you

823
00:50:53,997 --> 00:50:56,200
should have seen a little bit.

824
00:50:56,200 --> 00:51:01,571
The one we're going to
be using in this class

825
00:51:01,571 --> 00:51:05,000
predominantly is theta notation.

826
00:51:05,000 --> 00:51:09,900
And theta notation is
pretty easy notation

827
00:51:09,900 --> 00:51:16,000
to master because all you
do is, from a formula,

828
00:51:16,000 --> 00:51:24,000
just drop low order terms
and ignore leading constants.

829
00:51:30,000 --> 00:51:37,500
For example, if I have a formula
like 3n^3 = 90n^2 - 5n + 6046,

830
00:51:37,500 --> 00:51:43,494
I say, well, what low
order terms do I drop?

831
00:51:43,494 --> 00:51:48,500
Well, n^3 is a
bigger term n^2 than.

832
00:51:48,500 --> 00:51:56,000
I am going to drop all these
terms and ignore the leading

833
00:51:56,000 --> 00:52:01,000
constant, so I say
that's Theta(n^3).

834
00:52:01,000 --> 00:52:04,000
That's pretty easy.

835
00:52:04,000 --> 00:52:06,000
So, that's theta notation.

836
00:52:06,000 --> 00:52:11,000
Now, this is an engineering way
of manipulating theta notation.

837
00:52:11,000 --> 00:52:14,000
There is actually a
mathematical definition

838
00:52:14,000 --> 00:52:18,000
for this, which we
are going to talk

839
00:52:18,000 --> 00:52:22,000
about next time, which
is a definition in terms

840
00:52:22,000 --> 00:52:23,200
of sets of functions.

841
00:52:23,200 --> 00:52:25,500
And, you are going
to be responsible,

842
00:52:25,500 --> 00:52:30,571
this is both a math and a
computer science engineering

843
00:52:30,571 --> 00:52:31,142
class.

844
00:52:31,142 --> 00:52:34,416
Throughout the
course you are going

845
00:52:34,416 --> 00:52:37,328
to be responsible both
for mathematical rigor

846
00:52:37,328 --> 00:52:41,220
as if it were a math
course and engineering

847
00:52:41,220 --> 00:52:43,666
commonsense because it's
an engineering course.

848
00:52:43,666 --> 00:52:46,000
We are going to be doing both.

849
00:52:46,000 --> 00:52:50,000
This is the engineering way
of understanding what you do,

850
00:52:50,000 --> 00:52:54,000
so you're responsible for being
able to do these manipulations.

851
00:52:54,000 --> 00:52:57,000
You're also going to be
responsible for understanding

852
00:52:57,000 --> 00:52:59,400
the mathematical
definition of theta notion

853
00:52:59,400 --> 00:53:03,500
and of its related O
notation and omega notation.

854
00:53:03,500 --> 00:53:09,332
If I take a look as n
approached infinity,

855
00:53:09,332 --> 00:53:16,400
a Theta(n^2) algorithm
always beats, eventually,

856
00:53:16,400 --> 00:53:20,000
a Theta(n^3) algorithm.

857
00:53:20,000 --> 00:53:27,600
As n gets bigger, it doesn't
matter what these other terms

858
00:53:27,600 --> 00:53:34,636
were if I were describing
the absolute precise behavior

859
00:53:34,636 --> 00:53:37,816
in terms of a formula.

860
00:53:37,816 --> 00:53:44,000
If I had a Theta(n^2) algorithm,
it would always be faster

861
00:53:44,000 --> 00:53:47,776
for sufficiently large n
than a Theta(n^3) algorithm.

862
00:53:47,776 --> 00:53:51,776
It wouldn't matter what
those low order terms were.

863
00:53:51,776 --> 00:53:55,665
It wouldn't matter what
the leading constant was.

864
00:53:55,665 --> 00:53:59,000
This one will always be faster.

865
00:53:59,000 --> 00:54:04,500
Even if you ran the Theta(n^2)
algorithm on a slow computer

866
00:54:04,500 --> 00:54:09,000
and the Theta(n^3) algorithm
on a fast computer.

867
00:54:09,000 --> 00:54:12,000
The great thing about
asymptotic notation

868
00:54:12,000 --> 00:54:16,000
is it satisfies
our issue of being

869
00:54:16,000 --> 00:54:20,500
able to compare both
relative and absolute speed,

870
00:54:20,500 --> 00:54:25,666
because we are able
to do this no matter

871
00:54:25,666 --> 00:54:29,000
what the computer platform.

872
00:54:29,000 --> 00:54:34,000
On different platforms we may
get different constants here,

873
00:54:34,000 --> 00:54:39,000
machine dependent constants
for the actual running time,

874
00:54:39,000 --> 00:54:44,564
but if I look at the growth
as the size of the input

875
00:54:44,564 --> 00:54:49,000
gets larger, the asymptotics
generally won't change.

876
00:54:49,000 --> 00:54:53,444
For example, I will just
draw that as a picture.

877
00:54:53,444 --> 00:54:59,284
If I have n on this axis
and T(n) on this axis.

878
00:54:59,284 --> 00:55:04,000
This may be, for example, a
Theta(n^3) algorithm and this

879
00:55:04,000 --> 00:55:06,270
may be a Theta(n^2) algorithm.

880
00:55:06,270 --> 00:55:12,500
There is always going to be some
point n o where for everything

881
00:55:12,500 --> 00:55:17,500
larger the Theta(n^2) algorithm
is going to be cheaper than

882
00:55:17,500 --> 00:55:21,496
the Theta(n^3) algorithm not
matter how much advantage you

883
00:55:21,496 --> 00:55:26,180
give it at the beginning
in terms of the speed

884
00:55:26,180 --> 00:55:30,000
of the computer
you are running on.

885
00:55:30,000 --> 00:55:32,541
Now, from an engineering
point of view,

886
00:55:32,541 --> 00:55:36,664
there are some issues we have to
deal with because sometimes it

887
00:55:36,664 --> 00:55:40,997
could be that that n o is so
large that the computers aren't

888
00:55:40,997 --> 00:55:43,500
big enough to run the problem.

889
00:55:43,500 --> 00:55:47,140
That's why we, nevertheless,
are interested in some

890
00:55:47,140 --> 00:55:51,000
of the slower algorithms,
because some of the slower

891
00:55:51,000 --> 00:55:55,000
algorithms, even though they may
not asymptotically be slower,

892
00:55:55,000 --> 00:56:00,000
I mean asymptotically
they will be slower.

893
00:56:00,000 --> 00:56:03,000
They may still be faster on
reasonable sizes of things.

894
00:56:03,000 --> 00:56:07,000
And so we have to both balance
our mathematical understanding

895
00:56:07,000 --> 00:56:10,000
with our engineering
commonsense in order

896
00:56:10,000 --> 00:56:12,000
to do good programming.

897
00:56:12,000 --> 00:56:14,625
So, just having done
analysis of algorithms

898
00:56:14,625 --> 00:56:18,000
doesn't automatically make
you a good programmer.

899
00:56:18,000 --> 00:56:21,070
You also need to learn
how to program and use

900
00:56:21,070 --> 00:56:23,089
these tools in
practice to understand

901
00:56:23,089 --> 00:56:27,332
when they are relevant and
when they are not relevant.

902
00:56:27,332 --> 00:56:30,000
There is a saying.

903
00:56:30,000 --> 00:56:32,664
If you want to be
a good program,

904
00:56:32,664 --> 00:56:35,776
you just program ever
day for two years,

905
00:56:35,776 --> 00:56:38,444
you will be an
excellent programmer.

906
00:56:38,444 --> 00:56:42,444
If you want to be a
world class programmer,

907
00:56:42,444 --> 00:56:46,000
you can program every
day for ten years,

908
00:56:46,000 --> 00:56:49,744
or you can program
every day for two years

909
00:56:49,744 --> 00:56:51,888
and take an algorithms class.

910
00:56:51,888 --> 00:56:55,833
Let's get back to
what we were doing,

911
00:56:55,833 --> 00:57:00,000
which is analyzing
insertion sort.

912
00:57:00,000 --> 00:57:02,000
We are going to look
at the worse case.

913
00:57:16,000 --> 00:57:21,000
Which, as we mentioned before,
is when the input is reverse

914
00:57:21,000 --> 00:57:21,500
sorted.

915
00:57:21,500 --> 00:57:25,816
The biggest element comes
first and the smallest last

916
00:57:25,816 --> 00:57:30,714
because now every time you
do the insertion you've

917
00:57:30,714 --> 00:57:35,000
got to shuffle everything over.

918
00:57:35,000 --> 00:57:36,750
You can write down
the running time

919
00:57:36,750 --> 00:57:38,444
by looking at the
nesting of loops.

920
00:57:38,444 --> 00:57:40,000
What we do is we sum up.

921
00:57:40,000 --> 00:57:42,331
What we assume is
that every operation,

922
00:57:42,331 --> 00:57:45,000
every elemental operation
is going to take

923
00:57:45,000 --> 00:57:47,000
some constant amount of time.

924
00:57:47,000 --> 00:57:49,176
But we don't have
to worry about what

925
00:57:49,176 --> 00:57:51,875
that constant is because
we're going to be

926
00:57:51,875 --> 00:57:53,000
doing asymptotic analysis.

927
00:57:53,000 --> 00:57:54,840
As I say, the
beautify of the method

928
00:57:54,840 --> 00:57:57,500
is that it causes
all these things that

929
00:57:57,500 --> 00:58:01,000
are real distinctions
to sort of vanish.

930
00:58:01,000 --> 00:58:03,997
We sort of look at
them from 30,000 feet

931
00:58:03,997 --> 00:58:06,776
rather than from three
millimeters or something.

932
00:58:06,776 --> 00:58:10,500
Each of these operations
is going to sort of

933
00:58:10,500 --> 00:58:12,000
be a basic operation.

934
00:58:12,000 --> 00:58:16,600
One way to think about this, in
terms of counting operations,

935
00:58:16,600 --> 00:58:19,000
is counting memory references.

936
00:58:19,000 --> 00:58:23,000
How many times do you
actually access some variable?

937
00:58:23,000 --> 00:58:29,000
That's another way of sort
of thinking about this model.

938
00:58:29,000 --> 00:58:33,800
When we do that, well, we're
going to go through this loop,

939
00:58:33,800 --> 00:58:37,800
j is going from 2
to n, and then we're

940
00:58:37,800 --> 00:58:43,000
going to add up the work
that we do within the loop.

941
00:58:43,000 --> 00:58:48,136
We can sort of write that
in math as summation of j

942
00:58:48,136 --> 00:58:49,888
equals 2 to n.

943
00:58:49,888 --> 00:58:55,304
And then what is the work
that is going on in this loop?

944
00:58:55,304 --> 00:58:59,362
Well, the work that is
going on in this loop

945
00:58:59,362 --> 00:59:05,100
varies, but in the worst case
how many operations are going

946
00:59:05,100 --> 00:59:10,000
on here for each value of j?

947
00:59:10,000 --> 00:59:17,600
For a given value of j, how
much work goes on in this loop?

948
00:59:17,600 --> 00:59:20,000
Can somebody tell me?

949
00:59:20,000 --> 00:59:21,000
Asymptotically.

950
00:59:21,000 --> 00:59:28,544
It's j times some
constant, so it's theta j.

951
00:59:28,544 --> 00:59:33,768
So, there is theta
j work going on here

952
00:59:33,768 --> 00:59:38,000
because this loop starts
out with i being j minus 1,

953
00:59:38,000 --> 00:59:43,450
and then it's doing just
a constant amount of stuff

954
00:59:43,450 --> 00:59:47,927
for each step of the value
of i, and i is running

955
00:59:47,927 --> 00:59:50,664
from j minus one down to zero.

956
00:59:50,664 --> 00:59:57,330
So, we can say that is theta
j work that is going on.

957
00:59:57,330 --> 00:59:59,831
Do people follow that?

958
00:59:59,831 --> 01:00:00,330
OK.

959
01:00:00,330 --> 01:00:03,665
And now we have a
formula we can evaluate.

960
01:00:03,665 --> 01:00:05,713
What is the evaluation?

961
01:00:05,713 --> 01:00:11,000
If I want to simplify this
formula, what is that equal to?

962
01:00:19,901 --> 01:00:20,400
Sorry.

963
01:00:20,400 --> 01:00:22,000
In the back there.

964
01:00:28,000 --> 01:00:29,141
Yeah.

965
01:00:29,141 --> 01:00:30,282
OK.

966
01:00:30,282 --> 01:00:34,125
That's just Theta(n^2), good.

967
01:00:34,125 --> 01:00:38,312
Because when you're saying is
the sum of consecutive numbers,

968
01:00:38,312 --> 01:00:39,564
you mean what?

969
01:00:39,564 --> 01:00:41,664
What's the mathematic
term we have

970
01:00:41,664 --> 01:00:43,876
for that so we can communicate?

971
01:00:43,876 --> 01:00:48,158
You've got to know these
things so you can communicate.

972
01:00:48,158 --> 01:00:50,569
It's called what
type of sequence?

973
01:00:50,569 --> 01:00:53,479
It's actually a
series, but that's OK.

974
01:00:53,479 --> 01:00:56,127
What type of series
is this called?

975
01:00:56,127 --> 01:00:57,733
Arithmetic series, good.

976
01:00:57,733 --> 01:01:02,412
Wow, we've got some sharp
people who can communicate.

977
01:01:02,412 --> 01:01:05,943
This is an arithmetic series.

978
01:01:05,943 --> 01:01:11,937
You're basically summing 1 + 2 +
3 + 4, some constants in there,

979
01:01:11,937 --> 01:01:17,210
but basically it's 1 + 2
+ 3 + 4 + 5 + 6 up to n.

980
01:01:17,210 --> 01:01:18,376
That's Theta(n^2).

981
01:01:18,376 --> 01:01:25,043
If you don't know this math,
there is a chapter in the book,

982
01:01:25,043 --> 01:01:28,974
or you could have
taken the prerequisite.

983
01:01:28,974 --> 01:01:31,390
Arithmetic series.

984
01:01:31,390 --> 01:01:33,951
People have this
vague recollection.

985
01:01:33,951 --> 01:01:34,476
Oh, yeah.

986
01:01:34,476 --> 01:01:34,975
Good.

987
01:01:34,975 --> 01:01:38,048
Now, you have to learn
these manipulations.

988
01:01:38,048 --> 01:01:40,592
We will talk about
a bit next time,

989
01:01:40,592 --> 01:01:43,452
but you have to learn
your theta manipulations

990
01:01:43,452 --> 01:01:45,804
for what works with theta.

991
01:01:45,804 --> 01:01:48,765
And you have to be very
careful because theta

992
01:01:48,765 --> 01:01:50,231
is a weak notation.

993
01:01:50,231 --> 01:01:53,881
A strong notation is something
like Leibniz notation

994
01:01:53,881 --> 01:01:57,354
from calculus where
the chain rule is just

995
01:01:57,354 --> 01:01:58,857
canceling two things.

996
01:01:58,857 --> 01:02:03,544
It's just fabulous that you
can cancel in the chain rule.

997
01:02:03,544 --> 01:02:06,820
And Leibniz notation just
expresses that so directly you

998
01:02:06,820 --> 01:02:07,599
can manipulate.

999
01:02:07,599 --> 01:02:09,468
Theta notation is not like that.

1000
01:02:09,468 --> 01:02:12,303
If you think it is like
that you are in trouble.

1001
01:02:12,303 --> 01:02:15,033
You really have to think
of what is going on

1002
01:02:15,033 --> 01:02:16,162
under the theta notation.

1003
01:02:16,162 --> 01:02:18,644
And it is more of a
descriptive notation

1004
01:02:18,644 --> 01:02:20,867
than it is a
manipulative notation.

1005
01:02:20,867 --> 01:02:23,363
There are manipulations
you can do with it,

1006
01:02:23,363 --> 01:02:25,661
but unless you
understand what is really

1007
01:02:25,661 --> 01:02:29,325
going on under the theta
notation you will find yourself

1008
01:02:29,325 --> 01:02:30,179
in trouble.

1009
01:02:30,179 --> 01:02:34,575
And next time we will
talk a little bit more

1010
01:02:34,575 --> 01:02:35,977
about theta notation.

1011
01:02:35,977 --> 01:02:38,000
Is insertion sort fast?

1012
01:02:49,000 --> 01:02:53,000
Well, it turns out for small
n it is moderately fast.

1013
01:03:02,000 --> 01:03:11,000
But it is not at
all for large n.

1014
01:03:18,000 --> 01:03:21,626
So, I am going to give you
an algorithm that is faster.

1015
01:03:21,626 --> 01:03:22,942
It's called merge sort.

1016
01:03:22,942 --> 01:03:26,165
I wonder if I should
leave insertion sort up.

1017
01:03:26,165 --> 01:03:27,000
Why not.

1018
01:03:46,000 --> 01:03:49,768
I am going to write
on this later,

1019
01:03:49,768 --> 01:03:56,099
so if you are taking notes,
leave some space on the left.

1020
01:03:56,099 --> 01:04:02,000
Here is merge sort of an
array A from 1 up to n.

1021
01:04:02,000 --> 01:04:05,963
And it is basically three steps.

1022
01:04:05,963 --> 01:04:10,079
If n equals 1 we are done.

1023
01:04:10,079 --> 01:04:14,484
Sorting one element,
it is already sorted.

1024
01:04:14,484 --> 01:04:15,808
All right.

1025
01:04:15,808 --> 01:04:17,310
Recursive algorithm.

1026
01:04:17,310 --> 01:04:24,962
Otherwise, what we do is we
recursively sort A from 1

1027
01:04:24,962 --> 01:04:30,000
up to the ceiling of n over 2.

1028
01:04:30,000 --> 01:04:39,102
And the array A of the ceiling
of n over 2 plus one up to n.

1029
01:04:39,102 --> 01:04:44,502
So, we sort two
halves of the input.

1030
01:04:44,502 --> 01:04:53,502
And then, three, we take those
two lists that we have done

1031
01:04:53,502 --> 01:04:57,000
and we merge them.

1032
01:05:03,000 --> 01:05:05,601
And, to do that, we
use a merge subroutine

1033
01:05:05,601 --> 01:05:07,000
which I will show you.

1034
01:05:14,000 --> 01:05:20,492
The key subroutine here is
merge, and it works like this.

1035
01:05:20,492 --> 01:05:22,388
I have two lists.

1036
01:05:22,388 --> 01:05:25,710
Let's say one of them is 20.

1037
01:05:25,710 --> 01:05:30,000
I am doing this
in reverse order.

1038
01:05:30,000 --> 01:05:32,682
I have sorted this like this.

1039
01:05:32,682 --> 01:05:35,368
And then I sort another one.

1040
01:05:35,368 --> 01:05:39,795
I don't know why I do it
this order, but anyway.

1041
01:05:39,795 --> 01:05:41,710
Here is my other list.

1042
01:05:41,710 --> 01:05:45,445
I have my two lists
that I have sorted.

1043
01:05:45,445 --> 01:05:50,608
So, this is AA[1] to AA[|n/2|] and
AA[|n/2|+1] to AA[n] for the way

1044
01:05:50,608 --> 01:05:54,110
it will be called
in this program.

1045
01:05:54,110 --> 01:05:59,286
And now to merge these
two, what I want to do

1046
01:05:59,286 --> 01:06:04,000
is produce a sorted list
out of both of them.

1047
01:06:04,000 --> 01:06:07,900
What I do is first observe
where is the smallest

1048
01:06:07,900 --> 01:06:11,679
element of any two lists
that are already sorted?

1049
01:06:11,679 --> 01:06:15,783
It's in one of two places,
the head of the first list

1050
01:06:15,783 --> 01:06:18,387
or the head of the second list.

1051
01:06:18,387 --> 01:06:23,028
I look at those two elements
and say which one is smaller?

1052
01:06:23,028 --> 01:06:24,613
This one is smaller.

1053
01:06:24,613 --> 01:06:28,273
Then what I do is output
into my output array

1054
01:06:28,273 --> 01:06:30,263
the smaller of the two.

1055
01:06:30,263 --> 01:06:32,464
And I cross it off.

1056
01:06:32,464 --> 01:06:35,702
And now where is the
next smallest element?

1057
01:06:35,702 --> 01:06:40,320
And the answer is it's going to
be the head of one of these two

1058
01:06:40,320 --> 01:06:40,819
lists.

1059
01:06:40,819 --> 01:06:44,237
Then I cross out this
guy and put him here

1060
01:06:44,237 --> 01:06:45,648
and circle this one.

1061
01:06:45,648 --> 01:06:47,958
Now I look at these two guys.

1062
01:06:47,958 --> 01:06:52,004
This one is smaller so I output
that and circle that one.

1063
01:06:52,004 --> 01:06:55,131
Now I look at these
two guys, output 9.

1064
01:06:55,131 --> 01:06:58,893
So, every step here is some
fixed number of operations

1065
01:06:58,893 --> 01:07:04,729
that is independent of the size
of the arrays at each step.

1066
01:07:04,729 --> 01:07:10,474
Each individual step is just
me looking at two elements

1067
01:07:10,474 --> 01:07:15,483
and picking out the smallest
and advancing some pointers

1068
01:07:15,483 --> 01:07:19,747
into the array so
that I know where

1069
01:07:19,747 --> 01:07:23,522
the current head
of that list is.

1070
01:07:23,522 --> 01:07:30,000
And so, therefore, the time is
order n on n total elements.

1071
01:07:30,000 --> 01:07:34,235
The time to actually go through
this and merge two lists

1072
01:07:34,235 --> 01:07:35,470
is order n.

1073
01:07:35,470 --> 01:07:40,031
We sometimes call this linear
time because it's not quadratic

1074
01:07:40,031 --> 01:07:41,012
or whatever.

1075
01:07:41,012 --> 01:07:45,401
It is proportional to n,
proportional to the input size.

1076
01:07:45,401 --> 01:07:46,502
It's linear time.

1077
01:07:46,502 --> 01:07:50,098
I go through and just do
this simple operation,

1078
01:07:50,098 --> 01:07:54,291
just working up these
lists, and in the end

1079
01:07:54,291 --> 01:07:56,733
I have done essentially
n operations,

1080
01:07:56,733 --> 01:08:02,000
order n operations each of
which cost constant time.

1081
01:08:02,000 --> 01:08:05,451
That's a total of order n time.

1082
01:08:05,451 --> 01:08:06,871
Everybody with me?

1083
01:08:06,871 --> 01:08:07,371
OK.

1084
01:08:07,371 --> 01:08:10,099
So, this is a recursive program.

1085
01:08:10,099 --> 01:08:14,503
We can actually now
write what is called

1086
01:08:14,503 --> 01:08:17,309
a recurrence for this program.

1087
01:08:17,309 --> 01:08:21,885
The way we do that is
say let's let the time

1088
01:08:21,885 --> 01:08:24,759
to sort n elements to be T(n).

1089
01:08:24,759 --> 01:08:30,000
Then how long does it
take to do step one?

1090
01:08:35,000 --> 01:08:36,300
That's just constant time.

1091
01:08:36,300 --> 01:08:41,370
We just check to see if n is
1, and if it is we return.

1092
01:08:41,370 --> 01:08:45,380
That's independent of the size
of anything that we are doing.

1093
01:08:45,380 --> 01:08:48,856
It just takes a certain
number of machine instructions

1094
01:08:48,856 --> 01:08:52,875
on whatever machine and we
say it is constant time.

1095
01:08:52,875 --> 01:08:54,732
We call that theta one.

1096
01:08:54,732 --> 01:09:00,000
This is actually a little bit
of an abuse if you get into it.

1097
01:09:00,000 --> 01:09:04,092
And the reason is because
typically in order to say it

1098
01:09:04,092 --> 01:09:07,206
you need to say what
it is growing with.

1099
01:09:07,206 --> 01:09:10,945
Nevertheless, we use this
as an abuse of the notation

1100
01:09:10,945 --> 01:09:13,550
just to mean it is a constant.

1101
01:09:13,550 --> 01:09:16,604
So, that's an abuse
just so people know.

1102
01:09:16,604 --> 01:09:20,835
But it simplifies things if
I can just write theta one.

1103
01:09:20,835 --> 01:09:23,733
And it basically
means the same thing.

1104
01:09:23,733 --> 01:09:26,866
Now we recursively
sort these two things.

1105
01:09:26,866 --> 01:09:29,701
How can I describe that?

1106
01:09:29,701 --> 01:09:34,674
The time to do this, I
can describe recursively

1107
01:09:34,674 --> 01:09:40,004
as T of ceiling
of n over 2 plus T

1108
01:09:40,004 --> 01:09:44,582
of n minus ceiling of n over 2.

1109
01:09:44,582 --> 01:09:52,626
That is actually kind of messy,
so what we will do is just be

1110
01:09:52,626 --> 01:09:54,915
sloppy and write 2T(n/2).

1111
01:09:54,915 --> 01:10:00,000
So, this is just
us being sloppy.

1112
01:10:00,000 --> 01:10:02,352
And we will see on
Friday in recitation

1113
01:10:02,352 --> 01:10:04,534
that it is OK to be sloppy.

1114
01:10:04,534 --> 01:10:06,979
That's the great thing
about algorithms.

1115
01:10:06,979 --> 01:10:09,913
As long as you are
rigorous and precise,

1116
01:10:09,913 --> 01:10:12,502
you can be as
sloppy as you want.

1117
01:10:12,502 --> 01:10:15,846
[LAUGHTER] This is sloppy
because I didn't worry

1118
01:10:15,846 --> 01:10:19,051
about what was going
on, because it turns out

1119
01:10:19,051 --> 01:10:20,882
it doesn't make any difference.

1120
01:10:20,882 --> 01:10:25,798
And we are going to actually
see that that is the case.

1121
01:10:25,798 --> 01:10:28,660
And, finally, I have to merge
the two sorted lists which

1122
01:10:28,660 --> 01:10:30,058
have a total of n elements.

1123
01:10:30,058 --> 01:10:32,576
And we just analyze that
using the merge subroutine.

1124
01:10:32,576 --> 01:10:35,000
And that takes us
to theta n time.

1125
01:10:40,000 --> 01:10:43,933
That allows us now to write a
recurrence for the performance

1126
01:10:43,933 --> 01:10:45,000
of merge sort.

1127
01:10:57,000 --> 01:11:03,032
Which is to say that T
of n is equal to theta 1

1128
01:11:03,032 --> 01:11:09,488
if n equals 1 and 2T of
n over 2 plus theta of n

1129
01:11:09,488 --> 01:11:12,250
if n is bigger than 1.

1130
01:11:12,250 --> 01:11:16,324
Because either I
am doing step one

1131
01:11:16,324 --> 01:11:21,977
or I am doing all steps
one, two and three.

1132
01:11:21,977 --> 01:11:28,742
Here I am doing step one
and I return and I am done.

1133
01:11:28,742 --> 01:11:33,397
Or else I am doing step
one, I don't return,

1134
01:11:33,397 --> 01:11:36,730
and then I also do
steps two and three.

1135
01:11:36,730 --> 01:11:38,808
So, I add those together.

1136
01:11:38,808 --> 01:11:43,456
I could say theta n plus theta
1, but theta n plus theta 1

1137
01:11:43,456 --> 01:11:47,475
is just theta n because
theta 1 is a lower order

1138
01:11:47,475 --> 01:11:50,831
term than theta n and
I can throw it away.

1139
01:11:50,831 --> 01:11:56,111
It is either theta 1 or it is
2T of n over 2 plus theta n.

1140
01:11:56,111 --> 01:11:59,051
Now, typically we
won't be writing this.

1141
01:11:59,051 --> 01:12:01,478
Usually we omit this.

1142
01:12:01,478 --> 01:12:05,631
If it makes no difference to
the solution of the recurrence,

1143
01:12:05,631 --> 01:12:08,446
we will usually omit
constant base cases.

1144
01:12:08,446 --> 01:12:11,691
In algorithms, it's not true
generally in mathematics,

1145
01:12:11,691 --> 01:12:15,123
but in algorithms if you
are running something

1146
01:12:15,123 --> 01:12:19,145
on a constant size input it
takes constant time always.

1147
01:12:19,145 --> 01:12:22,172
So, we don't worry about
what this value is.

1148
01:12:22,172 --> 01:12:25,070
And it turns out it
has no real impact

1149
01:12:25,070 --> 01:12:28,171
on the asymptotic solution
of the recurrence.

1150
01:12:28,171 --> 01:12:32,037
How do we solve a
recurrence like this?

1151
01:12:32,037 --> 01:12:36,888
I now have T of n expressed
in terms of T of n over 2.

1152
01:12:36,888 --> 01:12:40,419
That's in the book and
it is also in Lecture 2.

1153
01:12:40,419 --> 01:12:43,901
We are going to do
Lecture 2 to solve that,

1154
01:12:43,901 --> 01:12:46,789
but in the meantime
what I am going

1155
01:12:46,789 --> 01:12:51,455
to do is give you a visual
way of understanding what

1156
01:12:51,455 --> 01:12:54,952
this costs, which is
one of the techniques

1157
01:12:54,952 --> 01:12:57,526
we will elaborate on next time.

1158
01:12:57,526 --> 01:13:02,000
It is called a recursion
tree technique.

1159
01:13:02,000 --> 01:13:07,681
And I will use it for the
actual recurrence that is almost

1160
01:13:07,681 --> 01:13:12,494
the same 2T(n/2), but I am
going to actually explicitly,

1161
01:13:12,494 --> 01:13:17,249
because I want you to
see where it occurs,

1162
01:13:17,249 --> 01:13:23,048
plus some constant times n where
c is a constant greater than

1163
01:13:23,048 --> 01:13:23,548
zero.

1164
01:13:23,548 --> 01:13:29,518
So, we are going to look at
this recurrence with a base

1165
01:13:29,518 --> 01:13:32,000
case of order one.

1166
01:13:32,000 --> 01:13:34,672
I am just making the
constant in here,

1167
01:13:34,672 --> 01:13:37,340
the upper bound
on the constant be

1168
01:13:37,340 --> 01:13:39,322
explicit rather than implicit.

1169
01:13:39,322 --> 01:13:43,092
And the way you do a recursion
tree is the following.

1170
01:13:43,092 --> 01:13:46,026
You start out by writing
down the left hand

1171
01:13:46,026 --> 01:13:47,345
side of the recurrence.

1172
01:13:47,345 --> 01:13:51,562
And then what you do is you
say well, that is equal to,

1173
01:13:51,562 --> 01:13:53,907
and now let's
write it as a tree.

1174
01:13:53,907 --> 01:13:58,603
I do c of n work plus now I
am going to have to do work

1175
01:13:58,603 --> 01:14:01,000
on each of my two children.

1176
01:14:01,000 --> 01:14:08,149
T of n over 2 and T of n over
If I sum up what is in here,

1177
01:14:08,149 --> 01:14:14,395
I get this because that is
what the recurrence says,

1178
01:14:14,395 --> 01:14:15,427
T(n)=2T(n/2)+cn.

1179
01:14:15,427 --> 01:14:17,014
I have 2T(n/2)+cn.

1180
01:14:17,014 --> 01:14:19,664
Then I do it again.

1181
01:14:19,664 --> 01:14:21,496
I have cn here.

1182
01:14:21,496 --> 01:14:23,786
I now have here cn/2.

1183
01:14:23,786 --> 01:14:25,618
And here is cn/2.

1184
01:14:25,618 --> 01:14:31,000
And each of these
now has a T(n/4).

1185
01:14:36,000 --> 01:14:39,972
And these each have a T(n/4).

1186
01:14:39,972 --> 01:14:43,285
And this has a T(n/4).

1187
01:14:43,285 --> 01:14:50,284
And I keep doing that, the
dangerous dot, dot, dots.

1188
01:14:50,284 --> 01:15:00,000
And, if I keep doing that, I end
up with it looking like this.

1189
01:15:18,000 --> 01:15:23,318
And I keep going down
until I get to a leaf.

1190
01:15:23,318 --> 01:15:27,896
And a leaf, I have
essentially a T(1).

1191
01:15:27,896 --> 01:15:29,702
That is T(1).

1192
01:15:29,702 --> 01:15:35,608
And so the first question
I ask here is, what

1193
01:15:35,608 --> 01:15:38,983
is the height of this tree?

1194
01:15:38,983 --> 01:15:39,489
Yeah.

1195
01:15:39,489 --> 01:15:41,008
It's log n.

1196
01:15:41,008 --> 01:15:45,880
It's actually very
close to exactly log n

1197
01:15:45,880 --> 01:15:51,144
because I am starting
out at the top with n

1198
01:15:51,144 --> 01:15:59,639
and then I go to n/2 and n/4
and all the way down until I

1199
01:15:59,639 --> 01:16:04,324
get to The number of halvings
of n until I get to 1 is

1200
01:16:04,324 --> 01:16:07,070
log n so the height
here is log n.

1201
01:16:07,070 --> 01:16:10,076
It's OK if it is
constant times log n.

1202
01:16:10,076 --> 01:16:11,161
It doesn't matter.

1203
01:16:11,161 --> 01:16:15,000
How many leaves are in
this tree, by the way?

1204
01:16:25,000 --> 01:16:28,328
How many leaves
does this tree have?

1205
01:16:28,328 --> 01:16:28,828
Yeah.

1206
01:16:28,828 --> 01:16:31,951
The number of
leaves, once again,

1207
01:16:31,951 --> 01:16:34,238
is actually pretty close.

1208
01:16:34,238 --> 01:16:35,438
It's actually n.

1209
01:16:35,438 --> 01:16:38,847
If you took it all the way down.

1210
01:16:38,847 --> 01:16:42,605
Let's make some simplifying
assumption. n is

1211
01:16:42,605 --> 01:16:47,849
a perfect power of 2, so it
is an integer power of 2.

1212
01:16:47,849 --> 01:16:52,235
Then this is exactly log
n to get down to T(1).

1213
01:16:52,235 --> 01:16:55,337
And then there are
exactly n leaves,

1214
01:16:55,337 --> 01:16:58,505
because the number
of leaves here,

1215
01:16:58,505 --> 01:17:05,000
the number of nodes at
this level is 1, 2, 4, 8.

1216
01:17:05,000 --> 01:17:08,627
And if I go down
height h, I have

1217
01:17:08,627 --> 01:17:13,963
2 to the h leaves, 2 to
the log n, that is just n.

1218
01:17:13,963 --> 01:17:17,172
We are doing math here, right?

1219
01:17:17,172 --> 01:17:20,567
Now let's figure
out how much work,

1220
01:17:20,567 --> 01:17:25,511
if I look at adding up
everything in this tree

1221
01:17:25,511 --> 01:17:31,518
I am going to get T(n),
so let's add that up.

1222
01:17:31,518 --> 01:17:36,138
Well, let's add it
up level by level.

1223
01:17:36,138 --> 01:17:40,763
How much do we have
in the first level?

1224
01:17:40,763 --> 01:17:41,982
Just cn.

1225
01:17:41,982 --> 01:17:47,786
If I add up the second
level, how much do I have?

1226
01:17:47,786 --> 01:17:48,285
cn.

1227
01:17:48,285 --> 01:17:53,059
How about if I add up
the third level? cn.

1228
01:17:53,059 --> 01:17:57,443
How about if I add
up all the leaves?

1229
01:17:57,443 --> 01:17:58,989
Theta n.

1230
01:17:58,989 --> 01:18:05,759
It is not necessarily cn
because the boundary case

1231
01:18:05,759 --> 01:18:09,397
may have a different constant.

1232
01:18:09,397 --> 01:18:14,988
It is actually theta n,
but cn all the way here.

1233
01:18:14,988 --> 01:18:18,924
If I add up the
total amount, that

1234
01:18:18,924 --> 01:18:23,583
is equal to cn times
log n, because that's

1235
01:18:23,583 --> 01:18:31,233
the height, that is how many
cn's I have here, plus theta n.

1236
01:18:31,233 --> 01:18:39,258
And this is a higher order term
than this, so this goes away,

1237
01:18:39,258 --> 01:18:45,783
get rid of the constants, that
is equal to theta(n lg n).

1238
01:18:45,783 --> 01:18:51,674
And theta(n lg n) is
asymptotically faster than

1239
01:18:51,674 --> 01:18:52,786
theta(n^2).

1240
01:18:52,786 --> 01:18:58,073
So, merge sort, on a
large enough input size,

1241
01:18:58,073 --> 01:19:03,000
is going to beat insertion sort.

1242
01:19:03,000 --> 01:19:07,291
Merge sort is going to
be a faster algorithm.

1243
01:19:07,291 --> 01:19:10,579
Sorry, you guys,
I didn't realize

1244
01:19:10,579 --> 01:19:13,014
you couldn't see over there.

1245
01:19:13,014 --> 01:19:16,682
You should speak up
if you cannot see.

1246
01:19:16,682 --> 01:19:22,662
So, this is a faster algorithm
because theta(n lg n) grows

1247
01:19:22,662 --> 01:19:25,048
more slowly than theta(n^2).

1248
01:19:25,048 --> 01:19:31,000
And merge sort asymptotically
beats insertion sort.

1249
01:19:31,000 --> 01:19:35,424
Even if you ran insertion
sort on a supercomputer,

1250
01:19:35,424 --> 01:19:40,842
somebody running on a PC with
merge sort for sufficient large

1251
01:19:40,842 --> 01:19:46,441
input will clobber them because
actually n^2 is way bigger than

1252
01:19:46,441 --> 01:19:50,053
n log n once you get
the n's to be large.

1253
01:19:50,053 --> 01:19:55,572
And, in practice, merge sort
tends to win here for n bigger

1254
01:19:55,572 --> 01:19:58,000
than, say, 30 or so.

1255
01:19:58,000 --> 01:20:02,092
If you have a very small
input like 30 elements,

1256
01:20:02,092 --> 01:20:06,272
insertion sort is a
perfectly decent sort to use.

1257
01:20:06,272 --> 01:20:11,497
But merge sort is going to be
a lot faster even for something

1258
01:20:11,497 --> 01:20:14,370
that is only a few
dozen elements.

1259
01:20:14,370 --> 01:20:18,289
It is going to actually
be a faster algorithm.

1260
01:20:18,289 --> 01:20:20,901
That's sort of the lessons, OK?

1261
01:20:20,901 --> 01:20:24,786
Remember that to get your
recitation assignments

1262
01:20:24,786 --> 01:20:27,702
and attend recitation on Friday.

1263
01:20:27,702 --> 01:20:33,952
Because we are going to be going
through a bunch of the things

1264
01:20:33,952 --> 01:20:36,871
that I have left
on the table here.

1265
01:20:36,871 --> 01:20:39,321
And see you next Monday.