1
00:00:00,060 --> 00:00:02,500
The following content is
provided under a Creative

2
00:00:02,500 --> 00:00:04,019
Commons license.

3
00:00:04,019 --> 00:00:06,360
Your support will help
MIT OpenCourseWare

4
00:00:06,360 --> 00:00:10,730
continue to offer high quality
educational resources for free.

5
00:00:10,730 --> 00:00:13,330
To make a donation or
view additional materials

6
00:00:13,330 --> 00:00:17,217
from hundreds of MIT courses,
visit MIT OpenCourseWare

7
00:00:17,217 --> 00:00:17,842
at ocw.mit.edu.

8
00:00:21,180 --> 00:00:22,900
PROFESSOR: Let's get started.

9
00:00:22,900 --> 00:00:24,170
Thanks for coming to lecture.

10
00:00:24,170 --> 00:00:25,830
Know there's a quiz coming up.

11
00:00:25,830 --> 00:00:30,500
There will be a tangible benefit
of attending this lecture.

12
00:00:30,500 --> 00:00:32,310
And it's not Frisbees.

13
00:00:32,310 --> 00:00:34,130
OK?

14
00:00:34,130 --> 00:00:36,270
We'll figure it out soon.

15
00:00:36,270 --> 00:00:40,500
So, two lectures on
approximation algorithms.

16
00:00:40,500 --> 00:00:44,060
One today and one,
I guess a week

17
00:00:44,060 --> 00:00:47,980
and two days from today on
Thursday after the break.

18
00:00:47,980 --> 00:00:49,420
Eric will give that one.

19
00:00:49,420 --> 00:00:52,310
So this is more of an
introductory lecture.

20
00:00:52,310 --> 00:00:56,740
Eric talked about NP complete
problems and NP hard problems.

21
00:00:56,740 --> 00:01:00,380
He talked about how you could
show that problems are NP

22
00:01:00,380 --> 00:01:02,180
complete or NP hard.

23
00:01:02,180 --> 00:01:05,330
So what happens when you
discover that a problem is NP

24
00:01:05,330 --> 00:01:07,040
complete or NP hard?

25
00:01:07,040 --> 00:01:10,850
Well, there's a
variety of strategies.

26
00:01:10,850 --> 00:01:13,990
You could just kind of give up,
and say this is intractable.

27
00:01:13,990 --> 00:01:16,210
I want a different job.

28
00:01:16,210 --> 00:01:18,840
You could say that
I'm just going

29
00:01:18,840 --> 00:01:23,520
to do the best I can without
any theoretical guarantees.

30
00:01:23,520 --> 00:01:25,010
I'm going to use a heuristic.

31
00:01:25,010 --> 00:01:27,830
I'm going to think of the
simplest, greedy heuristic.

32
00:01:27,830 --> 00:01:31,170
I'm going to code it up
and I'm going to move on.

33
00:01:31,170 --> 00:01:34,630
Or you could do
approximation algorithms.

34
00:01:34,630 --> 00:01:37,470
You could say, I'm going to
think up an interesting, greedy

35
00:01:37,470 --> 00:01:38,760
heuristic.

36
00:01:38,760 --> 00:01:44,010
But I'm going to prove
that this greedy heuristic,

37
00:01:44,010 --> 00:01:48,040
in every conceivable situation
with respect to the inputs,

38
00:01:48,040 --> 00:01:51,651
is going to be within
some factor of optimal.

39
00:01:51,651 --> 00:01:52,150
Right?

40
00:01:52,150 --> 00:01:53,900
And that's what we're
going to do today.

41
00:01:53,900 --> 00:01:57,030
We're going to take a bunch
of NP complete problems,

42
00:01:57,030 --> 00:02:01,590
and we're going to
essentially create

43
00:02:01,590 --> 00:02:05,280
simple heuristics with these
problems, simple strategies

44
00:02:05,280 --> 00:02:10,080
that are polynomial time, to
quote, "solve these problems."

45
00:02:10,080 --> 00:02:13,100
And what does it mean
to solve these problems?

46
00:02:13,100 --> 00:02:15,480
Well, you know that if
it's polynomial time,

47
00:02:15,480 --> 00:02:19,650
you're not guaranteed to get
the optimum answer every time.

48
00:02:19,650 --> 00:02:23,220
But you'll call it a solution--
an approximate solution--

49
00:02:23,220 --> 00:02:25,170
because you're within
a factor of two

50
00:02:25,170 --> 00:02:27,010
for every possible input.

51
00:02:27,010 --> 00:02:28,480
That's one example.

52
00:02:28,480 --> 00:02:34,110
Or, you have a more complicated
approximation factor

53
00:02:34,110 --> 00:02:36,120
that we'll get to
in a second, where

54
00:02:36,120 --> 00:02:37,780
it's not quite a factor of two.

55
00:02:37,780 --> 00:02:40,860
It might be a factor of two
for small size problems.

56
00:02:40,860 --> 00:02:43,290
Might be a factor of
10 for larger problems.

57
00:02:43,290 --> 00:02:44,800
And so on and so forth.

58
00:02:44,800 --> 00:02:46,870
And then the last
thing is, it'd be great

59
00:02:46,870 --> 00:02:51,150
if you could spend more time
and get better solutions.

60
00:02:51,150 --> 00:02:53,410
And those are
approximation schemes.

61
00:02:53,410 --> 00:02:56,370
And we'll look at
approximation schemes as well.

62
00:02:56,370 --> 00:02:59,440
So, just dive in each
of these problems,

63
00:02:59,440 --> 00:03:03,800
depending on whether it's
decision or optimization

64
00:03:03,800 --> 00:03:07,410
is NP complete or NP hard.

65
00:03:07,410 --> 00:03:10,300
I'll define these
problems as we go along.

66
00:03:10,300 --> 00:03:14,500
But basically, the name of the
game here is, grab a problem,

67
00:03:14,500 --> 00:03:17,760
define it, think of an
interesting heuristic,

68
00:03:17,760 --> 00:03:18,920
do a proof.

69
00:03:18,920 --> 00:03:19,420
OK?

70
00:03:19,420 --> 00:03:23,070
And that's essentially what
we're going to do three times.

71
00:03:23,070 --> 00:03:25,800
The good news is the
proofs aren't tortuous.

72
00:03:25,800 --> 00:03:27,810
They're not 30-minute proofs.

73
00:03:27,810 --> 00:03:29,810
And they should be
pretty intuitive.

74
00:03:29,810 --> 00:03:33,710
And we'll see if we can
extract them out of you.

75
00:03:33,710 --> 00:03:36,727
A painful extraction process.

76
00:03:36,727 --> 00:03:38,060
I went to the dentist yesterday.

77
00:03:40,489 --> 00:03:41,280
Not for extraction.

78
00:03:49,709 --> 00:03:51,750
I should've said, I went
to the dentist yesterday

79
00:03:51,750 --> 00:03:56,730
and I'm now going to take
it out on you, right?

80
00:03:56,730 --> 00:03:59,060
OK, so what's an
approximation algorithm?

81
00:03:59,060 --> 00:04:04,990
An algorithm, for
a problem of size

82
00:04:04,990 --> 00:04:11,090
n, and so it's going
to be parameterized.

83
00:04:11,090 --> 00:04:13,830
And the approximation factor
may also be parameterized.

84
00:04:13,830 --> 00:04:16,190
It'd be nice if it weren't,
if it were a constant factor

85
00:04:16,190 --> 00:04:17,260
approximation.

86
00:04:17,260 --> 00:04:19,411
But sometimes you can't do that.

87
00:04:19,411 --> 00:04:20,910
And in fact, sometimes
you can prove

88
00:04:20,910 --> 00:04:23,660
that constant
factor approximation

89
00:04:23,660 --> 00:04:26,610
algorithms don't exist.

90
00:04:26,610 --> 00:04:28,400
And if they do,
then p equals NP.

91
00:04:28,400 --> 00:04:28,900
All right?

92
00:04:28,900 --> 00:04:32,170
So it's gets very interesting.

93
00:04:32,170 --> 00:04:35,240
So, in this case, approximation
algorithms or schemes

94
00:04:35,240 --> 00:04:37,140
exist for these
three problems, which

95
00:04:37,140 --> 00:04:39,910
is why we are looking
at them today.

96
00:04:39,910 --> 00:04:42,780
But you got a problem of size n.

97
00:04:42,780 --> 00:04:56,510
And we're going to define an
approximation ratio, row of n,

98
00:04:56,510 --> 00:05:02,265
for any input-- if for
any input-- excuse me.

99
00:05:09,700 --> 00:05:16,740
The algorithm
produces a solution

100
00:05:16,740 --> 00:05:25,140
with cost C that satisfies
this little property, which

101
00:05:25,140 --> 00:05:32,040
says that the max of C
divided by Copt divided

102
00:05:32,040 --> 00:05:36,420
by-- oh, sorry-- and Copt
divided by C is less than

103
00:05:36,420 --> 00:05:39,430
or equal to row n.

104
00:05:39,430 --> 00:05:42,090
And the only reason you
have two terms in here

105
00:05:42,090 --> 00:05:44,240
is because you haven't
said whether it's

106
00:05:44,240 --> 00:05:46,950
a minimization problem or
a maximization problem.

107
00:05:46,950 --> 00:05:49,840
Right, so of it' a
minimization problem,

108
00:05:49,840 --> 00:05:52,982
you don't want to be too much
greater than the minimum.

109
00:05:52,982 --> 00:05:54,440
If it's a maximization
problem, you

110
00:05:54,440 --> 00:05:56,565
don't want to be too much
smaller than the maximum.

111
00:05:56,565 --> 00:05:58,610
And so you just stick
those two things in there.

112
00:05:58,610 --> 00:06:00,300
And you don't worry
about whether it's

113
00:06:00,300 --> 00:06:03,200
min or max in terms of
the objective function,

114
00:06:03,200 --> 00:06:05,580
and you want it to be
a particular ratio.

115
00:06:05,580 --> 00:06:06,610
OK?

116
00:06:06,610 --> 00:06:08,550
Now I did say row of n there.

117
00:06:08,550 --> 00:06:12,300
So this could be a constant or
it could be a function of no.

118
00:06:12,300 --> 00:06:15,570
If it's a function
of n, it's going

119
00:06:15,570 --> 00:06:17,682
to be an increasing
function of n.

120
00:06:17,682 --> 00:06:18,614
OK?

121
00:06:18,614 --> 00:06:20,030
Otherwise you could
just bound it,

122
00:06:20,030 --> 00:06:21,980
and have a constant, obviously.

123
00:06:21,980 --> 00:06:24,280
So, you might have
something like-- and we'll

124
00:06:24,280 --> 00:06:28,830
see one of these-- log n
approximation scheme, which

125
00:06:28,830 --> 00:06:34,470
says that you're going
to be within logarithmic

126
00:06:34,470 --> 00:06:37,210
of the answer-- the
minimum or maximum.

127
00:06:37,210 --> 00:06:41,300
But if it's a million, then
if you do log of base two,

128
00:06:41,300 --> 00:06:45,330
then you're within a factor
of 20, which isn't that great.

129
00:06:45,330 --> 00:06:47,125
But let's just say if
you're happy with it,

130
00:06:47,125 --> 00:06:48,500
and if it goes to
a billion, it's

131
00:06:48,500 --> 00:06:50,920
a factor 30, and
so on and so forth.

132
00:06:50,920 --> 00:06:52,910
Actually, it could grow.

133
00:06:52,910 --> 00:06:56,070
So that's an algorithm.

134
00:06:56,070 --> 00:07:00,820
If these terms are
used interchangeably,

135
00:07:00,820 --> 00:07:02,200
we'll try and differentiate.

136
00:07:02,200 --> 00:07:06,610
But we do have something that
we call an approximation scheme.

137
00:07:06,610 --> 00:07:08,660
And the big difference
between approximation

138
00:07:08,660 --> 00:07:10,810
algorithms an
approximation schemes

139
00:07:10,810 --> 00:07:13,880
is that I'm going to have a
little knob in an approximation

140
00:07:13,880 --> 00:07:17,400
scheme that's going
to let me do more work

141
00:07:17,400 --> 00:07:19,070
to get something better.

142
00:07:19,070 --> 00:07:19,690
OK?

143
00:07:19,690 --> 00:07:21,520
And that's essentially
what a scheme

144
00:07:21,520 --> 00:07:27,890
is, where we're going to take
an input-- an additional input--

145
00:07:27,890 --> 00:07:32,050
epsilon, strictly
greater than zero.

146
00:07:32,050 --> 00:07:42,410
And for any fixed
epsilon, the scheme--

147
00:07:42,410 --> 00:07:45,670
it's an approximation scheme
as opposed to an algorithm--

148
00:07:45,670 --> 00:07:51,330
is a 1 plus epsilon
approximation algorithm.

149
00:07:58,870 --> 00:08:02,840
And so here we just say that
this is a row n approximation

150
00:08:02,840 --> 00:08:07,270
algorithm if it
satisfies this property.

151
00:08:07,270 --> 00:08:10,480
And here we have a
family of algorithms

152
00:08:10,480 --> 00:08:15,810
that are parameterized by
n in terms of run time,

153
00:08:15,810 --> 00:08:18,960
as well as epsilon.

154
00:08:18,960 --> 00:08:21,970
And so you might
have a situation

155
00:08:21,970 --> 00:08:27,400
where you have
order n raised to q

156
00:08:27,400 --> 00:08:36,669
divided by epsilon running time
for an approximation algorithm.

157
00:08:36,669 --> 00:08:41,669
And what this means is that if
you're within 10% of optimal,

158
00:08:41,669 --> 00:08:45,180
then you're going to
put 0.1 down here.

159
00:08:45,180 --> 00:08:49,390
And this is going to be an
n raised to 20 algorithm.

160
00:08:49,390 --> 00:08:50,350
Polynomial time!

161
00:08:50,350 --> 00:08:51,060
Wonderful!

162
00:08:51,060 --> 00:08:52,880
Solve the world's problems.

163
00:08:52,880 --> 00:08:53,720
Not really.

164
00:08:53,720 --> 00:08:55,460
I mean, n raised to
20 is pretty bad.

165
00:08:55,460 --> 00:08:57,921
But it's not exponential.

166
00:08:57,921 --> 00:08:58,420
Right?

167
00:08:58,420 --> 00:09:03,890
So you see that there
is a growth here

168
00:09:03,890 --> 00:09:07,800
of polynomial degree
with respect to epsilon.

169
00:09:07,800 --> 00:09:11,860
And it's a pretty fast growth.

170
00:09:11,860 --> 00:09:14,860
If you want to go to
epsilon equals 0.01, I mean,

171
00:09:14,860 --> 00:09:17,230
even n raised to 20
is probably untenable.

172
00:09:17,230 --> 00:09:20,350
But certainly n raised to
200 is completely untenable.

173
00:09:20,350 --> 00:09:21,020
Right?

174
00:09:21,020 --> 00:09:33,800
So this is what's called a PTAS,
which is probabilistic time

175
00:09:33,800 --> 00:09:37,120
approximation scheme.

176
00:09:37,120 --> 00:09:40,610
And don't worry too much
about the probabilistic.

177
00:09:40,610 --> 00:09:43,651
It's a function of epsilon.

178
00:09:43,651 --> 00:09:45,400
That's the way you
want to think about it.

179
00:09:45,400 --> 00:09:46,820
The run time.

180
00:09:46,820 --> 00:09:51,920
And we'll look at a particular
scheme later in the lecture.

181
00:09:51,920 --> 00:09:56,150
But clearly this is
polynomial in n, OK?

182
00:09:56,150 --> 00:09:59,870
But it's not
polynomial in epsilon.

183
00:09:59,870 --> 00:10:00,370
All right?

184
00:10:00,370 --> 00:10:06,450
So a PTAS is going to
be poly in n, but not

185
00:10:06,450 --> 00:10:14,150
necessarily in epsilon.

186
00:10:14,150 --> 00:10:19,110
And I say not necessarily
because we still

187
00:10:19,110 --> 00:10:20,010
call it a PTAS.

188
00:10:20,010 --> 00:10:24,770
We just say, fully polynomial
time approximation scheme,

189
00:10:24,770 --> 00:10:30,030
FPTAS, if it's polynomial in
both n and 1 over epsilon.

190
00:10:30,030 --> 00:10:30,530
Right?

191
00:10:30,530 --> 00:10:41,790
So fully PTAS is poly
in n and 1 over epsilon.

192
00:10:41,790 --> 00:10:46,230
So a fully PTAS scheme
would be something like n

193
00:10:46,230 --> 00:10:48,970
divided by epsilon square.

194
00:10:48,970 --> 00:10:50,130
OK?

195
00:10:50,130 --> 00:10:53,830
So as epsilon shrinks,
obviously the run time

196
00:10:53,830 --> 00:10:57,470
is going to grow because you've
got 1 over epsilon square.

197
00:10:57,470 --> 00:11:00,350
But it's not anywhere
as bad as the n

198
00:11:00,350 --> 00:11:04,180
raised to 2 divided by epsilon
in terms of its growth rate.

199
00:11:04,180 --> 00:11:05,070
OK?

200
00:11:05,070 --> 00:11:07,070
So there's lots of
NP complete problems

201
00:11:07,070 --> 00:11:10,852
that have PTAS's and that some
of them have FPTAS's as well.

202
00:11:10,852 --> 00:11:11,810
And so on and so forth.

203
00:11:11,810 --> 00:11:12,480
Question?

204
00:11:12,480 --> 00:11:15,330
AUDIENCE: [INAUDIBLE]

205
00:11:15,330 --> 00:11:18,570
PROFESSOR: So we won't
get into that today.

206
00:11:18,570 --> 00:11:26,260
But you can think of
it as the probability

207
00:11:26,260 --> 00:11:32,420
over the space of possible
solutions that you have.

208
00:11:32,420 --> 00:11:36,082
You can distribution of inputs.

209
00:11:36,082 --> 00:11:37,540
The bottom line is
that we actually

210
00:11:37,540 --> 00:11:39,360
won't cover that today.

211
00:11:39,360 --> 00:11:43,400
So let's shelve for
the next lecture, OK?

212
00:11:43,400 --> 00:11:51,460
So just worry about the fact
that it's polynomial in n

213
00:11:51,460 --> 00:11:53,640
and not in epsilon
for the first part.

214
00:11:53,640 --> 00:11:57,070
And it's polynomial in n and
epsilon for the second part.

215
00:11:57,070 --> 00:11:58,090
OK?

216
00:11:58,090 --> 00:11:59,977
AUDIENCE: [INAUDIBLE]

217
00:11:59,977 --> 00:12:00,810
PROFESSOR: Oh I see.

218
00:12:00,810 --> 00:12:02,100
So, it's a good point.

219
00:12:02,100 --> 00:12:05,570
So for the purposes of this
lecture, thank you so much.

220
00:12:05,570 --> 00:12:07,900
I'm glad I didn't
have to get into that.

221
00:12:07,900 --> 00:12:09,810
Polynomial time.

222
00:12:09,810 --> 00:12:10,310
Good.

223
00:12:10,310 --> 00:12:12,000
Better answer.

224
00:12:12,000 --> 00:12:14,400
Either way, we're not
going to cover that.

225
00:12:14,400 --> 00:12:15,350
All right?

226
00:12:15,350 --> 00:12:15,850
Good.

227
00:12:15,850 --> 00:12:17,462
So, it's polynomial.

228
00:12:17,462 --> 00:12:19,420
I mean, you could have
probabilistic algorithms

229
00:12:19,420 --> 00:12:23,040
that have this kind of
behavior, of course.

230
00:12:23,040 --> 00:12:25,980
But we're not going to cover
that in today's lecture.

231
00:12:25,980 --> 00:12:28,090
But thanks for
pointing that out Eric.

232
00:12:28,090 --> 00:12:30,400
So, that's our set up.

233
00:12:30,400 --> 00:12:35,020
We essentially have
a situation where

234
00:12:35,020 --> 00:12:38,170
it'd be great if we could tackle
NP complete problems using

235
00:12:38,170 --> 00:12:40,900
this hammer, right?

236
00:12:40,900 --> 00:12:41,800
Any questions so far?

237
00:12:44,590 --> 00:12:47,660
All right.

238
00:12:47,660 --> 00:12:50,380
Vertex cover.

239
00:12:50,380 --> 00:12:51,630
So let's dive right in.

240
00:12:54,360 --> 00:12:59,110
Let's talk about a particular
problem, very simple problem.

241
00:12:59,110 --> 00:13:03,125
What you have is an
undirected graph, G(V,E).

242
00:13:12,630 --> 00:13:16,400
And all we want is
a set of vertices

243
00:13:16,400 --> 00:13:18,145
that cover all of the edges.

244
00:13:22,010 --> 00:13:30,780
So a set of vertices
that cover all edges.

245
00:13:30,780 --> 00:13:32,170
What does it mean to cover?

246
00:13:32,170 --> 00:13:34,200
It's the obvious thing.

247
00:13:34,200 --> 00:13:35,620
If I have something like this.

248
00:13:38,150 --> 00:13:45,310
As long as I have a vertex
in my set of vertices

249
00:13:45,310 --> 00:13:48,430
that I'm calling a
cover that touches

250
00:13:48,430 --> 00:13:52,290
one endpoint of an
edge, we're going

251
00:13:52,290 --> 00:13:55,800
to call that edge covered, OK?

252
00:13:55,800 --> 00:13:59,360
So, in this case
it's pretty clear

253
00:13:59,360 --> 00:14:02,570
that the vertex
cover is simply that.

254
00:14:02,570 --> 00:14:07,400
Because that vertex
touches all of the edges

255
00:14:07,400 --> 00:14:10,455
at least at one
of the endpoints.

256
00:14:10,455 --> 00:14:12,580
A vertex is going to touch
one endpoint of an edge.

257
00:14:12,580 --> 00:14:18,280
But this vertex cover that
I've shaded touches every edge.

258
00:14:18,280 --> 00:14:20,180
So that's a vertex cover.

259
00:14:20,180 --> 00:14:24,890
If, in fact, I had
an extra edge here,

260
00:14:24,890 --> 00:14:30,050
then I now have to pick
one-- or this one of that one

261
00:14:30,050 --> 00:14:32,080
in order to complete my cover.

262
00:14:32,080 --> 00:14:33,100
OK?

263
00:14:33,100 --> 00:14:34,210
That's it.

264
00:14:34,210 --> 00:14:36,030
That's vertex cover.

265
00:14:36,030 --> 00:14:40,810
Decision problem, NP
complete to figure out

266
00:14:40,810 --> 00:14:45,470
if there's a certain number that
is below a certain value that

267
00:14:45,470 --> 00:14:47,880
do the covering.

268
00:14:47,880 --> 00:14:50,600
You obviously have an
optimization problem

269
00:14:50,600 --> 00:14:51,990
associated with that.

270
00:14:51,990 --> 00:14:53,350
And so on and so forth.

271
00:14:53,350 --> 00:14:57,590
So that's our simple set up
for our first hard problem.

272
00:14:57,590 --> 00:14:59,090
All right?

273
00:14:59,090 --> 00:15:08,050
And so, just to write that
out, find a subset V prime,

274
00:15:08,050 --> 00:15:19,400
which is a subset of capital V,
such that if (U,V) is an edge

275
00:15:19,400 --> 00:15:30,040
of G-- belongs to E-- then we
have either U belonging to V

276
00:15:30,040 --> 00:15:34,981
prime, or V belonging
to V prime, or both.

277
00:15:34,981 --> 00:15:37,230
And it's quite possible that
your vertex cover is such

278
00:15:37,230 --> 00:15:42,180
that, for a given edge, you have
two vertices that touch each

279
00:15:42,180 --> 00:15:44,690
of the endpoints of the edge.

280
00:15:44,690 --> 00:15:46,940
And the optimization
problem, which

281
00:15:46,940 --> 00:15:52,330
is what we'd like to do
here, is find a V prime

282
00:15:52,330 --> 00:15:55,615
so the cardinality is minimum.

283
00:15:58,171 --> 00:15:58,670
OK?

284
00:16:02,030 --> 00:16:02,920
So that's it.

285
00:16:05,800 --> 00:16:11,620
So, we don't know of a
polynomial time algorithm

286
00:16:11,620 --> 00:16:13,260
to solve this problem.

287
00:16:13,260 --> 00:16:16,190
So we resort to heuristics.

288
00:16:16,190 --> 00:16:22,550
What is an intuitive
heuristic for this problem?

289
00:16:22,550 --> 00:16:28,540
Suppose I wanted to implement
a poly time, greedy algorithm

290
00:16:28,540 --> 00:16:30,090
for this problem.

291
00:16:30,090 --> 00:16:34,830
What would be the first
thing that you'd think of?

292
00:16:34,830 --> 00:16:35,902
Yeah, go ahead.

293
00:16:35,902 --> 00:16:37,159
AUDIENCE: [INAUDIBLE]

294
00:16:37,159 --> 00:16:38,450
PROFESSOR: Find the max degree.

295
00:16:38,450 --> 00:16:40,290
I love that answer.

296
00:16:40,290 --> 00:16:44,470
It's the wrong answer
for this problem.

297
00:16:44,470 --> 00:16:49,086
But I love it because it
sets me up for-- ta-DA!

298
00:16:49,086 --> 00:16:53,660
All this work I did
before lecture, OK?

299
00:16:53,660 --> 00:16:55,040
All right.

300
00:16:55,040 --> 00:17:02,490
So, it turns out, that it's
not an incorrect answer.

301
00:17:02,490 --> 00:17:07,040
It's really not the
best answer in terms

302
00:17:07,040 --> 00:17:13,420
of the heuristic you apply to
get an approximation algorithm.

303
00:17:13,420 --> 00:17:17,730
So we're still in the context of
an approximation algorithm, not

304
00:17:17,730 --> 00:17:19,589
an approximation scheme.

305
00:17:19,589 --> 00:17:24,970
And what we have here is a
perfectly fine heuristic that,

306
00:17:24,970 --> 00:17:25,470
who knows?

307
00:17:25,470 --> 00:17:28,450
It might actually work
better in practice

308
00:17:28,450 --> 00:17:30,710
than this other
approximation algorithm

309
00:17:30,710 --> 00:17:32,470
that I'm going to
talk about and prove.

310
00:17:32,470 --> 00:17:35,310
But the fact of the matter
is that this approximation

311
00:17:35,310 --> 00:17:38,820
algorithm that has,
as a heuristic,

312
00:17:38,820 --> 00:17:42,420
picking the maximum
degree continually.

313
00:17:42,420 --> 00:17:46,650
And completing your vertex cover
by picking the maximum degree

314
00:17:46,650 --> 00:17:53,470
continually is a log n
approximation algorithm.

315
00:17:53,470 --> 00:17:56,050
And what that means is
that I can construct--

316
00:17:56,050 --> 00:18:02,260
and that example is right
up there-- an example where,

317
00:18:02,260 --> 00:18:06,940
regardless of what n is,
this particular heuristic--

318
00:18:06,940 --> 00:18:10,900
the maximum degree
heuristic-- might be log n off

319
00:18:10,900 --> 00:18:12,410
from optimal.

320
00:18:12,410 --> 00:18:13,370
OK?

321
00:18:13,370 --> 00:18:16,080
Whereas, this other scheme
that we're going to talk about

322
00:18:16,080 --> 00:18:19,000
is going to be within a
factor of two of optimal

323
00:18:19,000 --> 00:18:21,580
regardless of the
input that you apply.

324
00:18:21,580 --> 00:18:22,110
Right?

325
00:18:22,110 --> 00:18:26,699
So, you have a domination
here with respect to the two

326
00:18:26,699 --> 00:18:27,740
approximation algorithms.

327
00:18:27,740 --> 00:18:29,720
You've got one that is log n.

328
00:18:29,720 --> 00:18:32,300
Row n is log n, as I've
defined over there.

329
00:18:32,300 --> 00:18:34,560
And on the other side,
you've got two, right?

330
00:18:34,560 --> 00:18:37,790
So if you're a theoretician, you
know what you're going to pick.

331
00:18:37,790 --> 00:18:39,236
You're going to pick two.

332
00:18:39,236 --> 00:18:40,860
It turns out, if
you're a practitioner,

333
00:18:40,860 --> 00:18:42,770
you might actually
pick this one, right?

334
00:18:42,770 --> 00:18:45,880
But this is a theory course.

335
00:18:45,880 --> 00:18:48,480
So what is going on here?

336
00:18:48,480 --> 00:18:50,650
Well, this is a
concocted example

337
00:18:50,650 --> 00:18:54,280
that shows you that a maximum
degree heuristic could

338
00:18:54,280 --> 00:19:01,380
be as far off as log n, right?

339
00:19:01,380 --> 00:19:10,620
And so if you look at
what's going on here,

340
00:19:10,620 --> 00:19:20,770
you end up with something where
you have a bunch of vertices

341
00:19:20,770 --> 00:19:23,080
up on top, OK?

342
00:19:23,080 --> 00:19:30,600
And you end up with case k
factorial vertices up on top.

343
00:19:30,600 --> 00:19:31,900
So k equals three in this case.

344
00:19:31,900 --> 00:19:34,500
I have six vertices up there.

345
00:19:34,500 --> 00:19:37,470
I got two down here because
this is 6 divided by 3,

346
00:19:37,470 --> 00:19:38,650
because k is 3.

347
00:19:38,650 --> 00:19:40,170
And then I got 6
divided by 3 here,

348
00:19:40,170 --> 00:19:42,230
so that's 2 and 6
divided by 1 here.

349
00:19:42,230 --> 00:19:43,560
And so that's 6.

350
00:19:43,560 --> 00:19:44,140
OK?

351
00:19:44,140 --> 00:19:46,750
And so these edges are
set up in such a way

352
00:19:46,750 --> 00:19:50,100
that it's a
pathological example.

353
00:19:50,100 --> 00:19:55,350
And I misspoke in terms of
the approximation algorithm.

354
00:19:55,350 --> 00:19:58,840
I will correct myself in just
a second, in terms of log n.

355
00:19:58,840 --> 00:20:03,110
It does grow with the
size of the graph.

356
00:20:03,110 --> 00:20:05,630
Well, I'll precisely tell
you what this approximation

357
00:20:05,630 --> 00:20:09,620
algorithm is in terms of the
row n factor in just a minute.

358
00:20:09,620 --> 00:20:12,820
But let's just take a
look at this problem

359
00:20:12,820 --> 00:20:16,240
here and see what happens when
you apply this maximum degree

360
00:20:16,240 --> 00:20:17,460
heuristic, right?

361
00:20:17,460 --> 00:20:19,760
And we have to take into
account the fact that,

362
00:20:19,760 --> 00:20:23,460
if you have ties, in
terms of maximum degree,

363
00:20:23,460 --> 00:20:25,770
you may end up doing
the wrong thing.

364
00:20:25,770 --> 00:20:27,990
Because you haven't defined
what the tiebreak is

365
00:20:27,990 --> 00:20:30,710
when you have two nodes
that have the same degree.

366
00:20:30,710 --> 00:20:33,032
You could do the wrong
thing and pick the bad node

367
00:20:33,032 --> 00:20:34,490
for this particular
problem, right?

368
00:20:34,490 --> 00:20:37,830
You have to do a
worst case analysis.

369
00:20:37,830 --> 00:20:42,650
So in the worst case, when
you create a vertex cover

370
00:20:42,650 --> 00:20:46,380
using maximum degree, what
is the worst case in terms

371
00:20:46,380 --> 00:20:48,570
of the number of
vertices that we picked

372
00:20:48,570 --> 00:20:51,380
for this particular example?

373
00:20:51,380 --> 00:20:51,880
Someone?

374
00:20:51,880 --> 00:20:54,296
What is the worst case in terms
of the number of vertices?

375
00:20:54,296 --> 00:20:55,210
Yeah, back there.

376
00:20:55,210 --> 00:20:56,070
AUDIENCE: Eleven?

377
00:20:56,070 --> 00:20:56,550
PROFESSOR: Eleven.

378
00:20:56,550 --> 00:20:57,895
And where did you get that from?

379
00:20:57,895 --> 00:21:00,127
AUDIENCE: [INAUDIBLE]

380
00:21:00,127 --> 00:21:02,210
PROFESSOR: You grab all
of the ones on the bottom.

381
00:21:02,210 --> 00:21:03,260
Fantastic.

382
00:21:03,260 --> 00:21:04,260
All right, there you go.

383
00:21:04,260 --> 00:21:05,480
Could you stand up?

384
00:21:08,170 --> 00:21:08,790
Whoa.

385
00:21:08,790 --> 00:21:09,290
All right.

386
00:21:11,900 --> 00:21:13,130
It was the dentist yesterday.

387
00:21:15,750 --> 00:21:19,340
So, that's exactly right.

388
00:21:19,340 --> 00:21:20,660
That's exactly right.

389
00:21:20,660 --> 00:21:25,540
So what could happen
is you could pick this,

390
00:21:25,540 --> 00:21:26,710
because that's degree 3.

391
00:21:26,710 --> 00:21:30,220
Notice that the maximum
degree here is 3, of any node.

392
00:21:30,220 --> 00:21:30,910
Right?

393
00:21:30,910 --> 00:21:33,620
So if I pick something of
degree three, I'm good.

394
00:21:33,620 --> 00:21:37,160
I'm in keeping
with my heuristic.

395
00:21:37,160 --> 00:21:41,250
I could pick all the
ones at the top, right?

396
00:21:41,250 --> 00:21:43,190
And then I'm done, right?

397
00:21:43,190 --> 00:21:47,400
That's a good-- that's
a good does solution.

398
00:21:47,400 --> 00:21:49,700
That's a good trajectory.

399
00:21:49,700 --> 00:21:53,057
But all I've said is, the
heuristic is maximum degree.

400
00:21:53,057 --> 00:21:55,390
So there's nothing that's
stopping me from picking this.

401
00:21:55,390 --> 00:21:57,720
And then once I pick that,
I could pick this one.

402
00:21:57,720 --> 00:22:00,850
And then I'm down to, once
I've taken away these two,

403
00:22:00,850 --> 00:22:04,760
remember that now the maximum
degree in the entire graph

404
00:22:04,760 --> 00:22:06,140
is two.

405
00:22:06,140 --> 00:22:06,650
Right?

406
00:22:06,650 --> 00:22:10,950
Because each of these things
is losing the degree--

407
00:22:10,950 --> 00:22:14,489
losing one from its
degree-- as I go along.

408
00:22:14,489 --> 00:22:17,030
So then I could pick this one,
this one, this one, et cetera.

409
00:22:17,030 --> 00:22:18,760
And so I could end up with 11.

410
00:22:18,760 --> 00:22:19,760
OK?

411
00:22:19,760 --> 00:22:23,470
So if you go do the
math really quickly--

412
00:22:23,470 --> 00:22:26,300
and this is where
I'll correct what

413
00:22:26,300 --> 00:22:35,380
I said before-- the algo could
pick all the bottom vertices.

414
00:22:40,410 --> 00:22:44,730
And so the solution and the
top vertices are optimal.

415
00:22:44,730 --> 00:22:47,890
Top optimal.

416
00:22:47,890 --> 00:22:49,620
So that's k factorial, right?

417
00:22:49,620 --> 00:22:53,120
According to my
parameterized graph.

418
00:22:53,120 --> 00:22:55,410
That's k factorial in terms
of the optimal solution

419
00:22:55,410 --> 00:22:56,740
for this graph.

420
00:22:56,740 --> 00:22:59,510
But if I pick the ones
that are the bottom,

421
00:22:59,510 --> 00:23:04,410
then it's k factorial
divided by k, plus 1 over k

422
00:23:04,410 --> 00:23:08,390
minus 1, plus da
da da da, plus 1.

423
00:23:08,390 --> 00:23:10,590
Which is our harmonic number.

424
00:23:10,590 --> 00:23:15,710
And that's approximately
k factorial log k, OK?

425
00:23:15,710 --> 00:23:17,450
And this is where I misspoke.

426
00:23:17,450 --> 00:23:19,510
I kept saying log n, log n.

427
00:23:19,510 --> 00:23:22,620
But that's not
completely correct.

428
00:23:22,620 --> 00:23:26,750
Because if I think of n as
being the size of the input,

429
00:23:26,750 --> 00:23:29,520
k factorial is n, right?

430
00:23:29,520 --> 00:23:33,300
And so if you see that
I have log k here,

431
00:23:33,300 --> 00:23:40,120
then remember that this is log
k where k factorial equals n.

432
00:23:40,120 --> 00:23:42,960
So this is another log
factor, roughly speaking.

433
00:23:42,960 --> 00:23:43,460
Right?

434
00:23:43,460 --> 00:23:48,810
So think of it approximately
as log log n approximation, OK?

435
00:23:48,810 --> 00:23:49,820
Which is pretty good.

436
00:23:49,820 --> 00:23:52,110
But it does grow with n, right?

437
00:23:52,110 --> 00:23:54,260
The point is this
does grow with n.

438
00:23:54,260 --> 00:23:57,410
So it's not the best
approximation scheme

439
00:23:57,410 --> 00:23:59,710
that you can think of.

440
00:23:59,710 --> 00:24:02,170
Because the approximation
factor grows

441
00:24:02,170 --> 00:24:04,200
with the size of your problem.

442
00:24:04,200 --> 00:24:07,960
So it'd be great if you could
come up with a constant factor

443
00:24:07,960 --> 00:24:11,210
approximation scheme that
would beat this one, certainly

444
00:24:11,210 --> 00:24:13,110
from a theoretical
standpoint, right?

445
00:24:13,110 --> 00:24:16,090
But this one, maximum
degree, chances

446
00:24:16,090 --> 00:24:18,700
are, if you're a practitioner,
this is what you'd code.

447
00:24:18,700 --> 00:24:20,340
Not the one I'm
going to describe.

448
00:24:20,340 --> 00:24:20,910
OK?

449
00:24:20,910 --> 00:24:23,170
But we're going to analyze
the one I've described.

450
00:24:23,170 --> 00:24:28,190
I've just shown you that
there is an example where

451
00:24:28,190 --> 00:24:30,380
you have this log k factor.

452
00:24:30,380 --> 00:24:32,260
We haven't done a
proof of the fact

453
00:24:32,260 --> 00:24:36,060
that there's no worse
example than this one.

454
00:24:36,060 --> 00:24:36,560
OK?

455
00:24:36,560 --> 00:24:38,920
So I'm just claiming,
at this point,

456
00:24:38,920 --> 00:24:45,280
that this is at best, a log
k approximation algorithm.

457
00:24:45,280 --> 00:24:48,040
We haven't actually shown
that it is, in fact, a log k

458
00:24:48,040 --> 00:24:49,040
approximation algorithm.

459
00:24:49,040 --> 00:24:50,630
At best, it's that.

460
00:24:50,630 --> 00:24:51,417
OK?

461
00:24:51,417 --> 00:24:52,000
Any questions?

462
00:24:54,690 --> 00:24:55,190
All right.

463
00:24:55,190 --> 00:24:57,230
So what's another heuristic?

464
00:24:57,230 --> 00:25:04,170
What's another heuristic
for doing vertex cover?

465
00:25:04,170 --> 00:25:07,440
We did this picking
the maximum degree.

466
00:25:07,440 --> 00:25:10,090
Nice and simple.

467
00:25:10,090 --> 00:25:12,460
But it didn't quite
work out for us.

468
00:25:16,960 --> 00:25:17,680
Any other ideas?

469
00:25:22,350 --> 00:25:24,540
So, I picked vertices.

470
00:25:24,540 --> 00:25:27,660
What else could I pick?

471
00:25:27,660 --> 00:25:29,270
I could pick edges, right?

472
00:25:29,270 --> 00:25:33,700
So, I could pick random edges.

473
00:25:33,700 --> 00:25:37,000
It turns out that
actually works better

474
00:25:37,000 --> 00:25:38,670
from a theoretical standpoint.

475
00:25:47,800 --> 00:25:51,700
So, what we're going
to do here is simply

476
00:25:51,700 --> 00:25:57,430
set the cover to be null.

477
00:25:57,430 --> 00:26:01,850
Go ahead and set all of
the edges to be E prime.

478
00:26:01,850 --> 00:26:06,865
And then we're going to
iterate over these edges.

479
00:26:17,260 --> 00:26:19,800
I'm not even
specifying the way I'm

480
00:26:19,800 --> 00:26:22,040
going to select these edges.

481
00:26:22,040 --> 00:26:26,460
And I still will be able to
do a proof of 2 approximation.

482
00:26:26,460 --> 00:26:26,960
OK?

483
00:26:31,050 --> 00:26:31,550
Oh!

484
00:26:31,550 --> 00:26:34,860
I forgot the best part.

485
00:26:34,860 --> 00:26:37,600
This is on your quiz.

486
00:26:37,600 --> 00:26:41,910
That was the tangible benefit
of attending the lecture.

487
00:26:41,910 --> 00:26:42,890
So copy that down.

488
00:26:56,420 --> 00:26:57,550
So this is very simple.

489
00:26:57,550 --> 00:27:01,850
It's not a complicated problem.

490
00:27:01,850 --> 00:27:05,510
This is not simple
heuristics that

491
00:27:05,510 --> 00:27:07,630
are going to be
particularly complicated.

492
00:27:07,630 --> 00:27:09,870
You just do some
selections, and then you

493
00:27:09,870 --> 00:27:11,050
iterate over the graph.

494
00:27:11,050 --> 00:27:13,160
And you take away
stuff from the graph.

495
00:27:13,160 --> 00:27:15,930
Typically, you take away
vertices as well as edges.

496
00:27:15,930 --> 00:27:19,560
And you keep going until
you got nothing left, right?

497
00:27:19,560 --> 00:27:21,570
And then you look at
your cover, and you say,

498
00:27:21,570 --> 00:27:22,865
what is the size of my cover?

499
00:27:31,900 --> 00:27:38,650
And here we return C, all right?

500
00:27:38,650 --> 00:27:41,330
So I won't spend
any time on that.

501
00:27:41,330 --> 00:27:42,250
You can read it.

502
00:27:42,250 --> 00:27:47,130
It's simple iterative
algorithm that fixed edges

503
00:27:47,130 --> 00:27:50,790
randomly and keeps going.

504
00:27:50,790 --> 00:27:55,350
So, now comes the
fun part, which

505
00:27:55,350 --> 00:28:02,000
is we need to show that that
little algorithm is always

506
00:28:02,000 --> 00:28:08,990
going to be within a
factor of 2 of optimal, OK?

507
00:28:08,990 --> 00:28:11,495
And you can play around
with this example.

508
00:28:14,020 --> 00:28:16,340
In fact, in this case,
you have 6 and 11.

509
00:28:16,340 --> 00:28:18,550
So that's a factor
of 2, of course.

510
00:28:18,550 --> 00:28:21,210
So even this algorithm is
better than a factor of 2.

511
00:28:21,210 --> 00:28:25,180
But it won't be if I expanded
the graph and increased k.

512
00:28:25,180 --> 00:28:27,540
But that algorithm that I
have up there is always going

513
00:28:27,540 --> 00:28:29,010
to be within a factor of 2.

514
00:28:29,010 --> 00:28:32,800
And we want to prove
that, all right?

515
00:28:32,800 --> 00:28:34,730
So how do we go
about proving that?

516
00:28:37,960 --> 00:28:47,760
We want to prove, in particular,
that a prox vertex cover is a 2

517
00:28:47,760 --> 00:28:53,750
approximation algorithm.

518
00:28:59,230 --> 00:29:02,687
So, any ideas?

519
00:29:02,687 --> 00:29:04,270
How would I prove
something like this?

520
00:29:07,180 --> 00:29:10,457
Where do you think this
factor of 2 comes from?

521
00:29:10,457 --> 00:29:11,790
Someone who hasn't answered yet.

522
00:29:14,869 --> 00:29:17,160
You answered a lot of questions
in the past, all right?

523
00:29:21,040 --> 00:29:21,540
No?

524
00:29:21,540 --> 00:29:24,000
Someone?

525
00:29:24,000 --> 00:29:24,931
All right.

526
00:29:24,931 --> 00:29:26,935
AUDIENCE: [INAUDIBLE]

527
00:29:26,935 --> 00:29:27,810
PROFESSOR: I'm sorry?

528
00:29:27,810 --> 00:29:30,440
AUDIENCE: [INAUDIBLE]

529
00:29:30,440 --> 00:29:33,040
PROFESSOR: That's an
excellent observation.

530
00:29:33,040 --> 00:29:38,510
The observation is
that the edges we pick

531
00:29:38,510 --> 00:29:40,160
do not intersect each other.

532
00:29:40,160 --> 00:29:43,740
So, I gave you a Frisbee
for the wrong answer.

533
00:29:43,740 --> 00:29:46,130
So for this correct one,
I won't give you one?

534
00:29:46,130 --> 00:29:47,400
That's fair, right?

535
00:29:47,400 --> 00:29:48,790
No.

536
00:29:48,790 --> 00:29:49,860
That's unfair.

537
00:29:49,860 --> 00:29:50,649
Right?

538
00:29:50,649 --> 00:29:51,190
There you go.

539
00:29:51,190 --> 00:29:51,690
Sorry.

540
00:29:54,240 --> 00:29:58,700
So, the key observation
was in this algorithm,

541
00:29:58,700 --> 00:30:00,640
I'm going to be picking edges.

542
00:30:00,640 --> 00:30:06,470
And the edges will not
share vertices, right?

543
00:30:06,470 --> 00:30:10,151
Because I delete the vertices
once I've picked an edge,

544
00:30:10,151 --> 00:30:10,650
correct?

545
00:30:10,650 --> 00:30:13,340
So there's no way that the
edges will share vertices.

546
00:30:13,340 --> 00:30:14,660
So what does that mean?

547
00:30:14,660 --> 00:30:19,350
Well, that means
that, I'm getting--

548
00:30:19,350 --> 00:30:23,320
let's say, I get A edges.

549
00:30:23,320 --> 00:30:30,550
So let A denote the
edges that are picked.

550
00:30:34,220 --> 00:30:38,470
So I'm going to get edges
that look like that.

551
00:30:38,470 --> 00:30:39,450
OK?

552
00:30:39,450 --> 00:30:43,400
And I got continuality
of A edges.

553
00:30:43,400 --> 00:30:48,220
I know that in my vertex
cover, that, obviously, I

554
00:30:48,220 --> 00:30:52,800
have to pick vertices
that cover all the edges.

555
00:30:52,800 --> 00:30:55,570
Now I'm picking edges,
and what's happening,

556
00:30:55,570 --> 00:30:59,420
of course, is that I'm
picking 2 A vertices.

557
00:31:04,150 --> 00:31:08,720
So my C-- and remember,
my cost was C.

558
00:31:08,720 --> 00:31:12,200
And I had Copt, that
corresponds to the optimum cost.

559
00:31:12,200 --> 00:31:15,360
And so the cost that
this algorithm produces

560
00:31:15,360 --> 00:31:17,770
is 2 times A, right?

561
00:31:17,770 --> 00:31:18,340
Make sense?

562
00:31:18,340 --> 00:31:21,950
Because I'm picking vertices,
we are picking edges.

563
00:31:21,950 --> 00:31:24,470
There's no overlap.

564
00:31:24,470 --> 00:31:27,740
And therefore, the cost
is 2 times A, right?

565
00:31:27,740 --> 00:31:33,500
So as long as I can now say
that Copt, which is the optimum,

566
00:31:33,500 --> 00:31:36,250
is less than or
equal to A, right?

567
00:31:36,250 --> 00:31:39,490
I have my factor of 2
approximation algorithm.

568
00:31:39,490 --> 00:31:40,590
So that's it.

569
00:31:40,590 --> 00:31:44,330
It's a simple
argument that says now

570
00:31:44,330 --> 00:31:59,270
show that Copt is at least
A. Copt, I'm minimizing.

571
00:31:59,270 --> 00:32:02,527
Copt should be at
least A. Right?

572
00:32:02,527 --> 00:32:03,860
I hope I said that right before.

573
00:32:03,860 --> 00:32:06,290
But I wrote that
down here correctly.

574
00:32:06,290 --> 00:32:09,190
So if I say that
Copt is at least A,

575
00:32:09,190 --> 00:32:12,770
then I got my proof
here of 2 approximation.

576
00:32:12,770 --> 00:32:14,690
Because I'm getting
2 A back, right?

577
00:32:14,690 --> 00:32:18,100
So if you go look at C divided
by-- so this means, of course,

578
00:32:18,100 --> 00:32:29,750
that C is less than or equal
to 2 Copt, if I can show--

579
00:32:29,750 --> 00:32:31,140
make that statement.

580
00:32:31,140 --> 00:32:35,100
And it turns out that's
a fairly easy statement

581
00:32:35,100 --> 00:32:42,070
to argue simply because of the
definition of vertex cover.

582
00:32:42,070 --> 00:32:49,510
Remember that I'm going to have
to cover every edge, correct?

583
00:32:49,510 --> 00:32:59,170
So I'm going to cover--
need to cover every edge,

584
00:32:59,170 --> 00:33:09,070
including all edges in A.
A is a subset of edges.

585
00:33:09,070 --> 00:33:10,669
I have to cover all the edges.

586
00:33:10,669 --> 00:33:12,460
Clearly, I have to
cover the A edges, which

587
00:33:12,460 --> 00:33:15,010
are a subset of all
of the edges, right?

588
00:33:15,010 --> 00:33:19,670
How am I going to cover
all of the A edges that

589
00:33:19,670 --> 00:33:24,320
happened to all be disjoint
in terms of their vertices?

590
00:33:24,320 --> 00:33:30,420
I'm going to have to pick one
vertex for each of these edges,

591
00:33:30,420 --> 00:33:31,100
right?

592
00:33:31,100 --> 00:33:32,850
I mean, I could pick
this one or that one.

593
00:33:32,850 --> 00:33:34,190
But I have to pick one of them.

594
00:33:34,190 --> 00:33:36,790
I could pick this
one or that one.

595
00:33:36,790 --> 00:33:38,900
And so on and so forth, right?

596
00:33:38,900 --> 00:33:43,830
So it is clear that, given
this disjoint collection

597
00:33:43,830 --> 00:33:46,740
of edges corresponding
to A, that Copt is

598
00:33:46,740 --> 00:33:49,440
greater than or equal to A, OK?

599
00:33:49,440 --> 00:33:50,980
And that's it.

600
00:33:50,980 --> 00:33:52,650
So I had to cover all.

601
00:33:52,650 --> 00:34:02,100
This requires, since no two
edges share an endpoint,

602
00:34:02,100 --> 00:34:13,610
this means that I need to pick a
different vertex from each edge

603
00:34:13,610 --> 00:34:26,150
in A. And that implies that Copt
is greater than or equal to A.

604
00:34:26,150 --> 00:34:27,350
All right?

605
00:34:27,350 --> 00:34:29,630
Any questions about that?

606
00:34:29,630 --> 00:34:30,510
We all good here?

607
00:34:30,510 --> 00:34:31,400
Yup?

608
00:34:31,400 --> 00:34:33,550
Understood the proof?

609
00:34:33,550 --> 00:34:36,590
So that's our first
approximation algorithm

610
00:34:36,590 --> 00:34:38,760
where we actually had a proof.

611
00:34:38,760 --> 00:34:41,510
And so, this is kind of cool.

612
00:34:41,510 --> 00:34:43,770
It obviously a pretty
simple algorithm.

613
00:34:43,770 --> 00:34:46,389
You're guaranteed to be
within a factor of 2.

614
00:34:46,389 --> 00:34:47,889
It doesn't mean
that that's the best

615
00:34:47,889 --> 00:34:50,679
heuristic you can come up with.

616
00:34:50,679 --> 00:34:55,739
It doesn't mean that
this is what you'd code.

617
00:34:55,739 --> 00:34:59,800
But this is the best
approximation algorithm

618
00:34:59,800 --> 00:35:05,050
that you're going to cover
for vertex cover, OK?

619
00:35:05,050 --> 00:35:10,370
So, what about other problems?

620
00:35:10,370 --> 00:35:14,340
What's the state of the world
with respect to approximation?

621
00:35:14,340 --> 00:35:17,550
There's lots of NP complete
and NP hard problems

622
00:35:17,550 --> 00:35:20,030
for which we know
approximation schemes.

623
00:35:20,030 --> 00:35:24,910
And we like to move towards
approximation schemes slowly.

624
00:35:24,910 --> 00:35:27,960
But I'd like to look at
a problem that perhaps

625
00:35:27,960 --> 00:35:32,020
is a little more compelling
than vertex cover

626
00:35:32,020 --> 00:35:34,282
before we get to
approximation schemes.

627
00:35:34,282 --> 00:35:35,740
And that's what's
called set cover.

628
00:35:44,220 --> 00:35:52,810
So set cover tries to
cover a set with subsets.

629
00:35:52,810 --> 00:35:58,050
And it's very useful
in optimization

630
00:35:58,050 --> 00:36:03,710
where you have overlapping
sets, maybe it's schedules,

631
00:36:03,710 --> 00:36:08,630
it's tasks, it's people getting
invited to dinner, et cetera,

632
00:36:08,630 --> 00:36:11,400
and you want to make sure
everybody gets invited.

633
00:36:11,400 --> 00:36:14,550
You want to invite
families, and there's

634
00:36:14,550 --> 00:36:17,840
overlapping families, because
people have relationships.

635
00:36:17,840 --> 00:36:20,680
And you want to
eventually minimize

636
00:36:20,680 --> 00:36:23,010
the number of dinners you
actually have to have.

637
00:36:23,010 --> 00:36:28,330
And that's, I don't know,
hopefully a motivating example.

638
00:36:28,330 --> 00:36:30,410
If it wasn't, too bad.

639
00:36:33,460 --> 00:36:36,220
So you do have a family of
possibly overlapping subsets.

640
00:36:40,410 --> 00:36:46,460
S1, S2, Sm, subset
of equal to x.

641
00:36:46,460 --> 00:36:48,570
So that's that big
set that we have.

642
00:36:48,570 --> 00:36:51,740
Such that I want to cover
all of the elements.

643
00:36:51,740 --> 00:36:56,680
So that's what this little
equation corresponds to.

644
00:36:56,680 --> 00:37:02,580
The union of all the
selected SI's should equal x.

645
00:37:02,580 --> 00:37:04,720
I need to cover it all.

646
00:37:04,720 --> 00:37:07,115
And I do want to minimize.

647
00:37:09,890 --> 00:37:17,042
It's called a C. Find
C subset 1, 2, m.

648
00:37:17,042 --> 00:37:18,750
So I'm selecting a
bunch of these things.

649
00:37:18,750 --> 00:37:26,530
So C is simply-- capital C here
is some subset of the indices.

650
00:37:26,530 --> 00:37:31,300
And the only reason I do
that is to say that I want

651
00:37:31,300 --> 00:37:33,430
to do this while minimizing.

652
00:37:33,430 --> 00:37:45,790
I wanted to I equals 1
through m while minimizing C.

653
00:37:45,790 --> 00:37:46,290
OK?

654
00:37:49,200 --> 00:37:50,550
Let me get this right.

655
00:37:50,550 --> 00:37:52,550
So this is what I have.

656
00:37:52,550 --> 00:37:59,002
Find C, subset of these,
such that-- I'm sorry.

657
00:37:59,002 --> 00:37:59,710
There's one more.

658
00:38:03,450 --> 00:38:10,190
Union of I belonging
to C, SI equals x.

659
00:38:10,190 --> 00:38:10,690
OK?

660
00:38:13,980 --> 00:38:16,340
Sorry for the mess here.

661
00:38:16,340 --> 00:38:18,370
But this last line
there-- so this

662
00:38:18,370 --> 00:38:22,390
is simply a specification
of the problem.

663
00:38:22,390 --> 00:38:24,290
I'm going to be given
x, and I'm going

664
00:38:24,290 --> 00:38:27,160
to be given a large
collection of subsets, such

665
00:38:27,160 --> 00:38:33,100
that the union of all of those
subsets are going to cover x.

666
00:38:33,100 --> 00:38:36,050
And now I'm saying,
I want to look inside

667
00:38:36,050 --> 00:38:39,810
and I want to
select all of them.

668
00:38:39,810 --> 00:38:42,190
I want to select a
bunch of these things.

669
00:38:42,190 --> 00:38:45,100
You know, C, which is some
subset of these indices,

670
00:38:45,100 --> 00:38:47,090
so 1 may not be in it.

671
00:38:47,090 --> 00:38:48,240
2 may be in it.

672
00:38:48,240 --> 00:38:50,120
4 may be in it, et cetera.

673
00:38:50,120 --> 00:38:55,600
And such that those subsets
that are in this capital C

674
00:38:55,600 --> 00:38:57,802
set add up to x.

675
00:38:57,802 --> 00:38:59,270
OK?

676
00:38:59,270 --> 00:39:07,740
So pictorially, may
make things clearer.

677
00:39:07,740 --> 00:39:16,410
You have, let's say, a grid
here corresponding to x.

678
00:39:16,410 --> 00:39:18,490
So each of these dots
that I'm drawing here

679
00:39:18,490 --> 00:39:20,260
are elements that
need to be covered.

680
00:39:25,130 --> 00:39:26,930
So that's my x.

681
00:39:26,930 --> 00:39:36,390
And I might have S1
corresponding to that.

682
00:39:36,390 --> 00:39:42,380
This is S3, right?

683
00:39:42,380 --> 00:39:45,940
And S2 is this
thing in the middle.

684
00:39:50,190 --> 00:39:52,600
That's S2.

685
00:39:52,600 --> 00:40:00,270
S5 is this one here.

686
00:40:00,270 --> 00:40:03,990
And I got a little S4 over here.

687
00:40:07,420 --> 00:40:16,623
And finally, I got--
let's see-- S6, yup.

688
00:40:19,870 --> 00:40:25,170
Which is kind of this funky
thing that goes like that.

689
00:40:30,050 --> 00:40:32,680
So this thing here is S6.

690
00:40:32,680 --> 00:40:34,600
All right?

691
00:40:34,600 --> 00:40:35,100
OK.

692
00:40:35,100 --> 00:40:36,340
What's the optimum?

693
00:40:36,340 --> 00:40:38,170
You got 30 seconds.

694
00:40:41,480 --> 00:40:45,890
What's the optimum cover?

695
00:40:45,890 --> 00:40:48,170
Yeah, go ahead.

696
00:40:48,170 --> 00:40:49,192
You had your hand up.

697
00:40:49,192 --> 00:40:50,067
AUDIENCE: [INAUDIBLE]

698
00:40:53,626 --> 00:40:54,250
PROFESSOR: Yep.

699
00:40:54,250 --> 00:40:55,125
That's exactly right.

700
00:40:55,125 --> 00:41:00,960
So the optimum S3, S4, S5.

701
00:41:00,960 --> 00:41:01,460
All right.

702
00:41:01,460 --> 00:41:02,335
AUDIENCE: [INAUDIBLE]

703
00:41:07,437 --> 00:41:08,020
PROFESSOR: Oh.

704
00:41:08,020 --> 00:41:08,660
S3.

705
00:41:08,660 --> 00:41:10,840
Yeah, S4 is this one here.

706
00:41:10,840 --> 00:41:12,270
S6, S5, OK good.

707
00:41:14,820 --> 00:41:16,400
S3.

708
00:41:16,400 --> 00:41:17,380
And then-- oh!

709
00:41:17,380 --> 00:41:18,420
You know what?

710
00:41:18,420 --> 00:41:19,860
You're right.

711
00:41:19,860 --> 00:41:20,780
Let's make you right.

712
00:41:26,210 --> 00:41:27,781
Don't erase that.

713
00:41:27,781 --> 00:41:28,280
Here you go.

714
00:41:33,990 --> 00:41:35,430
So that's 3, right?

715
00:41:35,430 --> 00:41:43,600
So C would be-- cardinality
of C would be 3.

716
00:41:43,600 --> 00:41:47,620
So it's a nontrivial problem.

717
00:41:47,620 --> 00:41:49,910
It's not clear how
you're going to do this.

718
00:41:49,910 --> 00:41:52,800
I've got to use a heuristic.

719
00:41:52,800 --> 00:41:55,640
Hard in terms of optimization.

720
00:41:55,640 --> 00:41:59,280
Optimal requires exponential
time, as far as we know.

721
00:41:59,280 --> 00:42:04,140
And we're just going to go
off and say, hey let's design

722
00:42:04,140 --> 00:42:06,250
an approximation
algorithm, right?

723
00:42:06,250 --> 00:42:08,590
So let's think of a heuristic.

724
00:42:08,590 --> 00:42:11,820
What's a good heuristic?

725
00:42:11,820 --> 00:42:15,280
What's a good heuristic
for this problem?

726
00:42:15,280 --> 00:42:17,420
I hope I haven't scared you.

727
00:42:17,420 --> 00:42:19,150
What's a good heuristic
for this problem?

728
00:42:19,150 --> 00:42:20,180
What's the obvious heuristic?

729
00:42:20,180 --> 00:42:20,680
Yeah?

730
00:42:20,680 --> 00:42:21,881
AUDIENCE: [INAUDIBLE]

731
00:42:21,881 --> 00:42:23,130
PROFESSOR: The largest subset.

732
00:42:23,130 --> 00:42:24,610
And in this
particular case, it's

733
00:42:24,610 --> 00:42:29,900
actually the best also
in terms of theory.

734
00:42:29,900 --> 00:42:44,376
So, approximate set
cover-- at least the best

735
00:42:44,376 --> 00:42:46,200
that we're concerned
about in this lecture.

736
00:42:55,360 --> 00:42:58,000
And what is
approximates set cover?

737
00:42:58,000 --> 00:43:05,130
It's pick largest SI.

738
00:43:05,130 --> 00:43:18,190
Remove all elements in
SI from x, and other Sj.

739
00:43:21,270 --> 00:43:23,260
So you're constantly shrinking.

740
00:43:23,260 --> 00:43:25,710
And then keep doing that.

741
00:43:25,710 --> 00:43:28,120
So you'll have a new problem.

742
00:43:28,120 --> 00:43:31,520
And you're going to specify that
new problem on every iteration,

743
00:43:31,520 --> 00:43:33,670
just like we did
for vertex cover

744
00:43:33,670 --> 00:43:35,490
and we've done many a time.

745
00:43:35,490 --> 00:43:40,840
If you do that over here,
notice that what you end up with

746
00:43:40,840 --> 00:43:46,830
is picking S1, because
S1 is the big boy here,

747
00:43:46,830 --> 00:43:50,610
in the sense that it's
got six elements in it.

748
00:43:50,610 --> 00:43:51,110
Right?

749
00:43:51,110 --> 00:43:52,670
Up on top.

750
00:43:52,670 --> 00:43:59,460
And so you'd pick
approx or heuristic algo

751
00:43:59,460 --> 00:44:07,660
would pick S1, S4, S5,
and S3 in that order.

752
00:44:07,660 --> 00:44:09,450
I won't go over it.

753
00:44:09,450 --> 00:44:12,060
It's not that important.

754
00:44:12,060 --> 00:44:15,790
The point is it doesn't get you
the optimum for this problem.

755
00:44:15,790 --> 00:44:19,300
And in general, you could
always concoct examples

756
00:44:19,300 --> 00:44:21,810
where any heuristic
fails, of course, right?

757
00:44:21,810 --> 00:44:23,870
Because this problem is hard.

758
00:44:23,870 --> 00:44:26,580
But it's four as
opposed to three.

759
00:44:26,580 --> 00:44:31,780
And the big question, again, as
always, is, what's the bound?

760
00:44:31,780 --> 00:44:35,980
What's the bound if you
applied this heuristic?

761
00:44:35,980 --> 00:44:37,690
And what can you
show with respect

762
00:44:37,690 --> 00:44:39,650
to the approximation algorithm?

763
00:44:39,650 --> 00:44:41,430
What is row n here?

764
00:44:41,430 --> 00:44:43,400
So that's what we
have to do here,

765
00:44:43,400 --> 00:44:45,360
in terms of what the bound is.

766
00:44:45,360 --> 00:44:48,710
And we're actually going
to do an analysis here

767
00:44:48,710 --> 00:44:50,990
that is pretty straightforward.

768
00:44:50,990 --> 00:44:53,160
It's got a little
bit of algebra in it.

769
00:44:53,160 --> 00:44:55,480
But if you go look
at that this, it's

770
00:44:55,480 --> 00:44:57,150
covered in CLRS, the textbook.

771
00:44:57,150 --> 00:45:00,520
But the analysis in there
uses harmonic numbers,

772
00:45:00,520 --> 00:45:04,190
and is substantially
more complicated

773
00:45:04,190 --> 00:45:08,430
for, in my mind, no reason.

774
00:45:08,430 --> 00:45:10,870
And so we have a
simpler analysis here

775
00:45:10,870 --> 00:45:14,630
that is simply going to
be a matter of counting.

776
00:45:14,630 --> 00:45:23,550
We are picking the maximum
number of elements every time.

777
00:45:23,550 --> 00:45:24,520
The best we can do.

778
00:45:24,520 --> 00:45:26,020
It's a greedy heuristic.

779
00:45:26,020 --> 00:45:34,000
We're trying to shrink our
problem as much as possible.

780
00:45:34,000 --> 00:45:35,830
Initially we have x.

781
00:45:35,830 --> 00:45:38,260
And then we're going
to get a new problem,

782
00:45:38,260 --> 00:45:41,540
let's call it x0 first,
for the initial problem.

783
00:45:41,540 --> 00:45:43,480
You're going to get
a new problem, x1.

784
00:45:43,480 --> 00:45:48,640
And we're maximally
shrinking x1 in relation

785
00:45:48,640 --> 00:45:52,320
to x0, in the sense that
we're going to remove

786
00:45:52,320 --> 00:45:54,030
as many elements as we can.

787
00:45:54,030 --> 00:45:56,740
Because that is
precisely our heuristic.

788
00:45:56,740 --> 00:46:01,864
So the big question is, as we
go from the biggest problem

789
00:46:01,864 --> 00:46:03,280
that we have, the
original problem

790
00:46:03,280 --> 00:46:07,020
to smaller and smaller problems,
when do we end up with nothing?

791
00:46:07,020 --> 00:46:08,880
When we end up with
nothing, that's

792
00:46:08,880 --> 00:46:11,570
when the number
of iterations that

793
00:46:11,570 --> 00:46:15,030
corresponds to the number
of SI's that we picked

794
00:46:15,030 --> 00:46:19,440
is going to be the collection
of SI's in our solution.

795
00:46:19,440 --> 00:46:23,210
And the cardinality of
that is our cost, right?

796
00:46:23,210 --> 00:46:27,230
So that's all pretty
straightforward, hopefully.

797
00:46:27,230 --> 00:46:30,980
So what we need to do, of
course, is show a proof.

798
00:46:30,980 --> 00:46:33,030
And the way we're
going to do this

799
00:46:33,030 --> 00:46:36,560
is by a fairly straightforward
counting argument.

800
00:46:36,560 --> 00:46:46,260
Assume there's a cover Copt,
since that Copt equals t.

801
00:46:46,260 --> 00:46:46,760
OK?

802
00:46:46,760 --> 00:46:49,010
So the cardinality
of Copt equals t.

803
00:46:49,010 --> 00:46:54,730
So I'm just assuming that this
t subset's in my optimum cover,

804
00:46:54,730 --> 00:46:56,570
OK?

805
00:46:56,570 --> 00:46:59,190
t subset's in my optimum cover.

806
00:46:59,190 --> 00:47:10,540
Now let x of k be the set
of elements in iteration k.

807
00:47:13,540 --> 00:47:16,070
And let's assume
that x0 equals x.

808
00:47:16,070 --> 00:47:18,870
So initially, I'm at 0.

809
00:47:18,870 --> 00:47:20,610
And I want to subscript
this because I

810
00:47:20,610 --> 00:47:25,090
want to point to
each of the problems

811
00:47:25,090 --> 00:47:28,670
that I'm going to
have as I shrink

812
00:47:28,670 --> 00:47:33,730
this set down to nothing.

813
00:47:33,730 --> 00:47:39,970
Now I know that for all
k, including of course x0,

814
00:47:39,970 --> 00:47:46,610
xk can be covered by t sets.

815
00:47:46,610 --> 00:47:47,360
OK?

816
00:47:47,360 --> 00:47:50,480
I mean, that's kind
of a vacuous statement

817
00:47:50,480 --> 00:47:54,350
because I assumed that x0
could be covered by t sets.

818
00:47:54,350 --> 00:47:57,920
And x0 is only shrinking
to x1, to x2, et cetera.

819
00:47:57,920 --> 00:47:59,950
And I'm just saying,
all of these things

820
00:47:59,950 --> 00:48:03,390
could be covered-- each of those
intermediate problems as well,

821
00:48:03,390 --> 00:48:04,540
can be covered-- by t.

822
00:48:04,540 --> 00:48:07,100
In fact, in the
solution that we have,

823
00:48:07,100 --> 00:48:11,670
the optimal solution,
if these x0's are

824
00:48:11,670 --> 00:48:13,210
coming from my heuristic.

825
00:48:13,210 --> 00:48:15,620
But if they were coming
from an optimum solution,

826
00:48:15,620 --> 00:48:18,936
then x0 would be covered by t.

827
00:48:18,936 --> 00:48:21,840
x1 would be covered
by t minus 1.

828
00:48:21,840 --> 00:48:23,990
And t minus 2, and
so on and so forth.

829
00:48:23,990 --> 00:48:27,180
But I don't have an
optimum algorithm here.

830
00:48:27,180 --> 00:48:28,992
I just have my
heuristic algorithm.

831
00:48:28,992 --> 00:48:30,950
And I'm just making a
fairly vacuous statement,

832
00:48:30,950 --> 00:48:33,290
as I said, based
on my definition,

833
00:48:33,290 --> 00:48:36,420
that the original problem
can be covered using t.

834
00:48:36,420 --> 00:48:38,110
And therefore, a
smaller problem,

835
00:48:38,110 --> 00:48:41,460
when I remove elements from it,
could also be covered using t.

836
00:48:41,460 --> 00:48:42,520
OK?

837
00:48:42,520 --> 00:48:45,640
So far, so good?

838
00:48:45,640 --> 00:48:57,680
So now, one of them covers what?

839
00:48:57,680 --> 00:48:59,720
What can I say
about one of them?

840
00:48:59,720 --> 00:49:03,580
What principle can I
use to make a claim

841
00:49:03,580 --> 00:49:06,190
about the number of
elements that are

842
00:49:06,190 --> 00:49:10,170
covered by one of these t sets?

843
00:49:13,540 --> 00:49:18,010
Remember a principle
from way back?

844
00:49:18,010 --> 00:49:19,700
6042?

845
00:49:19,700 --> 00:49:21,150
My favorite principle?

846
00:49:21,150 --> 00:49:21,650
Flapping.

847
00:49:21,650 --> 00:49:23,920
Think flapping.

848
00:49:23,920 --> 00:49:25,160
Pigeon hole.

849
00:49:25,160 --> 00:49:27,440
Pigeon hole principle, right?

850
00:49:27,440 --> 00:49:30,600
See, you have to remember
all of the material

851
00:49:30,600 --> 00:49:33,420
that you learned at MIT
for the rest of your life.

852
00:49:33,420 --> 00:49:34,270
OK?

853
00:49:34,270 --> 00:49:37,160
You never know when
it's going to be useful.

854
00:49:37,160 --> 00:49:39,090
OK, so here you go, pigeon hole.

855
00:49:39,090 --> 00:49:40,160
My favorite principle.

856
00:49:42,750 --> 00:49:46,110
It's such a trivial principle.

857
00:49:46,110 --> 00:49:53,369
So one of them covers at
least xk divided by t.

858
00:49:53,369 --> 00:49:54,910
I mean, why is this
even a principle?

859
00:49:54,910 --> 00:49:56,790
Right?

860
00:49:56,790 --> 00:49:57,290
Elements.

861
00:49:59,990 --> 00:50:02,810
OK, so that's it, right?

862
00:50:02,810 --> 00:50:05,730
That's the observation.

863
00:50:05,730 --> 00:50:12,100
And, now that implies that
an algorithm-- because it's

864
00:50:12,100 --> 00:50:14,480
going to pick the
maximum of these, right?

865
00:50:14,480 --> 00:50:17,020
The algorithm is going to
pick the maximum of these.

866
00:50:17,020 --> 00:50:28,320
So, it's going to pick
a set of current size

867
00:50:28,320 --> 00:50:32,347
greater than or equal
to xk divided by t.

868
00:50:32,347 --> 00:50:34,180
Otherwise, the algorithm
would be incorrect.

869
00:50:34,180 --> 00:50:36,490
It's not doing what
you told it to do.

870
00:50:36,490 --> 00:50:37,490
Right?

871
00:50:37,490 --> 00:50:39,300
It's got to pick the maximum.

872
00:50:39,300 --> 00:50:42,650
So keep chugging here.

873
00:50:42,650 --> 00:50:49,255
And one real observation, and
then the rest of it is algebra.

874
00:50:56,060 --> 00:51:00,745
So, what I can say
is that for all k,

875
00:51:00,745 --> 00:51:04,610
xk plus 1, which is
shrinking, is less than

876
00:51:04,610 --> 00:51:10,600
or equal to 1
minus 1 over t, xk.

877
00:51:10,600 --> 00:51:11,190
OK?

878
00:51:11,190 --> 00:51:13,221
That's the way I'm shrinking.

879
00:51:13,221 --> 00:51:13,720
Right?

880
00:51:13,720 --> 00:51:18,290
This is again, a fairly
conservative statement.

881
00:51:18,290 --> 00:51:19,680
Because the fact
of the matter is

882
00:51:19,680 --> 00:51:23,650
that I'm putting t in as
a constant here, right?

883
00:51:23,650 --> 00:51:26,800
But t is actually, in effect,
changing, obviously, halfway

884
00:51:26,800 --> 00:51:28,020
through the algorithm.

885
00:51:28,020 --> 00:51:33,790
I don't need t sets to cover the
x-- whatever it is-- xk over 2

886
00:51:33,790 --> 00:51:35,162
or whatever it is that I have.

887
00:51:35,162 --> 00:51:37,090
I need the t for x0.

888
00:51:37,090 --> 00:51:40,380
Maybe I did a bad selection
with my heuristic,

889
00:51:40,380 --> 00:51:45,306
and I still would need t, which
is optimum, remember, for x1.

890
00:51:45,306 --> 00:51:46,680
But halfway through
the algorithm

891
00:51:46,680 --> 00:51:51,090
after I picked a bunch of sets,
I'm still saying I need t, OK?

892
00:51:51,090 --> 00:51:53,137
Because I just need
that for my proof.

893
00:51:53,137 --> 00:51:54,470
That's all I need for the proof.

894
00:51:54,470 --> 00:51:57,430
In CLRS, this
actually varies and it

895
00:51:57,430 --> 00:51:59,380
turns into a harmonic series.

896
00:51:59,380 --> 00:52:00,640
We won't go there.

897
00:52:00,640 --> 00:52:01,230
OK?

898
00:52:01,230 --> 00:52:06,690
You can do the natural
logarithm of n plus 1, a proof,

899
00:52:06,690 --> 00:52:09,070
just doing the simpler analysis.

900
00:52:09,070 --> 00:52:10,220
OK?

901
00:52:10,220 --> 00:52:12,110
So you see what's
happening here.

902
00:52:12,110 --> 00:52:14,280
That's my shrinkage.

903
00:52:14,280 --> 00:52:17,610
That's the recurrence,
if you will,

904
00:52:17,610 --> 00:52:20,970
that tells me how my
problem size is shrinking.

905
00:52:20,970 --> 00:52:23,160
And when do I end?

906
00:52:23,160 --> 00:52:24,260
What is my stopping point?

907
00:52:27,400 --> 00:52:29,109
What's my stopping point?

908
00:52:29,109 --> 00:52:29,734
Mathematically?

909
00:52:35,650 --> 00:52:38,720
When do we end this lecture?

910
00:52:38,720 --> 00:52:40,440
When you give me the answer.

911
00:52:40,440 --> 00:52:41,850
No, not quite.

912
00:52:41,850 --> 00:52:42,860
So I end when?

913
00:52:42,860 --> 00:52:44,540
I got nothing to cover, right?

914
00:52:44,540 --> 00:52:56,990
So when one of these things
gets down to xk equals 0, right?

915
00:52:56,990 --> 00:52:59,020
When xk equals 0, I'm done.

916
00:53:03,310 --> 00:53:05,140
I'm constantly
taking stuff away.

917
00:53:05,140 --> 00:53:07,670
When xk equals 0,
I'm going to be done.

918
00:53:07,670 --> 00:53:08,490
OK?

919
00:53:08,490 --> 00:53:11,650
And I'm going to
move a little bit

920
00:53:11,650 --> 00:53:13,670
between discrete
and continuous here.

921
00:53:13,670 --> 00:53:15,370
It's all going to be fine.

922
00:53:15,370 --> 00:53:21,090
But what I have is, if I just
take that, I can turn this.

923
00:53:21,090 --> 00:53:22,470
This is a recurrence.

924
00:53:22,470 --> 00:53:25,470
I want to turn
that into a series.

925
00:53:25,470 --> 00:53:30,490
So I can say something like 1
minus 1 over t, raised to k,

926
00:53:30,490 --> 00:53:32,060
times n.

927
00:53:32,060 --> 00:53:34,150
And this is the
cardinality of x,

928
00:53:34,150 --> 00:53:36,930
which is the cardinality of x0.

929
00:53:36,930 --> 00:53:38,360
So that's what I have up there.

930
00:53:38,360 --> 00:53:39,860
And that's essentially
what happens.

931
00:53:39,860 --> 00:53:41,790
I constantly shrink
as I go along.

932
00:53:41,790 --> 00:53:44,330
And I have a constant
rate of shrinkage here.

933
00:53:44,330 --> 00:53:45,910
Which is the
conservative part of it.

934
00:53:45,910 --> 00:53:46,790
So keep that in mind.

935
00:53:46,790 --> 00:53:49,920
But it doesn't matter from
an analysis standpoint.

936
00:53:49,920 --> 00:53:50,570
OK?

937
00:53:50,570 --> 00:53:53,000
So if you look at that, and
you say, what happens here?

938
00:53:53,000 --> 00:53:54,583
Well, I can just say
that this is less

939
00:53:54,583 --> 00:53:56,640
than or equal to e
raised to minus--

940
00:53:56,640 --> 00:53:58,140
you knew you were
going to get an e,

941
00:53:58,140 --> 00:54:02,370
because you saw a natural
algorithm here, right?

942
00:54:02,370 --> 00:54:04,700
And so, that's what we got.

943
00:54:06,380 --> 00:54:10,090
And basically, that's it.

944
00:54:10,090 --> 00:54:12,150
You can do a little
bit of algebra.

945
00:54:12,150 --> 00:54:13,930
I'll just write
this out for you.

946
00:54:13,930 --> 00:54:16,740
But I won't really explain it.

947
00:54:16,740 --> 00:54:19,460
You're going to
have Xk equals 0.

948
00:54:19,460 --> 00:54:20,530
You're done.

949
00:54:20,530 --> 00:54:22,570
The cost, of course, is k.

950
00:54:22,570 --> 00:54:23,070
Right?

951
00:54:23,070 --> 00:54:27,410
The cost is k, because
you've selected k subsets.

952
00:54:27,410 --> 00:54:29,470
All right, so that's
your cost, all right?

953
00:54:29,470 --> 00:54:31,360
So, when you get to
the point, you're done.

954
00:54:31,360 --> 00:54:32,760
And the cost is k.

955
00:54:32,760 --> 00:54:35,850
So what you need
is, you need to say

956
00:54:35,850 --> 00:54:39,450
that e raised to
minus kt divided by n

957
00:54:39,450 --> 00:54:41,930
is strictly less than 1.

958
00:54:41,930 --> 00:54:45,690
Because that is
effectively when you have

959
00:54:45,690 --> 00:54:47,150
strictly less than 1 element.

960
00:54:47,150 --> 00:54:47,800
It's discrete.

961
00:54:47,800 --> 00:54:50,030
So that means you have zero
elements left to cover.

962
00:54:50,030 --> 00:54:51,980
That means you're done OK?

963
00:54:51,980 --> 00:54:55,840
So that's your
condition for stopping.

964
00:54:55,840 --> 00:55:00,580
So this done means that e
raised to minus kt times n

965
00:55:00,580 --> 00:55:02,750
is strictly less than 1.

966
00:55:02,750 --> 00:55:06,050
And if you go do
that, you'll get k

967
00:55:06,050 --> 00:55:13,460
over t, just about greater
than natural logarithm of m(n).

968
00:55:13,460 --> 00:55:19,210
The algorithm terminates if we
just do a little manipulation.

969
00:55:19,210 --> 00:55:23,520
And that implies that
k over t less than

970
00:55:23,520 --> 00:55:30,240
or equal to natural logarithm
of n plus 1 is valid.

971
00:55:30,240 --> 00:55:30,740
Right?

972
00:55:30,740 --> 00:55:32,910
k over t is going
to be less than

973
00:55:32,910 --> 00:55:34,990
or equal to natural
logarithm of n plus 1.

974
00:55:34,990 --> 00:55:37,530
Because the instant
it becomes greater,

975
00:55:37,530 --> 00:55:39,120
the algorithm terminates.

976
00:55:39,120 --> 00:55:39,620
Right?

977
00:55:39,620 --> 00:55:41,770
And that's how you got
your proof over here.

978
00:55:41,770 --> 00:55:43,790
Because this is
exactly what we want.

979
00:55:43,790 --> 00:55:48,170
k is our C from way back where
it's the cost of our heuristic

980
00:55:48,170 --> 00:55:50,250
or the cost of
our approximation.

981
00:55:50,250 --> 00:55:52,550
t is the optimum cost.

982
00:55:52,550 --> 00:55:54,040
That's what I defined it as.

983
00:55:54,040 --> 00:55:57,250
And this is a bound on k over t.

984
00:55:57,250 --> 00:55:58,730
All right?

985
00:55:58,730 --> 00:55:59,790
Cool.

986
00:55:59,790 --> 00:56:00,730
Any questions on this?

987
00:56:05,270 --> 00:56:07,000
OK.

988
00:56:07,000 --> 00:56:10,740
So this approximation ratio
gets worse for larger problems,

989
00:56:10,740 --> 00:56:14,410
just like this other
approximation's algorithm

990
00:56:14,410 --> 00:56:17,360
that we didn't actually prove
from the very first problem.

991
00:56:17,360 --> 00:56:20,580
That also got worse.

992
00:56:20,580 --> 00:56:22,520
We had a log k factor for that.

993
00:56:22,520 --> 00:56:24,670
And as your problem
size increased,

994
00:56:24,670 --> 00:56:26,970
obviously the approximation
factor increased.

995
00:56:26,970 --> 00:56:30,430
This is a little
clearer as to what

996
00:56:30,430 --> 00:56:32,550
the increase looks
like in relation

997
00:56:32,550 --> 00:56:35,650
to the original size of n.

998
00:56:35,650 --> 00:56:39,610
So it's just natural
logarithm of n plus 1.

999
00:56:39,610 --> 00:56:42,280
So, so far, we've done
approximation algorithms,

1000
00:56:42,280 --> 00:56:43,650
a couple of different varieties.

1001
00:56:43,650 --> 00:56:45,660
We had a constant
factor one, and then we

1002
00:56:45,660 --> 00:56:50,450
had a row of n that actually
had a dependence on n.

1003
00:56:50,450 --> 00:56:54,700
Now let's move, and we'll do
one last example on partition,

1004
00:56:54,700 --> 00:56:59,560
which it turns out has a trivial
constant factor approximation

1005
00:56:59,560 --> 00:57:01,120
scheme.

1006
00:57:01,120 --> 00:57:02,870
And this obvious
thing, and we'll

1007
00:57:02,870 --> 00:57:04,650
get to that in just a second.

1008
00:57:04,650 --> 00:57:08,250
But what is nice about
partition is you can actually

1009
00:57:08,250 --> 00:57:10,280
get a PTAS, right?

1010
00:57:10,280 --> 00:57:13,740
Polynomial time approximation
scheme, and FPTAS,

1011
00:57:13,740 --> 00:57:16,640
fully polynomial time
approximation scheme,

1012
00:57:16,640 --> 00:57:22,420
that essentially give you with
higher and higher run times.

1013
00:57:22,420 --> 00:57:24,310
They're going to
give you solutions

1014
00:57:24,310 --> 00:57:26,460
that are closer and
closer to optimal, right?

1015
00:57:26,460 --> 00:57:30,780
If you want to do the
FPTAS, we'll do the PTAS.

1016
00:57:30,780 --> 00:57:34,860
So partition is a trivial
little problem to define.

1017
00:57:34,860 --> 00:57:36,500
It's just you have
a set, and you

1018
00:57:36,500 --> 00:57:41,220
want to partition it into two
sets such that they're not

1019
00:57:41,220 --> 00:57:42,140
unbalanced.

1020
00:57:42,140 --> 00:57:47,900
So your cost is the imbalance
between the two sets.

1021
00:57:47,900 --> 00:57:49,570
And you want to
minimize that cost.

1022
00:57:49,570 --> 00:57:53,500
You'd love to have two sets that
are exactly the same weight.

1023
00:57:53,500 --> 00:57:58,610
But if one of them is extremely
unbalanced with respect

1024
00:57:58,610 --> 00:58:00,190
to the other, then it's bad.

1025
00:58:00,190 --> 00:58:02,376
Either way.

1026
00:58:02,376 --> 00:58:03,625
So, here we go with partition.

1027
00:58:10,810 --> 00:58:26,700
Set S of n items with
weights S1 through Sn,

1028
00:58:26,700 --> 00:58:35,804
assume S1 greater than S2, Sn,
without loss of generality.

1029
00:58:35,804 --> 00:58:37,220
So this is just
an ordering thing.

1030
00:58:37,220 --> 00:58:38,803
I mean, obviously,
there's some order.

1031
00:58:38,803 --> 00:58:40,960
I'm not even claiming
uniqueness here.

1032
00:58:40,960 --> 00:58:43,430
I'm just saying just assume
that this is the order.

1033
00:58:43,430 --> 00:58:46,910
The analysis is much
better if you do this--

1034
00:58:46,910 --> 00:58:49,230
if you make this assumption.

1035
00:58:49,230 --> 00:58:57,950
And I want to
partition into A and B

1036
00:58:57,950 --> 00:59:10,840
to minimize max of
sigma I belongs to A. Si

1037
00:59:10,840 --> 00:59:15,650
sigma I belongs to B Si.

1038
00:59:15,650 --> 00:59:25,140
And this is the weight of A.
And this is the weight of B.

1039
00:59:25,140 --> 00:59:29,800
And so there's only
so much you can do.

1040
00:59:29,800 --> 00:59:38,430
If you have 2L equals sigma
I equals 1 through n Si--

1041
00:59:38,430 --> 00:59:42,210
so I'm just calling that 2L--
the sum of all the weights.

1042
00:59:42,210 --> 00:59:47,070
Then my optimum
solution is what?

1043
00:59:47,070 --> 00:59:50,900
What is the lower bound
on the optimum solution?

1044
00:59:50,900 --> 00:59:53,140
If it's 2L?

1045
00:59:53,140 --> 00:59:55,870
What's the trivial lower bound?

1046
00:59:55,870 --> 00:59:57,190
Just L, right?

1047
00:59:57,190 --> 00:59:59,480
So I could have L
here and L there.

1048
00:59:59,480 --> 01:00:06,459
And if I had 2L here
and 0 here, then oh!

1049
01:00:06,459 --> 01:00:07,000
That's right.

1050
01:00:07,000 --> 01:00:07,791
I want to minimize.

1051
01:00:07,791 --> 01:00:10,285
Remember, don't-- maybe
that's what threw you off.

1052
01:00:10,285 --> 01:00:11,650
It threw me off right here.

1053
01:00:11,650 --> 01:00:13,940
I see a max here, and
I got a little worried.

1054
01:00:13,940 --> 01:00:16,450
But I want to minimize
the maximum of these two

1055
01:00:16,450 --> 01:00:18,340
quantities, OK?

1056
01:00:18,340 --> 01:00:22,380
And so the best I could
do is to keep them equal.

1057
01:00:22,380 --> 01:00:24,470
And if I get L for
both of them, that

1058
01:00:24,470 --> 01:00:27,170
would minimize the maximum
of those two quantities.

1059
01:00:27,170 --> 01:00:30,160
If I had 2L and 0, then the
maximum of those two quantities

1060
01:00:30,160 --> 01:00:33,320
is 2L, and obviously I
haven't done any minimization.

1061
01:00:33,320 --> 01:00:42,710
So now you see why there's
a trivial optimum solution

1062
01:00:42,710 --> 01:00:47,270
is greater than or
equal to L. Right?

1063
01:00:47,270 --> 01:00:50,070
And now you see why there's
a trivial two approximation

1064
01:00:50,070 --> 01:00:51,440
algorithm.

1065
01:00:51,440 --> 01:00:54,680
Because the worst I
could do is 2L, right?

1066
01:00:54,680 --> 01:00:57,610
I could dump all of them
on one side, constant time,

1067
01:00:57,610 --> 01:00:59,380
and the other one is 0.

1068
01:00:59,380 --> 01:01:01,440
And my cost is 2L.

1069
01:01:01,440 --> 01:01:04,820
So I'm within a factor of 2
in this problem trivially.

1070
01:01:04,820 --> 01:01:10,000
So, unfortunately that's not
the end of the lecture, OK?

1071
01:01:10,000 --> 01:01:12,100
So, we've got to do better.

1072
01:01:12,100 --> 01:01:13,680
I mean, clearly,
there's more here.

1073
01:01:13,680 --> 01:01:15,590
You'd like to get much closer.

1074
01:01:15,590 --> 01:01:17,180
This is a different
kind of problem.

1075
01:01:17,180 --> 01:01:19,820
And it would be wonderful if
we could get within epsilon.

1076
01:01:19,820 --> 01:01:20,320
Right?

1077
01:01:20,320 --> 01:01:21,040
I'm within 1%.

1078
01:01:21,040 --> 01:01:22,400
How long does it take me?

1079
01:01:22,400 --> 01:01:24,420
I'm within 0.01%, guaranteed.

1080
01:01:24,420 --> 01:01:25,490
How long does it take me?

1081
01:01:25,490 --> 01:01:25,990
Right?

1082
01:01:25,990 --> 01:01:28,470
That's what an approximation
scheme is, as opposed

1083
01:01:28,470 --> 01:01:30,000
to just a plain algorithm.

1084
01:01:30,000 --> 01:01:35,070
So if you're actually going
to talk about epsilon here,

1085
01:01:35,070 --> 01:01:38,380
and we're just doing a PTAS.

1086
01:01:38,380 --> 01:01:42,820
So we're going to see something
that is not polynomial in 1

1087
01:01:42,820 --> 01:01:43,890
over epsilon.

1088
01:01:43,890 --> 01:01:45,680
It's polynomial in n.

1089
01:01:45,680 --> 01:01:49,480
But not polynomial
in 1 over epsilon.

1090
01:01:49,480 --> 01:01:51,970
But there's an FPTAS
with this problem

1091
01:01:51,970 --> 01:01:53,740
that you're not responsible for.

1092
01:02:00,880 --> 01:02:03,130
So this is going to be an
interesting algorithm simply

1093
01:02:03,130 --> 01:02:06,390
because we now have to do
something with epsilon.

1094
01:02:06,390 --> 01:02:07,920
It's going to have
an extra input.

1095
01:02:07,920 --> 01:02:09,740
It's not going to be the
simple heuristic, where

1096
01:02:09,740 --> 01:02:12,239
I'm going to do maximum degree
or maximum number of elements

1097
01:02:12,239 --> 01:02:14,020
or anything like that.

1098
01:02:14,020 --> 01:02:16,671
I want to take this epsilon and
actually do something with it.

1099
01:02:16,671 --> 01:02:17,170
All right?

1100
01:02:17,170 --> 01:02:19,100
So how does this work?

1101
01:02:19,100 --> 01:02:25,430
Well, basically what happens in
PTAS's, or in a bunch of them,

1102
01:02:25,430 --> 01:02:30,390
is that you essentially do
an exponential amount of work

1103
01:02:30,390 --> 01:02:36,100
given a particular epsilon to
get a partial optimum solution.

1104
01:02:36,100 --> 01:02:43,610
So you can think of epsilon
as essentially being 1 divided

1105
01:02:43,610 --> 01:02:47,200
by m plus 1, where
m is some quantity.

1106
01:02:47,200 --> 01:02:51,540
And as m grows, the
complexity of your algorithm

1107
01:02:51,540 --> 01:02:52,840
is going to grow.

1108
01:02:52,840 --> 01:02:54,980
But obviously, as
m grows, you're

1109
01:02:54,980 --> 01:02:57,650
getting a tighter
and tighter epsilon.

1110
01:02:57,650 --> 01:03:01,840
You're getting guaranteed closer
and closer to your optimum.

1111
01:03:01,840 --> 01:03:03,470
And so we're going
to have two phases

1112
01:03:03,470 --> 01:03:08,230
here in this particular
approximation scheme.

1113
01:03:08,230 --> 01:03:19,730
The first phase is find an
optimal partition, A prime, B

1114
01:03:19,730 --> 01:03:26,030
prime, of S1 through Sm.

1115
01:03:26,030 --> 01:03:30,640
And we're just going to assume
that this exhaustive search,

1116
01:03:30,640 --> 01:03:33,930
which looks at all possible
subsets, and picks the best

1117
01:03:33,930 --> 01:03:34,890
one.

1118
01:03:34,890 --> 01:03:35,640
OK?

1119
01:03:35,640 --> 01:03:40,690
And how many subsets are
there for a set of size m?

1120
01:03:40,690 --> 01:03:42,310
It's 2 raised to m.

1121
01:03:42,310 --> 01:03:44,690
So this is going to be
an exponential order, 2

1122
01:03:44,690 --> 01:03:46,950
raised to m algorithm.

1123
01:03:46,950 --> 01:03:47,500
OK?

1124
01:03:47,500 --> 01:03:50,060
I'm just going to find
the optimum partition

1125
01:03:50,060 --> 01:03:52,780
through exhaustive search for m.

1126
01:03:52,780 --> 01:03:54,930
Right? m is less than n.

1127
01:03:59,577 --> 01:04:01,660
So I'm picking something
that's a smaller problem.

1128
01:04:01,660 --> 01:04:02,785
I'm going to seed this.

1129
01:04:02,785 --> 01:04:04,160
So, the way this
scheme works is,

1130
01:04:04,160 --> 01:04:07,760
I'm seeding my
actual algorithm--

1131
01:04:07,760 --> 01:04:13,390
my actual heuristic-- with
an initial partial solution.

1132
01:04:13,390 --> 01:04:19,615
And depending on how much
work I do to create the seed,

1133
01:04:19,615 --> 01:04:22,280
I'm going to end up
having higher complexity.

1134
01:04:22,280 --> 01:04:27,990
And obviously that's a function
of small m, or 1 over epsilon.

1135
01:04:27,990 --> 01:04:35,460
And so this takes-- this optimal
takes order to raise to m time.

1136
01:04:35,460 --> 01:04:38,940
And you can think
of that as order

1137
01:04:38,940 --> 01:04:42,420
2 raised to 1 over epsilon.

1138
01:04:42,420 --> 01:04:46,710
And so that's why it's a
PTAS, and not an FPTAS.

1139
01:04:46,710 --> 01:04:48,450
OK?

1140
01:04:48,450 --> 01:04:49,510
So this is PTAS.

1141
01:04:56,756 --> 01:04:57,880
What else do we need to do?

1142
01:04:57,880 --> 01:05:00,080
Well, I don't actually
have a solution yet.

1143
01:05:00,080 --> 01:05:02,520
Because if m is really
small-- and by the way,

1144
01:05:02,520 --> 01:05:05,690
m can be 0 as well.

1145
01:05:05,690 --> 01:05:06,680
Right?

1146
01:05:06,680 --> 01:05:09,460
Epsilon would then be
at 2 approximation,

1147
01:05:09,460 --> 01:05:12,970
1 divided by-- this
would be one half.

1148
01:05:12,970 --> 01:05:14,710
And so 1 over epsilon is 2.

1149
01:05:14,710 --> 01:05:17,120
So then you've got
your trivial algorithm

1150
01:05:17,120 --> 01:05:19,270
that we had, the 2
approximation scheme.

1151
01:05:19,270 --> 01:05:21,890
So that makes sense?

1152
01:05:21,890 --> 01:05:24,910
So the second phase
is you're going

1153
01:05:24,910 --> 01:05:31,460
to start with your seed
corresponding to A and B.

1154
01:05:31,460 --> 01:05:34,190
You're going to set them
to A prime and B prime.

1155
01:05:34,190 --> 01:05:42,310
And what I'm going to do is,
for I equals m plus 1 to n,

1156
01:05:42,310 --> 01:05:52,670
if w(a) less than
or equal to w(b),

1157
01:05:52,670 --> 01:06:02,540
A equals A union I--
running out of room-- else

1158
01:06:02,540 --> 01:06:10,750
B equals B union I. OK?

1159
01:06:10,750 --> 01:06:14,350
So it's not that hard
to see, hopefully.

1160
01:06:14,350 --> 01:06:18,130
All I'm doing here is, I'm just
going in a very greedy way.

1161
01:06:18,130 --> 01:06:21,370
I got my initial A
prime and B prime.

1162
01:06:21,370 --> 01:06:23,420
I set them to A and
B. And I say, oh, I

1163
01:06:23,420 --> 01:06:24,710
got this element here.

1164
01:06:24,710 --> 01:06:25,829
Which one is bigger?

1165
01:06:25,829 --> 01:06:26,620
This one is bigger?

1166
01:06:26,620 --> 01:06:28,334
I'm going to put the
element over here.

1167
01:06:28,334 --> 01:06:29,500
And then I look at it again.

1168
01:06:29,500 --> 01:06:30,550
I got another element.

1169
01:06:30,550 --> 01:06:31,530
Which one is bigger?

1170
01:06:31,530 --> 01:06:33,060
And I go this way, that way.

1171
01:06:33,060 --> 01:06:34,760
That's pretty much it.

1172
01:06:34,760 --> 01:06:36,200
So, again, all of
these algorithms

1173
01:06:36,200 --> 01:06:38,740
are really straightforward.

1174
01:06:38,740 --> 01:06:42,660
The interesting part's--
the fun part's-- in showing

1175
01:06:42,660 --> 01:06:46,530
the approximation guarantee.

1176
01:06:46,530 --> 01:06:48,610
So we're good here?

1177
01:06:48,610 --> 01:06:50,550
Yup?

1178
01:06:50,550 --> 01:06:51,120
All right.

1179
01:06:51,120 --> 01:06:54,090
So back to analysis.

1180
01:06:54,090 --> 01:06:54,790
One last time.

1181
01:07:00,210 --> 01:07:01,450
So, let's see.

1182
01:07:03,980 --> 01:07:13,030
We want to show that
a prox partition-- ah,

1183
01:07:13,030 --> 01:07:15,670
you know how to spell
partition-- is PTAS.

1184
01:07:18,610 --> 01:07:19,950
I guess I don't, but you do.

1185
01:07:23,200 --> 01:07:27,930
Let's assume that w(a)
is greater than or equal

1186
01:07:27,930 --> 01:07:31,790
to w(b) to end with.

1187
01:07:31,790 --> 01:07:32,290
Right?

1188
01:07:32,290 --> 01:07:34,030
So I'm just saying,
at the end here,

1189
01:07:34,030 --> 01:07:36,540
I'm just marking the one
that was larger that came out

1190
01:07:36,540 --> 01:07:38,610
of the max as A. Right?

1191
01:07:38,610 --> 01:07:40,054
Without loss of generality.

1192
01:07:40,054 --> 01:07:41,720
Just to make things
easier, I don't want

1193
01:07:41,720 --> 01:07:43,740
to keep interchanging things.

1194
01:07:43,740 --> 01:07:53,180
So our approximation ratio
is w(a) divided by L, right?

1195
01:07:53,180 --> 01:07:58,220
w(a) could at best be L if I got
a perfect partition, perfectly

1196
01:07:58,220 --> 01:07:59,220
balanced.

1197
01:07:59,220 --> 01:08:00,690
But it could be a
little bit more.

1198
01:08:00,690 --> 01:08:03,660
And that's my
approximation ratio, OK?

1199
01:08:03,660 --> 01:08:08,250
So I need to now figure
out how the approximation

1200
01:08:08,250 --> 01:08:13,960
ratio reflects on the run
time, and is related to m

1201
01:08:13,960 --> 01:08:16,010
and, therefore, epsilon.

1202
01:08:16,010 --> 01:08:17,710
All right?

1203
01:08:17,710 --> 01:08:19,470
So what I'm going
to do here is, I'm

1204
01:08:19,470 --> 01:08:31,149
going to look at a point in
time where I have A and B--

1205
01:08:31,149 --> 01:08:34,149
and remember that
w(a) is defined

1206
01:08:34,149 --> 01:08:35,830
to be greater than w(b).

1207
01:08:35,830 --> 01:08:38,880
But here, I'm
looking at some point

1208
01:08:38,880 --> 01:08:45,140
in time, which is not
necessarily at the very end

1209
01:08:45,140 --> 01:08:46,859
here.

1210
01:08:46,859 --> 01:08:47,569
It could be.

1211
01:08:47,569 --> 01:08:49,350
But you can think
of this as being,

1212
01:08:49,350 --> 01:08:55,000
I'm just going to say
B or intermediate B.

1213
01:08:55,000 --> 01:08:56,310
And this won't matter too much.

1214
01:08:56,310 --> 01:09:02,189
But I'm just being a little
bit of a stickler here.

1215
01:09:02,189 --> 01:09:05,560
I'll explain why I said
that in just a minute.

1216
01:09:05,560 --> 01:09:10,260
But the point is, I
have a situation where

1217
01:09:10,260 --> 01:09:12,230
I know that w(a) is
greater than w(b)

1218
01:09:12,230 --> 01:09:14,479
or greater than or equal to
w(b) because I assumed it.

1219
01:09:14,479 --> 01:09:15,939
That's how I marked it.

1220
01:09:15,939 --> 01:09:32,450
And I'm going to look at k is
the last element added to A.

1221
01:09:32,450 --> 01:09:34,439
It's been added.

1222
01:09:34,439 --> 01:09:43,859
Now this could have been
added in the first phase

1223
01:09:43,859 --> 01:09:46,120
or the second phase.

1224
01:09:46,120 --> 01:09:50,560
It's quite possible
that for a given m,

1225
01:09:50,560 --> 01:09:57,080
that if it's large, for example,
that A prime that you end up

1226
01:09:57,080 --> 01:10:00,640
with is your A to
begin with here,

1227
01:10:00,640 --> 01:10:05,280
and that you never
execute this statement OK?

1228
01:10:05,280 --> 01:10:07,900
It's quite possible, right?

1229
01:10:07,900 --> 01:10:12,764
You got your initial seed
and never added to it.

1230
01:10:12,764 --> 01:10:13,430
And that was it.

1231
01:10:13,430 --> 01:10:16,940
Because your m was large,
for example, right?

1232
01:10:16,940 --> 01:10:21,090
So what I'm saying here
is, k is the last element

1233
01:10:21,090 --> 01:10:22,854
added to A. OK?

1234
01:10:22,854 --> 01:10:24,270
So there's clearly
a last element.

1235
01:10:24,270 --> 01:10:26,100
I'm just marking that.

1236
01:10:26,100 --> 01:10:29,136
And we know that A is
greater than or equal to B.

1237
01:10:29,136 --> 01:10:32,360
Now it may be true that if
I'm looking at the snapshot

1238
01:10:32,360 --> 01:10:35,590
when just after I add
the k-th element to A,

1239
01:10:35,590 --> 01:10:40,200
I may not be done yet, in terms
of I still have a few elements.

1240
01:10:40,200 --> 01:10:42,910
And I may be adding
elements to B.

1241
01:10:42,910 --> 01:10:45,550
But regardless,
given my definition,

1242
01:10:45,550 --> 01:10:49,200
I know that the rate of B
is less than the rate of A.

1243
01:10:49,200 --> 01:10:52,870
Because even though the
last element overall

1244
01:10:52,870 --> 01:10:58,490
may be added to B,
w(b) is less than w(a),

1245
01:10:58,490 --> 01:11:03,220
and I'm only looking at the
last element added to A here.

1246
01:11:03,220 --> 01:11:04,570
OK?

1247
01:11:04,570 --> 01:11:10,090
So why all of this skulduggery?

1248
01:11:10,090 --> 01:11:16,300
Well, there's a method
here to the madness.

1249
01:11:16,300 --> 01:11:19,590
We're going to analyze
what possibly happens

1250
01:11:19,590 --> 01:11:22,680
in the first phase
and the second phase,

1251
01:11:22,680 --> 01:11:28,768
and get our approximation ratio.

1252
01:11:28,768 --> 01:11:29,768
Shouldn't take too long.

1253
01:11:39,450 --> 01:11:42,150
So there's two cases here
that we need to analyze.

1254
01:11:42,150 --> 01:11:43,370
The first one is easy.

1255
01:11:43,370 --> 01:11:45,250
The second one is a
little more involved.

1256
01:11:45,250 --> 01:11:50,240
I'm going to now assume
that k was the last element

1257
01:11:50,240 --> 01:11:53,620
and was added in
the first phase, OK?

1258
01:11:53,620 --> 01:12:07,840
If k is added to
A, what can I say?

1259
01:12:07,840 --> 01:12:11,426
If k was added in
the first phase to A,

1260
01:12:11,426 --> 01:12:17,440
and that's the last element
added throughout the algorithm,

1261
01:12:17,440 --> 01:12:20,212
by the time you get a
partition, what can you say?

1262
01:12:22,720 --> 01:12:24,220
What can you say
about the solution?

1263
01:12:24,220 --> 01:12:26,960
What strong statement can
you make about the solution?

1264
01:12:29,960 --> 01:12:32,290
What's the only interesting
strong statement

1265
01:12:32,290 --> 01:12:35,770
that you can make
about a solution?

1266
01:12:35,770 --> 01:12:39,540
So I got to A. Remember
what the first phase is.

1267
01:12:39,540 --> 01:12:42,240
What's the first phase?

1268
01:12:42,240 --> 01:12:44,810
Well, it's optimal, right?

1269
01:12:44,810 --> 01:12:49,120
So, after that,
what happened to A?

1270
01:12:49,120 --> 01:12:50,830
Well it didn't change.

1271
01:12:50,830 --> 01:12:51,630
Right?

1272
01:12:51,630 --> 01:12:54,237
So, what you got is optimum.

1273
01:12:54,237 --> 01:12:54,737
Right?

1274
01:12:54,737 --> 01:12:57,950
Because w(a) was defined to be
greater than or equal to w(b).

1275
01:12:57,950 --> 01:13:01,460
w(a) was optimum for the
smaller problem, whatever m was.

1276
01:13:01,460 --> 01:13:03,475
You never added
anything else to it.

1277
01:13:03,475 --> 01:13:04,100
So you're done.

1278
01:13:04,100 --> 01:13:04,990
It's optimum.

1279
01:13:04,990 --> 01:13:07,245
So in this case, your
approximation ratio

1280
01:13:07,245 --> 01:13:11,140
is 1 because you got the
optimum solution, right?

1281
01:13:11,140 --> 01:13:13,310
So if k is added to
A in the first place,

1282
01:13:13,310 --> 01:13:19,560
this means A equals A prime.

1283
01:13:19,560 --> 01:13:24,875
We have an optimal partition.

1284
01:13:29,350 --> 01:13:43,650
Since we can't do
better than w(a) prime

1285
01:13:43,650 --> 01:13:51,590
when we have n
greater than m items.

1286
01:13:51,590 --> 01:14:00,395
And we know w(a) prime is
optimal for the m items.

1287
01:14:03,160 --> 01:14:04,540
OK?

1288
01:14:04,540 --> 01:14:07,000
So that's cool.

1289
01:14:07,000 --> 01:14:07,700
That's good.

1290
01:14:07,700 --> 01:14:12,640
So we got an approximation
ratio of 1 there.

1291
01:14:12,640 --> 01:14:17,090
And remember that, this is not
taking overall exponential time

1292
01:14:17,090 --> 01:14:18,480
necessarily.

1293
01:14:18,480 --> 01:14:23,080
It's just a case where I've
picked some arbitrary m,

1294
01:14:23,080 --> 01:14:27,140
and it happens to be the
case that A equals A prime

1295
01:14:27,140 --> 01:14:28,740
at the end of the algorithm.

1296
01:14:28,740 --> 01:14:34,200
So I am taking exponential
time in m-- m as in Mary--

1297
01:14:34,200 --> 01:14:38,030
but I'm not taking
exponential time in n-- right?

1298
01:14:38,030 --> 01:14:41,290
n as in Nancy.

1299
01:14:41,290 --> 01:14:47,050
So the second part is where the
approximation ratio comes in.

1300
01:14:47,050 --> 01:14:55,105
k is added to A in
the second phase.

1301
01:15:01,092 --> 01:15:02,550
So here, what we're
going to do is,

1302
01:15:02,550 --> 01:15:09,390
we're going to say we know
w(a) minus Sk is less than

1303
01:15:09,390 --> 01:15:12,060
or equal to w(b).

1304
01:15:12,060 --> 01:15:14,640
This is the second phase
we're talking about here.

1305
01:15:14,640 --> 01:15:18,180
The only reason
Sk was added to A

1306
01:15:18,180 --> 01:15:24,060
was because you decided that A
was the side that was smaller,

1307
01:15:24,060 --> 01:15:25,590
right, or perhaps equal.

1308
01:15:25,590 --> 01:15:27,880
So that's the reason
you added into it.

1309
01:15:27,880 --> 01:15:30,620
So you know that w(a) minus
Sk is less than or equal

1310
01:15:30,620 --> 01:15:33,410
to w(b) at that time.

1311
01:15:33,410 --> 01:15:36,783
So think of A and B here
as being variables that

1312
01:15:36,783 --> 01:15:40,200
are obviously changing, right?

1313
01:15:40,200 --> 01:15:44,350
But what I'm saying
here is, even

1314
01:15:44,350 --> 01:15:46,920
if you look-- this is the
algorithm where A and B,

1315
01:15:46,920 --> 01:15:49,600
you constantly look at them
and decide which way to go.

1316
01:15:49,600 --> 01:15:53,360
But if you look
at the last step,

1317
01:15:53,360 --> 01:15:55,830
then you look at
the final values.

1318
01:15:55,830 --> 01:15:58,370
Then you could certainly
make the statement

1319
01:15:58,370 --> 01:16:01,520
for those final values,
that the w(a), which

1320
01:16:01,520 --> 01:16:04,970
is the resultant value, minus
Sk, should have been less than

1321
01:16:04,970 --> 01:16:06,500
or equal to w(b).

1322
01:16:06,500 --> 01:16:10,560
And you had a smaller
w(a) to begin with.

1323
01:16:10,560 --> 01:16:12,300
And you added Sk to it.

1324
01:16:12,300 --> 01:16:15,500
And that happened
in the second phase.

1325
01:16:15,500 --> 01:16:16,540
OK?

1326
01:16:16,540 --> 01:16:18,710
So this is why k was added.

1327
01:16:18,710 --> 01:16:27,750
This is why k was added to A.

1328
01:16:27,750 --> 01:16:29,280
I want to be a
little careful here

1329
01:16:29,280 --> 01:16:32,420
given that we're overloading
A and B without trying

1330
01:16:32,420 --> 01:16:35,480
to point out what each of these
statements actually means.

1331
01:16:35,480 --> 01:16:40,770
And ask questions
if you're confused.

1332
01:16:40,770 --> 01:16:45,670
I know that w(a)
minus Sk is less than

1333
01:16:45,670 --> 01:16:48,956
or equal to 2L minus w(a).

1334
01:16:48,956 --> 01:16:50,720
That's just a substitution.

1335
01:16:50,720 --> 01:16:52,670
Because w(a) plus
w(b) equals 2L.

1336
01:16:59,740 --> 01:17:05,290
And then, last little
trick, again it's algebra.

1337
01:17:05,290 --> 01:17:07,280
Nothing profound here.

1338
01:17:07,280 --> 01:17:11,330
We have our assumption that
we ordered these things.

1339
01:17:11,330 --> 01:17:14,390
So you had S1 through--
Sn, excuse me.

1340
01:17:14,390 --> 01:17:17,040
The whole thing was ordered.

1341
01:17:17,040 --> 01:17:27,750
And so we can say that
S1, S2, all the way to Sm,

1342
01:17:27,750 --> 01:17:30,690
is greater than or equal to Sk.

1343
01:17:30,690 --> 01:17:32,740
We are actually
doing this in order.

1344
01:17:32,740 --> 01:17:34,440
We are taking the
bigger elements

1345
01:17:34,440 --> 01:17:36,150
and then deciding where they go.

1346
01:17:36,150 --> 01:17:38,460
So we sorted those
things initially.

1347
01:17:38,460 --> 01:17:41,830
And so what we end up
with when we look at Sk,

1348
01:17:41,830 --> 01:17:45,250
we have taken care
of values prior to Sk

1349
01:17:45,250 --> 01:17:47,490
that are all greater
than or equal to Sk.

1350
01:17:47,490 --> 01:17:47,990
Right?

1351
01:17:47,990 --> 01:17:52,080
That's, again, using
our initial assumption.

1352
01:17:52,080 --> 01:17:56,340
So what that means
is 2L is greater

1353
01:17:56,340 --> 01:18:05,950
than or equal to m plus 1 Sk
since k is greater than m.

1354
01:18:05,950 --> 01:18:08,650
Again, this is not
particularly tight.

1355
01:18:08,650 --> 01:18:12,289
Because m could be really
pretty small in relation to n.

1356
01:18:12,289 --> 01:18:14,080
But I do know that I
can make the statement

1357
01:18:14,080 --> 01:18:17,980
that since the values
are decreasing,

1358
01:18:17,980 --> 01:18:21,450
that 2L, which is the sum of
all of those, is greater than

1359
01:18:21,450 --> 01:18:25,510
or equal to m plus 1 times
Sk, regardless of what m is.

1360
01:18:25,510 --> 01:18:26,580
Right?

1361
01:18:26,580 --> 01:18:29,250
And so, that's pretty much it.

1362
01:18:29,250 --> 01:18:34,640
Once you do that, you have
your approximation ratio.

1363
01:18:34,640 --> 01:18:37,670
Let's leave that up there
because that's the algorithm.

1364
01:18:37,670 --> 01:18:42,664
Finish this off with
a little algebra.

1365
01:18:52,050 --> 01:18:59,440
So w(a) divided by L is less
than or equal to L plus Sk

1366
01:18:59,440 --> 01:19:03,990
divided by 2, divided by L.
I'm basically substituting.

1367
01:19:03,990 --> 01:19:05,900
I have this and I have that.

1368
01:19:05,900 --> 01:19:08,900
And I'm playing around with it.

1369
01:19:08,900 --> 01:19:16,930
And I got 1 plus
Sk divided by-- 1

1370
01:19:16,930 --> 01:19:23,890
plus Sk divided by 2L, which
is-- now this I could say

1371
01:19:23,890 --> 01:19:25,720
is equal.

1372
01:19:25,720 --> 01:19:27,270
That's simply equal.

1373
01:19:27,270 --> 01:19:29,140
This, I have a less
than or equal to.

1374
01:19:29,140 --> 01:19:31,430
And then I can go less
than or equal to 1

1375
01:19:31,430 --> 01:19:37,830
plus Sk divided by m plus 1 Sk.

1376
01:19:37,830 --> 01:19:44,340
And then I got 1 plus
1 divided by m plus 1,

1377
01:19:44,340 --> 01:19:49,180
which is, of course,
1 plus epsilon.

1378
01:19:49,180 --> 01:19:52,470
So all I did here
was use this fact

1379
01:19:52,470 --> 01:19:55,800
and essentially relate 2L to Sk.

1380
01:19:55,800 --> 01:19:58,250
And once I could
relate 2L to Sk,

1381
01:19:58,250 --> 01:20:07,880
substituting for Sk in here, I
ended up with the approximation

1382
01:20:07,880 --> 01:20:12,490
ratio that I want, w(a) over
L, is L plus Sk divided by 2.

1383
01:20:12,490 --> 01:20:15,190
That simply comes from here.

1384
01:20:15,190 --> 01:20:19,060
And plug that in,
divided by L. And then

1385
01:20:19,060 --> 01:20:21,370
I have Sk divided by 2L.

1386
01:20:21,370 --> 01:20:23,850
And then this part
here, 2L is going

1387
01:20:23,850 --> 01:20:29,580
to get substituted by
m plus 1 Sk and voila.

1388
01:20:29,580 --> 01:20:30,780
I'm over here.

1389
01:20:30,780 --> 01:20:31,640
OK?

1390
01:20:31,640 --> 01:20:37,870
So the story behind
this particular problem

1391
01:20:37,870 --> 01:20:39,840
was it was in the quiz.

1392
01:20:39,840 --> 01:20:42,185
And Eric, I guess took the quiz.

1393
01:20:42,185 --> 01:20:44,010
Did you actually take the quiz?

1394
01:20:44,010 --> 01:20:45,430
Or edit the quiz?

1395
01:20:45,430 --> 01:20:49,330
And he says, this
problem is impossible.

1396
01:20:49,330 --> 01:20:50,400
It was a problem.

1397
01:20:50,400 --> 01:20:52,100
He said, this problem
is impossible.

1398
01:20:52,100 --> 01:20:55,090
I had to Google it to
find out the answer.

1399
01:20:55,090 --> 01:20:56,250
Or something like that.

1400
01:20:56,250 --> 01:20:59,100
And I said, well, I'm going
to give the answer in lecture,

1401
01:20:59,100 --> 01:20:59,600
right?

1402
01:20:59,600 --> 01:21:00,840
So there you go.

1403
01:21:00,840 --> 01:21:01,710
So remember this.

1404
01:21:01,710 --> 01:21:02,520
Write it down.

1405
01:21:02,520 --> 01:21:05,690
Four points for coming
to lecture today.

1406
01:21:05,690 --> 01:21:07,790
See you guys next time.