1
00:00:07,000 --> 00:00:09,000
Good morning,
everyone.

2
00:00:09,000 --> 00:00:14,000
Glad you are all here bright
and early.

3
00:00:14,000 --> 00:00:20,000
I'm counting the days till the
TA's outnumber the students.

4
00:00:20,000 --> 00:00:26,000
They'll show up.
We return to a familiar story.

5
00:00:26,000 --> 00:00:32,000
This is part two,
the Empire Strikes Back.

6
00:00:32,000 --> 00:00:33,000
So last time,
our adversary,

7
00:00:33,000 --> 00:00:36,000
the graph, came to us with a
problem.

8
00:00:36,000 --> 00:00:39,000
We have a source,
and we had a directed graph,

9
00:00:39,000 --> 00:00:43,000
and we had weights on the
edges, and they were all

10
00:00:43,000 --> 00:00:46,000
nonnegative.
And there was happiness.

11
00:00:46,000 --> 00:00:50,000
And we triumphed over the
Empire by designing Dijkstra's

12
00:00:50,000 --> 00:00:54,000
algorithm, and very efficiently
finding single source shortest

13
00:00:54,000 --> 00:01:00,000
paths, shortest path weight from
s to every other vertex.

14
00:01:00,000 --> 00:01:02,000
Today, however,
the Death Star has a new trick

15
00:01:02,000 --> 00:01:05,000
up its sleeve,
and we have negative weights,

16
00:01:05,000 --> 00:01:07,000
potentially.
And we're going to have to

17
00:01:07,000 --> 00:01:09,000
somehow deal with,
in particular,

18
00:01:09,000 --> 00:01:13,000
negative weight cycles.
And we saw that when we have a

19
00:01:13,000 --> 00:01:16,000
negative weight cycle,
we can just keep going around,

20
00:01:16,000 --> 00:01:19,000
and around, and around,
and go back in time farther,

21
00:01:19,000 --> 00:01:21,000
and farther,
and farther.

22
00:01:21,000 --> 00:01:24,000
And we can get to be
arbitrarily far back in the

23
00:01:24,000 --> 00:01:26,000
past.
And so there's no shortest

24
00:01:26,000 --> 00:01:29,000
path, because whatever path you
take you can get a shorter one.

25
00:01:29,000 --> 00:01:33,000
So we want to address that
issue today, and we're going to

26
00:01:33,000 --> 00:01:37,000
come up with a new algorithm
actually simpler than Dijkstra,

27
00:01:37,000 --> 00:01:39,000
but not as fast,
called the Bellman-Ford

28
00:01:39,000 --> 00:01:44,000
algorithm.
And, it's going to allow

29
00:01:44,000 --> 00:01:48,000
negative weights,
and in some sense allow

30
00:01:48,000 --> 00:01:54,000
negative weight cycles,
although maybe not as much as

31
00:01:54,000 --> 00:01:59,000
you might hope.
We have to leave room for a

32
00:01:59,000 --> 00:02:04,000
sequel, of course.
OK, so the Bellman-Ford

33
00:02:04,000 --> 00:02:09,000
algorithm, invented by two guys,
as you might expect,

34
00:02:09,000 --> 00:02:13,000
it computes the shortest path
weights.

35
00:02:13,000 --> 00:02:17,000
So, it makes no assumption
about the weights.

36
00:02:17,000 --> 00:02:22,000
Weights are arbitrary,
and it's going to compute the

37
00:02:22,000 --> 00:02:27,000
shortest path weights.
So, remember this notation:

38
00:02:27,000 --> 00:02:33,000
delta of s, v is the weight of
the shortest path from s to v.

39
00:02:33,000 --> 00:02:40,000
s was called a source vertex.
And, we want to compute these

40
00:02:40,000 --> 00:02:43,000
weights for all vertices,
little v.

41
00:02:43,000 --> 00:02:47,000
The claim is that computing
from s to everywhere is no

42
00:02:47,000 --> 00:02:51,000
harder than computing s to a
particular location.

43
00:02:51,000 --> 00:02:53,000
So, we're going to do for all
them.

44
00:02:53,000 --> 00:02:56,000
It's still going to be the case
here.

45
00:02:56,000 --> 00:02:59,000
And, it allows negative
weights.

46
00:02:59,000 --> 00:03:03,000
And this is the good case,
but there's an alternative,

47
00:03:03,000 --> 00:03:07,000
which is that Bellman-Ford may
just say, oops,

48
00:03:07,000 --> 00:03:11,000
there's a negative weight
cycle.

49
00:03:11,000 --> 00:03:14,000
And in that case it will just
say so.

50
00:03:14,000 --> 00:03:18,000
So, they say a negative weight
cycle exists.

51
00:03:18,000 --> 00:03:23,000
Therefore, some of these deltas
are minus infinity.

52
00:03:23,000 --> 00:03:27,000
And that seems weird.
So, Bellman-Ford as we'll

53
00:03:27,000 --> 00:03:33,000
present it today is intended for
the case, but there are no

54
00:03:33,000 --> 00:03:39,000
negative weights cycles,
which is more intuitive.

55
00:03:39,000 --> 00:03:42,000
It sort of allows them,
but it will just report them.

56
00:03:42,000 --> 00:03:45,000
In that case,
it will not give you delta

57
00:03:45,000 --> 00:03:48,000
values.
You can change the algorithm to

58
00:03:48,000 --> 00:03:52,000
give you delta values in that
case, but we are not going to

59
00:03:52,000 --> 00:03:54,000
see it here.
So, in exercise,

60
00:03:54,000 --> 00:03:57,000
after you see the algorithm,
exercise is:

61
00:03:57,000 --> 00:04:01,000
compute these deltas in all
cases.

62
00:04:12,000 --> 00:04:19,000
So, it's not hard to do.
But we don't have time for it

63
00:04:19,000 --> 00:04:24,000
here.
So, here's the algorithm.

64
00:04:24,000 --> 00:04:32,000
It's pretty straightforward.
As I said, it's easier than

65
00:04:32,000 --> 00:04:36,000
Dijkstra.
It's a relaxation algorithm.

66
00:04:36,000 --> 00:04:40,000
So the main thing that it does
is relax edges just like

67
00:04:40,000 --> 00:04:43,000
Dijkstra.
So, we'll be able to use a lot

68
00:04:43,000 --> 00:04:47,000
of dilemmas from Dijkstra.
And proof of correctness will

69
00:04:47,000 --> 00:04:51,000
be three times shorter because
the first two thirds we already

70
00:04:51,000 --> 00:04:55,000
have from Dijkstra.
But I'm jumping ahead a bit.

71
00:04:55,000 --> 00:04:57,000
So, the first part is
initialization.

72
00:04:57,000 --> 00:05:01,000
Again, d of v will represent
the estimated distance from s to

73
00:05:01,000 --> 00:05:05,000
v.
And we're going to be updating

74
00:05:05,000 --> 00:05:08,000
those estimates as the algorithm
goes along.

75
00:05:08,000 --> 00:05:10,000
And initially,
d of s is zero,

76
00:05:10,000 --> 00:05:14,000
which now may not be the right
answer conceivably.

77
00:05:14,000 --> 00:05:17,000
Everyone else is infinity,
which is certainly an upper

78
00:05:17,000 --> 00:05:20,000
bound.
OK, these are both upper bounds

79
00:05:20,000 --> 00:05:23,000
on the true distance.
So that's fine.

80
00:05:23,000 --> 00:05:27,000
That's initialization just like
before.

81
00:05:36,000 --> 00:05:39,000
And now we have a main loop
which happens v minus one times.

82
00:05:39,000 --> 00:05:41,000
We're not actually going to use
the index i.

83
00:05:41,000 --> 00:05:43,000
It's just a counter.

84
00:06:02,000 --> 00:06:07,000
And we're just going to look at
every edge and relax it.

85
00:06:07,000 --> 00:06:13,000
It's a very simple idea.
If you learn about relaxation,

86
00:06:13,000 --> 00:06:16,000
this is the first thing you
might try.

87
00:06:16,000 --> 00:06:20,000
The question is when do you
stop.

88
00:06:20,000 --> 00:06:25,000
It's sort of like I have this
friend to what he was like six

89
00:06:25,000 --> 00:06:31,000
years old he would claim,
oh, I know how to spell banana.

90
00:06:31,000 --> 00:06:37,000
I just don't know when to stop.
OK, same thing with relaxation.

91
00:06:37,000 --> 00:06:40,000
This is our relaxation step
just as before.

92
00:06:40,000 --> 00:06:43,000
We look at the edge;
we see whether it violates the

93
00:06:43,000 --> 00:06:47,000
triangle inequality according to
our current estimates we know

94
00:06:47,000 --> 00:06:51,000
the distance from s to v should
be at most distance from s to

95
00:06:51,000 --> 00:06:54,000
plus the weight of that edge
from u to v.

96
00:06:54,000 --> 00:06:55,000
If it isn't,
we set it equal.

97
00:06:55,000 --> 00:07:00,000
We've proved that this is
always an OK thing to do.

98
00:07:00,000 --> 00:07:03,000
We never violate,
I mean, these d of v's never

99
00:07:03,000 --> 00:07:07,000
get too small if we do a bunch
of relaxations.

100
00:07:07,000 --> 00:07:09,000
So, the idea is you take every
edge.

101
00:07:09,000 --> 00:07:12,000
You relax it.
I don't care which order.

102
00:07:12,000 --> 00:07:15,000
Just relax every edge,
one each.

103
00:07:15,000 --> 00:07:17,000
And that do that V minus one
times.

104
00:07:17,000 --> 00:07:21,000
The claim is that that should
be enough if you have no

105
00:07:21,000 --> 00:07:25,000
negative weights cycles.
So, if there's a negative

106
00:07:25,000 --> 00:07:30,000
weight cycle,
we need to figure it out.

107
00:07:30,000 --> 00:07:35,000
And, we'll do that in a fairly
straightforward way,

108
00:07:35,000 --> 00:07:40,000
which is we're going to do
exactly the same thing.

109
00:07:40,000 --> 00:07:44,000
So this is outside before loop
here.

110
00:07:44,000 --> 00:07:50,000
We'll have the same four loops
for each edge in our graph.

111
00:07:50,000 --> 00:07:54,000
We'll try to relax it.
And if you can relax it,

112
00:07:54,000 --> 00:08:02,000
the claim is that there has to
be a negative weight cycle.

113
00:08:02,000 --> 00:08:04,000
So this is the main thing that
needs proof.

114
00:08:28,000 --> 00:08:31,000
OK, and that's the algorithm.
So the claim is that at the

115
00:08:31,000 --> 00:08:35,000
ends we should have d of v,
let's see, L's so to speak.

116
00:08:35,000 --> 00:08:38,000
d of v equals delta of s comma
v for every vertex,

117
00:08:38,000 --> 00:08:40,000
v.
If we don't find a negative

118
00:08:40,000 --> 00:08:44,000
weight cycle according to this
rule, that we should have all

119
00:08:44,000 --> 00:08:47,000
the shortest path weights.
That's the claim.

120
00:08:47,000 --> 00:08:50,000
Now, the first question is,
in here, the running time is

121
00:08:50,000 --> 00:08:54,000
very easy to analyze.
So let's start with the running

122
00:08:54,000 --> 00:08:56,000
time.
We can compare it to Dijkstra,

123
00:08:56,000 --> 00:09:02,000
which is over here.
What is the running time of

124
00:09:02,000 --> 00:09:06,000
this algorithm?
V times E, exactly.

125
00:09:06,000 --> 00:09:12,000
OK, I'm going to assume,
because it's pretty reasonable,

126
00:09:12,000 --> 00:09:19,000
that V and E are both positive.
Then it's V times E.

127
00:09:19,000 --> 00:09:25,000
So, this is a little bit
slower, or a fair amount slower,

128
00:09:25,000 --> 00:09:30,000
than Dijkstra's algorithm.
There it is:

129
00:09:30,000 --> 00:09:35,000
E plus V log V is essentially,
ignoring the logs is pretty

130
00:09:35,000 --> 00:09:39,000
much linear time.
Here we have something that's

131
00:09:39,000 --> 00:09:43,000
at least quadratic in V,
assuming your graph is

132
00:09:43,000 --> 00:09:45,000
connected.
So, it's slower,

133
00:09:45,000 --> 00:09:48,000
but it's going to handle these
negative weights.

134
00:09:48,000 --> 00:09:52,000
Dijkstra can't handle negative
weights at all.

135
00:09:52,000 --> 00:09:56,000
So, let's do an example,
make it clear why you might

136
00:09:56,000 --> 00:10:03,000
hope this algorithm works.
And then we'll prove that it

137
00:10:03,000 --> 00:10:08,000
works, of course.
But the proof will be pretty

138
00:10:08,000 --> 00:10:12,000
easy.
So, I'm going to draw a graph

139
00:10:12,000 --> 00:10:18,000
that has negative weights,
but no negative weight cycles

140
00:10:18,000 --> 00:10:24,000
so that I get an interesting
answer.

141
00:10:55,000 --> 00:10:57,000
Good.
The other thing I need in order

142
00:10:57,000 --> 00:11:00,000
to make the output of this
algorithm well defined,

143
00:11:00,000 --> 00:11:03,000
it depends in which order you
visit the edges.

144
00:11:03,000 --> 00:11:07,000
So I'm going to assign an
arbitrary order to these edges.

145
00:11:07,000 --> 00:11:11,000
I could just ask you for an
order, but to be consistent with

146
00:11:11,000 --> 00:11:13,000
the notes, I'll put an ordering
on it.

147
00:11:13,000 --> 00:11:17,000
Let's say I put number four,
say that's the fourth edge I'll

148
00:11:17,000 --> 00:11:18,000
visit.
It doesn't matter.

149
00:11:18,000 --> 00:11:22,000
But it will affect what happens
during the algorithm for a

150
00:11:22,000 --> 00:11:25,000
particular graph.

151
00:11:43,000 --> 00:11:46,000
Do they get them all?
One, two, three,

152
00:11:46,000 --> 00:11:48,000
four, five, six,
seven, eight,

153
00:11:48,000 --> 00:11:51,000
OK.
And my source is going to be A.

154
00:11:51,000 --> 00:11:54,000
And, that's it.
So, I want to run this

155
00:11:54,000 --> 00:11:57,000
algorithm.
I'm just going to initialize

156
00:11:57,000 --> 00:12:01,000
everything.
So, I set the estimates for s

157
00:12:01,000 --> 00:12:06,000
to be zero, and everyone else to
be infinity.

158
00:12:06,000 --> 00:12:10,000
And to give me some notion of
time, over here I'm going to

159
00:12:10,000 --> 00:12:15,000
draw or write down what all of
these d values are as the

160
00:12:15,000 --> 00:12:20,000
algorithm proceeds because I'm
going to start crossing them out

161
00:12:20,000 --> 00:12:25,000
and rewriting them that the
figure will get a little bit

162
00:12:25,000 --> 00:12:28,000
messier.
But we can keep track of it

163
00:12:28,000 --> 00:12:31,000
over here.
It's initially zero and

164
00:12:31,000 --> 00:12:34,000
infinities.
Yeah?

165
00:12:34,000 --> 00:12:36,000
It doesn't matter.
So, for the algorithm you can

166
00:12:36,000 --> 00:12:40,000
go to the edges in a different
order every time if you want.

167
00:12:40,000 --> 00:12:42,000
We'll prove that,
but here I'm going to go

168
00:12:42,000 --> 00:12:44,000
through the same order every
time.

169
00:12:44,000 --> 00:12:47,000
Good question.
It turns out it doesn't matter

170
00:12:47,000 --> 00:12:49,000
here.
OK, so here's the starting

171
00:12:49,000 --> 00:12:51,000
point.
Now I'm going to relax every

172
00:12:51,000 --> 00:12:53,000
edge.
So, there's going to be a lot

173
00:12:53,000 --> 00:12:55,000
of edges here that don't do
anything.

174
00:12:55,000 --> 00:12:57,000
I try to relax n minus one.
I'd say, well,

175
00:12:57,000 --> 00:13:02,000
I know how to get from s to B
with weight infinity.

176
00:13:02,000 --> 00:13:04,000
Infinity plus two I can get to
from s to E.

177
00:13:04,000 --> 00:13:08,000
Well, infinity plus two is not
much better than infinity.

178
00:13:08,000 --> 00:13:11,000
OK, so I don't do anything,
don't update this to infinity.

179
00:13:11,000 --> 00:13:14,000
I mean, infinity plus two
sounds even worse.

180
00:13:14,000 --> 00:13:16,000
But infinity plus two is
infinity.

181
00:13:16,000 --> 00:13:20,000
OK, that's the edge number one.
So, no relaxation edge number

182
00:13:20,000 --> 00:13:24,000
two, same deal as number three,
same deal, edge number four we

183
00:13:24,000 --> 00:13:27,000
start to get something
interesting because I have a

184
00:13:27,000 --> 00:13:31,000
finite value here that says I
can get from A to B using a

185
00:13:31,000 --> 00:13:35,000
total weight of minus one.
So that seems good.

186
00:13:35,000 --> 00:13:41,000
I'll write down minus one here,
and update B to minus one.

187
00:13:41,000 --> 00:13:45,000
The rest stay the same.
So, I'm just going to keep

188
00:13:45,000 --> 00:13:50,000
doing this over and over.
That was edge number four.

189
00:13:50,000 --> 00:13:53,000
Number five,
we also get a relaxation.

190
00:13:53,000 --> 00:14:00,000
Four is better than infinity.
So, c gets a number of four.

191
00:14:00,000 --> 00:14:04,000
Then we get to edge number six.
That's infinity plus five is

192
00:14:04,000 --> 00:14:07,000
worse than four.
OK, so no relaxation there.

193
00:14:07,000 --> 00:14:11,000
Edge number seven is
interesting because I have a

194
00:14:11,000 --> 00:14:15,000
finite value here minus one plus
the weight of this edge,

195
00:14:15,000 --> 00:14:18,000
which is three.
That's a total of two,

196
00:14:18,000 --> 00:14:20,000
which is actually better than
four.

197
00:14:20,000 --> 00:14:24,000
So, this route,
A, B, c is actually better than

198
00:14:24,000 --> 00:14:26,000
the route I just found a second
ago.

199
00:14:26,000 --> 00:14:30,000
So, this is now a two.
This is all happening in one

200
00:14:30,000 --> 00:14:35,000
iteration of the main loop.
We actually found two good

201
00:14:35,000 --> 00:14:38,000
paths to c.
We found one better than the

202
00:14:38,000 --> 00:14:41,000
other.
OK, and that was edge number

203
00:14:41,000 --> 00:14:44,000
seven, and edge number eight is
over here.

204
00:14:44,000 --> 00:14:47,000
It doesn't matter.
OK, so that was round one of

205
00:14:47,000 --> 00:14:50,000
this outer loop,
so, the first value of i.

206
00:14:50,000 --> 00:14:52,000
i equals one.
OK, now we continue.

207
00:14:52,000 --> 00:14:56,000
Just keep going.
So, we start with edge number

208
00:14:56,000 --> 00:15:00,000
one.
Now, minus one plus two is one.

209
00:15:00,000 --> 00:15:04,000
That's better than infinity.
It'll start speeding up.

210
00:15:04,000 --> 00:15:08,000
It's repetitive.
It's actually not too much

211
00:15:08,000 --> 00:15:14,000
longer until we're done.
Number two, this is an infinity

212
00:15:14,000 --> 00:15:17,000
so we don't do anything.
Number three:

213
00:15:17,000 --> 00:15:22,000
minus one plus two is one;
better than infinity.

214
00:15:22,000 --> 00:15:25,000
This is vertex d,
and it's number three.

215
00:15:25,000 --> 00:15:31,000
Number four we've already done.
Nothing changed.

216
00:15:31,000 --> 00:15:35,000
Number five:
this is where we see the path

217
00:15:35,000 --> 00:15:38,000
four again, but that's worse
than two.

218
00:15:38,000 --> 00:15:43,000
So, we don't update anything.
Number six: one plus five is

219
00:15:43,000 --> 00:15:47,000
six, which is bigger than two,
so no good.

220
00:15:47,000 --> 00:15:49,000
Go around this way.
Number seven:

221
00:15:49,000 --> 00:15:53,000
same deal.
Number eight is interesting.

222
00:15:53,000 --> 00:15:58,000
So, we have a weight of one
here, a weight of minus three

223
00:15:58,000 --> 00:16:02,000
here.
So, the total is minus two,

224
00:16:02,000 --> 00:16:07,000
which is better than one.
So, that was d.

225
00:16:07,000 --> 00:16:13,000
And, I believe that's it.
So that was definitely the end

226
00:16:13,000 --> 00:16:18,000
of that round.
So, it's I plus two because we

227
00:16:18,000 --> 00:16:24,000
just looked at the eighth edge.
And, I'll cheat and check.

228
00:16:24,000 --> 00:16:30,000
Indeed, that is the last thing
that happens.

229
00:16:30,000 --> 00:16:33,000
We can check the couple of
outgoing edges from d because

230
00:16:33,000 --> 00:16:36,000
that's the only one whose value
just changed.

231
00:16:36,000 --> 00:16:39,000
And, there are no more
relaxations possible.

232
00:16:39,000 --> 00:16:43,000
So, that was in two rounds.
The claim is we got all the

233
00:16:43,000 --> 00:16:47,000
shortest path weights.
The algorithm would actually

234
00:16:47,000 --> 00:16:51,000
loop four times to guarantee
correctness because we have five

235
00:16:51,000 --> 00:16:53,000
vertices here and one less than
that.

236
00:16:53,000 --> 00:16:56,000
So, in fact,
in the execution here there are

237
00:16:56,000 --> 00:16:59,000
two more blank rounds at the
bottom.

238
00:16:59,000 --> 00:17:03,000
Nothing happens.
But, what the hell?

239
00:17:03,000 --> 00:17:06,000
OK, so that is Bellman-Ford.
I mean, it's certainly not

240
00:17:06,000 --> 00:17:08,000
doing anything wrong.
The question is,

241
00:17:08,000 --> 00:17:11,000
why is it guaranteed to
converge in V minus one steps

242
00:17:11,000 --> 00:17:13,000
unless there is a negative
weight cycle?

243
00:17:13,000 --> 00:17:15,000
Question?

244
00:17:24,000 --> 00:17:25,000
Right, so that's an
optimization.

245
00:17:25,000 --> 00:17:28,000
If you discover a whole round,
and nothing happens,

246
00:17:28,000 --> 00:17:31,000
so you can keep track of that
in the algorithm thing,

247
00:17:31,000 --> 00:17:33,000
you can stop.
In the worst case,

248
00:17:33,000 --> 00:17:35,000
it won't make a difference.
But in practice,

249
00:17:35,000 --> 00:17:37,000
you probably want to do that.
Yeah?

250
00:17:37,000 --> 00:17:40,000
Good question.
All right, so some simple

251
00:17:40,000 --> 00:17:42,000
observations,
I mean, we're only doing

252
00:17:42,000 --> 00:17:44,000
relaxation.
So, we can use a lot of our

253
00:17:44,000 --> 00:17:46,000
analysis from before.
In particular,

254
00:17:46,000 --> 00:17:49,000
the d values are only
decreasing monotonically.

255
00:17:49,000 --> 00:17:51,000
As we cross out values here,
we are always making it

256
00:17:51,000 --> 00:17:54,000
smaller, which is good.
Another nifty thing about this

257
00:17:54,000 --> 00:18:00,000
algorithm is that you can run it
even in a distributed system.

258
00:18:00,000 --> 00:18:02,000
If this is some actual network,
some computer network,

259
00:18:02,000 --> 00:18:05,000
and these are machines,
and they're communicating by

260
00:18:05,000 --> 00:18:07,000
these links, I mean,
it's a purely local thing.

261
00:18:07,000 --> 00:18:09,000
Relaxation is a local thing.
You don't need any global

262
00:18:09,000 --> 00:18:12,000
strategy, and you're asking
about, can we do a different

263
00:18:12,000 --> 00:18:15,000
order in each step?
Well, yeah, you could just keep

264
00:18:15,000 --> 00:18:16,000
relaxing edges,
and keep relaxing edges,

265
00:18:16,000 --> 00:18:19,000
and just keep going for the
entire lifetime of the network.

266
00:18:19,000 --> 00:18:21,000
And eventually,
you will find shortest paths.

267
00:18:21,000 --> 00:18:24,000
So, this algorithm is
guaranteed to finish in V rounds

268
00:18:24,000 --> 00:18:27,000
in a distributed system.
It might be more asynchronous.

269
00:18:27,000 --> 00:18:30,000
And, it's a little harder to
analyze.

270
00:18:30,000 --> 00:18:34,000
But it will still work
eventually.

271
00:18:34,000 --> 00:18:41,000
It's guaranteed to converge.
And so, Bellman-Ford is used in

272
00:18:41,000 --> 00:18:46,000
the Internet for finding
shortest paths.

273
00:18:46,000 --> 00:18:51,000
OK, so let's finally prove that
it works.

274
00:18:51,000 --> 00:18:56,000
This should only take a couple
of boards.

275
00:18:56,000 --> 00:19:03,000
So let's suppose we have a
graph and some edge weights that

276
00:19:03,000 --> 00:19:13,000
have no negative weight cycles.
Then the claim is that we

277
00:19:13,000 --> 00:19:19,000
terminate with the correct
answer.

278
00:19:19,000 --> 00:19:29,000
So, Bellman-Ford terminates
with all of these d of v values

279
00:19:29,000 --> 00:19:38,000
set to the delta values for
every vertex.

280
00:19:38,000 --> 00:19:42,000
OK, the proof is going to be
pretty immediate using the

281
00:19:42,000 --> 00:19:45,000
lemmas that we had from before
if you remember them.

282
00:19:45,000 --> 00:19:50,000
So, we're just going to look at
every vertex separately.

283
00:19:50,000 --> 00:19:54,000
So, I'll call the vertex v.
The claim is that this holds by

284
00:19:54,000 --> 00:19:58,000
the end of the algorithm.
So, remember what we need to

285
00:19:58,000 --> 00:20:02,000
prove is that at some point,
d of v equals delta of s comma

286
00:20:02,000 --> 00:20:06,000
v because we know it decreases
monotonically,

287
00:20:06,000 --> 00:20:10,000
and we know that it never gets
any smaller than the correct

288
00:20:10,000 --> 00:20:15,000
value because relaxations are
always safe.

289
00:20:15,000 --> 00:20:24,000
So, we just need to show at
some point this holds,

290
00:20:24,000 --> 00:20:32,000
and that it will hold at the
end.

291
00:20:32,000 --> 00:20:41,000
So, by monotonicity of the d
values, and by correctness part

292
00:20:41,000 --> 00:20:51,000
one, which was that the d of v's
are always greater than or equal

293
00:20:51,000 --> 00:20:58,000
to the deltas,
we only need to show that at

294
00:20:58,000 --> 00:21:04,000
some point we have equality.

295
00:21:18,000 --> 00:21:21,000
So that's our goal.
So what we're going to do is

296
00:21:21,000 --> 00:21:24,000
just look at v,
and the shortest path to v,

297
00:21:24,000 --> 00:21:30,000
and see what happens to the
algorithm relative to that path.

298
00:21:30,000 --> 00:21:35,000
So, I'm going to name the path.
Let's call it p.

299
00:21:35,000 --> 00:21:40,000
It starts at vertex v_0 and
goes to v_1, v_2,

300
00:21:40,000 --> 00:21:46,000
whatever, and ends at v_k.
And, this is not just any

301
00:21:46,000 --> 00:21:51,000
shortest path,
but it's one that starts at s.

302
00:21:51,000 --> 00:21:54,000
So, v_0's s,
and it ends at v.

303
00:21:54,000 --> 00:22:01,000
So, I'm going to give a couple
of names to s and v so I can

304
00:22:01,000 --> 00:22:04,000
talk about the path more
uniformly.

305
00:22:04,000 --> 00:22:11,000
So, this is a shortest path
from s to v.

306
00:22:11,000 --> 00:22:15,000
Now, I also want it to be not
just any shortest path from s to

307
00:22:15,000 --> 00:22:20,000
v, but among all shortest paths
from s to v I want it to be one

308
00:22:20,000 --> 00:22:23,000
with the fewest possible edges.

309
00:22:32,000 --> 00:22:36,000
OK, so shortest here means in
terms of the total weight of the

310
00:22:36,000 --> 00:22:38,000
path.
Subject to being shortest in

311
00:22:38,000 --> 00:22:42,000
weight, I wanted to also be
shortest in the number of edges.

312
00:22:42,000 --> 00:22:46,000
And, the reason I want that is
to be able to conclude that p is

313
00:22:46,000 --> 00:22:50,000
a simple path,
meaning that it doesn't repeat

314
00:22:50,000 --> 00:22:52,000
any vertices.
Now, can anyone tell me why I

315
00:22:52,000 --> 00:22:56,000
need to assume that the number
of edges is the smallest

316
00:22:56,000 --> 00:23:01,000
possible in order to guarantee
that p is simple?

317
00:23:01,000 --> 00:23:04,000
The claim is that not all
shortest paths are necessarily

318
00:23:04,000 --> 00:23:05,000
simple.
Yeah?

319
00:23:05,000 --> 00:23:07,000
Right, I can have a zero weight
cycle, exactly.

320
00:23:07,000 --> 00:23:10,000
So, we are hoping,
I mean, in fact in the theorem

321
00:23:10,000 --> 00:23:14,000
here, we're assuming that there
are no negative weight cycles.

322
00:23:14,000 --> 00:23:17,000
But there might be zero weight
cycles still.

323
00:23:17,000 --> 00:23:20,000
As a zero weight cycle,
you can put that in the middle

324
00:23:20,000 --> 00:23:23,000
of any shortest path to make it
arbitrarily long,

325
00:23:23,000 --> 00:23:26,000
repeat vertices over and over.
That's going to be annoying.

326
00:23:26,000 --> 00:23:30,000
What I want is that p is
simple.

327
00:23:30,000 --> 00:23:33,000
And, I can guarantee that
essentially by shortcutting.

328
00:23:33,000 --> 00:23:36,000
If ever I take a zero weight
cycle, I throw it away.

329
00:23:36,000 --> 00:23:39,000
And this is one mathematical
way of doing that.

330
00:23:39,000 --> 00:23:43,000
OK, now what else do we know
about this shortest path?

331
00:23:43,000 --> 00:23:47,000
Well, we know that subpaths are
shortest paths are shortest

332
00:23:47,000 --> 00:23:49,000
paths.
That's optimal substructure.

333
00:23:49,000 --> 00:23:53,000
So, we know what the shortest
path from s to v_i is sort of

334
00:23:53,000 --> 00:23:55,000
inductively.
It's the shortest path,

335
00:23:55,000 --> 00:23:58,000
I mean, it's the weight of that
path, which is,

336
00:23:58,000 --> 00:24:01,000
in particular,
the shortest path from s to v

337
00:24:01,000 --> 00:24:07,000
minus one plus the weight of the
last edge, v minus one to v_i.

338
00:24:07,000 --> 00:24:17,000
So, this is by optimal
substructure as we proved last

339
00:24:17,000 --> 00:24:23,000
time.
OK, and I think that's pretty

340
00:24:23,000 --> 00:24:30,000
much the warm-up.
So, I want to sort of do this

341
00:24:30,000 --> 00:24:33,000
inductively in I,
start out with v zero,

342
00:24:33,000 --> 00:24:37,000
and go up to v_k.
So, the first question is,

343
00:24:37,000 --> 00:24:40,000
what is d of v_0,
which is s?

344
00:24:40,000 --> 00:24:44,000
What is d of the source?
Well, certainly at the

345
00:24:44,000 --> 00:24:47,000
beginning of the algorithm,
it's zero.

346
00:24:47,000 --> 00:24:52,000
So, let's say equals zero
initially because that's what we

347
00:24:52,000 --> 00:24:55,000
set it to.
And it only goes down from

348
00:24:55,000 --> 00:24:57,000
there.
So, it certainly,

349
00:24:57,000 --> 00:25:01,000
at most, zero.
The real question is,

350
00:25:01,000 --> 00:25:06,000
what is delta of s comma v_0.
What is the shortest path

351
00:25:06,000 --> 00:25:09,000
weight from s to s?
It has to be zero,

352
00:25:09,000 --> 00:25:13,000
otherwise you have a negative
weight cycle,

353
00:25:13,000 --> 00:25:15,000
exactly.
My favorite answer,

354
00:25:15,000 --> 00:25:19,000
zero.
So, if we had another path from

355
00:25:19,000 --> 00:25:21,000
s to s, I mean,
that is a cycle.

356
00:25:21,000 --> 00:25:26,000
So, it's got to be zero.
So, these are actually equal at

357
00:25:26,000 --> 00:25:32,000
the beginning of the algorithm,
which is great.

358
00:25:32,000 --> 00:25:37,000
That means they will be for all
time because we just argued up

359
00:25:37,000 --> 00:25:41,000
here, only goes down,
never can get too small.

360
00:25:41,000 --> 00:25:45,000
So, we have d of v_0 set to the
right thing.

361
00:25:45,000 --> 00:25:49,000
Great: good for the base case
of the induction.

362
00:25:49,000 --> 00:25:53,000
Of course, what we really care
about is v_k,

363
00:25:53,000 --> 00:25:56,000
which is v.
So, let's talk about the v_i

364
00:25:56,000 --> 00:26:02,000
inductively, and then we will
get v_k as a result.

365
00:26:11,000 --> 00:26:14,000
So, yeah, let's do it by
induction.

366
00:26:14,000 --> 00:26:16,000
That's more fun.

367
00:26:27,000 --> 00:26:32,000
Let's say that d of v_i is
equal to delta of s v_i after I

368
00:26:32,000 --> 00:26:38,000
rounds of the algorithm.
So, this is actually referring

369
00:26:38,000 --> 00:26:42,000
to the I that is in the
algorithm here.

370
00:26:42,000 --> 00:26:46,000
These are rounds.
So, one round is an entire

371
00:26:46,000 --> 00:26:52,000
execution of all the edges,
relaxation of all the edges.

372
00:26:52,000 --> 00:26:56,000
So, this is certainly true for
I equals zero.

373
00:26:56,000 --> 00:27:00,000
We just proved that.
After zero rounds,

374
00:27:00,000 --> 00:27:06,000
at the beginning of the
algorithm, d of v_0 equals delta

375
00:27:06,000 --> 00:27:11,000
of s, v_0.
OK, so now, that's not really

376
00:27:11,000 --> 00:27:13,000
what I wanted,
but OK, fine.

377
00:27:13,000 --> 00:27:16,000
Now we'll prove it for d of v_i
plus one.

378
00:27:16,000 --> 00:27:20,000
Generally, I recommend you
assume something.

379
00:27:20,000 --> 00:27:24,000
In fact, why don't I follow my
own advice and change it?

380
00:27:24,000 --> 00:27:29,000
It's usually nicer to think of
induction as recursion.

381
00:27:29,000 --> 00:27:32,000
So, you assume that this is
true, let's say,

382
00:27:32,000 --> 00:27:37,000
for j less than the i that you
care about, and then you prove

383
00:27:37,000 --> 00:27:42,000
it for d of v_i.
It's usually a lot easier to

384
00:27:42,000 --> 00:27:44,000
think about it that way.
In particular,

385
00:27:44,000 --> 00:27:48,000
you can use strong induction
for all less than i.

386
00:27:48,000 --> 00:27:51,000
Here, we're only going to need
it for one less.

387
00:27:51,000 --> 00:27:56,000
We have some relation between I
and I minus one here in terms of

388
00:27:56,000 --> 00:27:59,000
the deltas.
And so, we want to argue

389
00:27:59,000 --> 00:28:05,000
something about the d values.
OK, well, let's think about

390
00:28:05,000 --> 00:28:08,000
what's going on here.
We know that,

391
00:28:08,000 --> 00:28:15,000
let's say, after I minus one
rounds, we have this inductive

392
00:28:15,000 --> 00:28:22,000
hypothesis, d of v_i minus one
equals delta of s v_i minus one.

393
00:28:22,000 --> 00:28:27,000
And, we want to conclude that
after i rounds,

394
00:28:27,000 --> 00:28:31,000
so we have one more round to do
this.

395
00:28:31,000 --> 00:28:38,000
We want to conclude that d of
v_i has the right answer,

396
00:28:38,000 --> 00:28:44,000
delta of s comma v_i.
Does that look familiar at all?

397
00:28:44,000 --> 00:28:47,000
So we want to relax every edge
in this round.

398
00:28:47,000 --> 00:28:49,000
In particular,
at some point,

399
00:28:49,000 --> 00:28:53,000
we have to relax the edge from
v_i minus one to v_i.

400
00:28:53,000 --> 00:28:56,000
We know that this path consists
of edges.

401
00:28:56,000 --> 00:29:00,000
That's the definition of a
path.

402
00:29:00,000 --> 00:29:10,000
So, during the i'th round,
we relax every edge.

403
00:29:10,000 --> 00:29:18,000
So, we better relax v_i minus
one v_i.

404
00:29:18,000 --> 00:29:30,000
And, what happens then?
It's a test of memory.

405
00:29:43,000 --> 00:29:46,000
Quick, the Death Star is
approaching.

406
00:29:46,000 --> 00:29:51,000
So, if we have the correct
value for v_i minus one,

407
00:29:51,000 --> 00:29:57,000
that we relax an outgoing edge
from there, and that edge is an

408
00:29:57,000 --> 00:30:01,000
edge of the shortest path from s
to v_i.

409
00:30:01,000 --> 00:30:07,000
What do we know?
d of v_i becomes the correct

410
00:30:07,000 --> 00:30:13,000
value, delta of s comma v_i.
This was called correctness

411
00:30:13,000 --> 00:30:18,000
lemma last time.
One of the things we proved

412
00:30:18,000 --> 00:30:24,000
about Dijkstra's algorithm,
but it was really just a fact

413
00:30:24,000 --> 00:30:29,000
about relaxation.
And it was a pretty simple

414
00:30:29,000 --> 00:30:32,000
proof.
And it comes from this fact.

415
00:30:32,000 --> 00:30:35,000
We know the shortest path
weight is this.

416
00:30:35,000 --> 00:30:38,000
So, certainly d of v_i was at
least this big,

417
00:30:38,000 --> 00:30:42,000
and let's suppose it's greater,
or otherwise we were done.

418
00:30:42,000 --> 00:30:44,000
We know d of v_i minus one is
set to this.

419
00:30:44,000 --> 00:30:48,000
And so, this is exactly the
condition that's being checked

420
00:30:48,000 --> 00:30:52,000
in the relaxation step.
And, the d of v_i value will be

421
00:30:52,000 --> 00:30:54,000
greater than this,
let's suppose.

422
00:30:54,000 --> 00:30:56,000
And then, we'll set it equal to
this.

423
00:30:56,000 --> 00:31:01,000
And that's exactly d of s v_i.
So, when we relax that edge,

424
00:31:01,000 --> 00:31:04,000
we've got to set it to the
right value.

425
00:31:04,000 --> 00:31:06,000
So, this is the end of the
proof, right?

426
00:31:06,000 --> 00:31:08,000
It's very simple.
The point is,

427
00:31:08,000 --> 00:31:11,000
you look at your shortest path.
Here it is.

428
00:31:11,000 --> 00:31:14,000
And if we assume there's no
negative weight cycles,

429
00:31:14,000 --> 00:31:17,000
this has the correct value
initially.

430
00:31:17,000 --> 00:31:20,000
d of s is going to be zero.
After the first round,

431
00:31:20,000 --> 00:31:23,000
you've got to relax this edge.
And then you get the right

432
00:31:23,000 --> 00:31:26,000
value for that vertex.
After the second round,

433
00:31:26,000 --> 00:31:30,000
you've got to relax this edge,
which gets you the right d

434
00:31:30,000 --> 00:31:36,000
value for this vertex and so on.
And so, no matter which

435
00:31:36,000 --> 00:31:40,000
shortest path you take,
you can apply this analysis.

436
00:31:40,000 --> 00:31:44,000
And you know that by,
if the length of this path,

437
00:31:44,000 --> 00:31:50,000
here we assumed it was k edges,
then after k rounds you've got

438
00:31:50,000 --> 00:31:53,000
to be done.
OK, so this was not actually

439
00:31:53,000 --> 00:31:57,000
the end of the proof.
Sorry.

440
00:31:57,000 --> 00:32:03,000
So this means after k rounds,
we have the right answer for

441
00:32:03,000 --> 00:32:08,000
v_k, which is v.
So, the only question is how

442
00:32:08,000 --> 00:32:12,000
big could k be?
And, it better be the right

443
00:32:12,000 --> 00:32:18,000
answer, at most,
v minus one is the claim by the

444
00:32:18,000 --> 00:32:24,000
algorithm that you only need to
do v minus one steps.

445
00:32:24,000 --> 00:32:30,000
And indeed, the number of edges
in a simple path in a graph is,

446
00:32:30,000 --> 00:32:37,000
at most, the number of vertices
minus one.

447
00:32:37,000 --> 00:32:40,000
k is, at most,
v minus one because p is

448
00:32:40,000 --> 00:32:43,000
simple.
So, that's why we had to assume

449
00:32:43,000 --> 00:32:47,000
that it wasn't just any shortest
path.

450
00:32:47,000 --> 00:32:52,000
It had to be a simple one so it
didn't repeat any vertices.

451
00:32:52,000 --> 00:32:55,000
So there are,
at most, V vertices in the

452
00:32:55,000 --> 00:33:01,000
path, so at most,
V minus one edges in the path.

453
00:33:01,000 --> 00:33:05,000
OK, and that's all there is to
Bellman-Ford.

454
00:33:05,000 --> 00:33:08,000
So: pretty simple in
correctness.

455
00:33:08,000 --> 00:33:15,000
Of course, we're using a lot of
the lemmas that we proved last

456
00:33:15,000 --> 00:33:21,000
time, which makes it easier.
OK, a consequence of this

457
00:33:21,000 --> 00:33:27,000
theorem, or of this proof is
that if Bellman-Ford fails to

458
00:33:27,000 --> 00:33:33,000
converge, and that's what the
algorithm is checking is whether

459
00:33:33,000 --> 00:33:39,000
this relaxation still requires
work after these d minus one

460
00:33:39,000 --> 00:33:44,000
steps.
Right, the end of this

461
00:33:44,000 --> 00:33:48,000
algorithm is run another round,
a V'th round,

462
00:33:48,000 --> 00:33:53,000
see whether anything changes.
So, we'll say that the

463
00:33:53,000 --> 00:33:58,000
algorithm fails to converge
after V minus one steps or

464
00:33:58,000 --> 00:34:01,000
rounds.
Then, there has to be a

465
00:34:01,000 --> 00:34:04,000
negative weight cycle.
OK, this is just a

466
00:34:04,000 --> 00:34:06,000
contrapositive of what we
proved.

467
00:34:06,000 --> 00:34:10,000
We proved that if you assume
there's no negative weight

468
00:34:10,000 --> 00:34:14,000
cycle, then we know that d of s
is zero, and then all this

469
00:34:14,000 --> 00:34:18,000
argument says is you've got to
converge after v minus one

470
00:34:18,000 --> 00:34:21,000
rounds.
There can't be anything left to

471
00:34:21,000 --> 00:34:24,000
do once you've reached the
shortest path weights because

472
00:34:24,000 --> 00:34:30,000
you're going monotonically;
you can never hit the bottom.

473
00:34:30,000 --> 00:34:33,000
You can never go to the floor.
So, if you fail to converge

474
00:34:33,000 --> 00:34:37,000
somehow after V minus one
rounds, you've got to have

475
00:34:37,000 --> 00:34:40,000
violated the assumption.
The only assumption we made was

476
00:34:40,000 --> 00:34:42,000
there's no negative weight
cycle.

477
00:34:42,000 --> 00:34:45,000
So, this tells us that
Bellman-Ford is actually

478
00:34:45,000 --> 00:34:48,000
correct.
When it says that there is a

479
00:34:48,000 --> 00:34:51,000
negative weight cycle,
it indeed means it.

480
00:34:51,000 --> 00:34:53,000
It's true.
OK, and you can modify

481
00:34:53,000 --> 00:34:56,000
Bellman-Ford in that case to
sort of run a little longer,

482
00:34:56,000 --> 00:35:01,000
and find where all the minus
infinities are.

483
00:35:01,000 --> 00:35:02,000
And that is,
in some sense,

484
00:35:02,000 --> 00:35:05,000
one of the things you have to
do in your problem set,

485
00:35:05,000 --> 00:35:08,000
I believe.
So, I won't cover it here.

486
00:35:08,000 --> 00:35:11,000
But, it's a good exercise in
any case to figure out how you

487
00:35:11,000 --> 00:35:14,000
would find where the minus
infinities are.

488
00:35:14,000 --> 00:35:18,000
What are all the vertices
reachable from negative weight

489
00:35:18,000 --> 00:35:20,000
cycle?
Those are the ones that have

490
00:35:20,000 --> 00:35:22,000
minus infinities.
OK, so you might say,

491
00:35:22,000 --> 00:35:26,000
well, that was awfully fast.
Actually, it's not over yet.

492
00:35:26,000 --> 00:35:29,000
The episode is not yet ended.
We're going to use Bellman-Ford

493
00:35:29,000 --> 00:35:35,000
to solve the even bigger and
greater shortest path problems.

494
00:35:35,000 --> 00:35:39,000
And in the remainder of today's
lecture, we will see it applied

495
00:35:39,000 --> 00:35:42,000
to a more general problem,
in some sense,

496
00:35:42,000 --> 00:35:45,000
called linear programming.
And the next lecture,

497
00:35:45,000 --> 00:35:49,000
we'll really use it to do some
amazing stuff with all pairs

498
00:35:49,000 --> 00:35:52,000
shortest paths.
Let's go over here.

499
00:35:52,000 --> 00:35:55,000
So, our goal,
although it won't be obvious

500
00:35:55,000 --> 00:35:59,000
today, is to be able to compute
the shortest paths between every

501
00:35:59,000 --> 00:36:03,000
pair of vertices,
which we could certainly do at

502
00:36:03,000 --> 00:36:08,000
this point just by running
Bellman-Ford v times.

503
00:36:08,000 --> 00:36:15,000
OK, but we want to do better
than that, of course.

504
00:36:15,000 --> 00:36:21,000
And, that will be the climax of
the trilogy.

505
00:36:21,000 --> 00:36:30,000
OK, today we just discovered
who Luke's father is.

506
00:36:30,000 --> 00:36:37,000
So, it turns out the father of
shortest paths is linear

507
00:36:37,000 --> 00:36:42,000
programming.
Actually, simultaneously the

508
00:36:42,000 --> 00:36:50,000
father and the mother because
programs do not have gender.

509
00:36:50,000 --> 00:36:57,000
OK, my father likes to say,
we both took improv comedy

510
00:36:57,000 --> 00:37:05,000
lessons so we have degrees in
improvisation.

511
00:37:05,000 --> 00:37:07,000
And he said,
you know, we went to improv

512
00:37:07,000 --> 00:37:10,000
classes in order to learn how to
make our humor better.

513
00:37:10,000 --> 00:37:13,000
And, the problem is,
it didn't actually make our

514
00:37:13,000 --> 00:37:16,000
humor better.
It just made us less afraid to

515
00:37:16,000 --> 00:37:17,000
use it.
[LAUGHTER] So,

516
00:37:17,000 --> 00:37:20,000
you are subjected to all this
improv humor.

517
00:37:20,000 --> 00:37:22,000
I didn't see the connection of
Luke's father,

518
00:37:22,000 --> 00:37:25,000
but there you go.
OK, so, linear programming is a

519
00:37:25,000 --> 00:37:29,000
very general problem,
a very big tool.

520
00:37:29,000 --> 00:37:32,000
Has anyone seen linear
programming before?

521
00:37:32,000 --> 00:37:36,000
OK, one person.
And, I'm sure you will,

522
00:37:36,000 --> 00:37:40,000
at some time in your life,
do anything vaguely computing

523
00:37:40,000 --> 00:37:45,000
optimization related,
linear programming comes up at

524
00:37:45,000 --> 00:37:48,000
some point.
It's a very useful tool.

525
00:37:48,000 --> 00:37:53,000
You're given a matrix and two
vectors: not too exciting yet.

526
00:37:53,000 --> 00:37:57,000
What you want to do is find a
vector.

527
00:37:57,000 --> 00:38:02,000
This is a very dry description.
We'll see what makes it so

528
00:38:02,000 --> 00:38:04,000
interesting in a moment.

529
00:38:17,000 --> 00:38:21,000
So, you want to maximize some
objective, and you have some

530
00:38:21,000 --> 00:38:24,000
constraints.
And they're all linear.

531
00:38:24,000 --> 00:38:28,000
So, the objective is a linear
function in the variables x,

532
00:38:28,000 --> 00:38:32,000
and your constraints are a
bunch of linear constraints,

533
00:38:32,000 --> 00:38:36,000
inequality constraints,
that's one makes an

534
00:38:36,000 --> 00:38:39,000
interesting.
It's not just solving a linear

535
00:38:39,000 --> 00:38:43,000
system as you've seen in linear
algebra, or whatever.

536
00:38:43,000 --> 00:38:46,000
Or, of course,
it could be that there is no

537
00:38:46,000 --> 00:38:49,000
such x.
OK: vaguely familiar you might

538
00:38:49,000 --> 00:38:52,000
think to the theorem about
Bellman-Ford.

539
00:38:52,000 --> 00:38:56,000
And, we'll show that there's
some kind of connection here

540
00:38:56,000 --> 00:39:01,000
that either you want to find
something, or show that it

541
00:39:01,000 --> 00:39:06,000
doesn't exist.
Well, that's still a pretty

542
00:39:06,000 --> 00:39:09,000
vague connection,
but I also want to maximize

543
00:39:09,000 --> 00:39:13,000
something, or are sort of
minimize the shortest paths,

544
00:39:13,000 --> 00:39:17,000
OK, somewhat similar.
We have these constraints.

545
00:39:17,000 --> 00:39:19,000
So, yeah.
This may be intuitive to you,

546
00:39:19,000 --> 00:39:22,000
I don't know.
I prefer a more geometric

547
00:39:22,000 --> 00:39:27,000
picture, and I will try to draw
such a geometric picture,

548
00:39:27,000 --> 00:39:30,000
and I've never tried to do this
on a blackboard,

549
00:39:30,000 --> 00:39:36,000
so it should be interesting.
I think I'm going to fail

550
00:39:36,000 --> 00:39:39,000
miserably.
It sort of looks like a

551
00:39:39,000 --> 00:39:41,000
dodecahedron,
right?

552
00:39:41,000 --> 00:39:44,000
Sort of, kind of,
not really.

553
00:39:44,000 --> 00:39:47,000
A bit rough on the bottom,
OK.

554
00:39:47,000 --> 00:39:51,000
So, if you have a bunch of
linear constraints,

555
00:39:51,000 --> 00:39:56,000
this is supposed to be in 3-D.
Now I labeled it.

556
00:39:56,000 --> 00:40:00,000
It's now in 3-D.
Good.

557
00:40:00,000 --> 00:40:02,000
So, you have these linear
constraints.

558
00:40:02,000 --> 00:40:06,000
That turns out to define
hyperplanes in n dimensions.

559
00:40:06,000 --> 00:40:11,000
OK, so you have this base here
that's three-dimensional space.

560
00:40:11,000 --> 00:40:14,000
So, n equals three.
And, these hyperplanes,

561
00:40:14,000 --> 00:40:17,000
if you're looking at one side
of the hyperplane,

562
00:40:17,000 --> 00:40:21,000
that's the less than or equal
to, if you take the

563
00:40:21,000 --> 00:40:24,000
intersection,
you get some convex polytope or

564
00:40:24,000 --> 00:40:27,000
polyhedron.
In 3-D, you might get a

565
00:40:27,000 --> 00:40:29,000
dodecahedron or whatever.
And, your goal,

566
00:40:29,000 --> 00:40:33,000
you have some objective vector
c, let's say,

567
00:40:33,000 --> 00:40:37,000
up.
Suppose that's the c vector.

568
00:40:37,000 --> 00:40:42,000
Your goal is to find the
highest point in this polytope.

569
00:40:42,000 --> 00:40:47,000
So here, it's maybe this one.
OK, this is the target.

570
00:40:47,000 --> 00:40:49,000
This is the optimal,
x.

571
00:40:49,000 --> 00:40:54,000
That is the geometric view.
If you prefer the algebraic

572
00:40:54,000 --> 00:41:00,000
view, you want to maximize the c
transpose times x.

573
00:41:00,000 --> 00:41:01,000
So, this is m.
This is n.

574
00:41:01,000 --> 00:41:04,000
Check out the dimensions work
out.

575
00:41:04,000 --> 00:41:08,000
So that's saying you want to
maximize the dot product.

576
00:41:08,000 --> 00:41:13,000
You want to maximize the extent
to which x is in the direction

577
00:41:13,000 --> 00:41:16,000
c.
And, you want to maximize that

578
00:41:16,000 --> 00:41:20,000
subject to some constraints,
which looks something like

579
00:41:20,000 --> 00:41:22,000
this, maybe.
So, this is A,

580
00:41:22,000 --> 00:41:25,000
and it's m by n.
You want to multiply it by,

581
00:41:25,000 --> 00:41:30,000
it should be something of
height n.

582
00:41:30,000 --> 00:41:32,000
That's x.
Let me put x down here,

583
00:41:32,000 --> 00:41:36,000
n by one.
And, it should be less than or

584
00:41:36,000 --> 00:41:39,000
equal to something of this
height, which is B,

585
00:41:39,000 --> 00:41:44,000
the right hand side.
OK, that's the algebraic view,

586
00:41:44,000 --> 00:41:48,000
which is to check out all the
dimensions are working out.

587
00:41:48,000 --> 00:41:52,000
But, you can read these off in
each row here,

588
00:41:52,000 --> 00:41:57,000
when multiplied by this column,
gives you one value here.

589
00:41:57,000 --> 00:42:03,000
And as just a linear
constraints on all the x sides.

590
00:42:03,000 --> 00:42:08,000
So, you want to maximize this
linear function of x_1 up to x_n

591
00:42:08,000 --> 00:42:11,000
subject to these constraints,
OK?

592
00:42:11,000 --> 00:42:16,000
Pretty simple,
but pretty powerful in general.

593
00:42:16,000 --> 00:42:21,000
So, it turns out that with,
you can formulate a huge number

594
00:42:21,000 --> 00:42:26,000
of problems such as shortest
paths as a linear program.

595
00:42:26,000 --> 00:42:31,000
So, it's a general tool.
And in this class,

596
00:42:31,000 --> 00:42:37,000
we will not cover any
algorithms for solving linear

597
00:42:37,000 --> 00:42:40,000
programming.
It's a bit tricky.

598
00:42:40,000 --> 00:42:44,000
I'll just mention that they are
out there.

599
00:42:44,000 --> 00:42:50,000
So, there's many efficient
algorithms, and lots of code

600
00:42:50,000 --> 00:42:55,000
that does this.
It's a very practical setup.

601
00:42:55,000 --> 00:43:02,000
So, lots of algorithms to solve
LP's, linear programs.

602
00:43:02,000 --> 00:43:05,000
Linear programming is usually
called LP.

603
00:43:05,000 --> 00:43:08,000
And, I'll mention a few of
them.

604
00:43:08,000 --> 00:43:14,000
There's the simplex algorithm.
This is one of the first.

605
00:43:14,000 --> 00:43:18,000
I think it is the first,
the ellipsoid algorithm.

606
00:43:18,000 --> 00:43:24,000
There's interior point methods,
and there's random sampling.

607
00:43:24,000 --> 00:43:29,000
I'll just say a little bit
about each of these because

608
00:43:29,000 --> 00:43:36,000
we're not going to talk about
any of them in depth.

609
00:43:36,000 --> 00:43:38,000
The simplex algorithm,
this is, I mean,

610
00:43:38,000 --> 00:43:41,000
one of the first algorithms in
the world in some sense,

611
00:43:41,000 --> 00:43:43,000
certainly one of the most
popular.

612
00:43:43,000 --> 00:43:47,000
It's still used today.
Almost all linear programming

613
00:43:47,000 --> 00:43:50,000
code uses the simplex algorithm.
It happens to run an

614
00:43:50,000 --> 00:43:53,000
exponential time in the
worst-case, so it's actually

615
00:43:53,000 --> 00:43:56,000
pretty bad theoretically.
But in practice,

616
00:43:56,000 --> 00:43:59,000
it works really well.
And there is some recent work

617
00:43:59,000 --> 00:44:03,000
that tries to understand this.
It's still exponential in the

618
00:44:03,000 --> 00:44:06,000
worst case.
But, it's practical.

619
00:44:06,000 --> 00:44:10,000
There's actually an open
problem whether there exists a

620
00:44:10,000 --> 00:44:13,000
variation of simplex that runs
in polynomial time.

621
00:44:13,000 --> 00:44:17,000
But, I won't go into that.
That's a major open problem in

622
00:44:17,000 --> 00:44:22,000
this area of linear programming.
The ellipsoid algorithm was the

623
00:44:22,000 --> 00:44:26,000
first algorithm to solve linear
programming in polynomial time.

624
00:44:26,000 --> 00:44:30,000
So, for a long time,
people didn't know.

625
00:44:30,000 --> 00:44:32,000
Around this time,
people started realizing

626
00:44:32,000 --> 00:44:36,000
polynomial time is a good thing.
That happened around the late

627
00:44:36,000 --> 00:44:37,000
60s.
Polynomial time is good.

628
00:44:37,000 --> 00:44:41,000
And, the ellipsoid algorithm is
the first one to do it.

629
00:44:41,000 --> 00:44:44,000
It's a very general algorithm,
and very powerful,

630
00:44:44,000 --> 00:44:46,000
theoretically:
completely impractical.

631
00:44:46,000 --> 00:44:49,000
But, it's cool.
It lets you do things like you

632
00:44:49,000 --> 00:44:52,000
can solve a linear program that
has exponentially many

633
00:44:52,000 --> 00:44:56,000
constraints in polynomial time.
You've got all sorts of crazy

634
00:44:56,000 --> 00:44:57,000
things.
So, I'll just say it's

635
00:44:57,000 --> 00:45:01,000
polynomial time.
I can't say something nice

636
00:45:01,000 --> 00:45:04,000
about it; don't say it at all.
It's impractical.

637
00:45:04,000 --> 00:45:07,000
Interior point methods are sort
of the mixture.

638
00:45:07,000 --> 00:45:11,000
They run in polynomial time.
You can guarantee that.

639
00:45:11,000 --> 00:45:14,000
And, they are also pretty
practical, and there's sort of

640
00:45:14,000 --> 00:45:18,000
this competition these days
about whether simplex or

641
00:45:18,000 --> 00:45:21,000
interior point is better.
And, I don't know what it is

642
00:45:21,000 --> 00:45:24,000
today but a few years ago they
were neck and neck.

643
00:45:24,000 --> 00:45:27,000
And, random sampling is a brand
new approach.

644
00:45:27,000 --> 00:45:31,000
This is just from a couple
years ago by two MIT professors,

645
00:45:31,000 --> 00:45:35,000
Dimitris Bertsimas and Santosh
Vempala, I guess the other is in

646
00:45:35,000 --> 00:45:39,000
applied math.
So, just to show you,

647
00:45:39,000 --> 00:45:41,000
there's active work in this
area.

648
00:45:41,000 --> 00:45:44,000
People are still finding new
ways to solve linear programs.

649
00:45:44,000 --> 00:45:47,000
This is completely randomized,
and very simple,

650
00:45:47,000 --> 00:45:50,000
and very general.
It hasn't been implemented,

651
00:45:50,000 --> 00:45:52,000
so we don't know how practical
it is yet.

652
00:45:52,000 --> 00:45:54,000
But, it has potential.
OK: pretty neat.

653
00:45:54,000 --> 00:45:57,000
OK, we're going to look at a
somewhat simpler version of

654
00:45:57,000 --> 00:46:02,000
linear programming.
The first restriction we are

655
00:46:02,000 --> 00:46:05,000
going to make is actually not
much of a restriction.

656
00:46:05,000 --> 00:46:09,000
But, nonetheless we will
consider it, it's a little bit

657
00:46:09,000 --> 00:46:13,000
easier to think about.
So here, we had some polytope

658
00:46:13,000 --> 00:46:16,000
we wanted to maximize some
objective.

659
00:46:16,000 --> 00:46:19,000
In a feasibility problem,
I just want to know,

660
00:46:19,000 --> 00:46:23,000
is the polytope empty?
Can you find any point in that

661
00:46:23,000 --> 00:46:26,000
polytope?
Can you find any set of values,

662
00:46:26,000 --> 00:46:30,000
x, that satisfy these
constraints?

663
00:46:30,000 --> 00:46:34,000
OK, so there's no objective.
c, just find x such that AX is

664
00:46:34,000 --> 00:46:39,000
less than or equal to B.
OK, it turns out you can prove

665
00:46:39,000 --> 00:46:43,000
a very general theorem that if
you can solve linear

666
00:46:43,000 --> 00:46:47,000
feasibility, you can also solve
linear programming.

667
00:46:47,000 --> 00:46:52,000
We won't prove that here,
but this is actually no easier

668
00:46:52,000 --> 00:46:56,000
than the original problem even
though it feels easier,

669
00:46:56,000 --> 00:47:03,000
and it's easier to think about.
I was just saying actually no

670
00:47:03,000 --> 00:47:08,000
easier than LP.
OK, the next restriction we're

671
00:47:08,000 --> 00:47:11,000
going to make is a real
restriction.

672
00:47:11,000 --> 00:47:17,000
And it simplifies the problem
quite a bit.

673
00:47:30,000 --> 00:47:35,000
And that's to look at different
constraints.

674
00:47:35,000 --> 00:47:40,000
And, if all this seemed a bit
abstract so far,

675
00:47:40,000 --> 00:47:45,000
we will now ground ourselves
little bit.

676
00:47:45,000 --> 00:47:51,000
A system of different
constraints is a linear

677
00:47:51,000 --> 00:47:57,000
feasibility problem.
So, it's an LP where there's no

678
00:47:57,000 --> 00:48:06,000
objective.
And, it's with a restriction,

679
00:48:06,000 --> 00:48:17,000
so, where each row of the
matrix, so, the matrix,

680
00:48:17,000 --> 00:48:26,000
A, has one one,
and it has one minus one,

681
00:48:26,000 --> 00:48:36,000
and everything else in the row
is zero.

682
00:48:36,000 --> 00:48:40,000
OK, in other words,
each constraint has its very

683
00:48:40,000 --> 00:48:45,000
simple form.
It involves two variables and

684
00:48:45,000 --> 00:48:49,000
some number.
So, we have something like x_j

685
00:48:49,000 --> 00:48:53,000
minus x_i is less than or equal
to w_ij.

686
00:48:53,000 --> 00:49:00,000
So, this is just a number.
These are two variables.

687
00:49:00,000 --> 00:49:02,000
There's a minus sign,
no values up here,

688
00:49:02,000 --> 00:49:06,000
no coefficients,
no other of the X_k's appear,

689
00:49:06,000 --> 00:49:09,000
just two of them.
And, you have a bunch of

690
00:49:09,000 --> 00:49:13,000
constraints of this form,
one per row of the matrix.

691
00:49:13,000 --> 00:49:16,000
Geometrically,
I haven't thought about what

692
00:49:16,000 --> 00:49:18,000
this means.
I think it means the

693
00:49:18,000 --> 00:49:22,000
hyperplanes are pretty simple.
Sorry I can't do better than

694
00:49:22,000 --> 00:49:25,000
that.
It's a little hard to see this

695
00:49:25,000 --> 00:49:30,000
in high dimensions.
But, it will start to

696
00:49:30,000 --> 00:49:38,000
correspond to something we've
seen, namely the board that its

697
00:49:38,000 --> 00:49:45,000
next to, very shortly.
OK, so let's do a very quick

698
00:49:45,000 --> 00:49:50,000
example mainly to have something
to point at.

699
00:49:50,000 --> 00:49:59,000
Here's a very simple system of
difference constraints --

700
00:50:11,000 --> 00:50:13,000
-- OK, and a solution.
Why not?

701
00:50:13,000 --> 00:50:18,000
It's not totally trivial to
solve this, but here's a

702
00:50:18,000 --> 00:50:21,000
solution.
And the only thing to check is

703
00:50:21,000 --> 00:50:25,000
that each of these constraints
is satisfied.

704
00:50:25,000 --> 00:50:29,000
x_1 minus x_2 is three,
which is less than or equal to

705
00:50:29,000 --> 00:50:35,000
three, and so on.
There could be negative values.

706
00:50:35,000 --> 00:50:42,000
There could be positive values.
It doesn't matter.

707
00:50:42,000 --> 00:50:49,000
I'd like to transform this
system of difference constraints

708
00:50:49,000 --> 00:50:55,000
into a graph because we know a
lot about graphs.

709
00:50:55,000 --> 00:51:03,000
So, we're going to call this
the constraint graph.

710
00:51:03,000 --> 00:51:08,000
And, it's going to represent
these constraints.

711
00:51:08,000 --> 00:51:13,000
How'd I do it?
Well, I take every constraint,

712
00:51:13,000 --> 00:51:20,000
which in general looks like
this, and I convert it into an

713
00:51:20,000 --> 00:51:24,000
edge.
OK, so if I write it as x_j

714
00:51:24,000 --> 00:51:29,000
minus x_i is less than or equal
to some w_ij,

715
00:51:29,000 --> 00:51:36,000
w seems suggestive of weights.
That's exactly why I called it

716
00:51:36,000 --> 00:51:38,000
w.
I'm going to make that an edge

717
00:51:38,000 --> 00:51:41,000
from v_i to v_j.
So, the order flips a little

718
00:51:41,000 --> 00:51:44,000
bit.
And, the weight of that edge is

719
00:51:44,000 --> 00:51:46,000
w_ij.
So, just do that.

720
00:51:46,000 --> 00:51:49,000
Make n vertices.
So, you have the number of

721
00:51:49,000 --> 00:51:53,000
vertices equals n.
The number of edges equals the

722
00:51:53,000 --> 00:51:56,000
number of constraints,
which is m, the height of the

723
00:51:56,000 --> 00:52:01,000
matrix, and just transform.
So, for example,

724
00:52:01,000 --> 00:52:06,000
here we have three variables.
So, we have three vertices,

725
00:52:06,000 --> 00:52:09,000
v_1, v_2, v_3.
We have x_1 minus x_2.

726
00:52:09,000 --> 00:52:14,000
So, we have an edge from v_2 to
v_1 of weight three.

727
00:52:14,000 --> 00:52:18,000
We have x_2 minus x_3.
So, we have an edge from v_3 to

728
00:52:18,000 --> 00:52:23,000
v_2 of weight minus two.
And, we have x_1 minus x_3.

729
00:52:23,000 --> 00:52:27,000
So, we have an edge from v_3 to
v_1 of weight two.

730
00:52:27,000 --> 00:52:32,000
I hope I got the directions
right.

731
00:52:32,000 --> 00:52:34,000
Yep.
So, there it is,

732
00:52:34,000 --> 00:52:40,000
a graph: currently no obvious
connection to shortest paths,

733
00:52:40,000 --> 00:52:42,000
right?
But in fact,

734
00:52:42,000 --> 00:52:47,000
this constraint is closely
related to shortest paths.

735
00:52:47,000 --> 00:52:52,000
So let me just rewrite it.
You could say,

736
00:52:52,000 --> 00:52:59,000
well, an x_j is less than or
equal to x_i plus w_ij.

737
00:52:59,000 --> 00:53:03,000
Or, you could think of it as
d[j] less than or equal to d[i]

738
00:53:03,000 --> 00:53:07,000
plus w_ij.
This is a conceptual balloon.

739
00:53:07,000 --> 00:53:10,000
Look awfully familiar?
A lot like the triangle

740
00:53:10,000 --> 00:53:13,000
inequality, a lot like
relaxation.

741
00:53:13,000 --> 00:53:17,000
So, there's a very close
connection between these two

742
00:53:17,000 --> 00:53:21,000
problems as we will now prove.

743
00:53:43,000 --> 00:53:45,000
So, we're going to have two
theorems.

744
00:53:45,000 --> 00:53:49,000
And, they're going to look
similar to the correctness of

745
00:53:49,000 --> 00:53:53,000
Bellman-Ford in that they talk
about negative weight cycles.

746
00:53:53,000 --> 00:53:54,000
Here we go.
It turns out,

747
00:53:54,000 --> 00:53:57,000
I mean, we have this constraint
graph.

748
00:53:57,000 --> 00:54:02,000
It can have negative weights.
It can have positive weights.

749
00:54:02,000 --> 00:54:05,000
It turns out what matters is if
you have a negative weight

750
00:54:05,000 --> 00:54:07,000
cycle.
So, the first thing to prove is

751
00:54:07,000 --> 00:54:11,000
that if you have a negative
weight cycle that something bad

752
00:54:11,000 --> 00:54:13,000
happens.
OK, what could happen bad?

753
00:54:13,000 --> 00:54:16,000
Well, we're just trying to
satisfy this system of

754
00:54:16,000 --> 00:54:19,000
constraints.
So, the bad thing is that there

755
00:54:19,000 --> 00:54:22,000
might not be any solution.
These constraints may be

756
00:54:22,000 --> 00:54:24,000
infeasible.
And that's the claim.

757
00:54:24,000 --> 00:54:29,000
The claim is that this is
actually an if and only if.

758
00:54:29,000 --> 00:54:33,000
But first we'll proved the if.
If you have a negative weight

759
00:54:33,000 --> 00:54:38,000
cycle, you're doomed.
The difference constraints are

760
00:54:38,000 --> 00:54:41,000
unsatisfiable.
That's a more intuitive way to

761
00:54:41,000 --> 00:54:43,000
say it.
In the LP world,

762
00:54:43,000 --> 00:54:48,000
they call it infeasible.
But unsatisfiable makes a lot

763
00:54:48,000 --> 00:54:51,000
more sense.
There's no way to assign the

764
00:54:51,000 --> 00:54:56,000
x_i's in order to satisfy all
the constraints simultaneously.

765
00:54:56,000 --> 00:55:01,000
So, let's just take a look.
Consider a negative weight

766
00:55:01,000 --> 00:55:03,000
cycle.
It starts at some vertex,

767
00:55:03,000 --> 00:55:07,000
goes through some vertices,
and at some point comes back.

768
00:55:07,000 --> 00:55:11,000
I don't care whether it repeats
vertices, just as long as this

769
00:55:11,000 --> 00:55:15,000
cycle, from v_1 to v_1 is a
negative weight cycle strictly

770
00:55:15,000 --> 00:55:17,000
negative weight.

771
00:55:26,000 --> 00:55:30,000
OK, and what I'm going to do is
just write down all the

772
00:55:30,000 --> 00:55:34,000
constraints.
Each of these edges corresponds

773
00:55:34,000 --> 00:55:37,000
to a constraint,
which must be in the set of

774
00:55:37,000 --> 00:55:40,000
constraints because we had that
graph.

775
00:55:40,000 --> 00:55:45,000
So, these are all edges.
Let's look at what they give

776
00:55:45,000 --> 00:55:48,000
us.
So, we have an edge from v_1 to

777
00:55:48,000 --> 00:55:50,000
v_2.
That corresponds to x_2 minus

778
00:55:50,000 --> 00:55:53,000
x_1 is, at most,
something, w_12.

779
00:55:53,000 --> 00:55:57,000
Then we have x_3 minus x_2.
That's the weight w_23,

780
00:55:57,000 --> 00:56:04,000
and so on.
And eventually we get up to

781
00:56:04,000 --> 00:56:08,000
something like x_k minus
x_(k-1).

782
00:56:08,000 --> 00:56:15,000
That's this edge:
w_(k-1),k , and lastly we have

783
00:56:15,000 --> 00:56:23,000
this edge, which wraps around.
So, it's x_1 minus x_k,

784
00:56:23,000 --> 00:56:30,000
w_k1 if I've got the signs
right.

785
00:56:30,000 --> 00:56:35,000
Good, so here's a bunch of
constraints.

786
00:56:35,000 --> 00:56:40,000
What do you suggest I do with
them?

787
00:56:40,000 --> 00:56:47,000
Anything interesting about
these constraints,

788
00:56:47,000 --> 00:56:52,000
say, the left hand sides?
Sorry?

789
00:56:52,000 --> 00:57:00,000
It sounded like the right word.
What was it?

790
00:57:00,000 --> 00:57:01,000
Telescopes, yes,
good.

791
00:57:01,000 --> 00:57:04,000
Everything cancels.
If I added these up,

792
00:57:04,000 --> 00:57:08,000
there's an x_2 and a minus x_2.
There's a minus x_1 and an x_1.

793
00:57:08,000 --> 00:57:12,000
There's a minus XK and an XK.
Everything here cancels if I

794
00:57:12,000 --> 00:57:15,000
add up the left hand sides.
So, what happens if I add up

795
00:57:15,000 --> 00:57:18,000
the right hand sides?
Over here I get zero,

796
00:57:18,000 --> 00:57:20,000
my favorite answer.
And over here,

797
00:57:20,000 --> 00:57:24,000
we get all the weights of all
the edges in the negative weight

798
00:57:24,000 --> 00:57:30,000
cycle, which is the weight of
the cycle, which is negative.

799
00:57:30,000 --> 00:57:33,000
So, zero is strictly less than
zero: contradiction.

800
00:57:33,000 --> 00:57:35,000
Contradiction:
wait a minute,

801
00:57:35,000 --> 00:57:37,000
we didn't assume anything that
was false.

802
00:57:37,000 --> 00:57:40,000
So, it's not really a
contradiction in the

803
00:57:40,000 --> 00:57:43,000
mathematical sense.
We didn't contradict the world.

804
00:57:43,000 --> 00:57:47,000
We just said that these
constraints are contradictory.

805
00:57:47,000 --> 00:57:50,000
In other words,
if you pick any values of the

806
00:57:50,000 --> 00:57:53,000
x_i's, there is no way that
these can all be true because

807
00:57:53,000 --> 00:57:55,000
that you would get a
contradiction.

808
00:57:55,000 --> 00:57:59,000
So, it's impossible for these
things to be satisfied by some

809
00:57:59,000 --> 00:58:01,000
real x_i's.
So, these must be

810
00:58:01,000 --> 00:58:07,000
unsatisfiable.
Let's say there's no satisfying

811
00:58:07,000 --> 00:58:11,000
assignment, a little more
precise, x_1 up to x_m,

812
00:58:11,000 --> 00:58:14,000
no weights.
Can we satisfy those

813
00:58:14,000 --> 00:58:18,000
constraints?
Because they add up to zero on

814
00:58:18,000 --> 00:58:23,000
the left-hand side,
and negative on the right-hand

815
00:58:23,000 --> 00:58:26,000
side.
OK, so that's an easy proof.

816
00:58:26,000 --> 00:58:33,000
The reverse direction will be
only slightly harder.

817
00:58:33,000 --> 00:58:34,000
OK, so, cool.
We have this connection.

818
00:58:34,000 --> 00:58:37,000
So motivation is,
suppose you'd want to solve

819
00:58:37,000 --> 00:58:40,000
these difference constraints.
And we'll see one such

820
00:58:40,000 --> 00:58:42,000
application.
I Googled around for difference

821
00:58:42,000 --> 00:58:44,000
constraints.
There is a fair number of

822
00:58:44,000 --> 00:58:46,000
papers that care about
difference constraints.

823
00:58:46,000 --> 00:58:49,000
And, they all use shortest
paths to solve them.

824
00:58:49,000 --> 00:58:51,000
So, if we can prove a
connection between shortest

825
00:58:51,000 --> 00:58:54,000
paths, which we know how to
compute, and difference

826
00:58:54,000 --> 00:58:56,000
constraints, then we'll have
something cool.

827
00:58:56,000 --> 00:59:00,000
And, next class will see even
more applications of difference

828
00:59:00,000 --> 00:59:05,000
constraints.
It turns out they're really

829
00:59:05,000 --> 00:59:09,000
useful for all pairs shortest
paths.

830
00:59:09,000 --> 00:59:16,000
OK, but for now let's just
prove this equivalence and

831
00:59:16,000 --> 00:59:21,000
finish it off.
So, the reverse direction is if

832
00:59:21,000 --> 00:59:29,000
there's no negative weight cycle
in this constraint graph,

833
00:59:29,000 --> 00:59:35,000
then the system better be
satisfiable.

834
00:59:35,000 --> 00:59:42,000
The claim is that these
negative weight cycles are the

835
00:59:42,000 --> 00:59:49,000
only barriers for finding a
solution to these difference

836
00:59:49,000 --> 00:59:54,000
constraints.
I have this feeling somewhere

837
00:59:54,000 --> 00:59:58,000
here.
I had to talk about the

838
00:59:58,000 --> 01:00:03,000
constraint graph.
Good.

839
01:00:13,000 --> 01:00:19,830
Satisfied, good.
So, here we're going to see a

840
01:00:19,830 --> 01:00:28,482
technique that is very useful
when thinking about shortest

841
01:00:28,482 --> 01:00:32,788
paths.
And, it's a bit hard to guess,

842
01:00:32,788 --> 01:00:36,505
especially if you haven't seen
it before.

843
01:00:36,505 --> 01:00:40,780
This is useful in problem sets,
and in quizzes,

844
01:00:40,780 --> 01:00:45,334
and finals, and everything.
So, keep this in mind.

845
01:00:45,334 --> 01:00:50,539
I mean, I'm using it to prove
this rather simple theorem,

846
01:00:50,539 --> 01:00:56,115
but the idea of changing the
graph, so I'm going to call this

847
01:00:56,115 --> 01:01:00,483
constraint graph G.
Changing the graph is a very

848
01:01:00,483 --> 01:01:04,386
powerful idea.
So, we're going to add a new

849
01:01:04,386 --> 01:01:07,732
vertex, s, or source,
use the source,

850
01:01:07,732 --> 01:01:13,215
Luke, and we're going to add a
bunch of edges from s because

851
01:01:13,215 --> 01:01:17,397
being a source,
it better be connected to some

852
01:01:17,397 --> 01:01:23,529
things.
So, we are going to add a zero

853
01:01:23,529 --> 01:01:29,764
weight edge, or weight zero edge
from s to everywhere,

854
01:01:29,764 --> 01:01:36,000
so, to every other vertex in
the constraint graph.

855
01:01:36,000 --> 01:01:40,121
Those vertices are called v_i,
v_1 up to v_n.

856
01:01:40,121 --> 01:01:45,928
So, I have my constraint graph.
But I'll copy this one so I can

857
01:01:45,928 --> 01:01:49,768
change it.
It's always good to backup your

858
01:01:49,768 --> 01:01:53,046
work before you make changes,
right?

859
01:01:53,046 --> 01:01:57,542
So now, I want to add a new
vertex, s, over here,

860
01:01:57,542 --> 01:02:01,195
my new source.
I just take my constraint

861
01:02:01,195 --> 01:02:06,909
graph, whatever it looks like,
add in weight zero edges to all

862
01:02:06,909 --> 01:02:11,171
the other vertices.
Simple enough.

863
01:02:11,171 --> 01:02:14,100
Now, what did I do?
What did you do?

864
01:02:14,100 --> 01:02:18,953
Well, I have a candidate source
now which can reach all the

865
01:02:18,953 --> 01:02:21,799
vertices.
So, shortest path from s,

866
01:02:21,799 --> 01:02:24,728
hopefully, well,
paths from s exist.

867
01:02:24,728 --> 01:02:30,000
I can get from s to everywhere
in weight at most zero.

868
01:02:30,000 --> 01:02:31,851
OK, maybe less.
Could it be less?

869
01:02:31,851 --> 01:02:34,338
Well, you know,
like v_2, I can get to it by

870
01:02:34,338 --> 01:02:36,710
zero minus two.
So, that's less than zero.

871
01:02:36,710 --> 01:02:38,677
So I've got to be a little
careful.

872
01:02:38,677 --> 01:02:40,933
What if there's a negative
weight cycle?

873
01:02:40,933 --> 01:02:42,785
Oh no?
Then there wouldn't be any

874
01:02:42,785 --> 01:02:44,347
shortest paths.
Fortunately,

875
01:02:44,347 --> 01:02:47,413
we assume that there's no
negative weight cycle in the

876
01:02:47,413 --> 01:02:49,785
original graph.
And if you think about it,

877
01:02:49,785 --> 01:02:53,082
if there's no negative weight
cycle in the original graph,

878
01:02:53,082 --> 01:02:55,396
we add an edge from s to
everywhere else.

879
01:02:55,396 --> 01:02:58,520
We're not making any new
negative weight cycles because

880
01:02:58,520 --> 01:03:01,586
you can start at s and go
somewhere at a cost of zero,

881
01:03:01,586 --> 01:03:05,000
which doesn't affect any
weights.

882
01:03:05,000 --> 01:03:08,920
And then, you are forced to
stay in the old graph.

883
01:03:08,920 --> 01:03:12,840
So, there can't be any new
negative weight cycles.

884
01:03:12,840 --> 01:03:17,000
So, the modified graph has no
negative weight cycles.

885
01:03:17,000 --> 01:03:20,519
That's good because it also has
paths from s,

886
01:03:20,519 --> 01:03:25,000
and therefore it also has
shortest paths from s.

887
01:03:25,000 --> 01:03:30,376
The modified graph has no
negative weight because it

888
01:03:30,376 --> 01:03:34,487
didn't before.
And, it has paths from s.

889
01:03:34,487 --> 01:03:38,387
There's a path from s to every
vertex.

890
01:03:38,387 --> 01:03:44,923
There may not have been before.
Before, I couldn't get from v_2

891
01:03:44,923 --> 01:03:49,561
to v_3, for example.
Well, that's still true.

892
01:03:49,561 --> 01:03:53,145
But from s I can get to
everywhere.

893
01:03:53,145 --> 01:03:58,521
So, that means that this graph,
this modified graph,

894
01:03:58,521 --> 01:04:04,974
has shortest paths.
Shortest paths exist from s.

895
01:04:04,974 --> 01:04:09,860
In other words,
if I took all the shortest path

896
01:04:09,860 --> 01:04:14,641
weights, like I ran Bellman-Ford
from s, then,

897
01:04:14,641 --> 01:04:19,421
I would get a bunch of finite
numbers, d of v,

898
01:04:19,421 --> 01:04:22,926
for every value,
for every vertex.

899
01:04:22,926 --> 01:04:27,175
That seems like a good idea.
Let's do it.

900
01:04:27,175 --> 01:04:33,757
So, shortest paths exist.
Let's just assign x_i to be the

901
01:04:33,757 --> 01:04:36,782
shortest path weight from s to
v_i.

902
01:04:36,782 --> 01:04:39,806
Why not?
That's a good choice for a

903
01:04:39,806 --> 01:04:43,898
number, the shortest path weight
from s to v_i.

904
01:04:43,898 --> 01:04:47,990
This is finite because it's
less than infinity,

905
01:04:47,990 --> 01:04:51,549
and it's greater than minus
infinity, so,

906
01:04:51,549 --> 01:04:55,730
some finite number.
That's what we need to do in

907
01:04:55,730 --> 01:05:00,000
order to satisfy these
constraints.

908
01:05:00,000 --> 01:05:03,933
The claim is that this is a
satisfying assignment.

909
01:05:03,933 --> 01:05:05,860
Why?
Triangle inequality.

910
01:05:05,860 --> 01:05:09,311
Somewhere here we wrote
triangle inequality.

911
01:05:09,311 --> 01:05:12,924
This looks a lot like the
triangle inequality.

912
01:05:12,924 --> 01:05:16,456
In fact, I think that's the end
of the proof.

913
01:05:16,456 --> 01:05:19,908
Let's see here.
What we want to be true with

914
01:05:19,908 --> 01:05:24,564
this assignment is that x_j
minus x_i is less than or equal

915
01:05:24,564 --> 01:05:28,497
to w_ij whenever ij is an edge.
Or, let's say v_i,

916
01:05:28,497 --> 01:05:31,949
v_j, for every such constraint,
so, for v_i,

917
01:05:31,949 --> 01:05:37,313
v_j in the edge set.
OK, so what is this true?

918
01:05:37,313 --> 01:05:42,217
Well, let's just expand it out.
So, x_i is this delta,

919
01:05:42,217 --> 01:05:46,935
and x_j is some other delta.
So, we have delta of s,

920
01:05:46,935 --> 01:05:51,654
vj minus delta of s_vi.
And, on the right-hand side,

921
01:05:51,654 --> 01:05:56,743
well, w_ij, that was the weight
of the edge from I to J.

922
01:05:56,743 --> 01:06:01,000
So, this is the weight of v_i
to v_j.

923
01:06:01,000 --> 01:06:03,659
OK, I will rewrite this
slightly.

924
01:06:03,659 --> 01:06:07,315
Delta s, vj is less than or
equal to delta s,

925
01:06:07,315 --> 01:06:09,060
vi plus w of v_i,
v_j.

926
01:06:09,060 --> 01:06:12,965
And that's the triangle
inequality more or less.

927
01:06:12,965 --> 01:06:18,117
The shortest path from s to v_j
is, at most, shortest path from

928
01:06:18,117 --> 01:06:22,022
s to v_i plus a particular path
from v_i to v_j,

929
01:06:22,022 --> 01:06:24,765
namely the single edge v_i to
v_j.

930
01:06:24,765 --> 01:06:30,000
This could only be longer than
the shortest path.

931
01:06:30,000 --> 01:06:33,372
And so, that makes the
right-hand side bigger,

932
01:06:33,372 --> 01:06:37,644
which makes this inequality
more true, meaning it was true

933
01:06:37,644 --> 01:06:39,967
before.
And now it's still true.

934
01:06:39,967 --> 01:06:42,441
And, that proves it.
This is true.

935
01:06:42,441 --> 01:06:45,513
And, these were all equivalent
statements.

936
01:06:45,513 --> 01:06:48,961
This we know to be true by
triangle inequality.

937
01:06:48,961 --> 01:06:52,408
Therefore, these constraints
are all satisfied.

938
01:06:52,408 --> 01:06:54,357
Magic.
I'm so excited here.

939
01:06:54,357 --> 01:06:59,004
So, we've proved that having a
negative weight cycle is exactly

940
01:06:59,004 --> 01:07:05,000
when these system of difference
constraints are unsatisfiable.

941
01:07:05,000 --> 01:07:08,241
So, if we want to satisfy them,
if we want to find the right

942
01:07:08,241 --> 01:07:10,000
answer to x, we run
Bellman-Ford.

943
01:07:10,000 --> 01:07:12,417
Either it says,
oh, no negative weight cycle.

944
01:07:12,417 --> 01:07:14,945
Then you are hosed.
Then, there is no solution.

945
01:07:14,945 --> 01:07:17,252
But that's the best you could
hope to know.

946
01:07:17,252 --> 01:07:19,670
Otherwise, it says,
oh, there was no negative

947
01:07:19,670 --> 01:07:22,087
weight cycle,
and here are your shortest path

948
01:07:22,087 --> 01:07:23,736
weights.
You just plug them in,

949
01:07:23,736 --> 01:07:26,868
and bam, you have your x_i's
that satisfy the constraints.

950
01:07:26,868 --> 01:07:30,000
Awesome.
Now, it wasn't just any graph.

951
01:07:30,000 --> 01:07:32,877
I mean, we started with
constraints, algebra,

952
01:07:32,877 --> 01:07:35,886
we converted it into a graph by
this transform.

953
01:07:35,886 --> 01:07:37,978
Then we added a source vertex,
s.

954
01:07:37,978 --> 01:07:41,641
So, I mean, we had to build a
graph to solve our problem,

955
01:07:41,641 --> 01:07:43,210
very powerful idea.
Cool.

956
01:07:43,210 --> 01:07:47,135
This is the idea of reduction.
You can reduce the problem you

957
01:07:47,135 --> 01:07:50,601
want to solve into some problem
you know how to solve.

958
01:07:50,601 --> 01:07:54,656
You know how to solve shortest
paths when there are no negative

959
01:07:54,656 --> 01:07:57,337
weight cycles,
or find out that there is a

960
01:07:57,337 --> 01:08:01,000
negative weight cycle by
Bellman-Ford.

961
01:08:01,000 --> 01:08:06,099
So, now we know how to solve
difference constraints.

962
01:08:06,099 --> 01:08:09,400
It turns out you can do even
more.

963
01:08:09,400 --> 01:08:15,000
Bellman-Ford does a little bit
more than just solve these

964
01:08:15,000 --> 01:08:18,899
constraints.
But first let me write down

965
01:08:18,899 --> 01:08:22,899
what I've been jumping up and
down about.

966
01:08:22,899 --> 01:08:27,000
The corollary is you can use
Bellman-Ford.

967
01:08:27,000 --> 01:08:34,484
I mean, you make this graph.
Then you apply Bellman-Ford,

968
01:08:34,484 --> 01:08:41,330
and it will solve your system
of difference constraints.

969
01:08:41,330 --> 01:08:45,685
So, let me put in some numbers
here.

970
01:08:45,685 --> 01:08:49,792
You have m difference
constraints.

971
01:08:49,792 --> 01:08:56,265
And, you have n variables.
And, it will solve them in

972
01:08:56,265 --> 01:09:02,416
order m times n time.
Actually, these numbers go up

973
01:09:02,416 --> 01:09:07,332
slightly because we are adding n
edges, and we're adding one

974
01:09:07,332 --> 01:09:12,000
vertex, but assuming all of
these numbers are nontrivial,

975
01:09:12,000 --> 01:09:14,916
m is at least n.
It's order MN time.

976
01:09:14,916 --> 01:09:20,082
OK, trying to avoid cases where
some of them are close to zero.

977
01:09:20,082 --> 01:09:22,250
Good.
So, some other facts,

978
01:09:22,250 --> 01:09:26,250
that's what I just said.
And we'll leave these as

979
01:09:26,250 --> 01:09:31,000
exercises because they're not
too essential.

980
01:09:31,000 --> 01:09:35,627
The main thing we need is this.
But, some other cool facts is

981
01:09:35,627 --> 01:09:39,484
that Bellman-Ford actually
optimizes some objective

982
01:09:39,484 --> 01:09:42,492
functions.
So, we are saying it's just a

983
01:09:42,492 --> 01:09:46,193
feasibility problem.
We just want to know whether

984
01:09:46,193 --> 01:09:48,738
these constraints are
satisfiable.

985
01:09:48,738 --> 01:09:52,750
In fact, you can add a
particular objective function.

986
01:09:52,750 --> 01:09:56,837
So, you can't give it an
arbitrary objective function,

987
01:09:56,837 --> 01:10:04,647
but here's one of interest.
x_1 plus x_2 plus x_n,

988
01:10:04,647 --> 01:10:15,000
OK, but not just that.
We have some constraints.

989
01:10:24,000 --> 01:10:27,395
OK, this is a linear program.
I want to maximize the sum of

990
01:10:27,395 --> 01:10:30,849
the x_i's subject to all the
x_i's being nonpositive and the

991
01:10:30,849 --> 01:10:33,542
difference constraints.
So, this we had before.

992
01:10:33,542 --> 01:10:35,943
This is fine.
We noticed at some point you

993
01:10:35,943 --> 01:10:38,811
could get from s to everywhere
with cost, at most,

994
01:10:38,811 --> 01:10:40,509
zero.
So, we know that in this

995
01:10:40,509 --> 01:10:42,851
assignment all of the x_i's are
negative.

996
01:10:42,851 --> 01:10:45,602
That's not necessary,
but it's true when you run

997
01:10:45,602 --> 01:10:47,943
Bellman-Ford.
So if you solve your system

998
01:10:47,943 --> 01:10:50,754
using Bellman-Ford,
which is no less general than

999
01:10:50,754 --> 01:10:53,272
anything else,
you happen to get nonpositive

1000
01:10:53,272 --> 01:10:54,969
x_i's.
And so, subject to that

1001
01:10:54,969 --> 01:10:58,072
constraint, it actually makes
them is close to zero as

1002
01:10:58,072 --> 01:11:04,009
possible in the L1 norm.
In the sum of these values,

1003
01:11:04,009 --> 01:11:08,577
it tries to make the sum as
close to zero,

1004
01:11:08,577 --> 01:11:15,154
it tries to make the values as
small as possible in absolute

1005
01:11:15,154 --> 01:11:20,393
value in this sense.
OK, it does more than that.

1006
01:11:20,393 --> 01:11:25,297
It cooks, it cleans,
it finds shortest paths.

1007
01:11:25,297 --> 01:11:31,761
It also minimizes the spread,
the maximum over all i of x_i

1008
01:11:31,761 --> 01:11:37,000
minus the minimum over all i of
x_i.

1009
01:11:37,000 --> 01:11:40,840
So, I mean, if you have your
real line, and here are the

1010
01:11:40,840 --> 01:11:44,402
x_i's wherever they are.
It minimizes this distance.

1011
01:11:44,402 --> 01:11:46,567
And zero is somewhere over
here.

1012
01:11:46,567 --> 01:11:50,268
So, it tries to make the x_i's
as compact as possible.

1013
01:11:50,268 --> 01:11:54,458
This is actually the L infinity
norm, if you know stuff about

1014
01:11:54,458 --> 01:11:56,972
norms from your linear algebra
class.

1015
01:11:56,972 --> 01:12:00,673
OK, this is the L1 norm.
I think it minimizes every LP

1016
01:12:00,673 --> 01:12:05,170
norm.
Good, so let's use this for

1017
01:12:05,170 --> 01:12:09,163
something.
Yeah, let's solve a real

1018
01:12:09,163 --> 01:12:13,978
problem, and then we'll be done
for today.

1019
01:12:13,978 --> 01:12:20,790
Next class we'll see the really
cool stuff, the really cool

1020
01:12:20,790 --> 01:12:27,366
application of all of this.
For now, and we'll see a cool

1021
01:12:27,366 --> 01:12:32,886
but relatively simple
application, which is VLSI

1022
01:12:32,886 --> 01:12:37,528
layout.
We talked a little bit about

1023
01:12:37,528 --> 01:12:40,779
VLSI way back and divide and
conquer.

1024
01:12:40,779 --> 01:12:45,655
You have a bunch of chips,
or you want to arrange them,

1025
01:12:45,655 --> 01:12:50,441
and minimize some objectives.
So, here's a particular,

1026
01:12:50,441 --> 01:12:54,505
tons of problems that come out
of VLSI layout.

1027
01:12:54,505 --> 01:12:59,020
Here's one of them.
You have a bunch of features of

1028
01:12:59,020 --> 01:13:04,583
an integrated circuit.
You want to somehow arrange

1029
01:13:04,583 --> 01:13:09,845
them on your circuit without
putting any two of them too

1030
01:13:09,845 --> 01:13:13,768
close to each other.
You have some minimum

1031
01:13:13,768 --> 01:13:19,030
separation like at least they
should not get top of each

1032
01:13:19,030 --> 01:13:22,283
other.
Probably, you also need some

1033
01:13:22,283 --> 01:13:26,589
separation to put wires in
between, and so on,

1034
01:13:26,589 --> 01:13:33,000
so, without putting any two
features too close together.

1035
01:13:33,000 --> 01:13:37,152
OK, so just to give you an
idea, so I have some objects and

1036
01:13:37,152 --> 01:13:41,089
I'm going to be a little bit
vague about how this works.

1037
01:13:41,089 --> 01:13:43,738
You have some features.
This is stuff,

1038
01:13:43,738 --> 01:13:47,460
some chips, whatever.
We don't really care what their

1039
01:13:47,460 --> 01:13:50,825
shapes look like.
I just want to be able to move

1040
01:13:50,825 --> 01:13:55,192
them around so that the gap at
any point, so let me just think

1041
01:13:55,192 --> 01:13:58,199
about this gap.
This gap should be at least

1042
01:13:58,199 --> 01:14:01,134
some delta.
Or, I don't want to use delta.

1043
01:14:01,134 --> 01:14:05,000
Let's say epsilon,
good, small number.

1044
01:14:05,000 --> 01:14:08,827
So, I just need some separation
between all of my parts.

1045
01:14:08,827 --> 01:14:12,378
And for this problem,
I'm going to be pretty simple,

1046
01:14:12,378 --> 01:14:15,719
just say that the parts are
only allowed to slide

1047
01:14:15,719 --> 01:14:18,433
horizontally.
So, it's a one-dimensional

1048
01:14:18,433 --> 01:14:20,730
problem.
These objects are in 2-d,

1049
01:14:20,730 --> 01:14:23,654
or whatever,
but I can only slide them an x

1050
01:14:23,654 --> 01:14:25,672
coordinate.
So, to model that,

1051
01:14:25,672 --> 01:14:29,570
I'm going to look at the left
edge of every part and say,

1052
01:14:29,570 --> 01:14:32,981
well, these two left edges
should be at least some

1053
01:14:32,981 --> 01:14:36,848
separation.
So, I think of it as whatever

1054
01:14:36,848 --> 01:14:38,952
the distance is plus some
epsilon.

1055
01:14:38,952 --> 01:14:41,501
But, you know,
if you have some funky 2-d

1056
01:14:41,501 --> 01:14:45,135
shapes you have to compute,
well, this is a little bit too

1057
01:14:45,135 --> 01:14:47,621
close because these come into
alignment.

1058
01:14:47,621 --> 01:14:51,063
But, there's some constraint,
well, for any two pieces,

1059
01:14:51,063 --> 01:14:53,677
I could figure out how close
they can get.

1060
01:14:53,677 --> 01:14:57,309
They should get no closer.
So, I'm going to call this x_1.

1061
01:14:57,309 --> 01:15:00,243
I'll call this x_2.
So, we have some constraint

1062
01:15:00,243 --> 01:15:03,111
like x_2 minus x_1 is at least d
plus epsilon,

1063
01:15:03,111 --> 01:15:07,000
or whatever you compute that
weight to be.

1064
01:15:07,000 --> 01:15:09,735
OK, so for every pair of
pieces, I can do this,

1065
01:15:09,735 --> 01:15:13,066
compute some constraint on how
far apart they have to be.

1066
01:15:13,066 --> 01:15:15,861
And, now I'd like to assign
these x coordinates.

1067
01:15:15,861 --> 01:15:18,596
Right now, I'm assuming they're
just variables.

1068
01:15:18,596 --> 01:15:22,105
I want to slide these pieces
around horizontally in order to

1069
01:15:22,105 --> 01:15:25,257
compactify them as much as
possible so they fit in the

1070
01:15:25,257 --> 01:15:28,350
smallest chip that I can make
because it costs money,

1071
01:15:28,350 --> 01:15:31,145
and time, and everything,
and power, everything.

1072
01:15:31,145 --> 01:15:34,000
You always want your chip
small.

1073
01:15:34,000 --> 01:15:40,225
So, Bellman-Ford does that.
All right, so Bellman-Ford

1074
01:15:40,225 --> 01:15:47,626
solves these constraints because
it's just a bunch of difference

1075
01:15:47,626 --> 01:15:51,972
constraints.
And we know that they are

1076
01:15:51,972 --> 01:15:57,963
solvable because you could
spread all the pieces out

1077
01:15:57,963 --> 01:16:03,250
arbitrarily far.
And, it minimizes the spread,

1078
01:16:03,250 --> 01:16:10,298
minimizes the size of the chip
I need, a max of x_i minus the

1079
01:16:10,298 --> 01:16:14,879
min of x_i.
So, this is it maximizes

1080
01:16:14,879 --> 01:16:18,167
compactness, or minimizes size
of the chip.

1081
01:16:18,167 --> 01:16:22,943
OK, this is a one-dimensional
problem, so it may seem a little

1082
01:16:22,943 --> 01:16:27,014
artificial, but the two
dimensional problem is really

1083
01:16:27,014 --> 01:16:29,049
hard to solve.
And this is,

1084
01:16:29,049 --> 01:16:33,355
in fact, the best you can do
with a nice polynomial time

1085
01:16:33,355 --> 01:16:37,419
algorithm.
There are other applications if

1086
01:16:37,419 --> 01:16:42,024
you're scheduling events in,
like, a multimedia environment,

1087
01:16:42,024 --> 01:16:46,629
and you want to guarantee that
this audio plays at least two

1088
01:16:46,629 --> 01:16:50,922
seconds after this video,
but then there are things that

1089
01:16:50,922 --> 01:16:55,605
are playing at the same time,
and they have to be within some

1090
01:16:55,605 --> 01:16:59,351
gap of each other,
so, lots of papers about using

1091
01:16:59,351 --> 01:17:02,786
Bellman-Ford,
solve difference constraints to

1092
01:17:02,786 --> 01:17:06,766
enable multimedia environments.
OK, so there you go.

1093
01:17:06,766 --> 01:17:11,449
And next class we'll see more
applications of Bellman-Ford to

1094
01:17:11,449 --> 01:17:14,181
all pairs shortest paths.
Questions?

1095
01:17:14,181 --> 01:17:17,000
Great.