1
00:00:07,000 --> 00:00:10,000
We're going to talk about
shortest paths,

2
00:00:10,000 --> 00:00:14,000
and we're going to talk about
shortest paths for three

3
00:00:14,000 --> 00:00:17,000
lectures.
So, this is a trilogy.

4
00:00:17,000 --> 00:00:20,000
Today will be Shortest Paths
One.

5
00:00:20,000 --> 00:00:25,000
I've been watching far too many
versions of Star Wars this

6
00:00:25,000 --> 00:00:28,000
weekend.
I saw the musical yesterday,

7
00:00:28,000 --> 00:00:31,000
matinee.
That was an MIT musical.

8
00:00:31,000 --> 00:00:35,000
That was fun,
of all three movies in about

9
00:00:35,000 --> 00:00:38,000
four hours.
That was a bit long and then I

10
00:00:38,000 --> 00:00:42,000
saw the one-man show on Friday.
One-man Star Wars:

11
00:00:42,000 --> 00:00:45,000
the original three movies in
one hour.

12
00:00:45,000 --> 00:00:48,000
That was the opposite of too
long.

13
00:00:48,000 --> 00:00:51,000
Both were fun.
So I get my trilogy fix.

14
00:00:51,000 --> 00:00:54,000
All episodes,
first we're going to start with

15
00:00:54,000 --> 00:00:58,000
The New Hope,
and we're going to talk about

16
00:00:58,000 --> 00:01:02,000
the shortest paths problem and
solve one particular problem of

17
00:01:02,000 --> 00:01:09,000
it, a very interesting version.
And then we're going to look at

18
00:01:09,000 --> 00:01:12,000
increasingly more general
versions as we go on.

19
00:01:12,000 --> 00:01:15,000
Shortest paths are sort of an
application of dynamic

20
00:01:15,000 --> 00:01:19,000
programming, which we saw last
week, and greedy algorithms,

21
00:01:19,000 --> 00:01:23,000
which we also saw last week.
So, were going to build that

22
00:01:23,000 --> 00:01:27,000
and get some pretty interesting
algorithms for an important

23
00:01:27,000 --> 00:01:30,000
problem, which is how to get
from Alderon to,

24
00:01:30,000 --> 00:01:33,000
I don't know,
Cambridge as quickly as

25
00:01:33,000 --> 00:01:37,000
possible, OK,
when you live in a graph.

26
00:01:37,000 --> 00:01:41,000
So, there's geometric shortest
paths which is a little bit

27
00:01:41,000 --> 00:01:44,000
harder.
Here, we're just going to look

28
00:01:44,000 --> 00:01:48,000
at shortest paths in graphs.
Now, hopefully you all know

29
00:01:48,000 --> 00:01:52,000
what a path in a graph is.
But, so, very quick review in

30
00:01:52,000 --> 00:01:56,000
particular because we're going
to be looking at weighted

31
00:01:56,000 --> 00:01:59,000
graphs.
So, the usual setup:

32
00:01:59,000 --> 00:02:03,000
suppose we have directed graph,
G, have some vertices,

33
00:02:03,000 --> 00:02:07,000
some edges.
We have edge weights,

34
00:02:07,000 --> 00:02:12,000
make it a little more
interesting.

35
00:02:12,000 --> 00:02:18,000
So, this is just a real number
on each edge.

36
00:02:18,000 --> 00:02:25,000
So, edge weights are usually
given by function,

37
00:02:25,000 --> 00:02:27,000
w.
For every edge,

38
00:02:27,000 --> 00:02:32,000
you get a real number.

39
00:02:40,000 --> 00:02:43,000
And then, if we look at the
paths in the graph,

40
00:02:43,000 --> 00:02:47,000
so we're going to use some
simple notation for paths called

41
00:02:47,000 --> 00:02:51,000
a path, p, starts at some
vertex, and it goes to some

42
00:02:51,000 --> 00:02:52,000
other vertex,
and so on.

43
00:02:52,000 --> 00:02:56,000
Say the last vertex is v_k,
and each of these should be a

44
00:02:56,000 --> 00:03:00,000
directed edge in the digraph.
So, this is a directed path.

45
00:03:00,000 --> 00:03:04,000
It has to respect edges in
here.

46
00:03:04,000 --> 00:03:10,000
And, we'll say that the weight
of such a path is just the sum

47
00:03:10,000 --> 00:03:14,000
of the weights of the edges
along the path.

48
00:03:14,000 --> 00:03:18,000
And, we'll call that w(p).
This is sum,

49
00:03:18,000 --> 00:03:23,000
i equals one to k minus one of
w(v_i, v_(i+1)) plus one.

50
00:03:23,000 --> 00:03:27,000
OK, so just to rub it in,
and in particular,

51
00:03:27,000 --> 00:03:32,000
how general this can be,
we have some path,

52
00:03:32,000 --> 00:03:37,000
it starts at some vertex,
there's some edge weights along

53
00:03:37,000 --> 00:03:42,000
the way.
This is some arbitrary path in

54
00:03:42,000 --> 00:03:46,000
the graph, in some hypothetical
graph.

55
00:03:58,000 --> 00:04:01,000
OK, this is mainly to point out
that some of the edge weights

56
00:04:01,000 --> 00:04:04,000
could be negative.
Some of them could be zero.

57
00:04:04,000 --> 00:04:08,000
This sum here is minus two.
So, the weight of this path is

58
00:04:08,000 --> 00:04:09,000
minus two.
And, presumably,

59
00:04:09,000 --> 00:04:12,000
the graph is much bigger than
this.

60
00:04:12,000 --> 00:04:14,000
This is just one path in the
graph.

61
00:04:14,000 --> 00:04:18,000
We're usually thinking about
simple paths that can't repeat a

62
00:04:18,000 --> 00:04:20,000
vertex.
But, sometimes we allow that.

63
00:04:20,000 --> 00:04:23,000
And then, what we care about is
the shortest path,

64
00:04:23,000 --> 00:04:26,000
or a shortest path.
Again, this may not be unique,

65
00:04:26,000 --> 00:04:31,000
but we'll still usually call it
the shortest path.

66
00:04:31,000 --> 00:04:36,000
So, we want the shortest path
from some A to some B.

67
00:04:36,000 --> 00:04:39,000
Or, we'll call the vertices u
and v.

68
00:04:39,000 --> 00:04:45,000
And we want this to be some
path of minimum possible weight,

69
00:04:45,000 --> 00:04:49,000
subject to starting at u,
and going to v.

70
00:04:49,000 --> 00:04:53,000
OK, so that's what we're
looking for.

71
00:04:53,000 --> 00:04:58,000
In general, give you a vertex,
u, give you a vertex,

72
00:04:58,000 --> 00:05:04,000
v, find a shortest path as
quickly as possible.

73
00:05:04,000 --> 00:05:06,000
What's a good algorithm for
that?

74
00:05:06,000 --> 00:05:10,000
That's the topic for the next
three lectures.

75
00:05:10,000 --> 00:05:15,000
We'll usually think about a
slightly simpler problem,

76
00:05:15,000 --> 00:05:19,000
which is just computing the
weight of that path,

77
00:05:19,000 --> 00:05:24,000
which is essentially computing
the distance from A to B.

78
00:05:24,000 --> 00:05:28,000
So, we'll call this the
shortest path weight from u to

79
00:05:28,000 --> 00:05:33,000
v.
And, we'll denote it by delta

80
00:05:33,000 --> 00:05:38,000
of (u,v), delta .
So, I mean, it's the weight of

81
00:05:38,000 --> 00:05:43,000
the shortest path,
or a weight of every shortest

82
00:05:43,000 --> 00:05:45,000
path.
Or, in other words,

83
00:05:45,000 --> 00:05:51,000
it's the Min over the weight of
each path from u to v.

84
00:05:51,000 --> 00:05:56,000
So, p here is a path.
OK, so you just consider,

85
00:05:56,000 --> 00:06:02,000
there could be a lot of
different paths.

86
00:06:02,000 --> 00:06:04,000
There could,
in principle,

87
00:06:04,000 --> 00:06:09,000
be infinitely many,
if you're allowed to repeat

88
00:06:09,000 --> 00:06:13,000
vertices.
You look at all those paths

89
00:06:13,000 --> 00:06:17,000
hypothetically.
You take the minimum weight.

90
00:06:17,000 --> 00:06:19,000
Question?
Good.

91
00:06:19,000 --> 00:06:25,000
My next question was going to
be, when do shortest paths not

92
00:06:25,000 --> 00:06:28,000
exist?
And you've hit upon one

93
00:06:28,000 --> 00:06:36,000
version, which is when you have
negative edge weights.

94
00:06:36,000 --> 00:06:40,000
So, in principle,
when you have negative edge

95
00:06:40,000 --> 00:06:45,000
weights, some shortest paths may
not exist in the sense that

96
00:06:45,000 --> 00:06:50,000
there is no shortest paths.
There are no shortest paths.

97
00:06:50,000 --> 00:06:54,000
There is no shortest path from
u to v.

98
00:06:54,000 --> 00:06:58,000
OK, in particular,
if I have two vertices,

99
00:06:58,000 --> 00:07:02,000
u and v, and I want the
shortest path between them,

100
00:07:02,000 --> 00:07:06,000
and I have negative edge
weights, well,

101
00:07:06,000 --> 00:07:12,000
this is fine.
I mean, I can still compute the

102
00:07:12,000 --> 00:07:17,000
weight of a path that has
negative weights.

103
00:07:17,000 --> 00:07:24,000
But when specifically won't I
have a single shortest path from

104
00:07:24,000 --> 00:07:26,000
u to v?
So, go ahead.

105
00:07:26,000 --> 00:07:29,000
Good.
So, if I can find the cycle

106
00:07:29,000 --> 00:07:34,000
somewhere along here whose total
weight, say, the sum of all the

107
00:07:34,000 --> 00:07:37,000
weights of these images is
negative, then I get there,

108
00:07:37,000 --> 00:07:40,000
I go around as many times as I
want.

109
00:07:40,000 --> 00:07:44,000
I keep decreasing the weight
because the weight is negative.

110
00:07:44,000 --> 00:07:48,000
I decrease it by some fixed
amount, and then I can go to v.

111
00:07:48,000 --> 00:07:52,000
So, as long as there is a
negative weights cycle reachable

112
00:07:52,000 --> 00:07:56,000
from u that can also reach v,
then there's no shortest path

113
00:07:56,000 --> 00:08:00,000
because if I take any particular
path, I can make it shorter by

114
00:08:00,000 --> 00:08:04,000
going around a couple more
times.

115
00:08:04,000 --> 00:08:07,000
So, in some sense,
this is not really a minimum.

116
00:08:07,000 --> 00:08:11,000
It's more like an infimum for
those who like to get fancy

117
00:08:11,000 --> 00:08:14,000
about such things.
But we'll just say that delta

118
00:08:14,000 --> 00:08:16,000
of (u,v) is minus infinity in
this case.

119
00:08:16,000 --> 00:08:19,000
There's a negative weights
cycle from u to v.

120
00:08:19,000 --> 00:08:23,000
So, that's one case we have to
worry about in some sense.

121
00:08:23,000 --> 00:08:27,000
But, as long as there are no
negative weight cycles,

122
00:08:27,000 --> 00:08:30,000
delta of (u,v) will be
something bigger than minus

123
00:08:30,000 --> 00:08:34,000
infinity, bounded below by some
finite value even if you could

124
00:08:34,000 --> 00:08:38,000
have negative weights,
but still no negative weights

125
00:08:38,000 --> 00:08:41,000
cycle for example,
there might not be any cycles

126
00:08:41,000 --> 00:08:45,000
in your graph.
So that's still interesting.

127
00:08:45,000 --> 00:08:50,000
And, I guess it's useful to
note that you can get from A to

128
00:08:50,000 --> 00:08:53,000
B in negative infinite time.
It's time travel,

129
00:08:53,000 --> 00:08:56,000
if the weights happen that
correspond to time.

130
00:08:56,000 --> 00:09:00,000
But when else might shortest
paths not exist?

131
00:09:00,000 --> 00:09:04,000
So, this is one case,
but there's another,

132
00:09:04,000 --> 00:09:07,000
simpler case.
It's not connected.

133
00:09:07,000 --> 00:09:12,000
There might not be any path
from u to v.

134
00:09:12,000 --> 00:09:17,000
This path might be empty.
There may be no path from u to

135
00:09:17,000 --> 00:09:21,000
v.
Here we have to define what

136
00:09:21,000 --> 00:09:25,000
happens, and here,
we'll say it's infinity if

137
00:09:25,000 --> 00:09:32,000
there's no path from u to v.
So, there are these exceptional

138
00:09:32,000 --> 00:09:35,000
cases plus infinity and minus
infinity, which are pretty

139
00:09:35,000 --> 00:09:39,000
intuitive because it takes a
really long time to get from u

140
00:09:39,000 --> 00:09:44,000
to v if there's no path there.
You can't get there from here.

141
00:09:44,000 --> 00:09:47,000
OK, but that's the definition.
Most of the time,

142
00:09:47,000 --> 00:09:50,000
this is the case we care about,
of course.

143
00:09:50,000 --> 00:09:53,000
Usually this is a finite set.
OK, good, so that's the

144
00:09:53,000 --> 00:09:56,000
definition.
We're going to get a few basic

145
00:09:56,000 --> 00:10:00,000
structural properties about
shortest paths that will allow

146
00:10:00,000 --> 00:10:04,000
us to obtain good algorithms
finding these paths when they

147
00:10:04,000 --> 00:10:07,000
exist.
And, in particular,

148
00:10:07,000 --> 00:10:10,000
we want to use ideas from
dynamic programming.

149
00:10:10,000 --> 00:10:14,000
So, if I want to use dynamic
programming to solve shortest

150
00:10:14,000 --> 00:10:17,000
paths, what do I need to
establish?

151
00:10:17,000 --> 00:10:19,000
What's the first thing I should
check?

152
00:10:19,000 --> 00:10:23,000
You've all implemented dynamic
programming by now,

153
00:10:23,000 --> 00:10:27,000
so should make complete sense
hopefully, at least more sense

154
00:10:27,000 --> 00:10:30,000
than it did a couple of weeks
ago, last week,

155
00:10:30,000 --> 00:10:34,000
when we learned it.
Dynamic programming is

156
00:10:34,000 --> 00:10:39,000
something that grows on you.
Every year I think I understand

157
00:10:39,000 --> 00:10:42,000
it better than the previous
year.

158
00:10:42,000 --> 00:10:45,000
But, in particular,
when you learned dynamic

159
00:10:45,000 --> 00:10:50,000
programming in this class,
there is this nice key property

160
00:10:50,000 --> 00:10:52,000
that you should check.
Yeah?

161
00:10:52,000 --> 00:10:54,000
Optimal substructure:
good.

162
00:10:54,000 --> 00:10:58,000
This is the phrase you should
keep in mind.

163
00:10:58,000 --> 00:11:03,000
It's not really enough for
dynamic programming to be useful

164
00:11:03,000 --> 00:11:07,000
in an efficient way,
but it at least tells you that

165
00:11:07,000 --> 00:11:12,000
you should be able to try to
apply it.

166
00:11:12,000 --> 00:11:15,000
That's a pretty weak statement,
but it's something that you

167
00:11:15,000 --> 00:11:18,000
should check.
It's definitely pretty much a

168
00:11:18,000 --> 00:11:22,000
necessary condition for dynamic
programming to make sense.

169
00:11:22,000 --> 00:11:26,000
And so, optimal some structure
here means that if I take some

170
00:11:26,000 --> 00:11:29,000
shortest path,
and I look at a subpath of that

171
00:11:29,000 --> 00:11:32,000
shortest path,
I claimed that it too is a

172
00:11:32,000 --> 00:11:34,000
shortest path,
OK, with its respective

173
00:11:34,000 --> 00:11:39,000
endpoints; obviously not between
the same endpoints.

174
00:11:39,000 --> 00:11:43,000
But if I have some shortest
path between two endpoints,

175
00:11:43,000 --> 00:11:47,000
I take any subpath and that's
also the shortest path.

176
00:11:47,000 --> 00:11:51,000
This is one version of optimal
substructure.

177
00:11:51,000 --> 00:11:55,000
This one turns out to be true
for this setup.

178
00:11:55,000 --> 00:11:59,000
And, how should I prove an
optimal substructure property?

179
00:11:59,000 --> 00:12:04,000
Cut and paste.
Yep, that works here too.

180
00:12:04,000 --> 00:12:09,000
I mean, this isn't always true.
But it's a good technique here.

181
00:12:09,000 --> 00:12:14,000
So, we're going to think about,
and I'll do essentially a proof

182
00:12:14,000 --> 00:12:17,000
by picture here.
So, suppose you have some

183
00:12:17,000 --> 00:12:22,000
subpath of some shortest path.
So, let's say the subpath is x

184
00:12:22,000 --> 00:12:25,000
to y.
And, the path goes from u to v.

185
00:12:25,000 --> 00:12:30,000
So, we assume that (u,v) is a
shortest path.

186
00:12:30,000 --> 00:12:33,000
We want to prove that (x,y) is
a shortest path.

187
00:12:33,000 --> 00:12:36,000
Well, suppose (x,y) isn't a
shortest path.

188
00:12:36,000 --> 00:12:40,000
Then there is some shorter path
that goes from x to y.

189
00:12:40,000 --> 00:12:45,000
But, if you have some shorter
path from x to y than this one.

190
00:12:45,000 --> 00:12:49,000
Then I should just erase this
part of the shortest path from u

191
00:12:49,000 --> 00:12:52,000
to v, and replace it with this
shorter one.

192
00:12:52,000 --> 00:12:56,000
So, this is some hypothetical
shorter path.

193
00:12:56,000 --> 00:12:59,000
So, suppose this existed.
If that existed,

194
00:12:59,000 --> 00:13:02,000
then I should just cut the old
path from x to y,

195
00:13:02,000 --> 00:13:07,000
and paste in this new one from
x to y.

196
00:13:07,000 --> 00:13:10,000
It's strictly shorter.
Therefore, I get a strictly

197
00:13:10,000 --> 00:13:14,000
shorter path from u to v.
But I assumed u to v was a

198
00:13:14,000 --> 00:13:16,000
shortest path:
contradiction.

199
00:13:16,000 --> 00:13:18,000
OK, so there is no shorter
path.

200
00:13:18,000 --> 00:13:21,000
And that proves the lemma that
we have this:

201
00:13:21,000 --> 00:13:25,000
subpaths of shortest paths are
shortest paths.

202
00:13:25,000 --> 00:13:29,000
OK, this should now be a pretty
familiar proof technique.

203
00:13:29,000 --> 00:13:34,000
But, there is yet another
instance of cut and paste.

204
00:13:34,000 --> 00:13:36,000
OK, so that's a good sign for
computing shortest paths.

205
00:13:36,000 --> 00:13:39,000
I mean, in terms of dynamic
programming, we won't look

206
00:13:39,000 --> 00:13:42,000
directly at dynamic programming
here because we are going to aim

207
00:13:42,000 --> 00:13:44,000
for greedy, which is even
stronger.

208
00:13:44,000 --> 00:13:47,000
But, next Monday we'll see some
dynamic programming approaches.

209
00:13:47,000 --> 00:13:49,000
Intuitively,
there are some pretty natural

210
00:13:49,000 --> 00:13:52,000
sub-problems here.
I mean, going from u to v,

211
00:13:52,000 --> 00:13:55,000
if I want to find what is the
shortest path from u to v,

212
00:13:55,000 --> 00:13:56,000
well, that's a particular
problem.

213
00:13:56,000 --> 00:13:59,000
Maybe it involves computing
shortest paths from u to some

214
00:13:59,000 --> 00:14:01,000
intermediate point,
x, and then from x to u,

215
00:14:01,000 --> 00:14:05,000
something like that.
That feels good.

216
00:14:05,000 --> 00:14:07,000
That's like,
quadratically,

217
00:14:07,000 --> 00:14:11,000
many subproblems.
And, V^2 subproblems,

218
00:14:11,000 --> 00:14:16,000
it sounds like that would lead
to a dynamic program.

219
00:14:16,000 --> 00:14:21,000
You can make it work out;
it's just a little bit trickier

220
00:14:21,000 --> 00:14:25,000
than that.
We'll see that next Monday.

221
00:14:25,000 --> 00:14:29,000
But thinking about this
intermediate point we get

222
00:14:29,000 --> 00:14:35,000
something called the triangle
inequality.

223
00:14:35,000 --> 00:14:40,000
So, you've probably heard some
form of the triangle inequality

224
00:14:40,000 --> 00:14:42,000
before.
It holds in all sorts of

225
00:14:42,000 --> 00:14:46,000
geometric spaces,
but it also holds for shortest

226
00:14:46,000 --> 00:14:51,000
paths, which is slightly less
obvious, or more obvious,

227
00:14:51,000 --> 00:14:54,000
I guess, depending on your
inclination.

228
00:14:54,000 --> 00:14:59,000
So, if you have any triple of
vertices, the shortest path from

229
00:14:59,000 --> 00:15:03,000
u to v is, at most,
the shortest path from u to x

230
00:15:03,000 --> 00:15:08,000
plus the shortest path from x to
v.

231
00:15:08,000 --> 00:15:10,000
Of course, here I need a
shortest path weight from u to

232
00:15:10,000 --> 00:15:12,000
x, and shortest path weight from
x to v.

233
00:15:12,000 --> 00:15:15,000
So, this should be pretty
natural just from the statement,

234
00:15:15,000 --> 00:15:18,000
even more natural if you draw
the picture.

235
00:15:18,000 --> 00:15:19,000
So, we have some vertex,
u.

236
00:15:19,000 --> 00:15:22,000
I'm using wiggly lines to
denote potentially long paths as

237
00:15:22,000 --> 00:15:24,000
opposed to edges.
We have some intermediate

238
00:15:24,000 --> 00:15:27,000
point, x, and we have some
target, v, and we are

239
00:15:27,000 --> 00:15:30,000
considering these three shortest
paths.

240
00:15:30,000 --> 00:15:35,000
This is the shortest path from
u to v, or this is its weights.

241
00:15:35,000 --> 00:15:38,000
This is the shortest path from
u to x.

242
00:15:38,000 --> 00:15:42,000
And here's its weight,
and the shortest path from x to

243
00:15:42,000 --> 00:15:44,000
v.
And here's its weight.

244
00:15:44,000 --> 00:15:48,000
And, the point is,
this should be the shortest

245
00:15:48,000 --> 00:15:51,000
path or a shortest path from u
to v.

246
00:15:51,000 --> 00:15:55,000
And, in particular,
one such path is you go from u

247
00:15:55,000 --> 00:16:00,000
to x, and then you go from x to
v.

248
00:16:00,000 --> 00:16:04,000
So, I mean, this sum is just
measuring the length of this

249
00:16:04,000 --> 00:16:07,000
particular path.
Take the shortest path here;

250
00:16:07,000 --> 00:16:12,000
take the shortest path here.
And, this is supposed to be the

251
00:16:12,000 --> 00:16:15,000
Min over all paths.
So, certainly this is,

252
00:16:15,000 --> 00:16:19,000
at most, this particular path,
the sum of these two values,

253
00:16:19,000 --> 00:16:22,000
OK, another proof by picture.
Clear?

254
00:16:22,000 --> 00:16:26,000
OK, this stuff is easy.
I assume we'll get into some

255
00:16:26,000 --> 00:16:31,000
more set exciting algorithms in
particular, which is always more

256
00:16:31,000 --> 00:16:36,000
exciting.
Today, we're going to look at a

257
00:16:36,000 --> 00:16:41,000
particular version of shortest
paths called,

258
00:16:41,000 --> 00:16:46,000
or the shortest paths problem
called the single source

259
00:16:46,000 --> 00:16:51,000
shortest path problem.
OK, it's a little bit more

260
00:16:51,000 --> 00:16:56,000
general than go from A to B.
The problem is,

261
00:16:56,000 --> 00:17:03,000
you're given a source vertex,
and you want to know how to get

262
00:17:03,000 --> 00:17:09,000
from that source vertex to
everywhere else.

263
00:17:09,000 --> 00:17:12,000
So, we'll call this source
vertex s.

264
00:17:12,000 --> 00:17:15,000
And from that source,
we want to find,

265
00:17:15,000 --> 00:17:21,000
let's say, the shortest path
weights from s to everyone.

266
00:17:21,000 --> 00:17:24,000
In particular,
we'd also like to know the

267
00:17:24,000 --> 00:17:30,000
shortest paths,
but that isn't too much harder.

268
00:17:30,000 --> 00:17:33,000
So, that's delta of s,
v for all vertices,

269
00:17:33,000 --> 00:17:35,000
v.
OK, so this is actually a

270
00:17:35,000 --> 00:17:39,000
little bit harder than the
problem we started with a

271
00:17:39,000 --> 00:17:41,000
getting from Alderon to
Cambridge.

272
00:17:41,000 --> 00:17:46,000
Now, we want to get from
Alderon to the entire universe.

273
00:17:46,000 --> 00:17:50,000
OK, it turns out,
this is one of the weird things

274
00:17:50,000 --> 00:17:53,000
about shortest paths,
according to the

275
00:17:53,000 --> 00:17:57,000
state-of-the-art we know today,
it seems like the following

276
00:17:57,000 --> 00:18:02,000
statement will remain true for
all time.

277
00:18:02,000 --> 00:18:05,000
But we don't know.
The best algorithms for solving

278
00:18:05,000 --> 00:18:08,000
the A to B problem,
given s, given t,

279
00:18:08,000 --> 00:18:11,000
go from s to t,
is no easier than this problem.

280
00:18:11,000 --> 00:18:15,000
It's the best ways we know how
to solve going from A to B is to

281
00:18:15,000 --> 00:18:18,000
solve how to go from A to
everywhere else.

282
00:18:18,000 --> 00:18:22,000
So, we sort of can't help
ourselves, but to solve this

283
00:18:22,000 --> 00:18:26,000
problem it turns out.
Today, we're going to look at a

284
00:18:26,000 --> 00:18:30,000
further restriction on this
problem because this is a bit

285
00:18:30,000 --> 00:18:34,000
tricky.
Will solve it next class.

286
00:18:34,000 --> 00:18:39,000
But, today we're going to get
rid of the negative weight cycle

287
00:18:39,000 --> 00:18:42,000
issue by forbidding negative
weights.

288
00:18:42,000 --> 00:18:47,000
So, we're going to assume that
all of the edge weights are

289
00:18:47,000 --> 00:18:50,000
nonnegative, so,
for all vertices,

290
00:18:50,000 --> 00:18:53,000
u and v.
So, in particular,

291
00:18:53,000 --> 00:18:58,000
shortest paths exist,
provided paths exist.

292
00:19:12,000 --> 00:19:15,000
And, we don't have to worry
about these minus infinities.

293
00:19:15,000 --> 00:19:18,000
Delta of (u,v) is always bigger
than minus infinity.

294
00:19:18,000 --> 00:19:21,000
It still might be plus infinity
if there is no path,

295
00:19:21,000 --> 00:19:23,000
but this will make life a lot
easier.

296
00:19:23,000 --> 00:19:26,000
And the algorithm we'll cover
today really requires this

297
00:19:26,000 --> 00:19:30,000
property.
You can't get away without it.

298
00:19:30,000 --> 00:19:36,000
Next class, we'll get away
without it with a fancier and

299
00:19:36,000 --> 00:19:40,000
slower algorithm.
So, as I hinted at,

300
00:19:40,000 --> 00:19:47,000
the main idea we're going to
use for the algorithm today is

301
00:19:47,000 --> 00:19:54,000
greedy, which should be faster
than dynamic programming

302
00:19:54,000 --> 00:19:58,000
generally.
And, the tricky part will be

303
00:19:58,000 --> 00:20:06,000
proving that the greedy
algorithm actually works.

304
00:20:06,000 --> 00:20:11,000
So, I think there's pretty much
only one natural way to go

305
00:20:11,000 --> 00:20:16,000
about, well, there's one way
that works to go about greedy,

306
00:20:16,000 --> 00:20:19,000
let's say.
This may be not the obvious

307
00:20:19,000 --> 00:20:22,000
one.
So, let me give you a little

308
00:20:22,000 --> 00:20:26,000
bit of setup.
The invariant we are going to

309
00:20:26,000 --> 00:20:31,000
maintain is that at all times,
we have estimates on the

310
00:20:31,000 --> 00:20:36,000
distances from the source to
every vertex.

311
00:20:36,000 --> 00:20:39,000
When I say distance,
I mean shortest path weight.

312
00:20:39,000 --> 00:20:43,000
I'm going to use weight and
distance interchangeably here

313
00:20:43,000 --> 00:20:46,000
for more intuition.
And, in particular,

314
00:20:46,000 --> 00:20:50,000
I want to maintain the set of
vertices where those estimates

315
00:20:50,000 --> 00:20:53,000
are actually the right answer.

316
00:21:10,000 --> 00:21:13,000
OK, this is little s.
This is big S.

317
00:21:13,000 --> 00:21:18,000
So, the big S will be the set
of all vertices where I know the

318
00:21:18,000 --> 00:21:21,000
answer.
What is the shortest path

319
00:21:21,000 --> 00:21:25,000
distance from little S to that
vertex in big S?

320
00:21:25,000 --> 00:21:29,000
So, for starters,
which distance do I know?

321
00:21:29,000 --> 00:21:31,000
Sorry?
s.

322
00:21:31,000 --> 00:21:34,000
I know the shortest path
distance from s to s because if

323
00:21:34,000 --> 00:21:37,000
I assume that all of my weights
are nonnegative,

324
00:21:37,000 --> 00:21:40,000
I really can't get from s to s
any faster than not doing

325
00:21:40,000 --> 00:21:43,000
anything.
OK, if I had a negative weight

326
00:21:43,000 --> 00:21:46,000
cycle, maybe the distance from s
to s is minus infinity.

327
00:21:46,000 --> 00:21:50,000
OK, but I can't have negative
weights so there's no way I can

328
00:21:50,000 --> 00:21:53,000
get from s to s any faster than
zero time.

329
00:21:53,000 --> 00:21:56,000
There might be a longer path
that still has zero cost,

330
00:21:56,000 --> 00:22:00,000
but it can't be any better than
zero.

331
00:22:00,000 --> 00:22:02,000
So, in particular,
I know that.

332
00:22:02,000 --> 00:22:05,000
So, initially,
S is certainly an s.

333
00:22:05,000 --> 00:22:09,000
OK, and the idea is we're going
to accumulate more and more

334
00:22:09,000 --> 00:22:14,000
vertices that we know.
So, at some point we know the

335
00:22:14,000 --> 00:22:16,000
distances from some of the
vertices.

336
00:22:16,000 --> 00:22:21,000
So, we have some cloud here.
This is S, and this is

337
00:22:21,000 --> 00:22:23,000
everything else.
This is the graph,

338
00:22:23,000 --> 00:22:26,000
G.
This is the subset of the

339
00:22:26,000 --> 00:22:29,000
vertices.
And, there's some edges that go

340
00:22:29,000 --> 00:22:33,000
out from there.
And, so we have estimates on

341
00:22:33,000 --> 00:22:36,000
how to get to these vertices.
Some of them,

342
00:22:36,000 --> 00:22:39,000
we may not have even seen yet.
They may not be connected to

343
00:22:39,000 --> 00:22:42,000
this portion of S.
I mean: not directly.

344
00:22:42,000 --> 00:22:44,000
They might be connected by some
longer path.

345
00:22:44,000 --> 00:22:48,000
They might be in a completely
different connected component.

346
00:22:48,000 --> 00:22:50,000
We don't know yet.
Some of them,

347
00:22:50,000 --> 00:22:53,000
we have estimates for because
we've sort of seen how to get

348
00:22:53,000 --> 00:22:55,000
there from S.
And the idea is,

349
00:22:55,000 --> 00:22:58,000
among all of these nodes where
we have estimates,

350
00:22:58,000 --> 00:23:01,000
and on to get from little S,
which is some vertex in here,

351
00:23:01,000 --> 00:23:04,000
to these vertices,
we're going to take the one for

352
00:23:04,000 --> 00:23:10,000
which the estimate is smallest.
That's the greedy choice.

353
00:23:10,000 --> 00:23:15,000
And, we're just going to add
that vertex to S.

354
00:23:15,000 --> 00:23:18,000
So, S grows one vertex per
step.

355
00:23:18,000 --> 00:23:23,000
Each step, we're going to add
to S, the vertex.

356
00:23:23,000 --> 00:23:27,000
Of course, again,
this is not a unique,

357
00:23:27,000 --> 00:23:32,000
it's a vertex,
v, in V minus S.

358
00:23:32,000 --> 00:23:39,000
So, it's something we haven't
yet computed yet whose estimated

359
00:23:39,000 --> 00:23:47,000
distance from S is minimum.
So, we look at all the vertices

360
00:23:47,000 --> 00:23:55,000
we haven't yet added to S.
Just take the one where we have

361
00:23:55,000 --> 00:24:01,000
the estimated smallest distance.
The intuition is that that

362
00:24:01,000 --> 00:24:03,000
should be a good choice.
So, if I pick the one that's

363
00:24:03,000 --> 00:24:06,000
closest to little s among all
the ones that I've seen,

364
00:24:06,000 --> 00:24:09,000
among all the paths that I've
seen, I sort of have to buy into

365
00:24:09,000 --> 00:24:11,000
that those are good paths.
But, I mean,

366
00:24:11,000 --> 00:24:13,000
maybe there's some path I
didn't see.

367
00:24:13,000 --> 00:24:16,000
Maybe you go out to here and
then you take some other path to

368
00:24:16,000 --> 00:24:18,000
some vertex, which we've already
seen.

369
00:24:18,000 --> 00:24:20,000
OK, the worry is,
well, I'd better not say that

370
00:24:20,000 --> 00:24:23,000
that's the shortest path because
there may have been some other

371
00:24:23,000 --> 00:24:25,000
way to get there.
Right, as soon as I add

372
00:24:25,000 --> 00:24:27,000
something to S,
I declare I've solved the

373
00:24:27,000 --> 00:24:32,000
problem for that vertex.
I can't change my answer later.

374
00:24:32,000 --> 00:24:35,000
OK, the estimates can change
until they get added to S.

375
00:24:35,000 --> 00:24:39,000
So, I don't want to add this
vertex to S because I haven't

376
00:24:39,000 --> 00:24:43,000
considered this path.
Well, if all my weights are

377
00:24:43,000 --> 00:24:47,000
nonnegative, and I take the
vertex here that has the

378
00:24:47,000 --> 00:24:50,000
shortest estimate from S,
so let's suppose this one is

379
00:24:50,000 --> 00:24:54,000
the shortest one,
then this can't be a shorter

380
00:24:54,000 --> 00:24:57,000
path because the distance
estimate, at least,

381
00:24:57,000 --> 00:25:02,000
from S to that vertex is larger
from S to that vertex.

382
00:25:02,000 --> 00:25:05,000
So, no way can I make the path
longer and decrease the

383
00:25:05,000 --> 00:25:07,000
distance.
That's the intuition.

384
00:25:07,000 --> 00:25:10,000
OK, it's a little bit fuzzy
here because I don't have any

385
00:25:10,000 --> 00:25:14,000
induction hypotheses set up,
and it's going to be a lot more

386
00:25:14,000 --> 00:25:17,000
work to prove that.
But that's the intuition why

387
00:25:17,000 --> 00:25:21,000
this is the right thing to do.
OK, you have to prove something

388
00:25:21,000 --> 00:25:25,000
about the distance estimates for
that to be a proof.

389
00:25:25,000 --> 00:25:27,000
But, intuitively,
it feels good.

390
00:25:27,000 --> 00:25:32,000
It was a good starting point.
OK, and then presumably we have

391
00:25:32,000 --> 00:25:34,000
to maintain these distance
estimates.

392
00:25:34,000 --> 00:25:38,000
So, the heart of the algorithm
is updating distance estimates,

393
00:25:38,000 --> 00:25:41,000
I mean, choosing the best
vertex to add to S,

394
00:25:41,000 --> 00:25:44,000
that's one step.
Then, updating the distance

395
00:25:44,000 --> 00:25:46,000
estimates is sort of where the
work is.

396
00:25:46,000 --> 00:25:50,000
And, it turns out we'll only
need to update distance

397
00:25:50,000 --> 00:25:53,000
estimates of some of the
vertices, the ones that are

398
00:25:53,000 --> 00:25:56,000
adjacent to v.
v was the vertex we just added

399
00:25:56,000 --> 00:25:59,000
to S.
So, once we add somebody to S,

400
00:25:59,000 --> 00:26:02,000
so we grow S by a little bit,
then we look at all the new

401
00:26:02,000 --> 00:26:07,000
edges that go out of S from that
vertex.

402
00:26:07,000 --> 00:26:14,000
We update something.
That's the idea.

403
00:26:14,000 --> 00:26:24,000
So, that's the idea for how
we're going to use greedy.

404
00:26:24,000 --> 00:26:32,000
Now I'll give you the
algorithm.

405
00:26:32,000 --> 00:26:40,000
So, this is called Dijkstra's
algorithm.

406
00:26:40,000 --> 00:26:47,000
Dijkstra is a famous,
recently late,

407
00:26:47,000 --> 00:26:57,000
if that makes sense,
computer scientist from the

408
00:26:57,000 --> 00:27:04,000
Netherlands.
And, this is probably the

409
00:27:04,000 --> 00:27:12,000
algorithm he is most famous for.
So, the beginning of the

410
00:27:12,000 --> 00:27:18,000
algorithm is just some
initialization,

411
00:27:18,000 --> 00:27:24,000
not too exciting.
OK, but let me tell you what

412
00:27:24,000 --> 00:27:34,000
some of the variables mean.
OK, so d is some array indexed

413
00:27:34,000 --> 00:27:42,000
by vertices, and the idea is
that d of x is the distance

414
00:27:42,000 --> 00:27:47,000
estimate for x,
so, from S to x.

415
00:27:47,000 --> 00:27:54,000
so in particular,
it's going to equal the real

416
00:27:54,000 --> 00:28:03,000
shortest path weight from S to x
when we've added x to our set

417
00:28:03,000 --> 00:28:07,000
capital, S.
OK, so this is,

418
00:28:07,000 --> 00:28:10,000
in particular,
going to be the output to the

419
00:28:10,000 --> 00:28:11,000
algorithm.
Did you have a question?

420
00:28:11,000 --> 00:28:13,000
Or were you just stretching?
Good.

421
00:28:13,000 --> 00:28:15,000
So, in d of x,
when we are done,

422
00:28:15,000 --> 00:28:17,000
d of x is the output.
For every vertex,

423
00:28:17,000 --> 00:28:20,000
it's going to give us the
shortest path weight from S to

424
00:28:20,000 --> 00:28:21,000
that vertex.
Along the way,

425
00:28:21,000 --> 00:28:24,000
it's going to be some estimated
distance from S to that vertex.

426
00:28:24,000 --> 00:28:26,000
And, we're going to improve it
over time.

427
00:28:26,000 --> 00:28:29,000
This is an infinity.
So initially,

428
00:28:29,000 --> 00:28:32,000
we know that the distance,
we know the distance from S to

429
00:28:32,000 --> 00:28:35,000
S is zero.
So, we're going to set that to

430
00:28:35,000 --> 00:28:37,000
be our estimate.
It's going to be accurate.

431
00:28:37,000 --> 00:28:40,000
Everything else we're going to
just set to infinity because we

432
00:28:40,000 --> 00:28:43,000
may not be connected.
From the beginning,

433
00:28:43,000 --> 00:28:45,000
we don't know much.
S, initially,

434
00:28:45,000 --> 00:28:47,000
is going to be infinity.
Immediately,

435
00:28:47,000 --> 00:28:49,000
we're going to add little s to
big S.

436
00:28:49,000 --> 00:28:52,000
And then, the interesting part
here is Q, which is going to

437
00:28:52,000 --> 00:28:55,000
consist of, initially all the
vertices in the graph.

438
00:28:55,000 --> 00:29:00,000
And, it's going to not just be
a queue as the letter suggests.

439
00:29:00,000 --> 00:29:03,000
It's going to be a priority
queue.

440
00:29:03,000 --> 00:29:07,000
So, it's going to maintain,
in particular,

441
00:29:07,000 --> 00:29:12,000
the vertex that has the
smallest distance estimate.

442
00:29:12,000 --> 00:29:17,000
So, this is a priority queue.
This is really an abuse of

443
00:29:17,000 --> 00:29:23,000
notation for a data structure.
OK, so this could be a heap or

444
00:29:23,000 --> 00:29:27,000
whatever.
The vertices are keyed on d,

445
00:29:27,000 --> 00:29:32,000
our distance estimate.
So, in particular,

446
00:29:32,000 --> 00:29:35,000
S will have the,
this is going to be a Min heap.

447
00:29:35,000 --> 00:29:38,000
S will be the guy who has the
minimum.

448
00:29:38,000 --> 00:29:41,000
Everyone else has the same key
initially.

449
00:29:41,000 --> 00:29:45,000
And, we're going to repeatedly
extract the minimum element from

450
00:29:45,000 --> 00:29:50,000
this queue and do other things.
OK, so this is initialization.

451
00:29:50,000 --> 00:29:53,000
OK, I'm going to call that
initialization.

452
00:29:53,000 --> 00:29:57,000
It's a pretty simple thing.
It just takes linear time,

453
00:29:57,000 --> 00:30:02,000
nothing fancy going on.
The heart of the algorithm is

454
00:30:02,000 --> 00:30:06,000
all in six lines.
And, so this is not really a

455
00:30:06,000 --> 00:30:09,000
step.
The first step here that we

456
00:30:09,000 --> 00:30:15,000
need to do is we take the vertex
whose distance estimate is

457
00:30:15,000 --> 00:30:17,000
minimum.
So that, among all the

458
00:30:17,000 --> 00:30:21,000
vertices, not yet,
and that's currently S is

459
00:30:21,000 --> 00:30:23,000
empty.
Q has everyone.

460
00:30:23,000 --> 00:30:29,000
In general, Q will have
everyone except S.

461
00:30:29,000 --> 00:30:33,000
So, we'll take the vertex,
u, that has the minimum key in

462
00:30:33,000 --> 00:30:38,000
that priority queue.
So, extract the Min from Q.

463
00:30:48,000 --> 00:31:02,000
OK.
We're going to add a little u

464
00:31:02,000 --> 00:31:07,000
to S, claim that that is now,
I mean, that's exactly what

465
00:31:07,000 --> 00:31:11,000
we're saying here.
We add to S that vertex that

466
00:31:11,000 --> 00:31:17,000
has minimum distance estimate.
And now, we need to update the

467
00:31:17,000 --> 00:31:21,000
distances.
So, we're going to look at each

468
00:31:21,000 --> 00:31:27,000
adjacent vertex for each v in
the adjacency list for u.

469
00:31:27,000 --> 00:31:30,000
We look at a few distances.

470
00:31:47,000 --> 00:31:49,000
So that's the algorithm or more
or less.

471
00:31:49,000 --> 00:31:53,000
This is the key.
I should define it a little bit

472
00:31:53,000 --> 00:31:56,000
what's going on here.
We talked mainly about

473
00:31:56,000 --> 00:32:00,000
undirected graph last time.
Here, we're thinking about

474
00:32:00,000 --> 00:32:03,000
undirected graphs.
And, the adjacency list for u

475
00:32:03,000 --> 00:32:07,000
here is just going to mean,
give me all the vertices for

476
00:32:07,000 --> 00:32:09,000
which there is an edge from u to
v.

477
00:32:09,000 --> 00:32:11,000
So, this is the outgoing
adjacency list,

478
00:32:11,000 --> 00:32:13,000
not the incoming adjacency
list.

479
00:32:13,000 --> 00:32:16,000
Undirected graphs:
you list everything.

480
00:32:16,000 --> 00:32:19,000
Directed graphs:
here, we're only going to care

481
00:32:19,000 --> 00:32:21,000
about those ones.
So, for every edge,

482
00:32:21,000 --> 00:32:24,000
(u,v), is what this is saying,
we are going to compare the

483
00:32:24,000 --> 00:32:28,000
current estimate for v,
and this candidate estimate,

484
00:32:28,000 --> 00:32:30,000
which intuitively means you go
from s to u.

485
00:32:30,000 --> 00:32:34,000
That's d of u because we now
know that that's the right

486
00:32:34,000 --> 00:32:37,000
answer.
This, in fact,

487
00:32:37,000 --> 00:32:39,000
equals, we hope,
assuming the algorithm is

488
00:32:39,000 --> 00:32:44,000
correct, this should be the
shortest path weight from s to u

489
00:32:44,000 --> 00:32:48,000
because we just added u to S.
And whenever we add something

490
00:32:48,000 --> 00:32:50,000
to S, it should have the right
value.

491
00:32:50,000 --> 00:32:54,000
So, we could say,
well, you take the shortest

492
00:32:54,000 --> 00:32:57,000
path from S to u,
and then you follow this edge

493
00:32:57,000 --> 00:32:59,000
from u to v.
That has weight,

494
00:32:59,000 --> 00:33:02,000
w, of (u,v).
That's one possible path from S

495
00:33:02,000 --> 00:33:05,000
to v.
And, if that's a shorter path

496
00:33:05,000 --> 00:33:09,000
than the one we currently have
in our estimate,

497
00:33:09,000 --> 00:33:12,000
if this is smaller than that,
then we should update the

498
00:33:12,000 --> 00:33:16,000
estimate to be that sum because
that's a better path,

499
00:33:16,000 --> 00:33:19,000
so, add it to our database of
paths, so to speak:

500
00:33:19,000 --> 00:33:23,000
OK, very intuitive operation;
clearly should not do anything

501
00:33:23,000 --> 00:33:25,000
bad.
I mean, these should be paths

502
00:33:25,000 --> 00:33:30,000
that makes sense.
We'll prove that in a moment.

503
00:33:30,000 --> 00:33:33,000
That's the first part of
correctness, that this never

504
00:33:33,000 --> 00:33:36,000
screws up.
And then, the tricky part is to

505
00:33:36,000 --> 00:33:40,000
show that it finds all the paths
that we care about.

506
00:33:40,000 --> 00:33:43,000
This step is called a
relaxation step.

507
00:33:43,000 --> 00:33:47,000
Relaxation is always a
difficult technique to teach to

508
00:33:47,000 --> 00:33:50,000
MIT students.
It doesn't come very naturally.

509
00:33:50,000 --> 00:33:55,000
But it's very simple operation.
It comes from optimization

510
00:33:55,000 --> 00:34:00,000
terminology, programming
terminology, so to speak.

511
00:34:00,000 --> 00:34:04,000
And, does this inequality look
familiar at all especially when

512
00:34:04,000 --> 00:34:08,000
you start writing it this way?
You say, the shortest path from

513
00:34:08,000 --> 00:34:12,000
S to v and the shortest path
from S to u in some edge from u

514
00:34:12,000 --> 00:34:15,000
to v, does that look like
anything we've seen?

515
00:34:15,000 --> 00:34:18,000
In fact, it was on this board
but I just erased it.

516
00:34:18,000 --> 00:34:20,000
Triangle inequality,
yeah.

517
00:34:20,000 --> 00:34:24,000
So, this is trying to make the
triangle inequality true.

518
00:34:24,000 --> 00:34:28,000
Certainly, the shortest path
from S to v should be less than

519
00:34:28,000 --> 00:34:32,000
or equal to, not greater than.
The shortest path from S to u,

520
00:34:32,000 --> 00:34:36,000
plus whatever path from u to v,
the shortest path should be,

521
00:34:36,000 --> 00:34:40,000
at most, that.
So, this is sort of a somewhat

522
00:34:40,000 --> 00:34:42,000
more general triangle
inequality.

523
00:34:42,000 --> 00:34:44,000
And, we want to,
certainly it should be true.

524
00:34:44,000 --> 00:34:46,000
So, if it's not true,
we fix it.

525
00:34:46,000 --> 00:34:48,000
If it's greater than,
we make it equal.

526
00:34:48,000 --> 00:34:51,000
But we don't want to make it
less than because that's not

527
00:34:51,000 --> 00:34:52,000
always true.
OK, but certainly,

528
00:34:52,000 --> 00:34:54,000
it should be less than or equal
to.

529
00:34:54,000 --> 00:34:56,000
So, this is fixing the triangle
inequality.

530
00:34:56,000 --> 00:35:00,000
It's trying to make that
constraint more true.

531
00:35:00,000 --> 00:35:03,000
In optimization,
that's called relaxing the

532
00:35:03,000 --> 00:35:06,000
constraint.
OK, so we're sort of relaxing

533
00:35:06,000 --> 00:35:11,000
the triangle inequality here.
In the end, we should have all

534
00:35:11,000 --> 00:35:14,000
the shortest paths.
That's a claim.

535
00:35:14,000 --> 00:35:19,000
So: a very simple algorithm.
Let's try it out on a graph,

536
00:35:19,000 --> 00:35:23,000
and that should make it more
intuitive why it's working,

537
00:35:23,000 --> 00:35:28,000
and that the rest of the
lecture will be proving that it

538
00:35:28,000 --> 00:35:32,000
works.
Yeah, this is enough room.

539
00:35:32,000 --> 00:35:36,000
So, oh, I should mention one
other thing here.

540
00:35:36,000 --> 00:35:38,000
Sorry.
Whenever we change d of v,

541
00:35:38,000 --> 00:35:43,000
this is changing the key of v
in the priority queue.

542
00:35:43,000 --> 00:35:48,000
So, implicitly what's happening
here in this assignment,

543
00:35:48,000 --> 00:35:53,000
this is getting a bit messy,
is a decreased key operation,

544
00:35:53,000 --> 00:35:58,000
OK, which we talked briefly
about last class in the context

545
00:35:58,000 --> 00:36:05,000
of minimum spanning trees where
we were also decreasing the key.

546
00:36:05,000 --> 00:36:08,000
The point is we were changing
the key of one element industry

547
00:36:08,000 --> 00:36:11,000
like station step in the
priority queue so that if it now

548
00:36:11,000 --> 00:36:14,000
becomes the minimum,
we should extract here.

549
00:36:14,000 --> 00:36:17,000
And, we are only ever
decreasing keys because we are

550
00:36:17,000 --> 00:36:20,000
always replacing larger values
with smaller values.

551
00:36:20,000 --> 00:36:23,000
So, we'll come back to that
later when we analyze the

552
00:36:23,000 --> 00:36:25,000
running time.
But, there is some data

553
00:36:25,000 --> 00:36:28,000
structure work going on here.
Again, we are abusing notation

554
00:36:28,000 --> 00:36:33,000
a bit.
OK, so here is a graph with

555
00:36:33,000 --> 00:36:36,000
edge weights.

556
00:37:06,000 --> 00:37:10,000
OK, and I want my priority
queue over here.

557
00:37:10,000 --> 00:37:14,000
And, I'm also going to draw my
estimates.

558
00:37:14,000 --> 00:37:19,000
OK, now I don't want to cheat.
So, we're going to run the

559
00:37:19,000 --> 00:37:23,000
algorithm on this graph.
s will be A,

560
00:37:23,000 --> 00:37:30,000
and I want to know the shortest
path from A to everyone else.

561
00:37:30,000 --> 00:37:32,000
So, you can check,
OK, paths exist.

562
00:37:32,000 --> 00:37:35,000
So, hopefully everything should
end up a finite value by the

563
00:37:35,000 --> 00:37:37,000
end.
All the weights are

564
00:37:37,000 --> 00:37:39,000
nonnegative, so this algorithm
should work.

565
00:37:39,000 --> 00:37:42,000
The algorithm doesn't even need
connectivity,

566
00:37:42,000 --> 00:37:45,000
but it does mean that all the
weights are nonnegative.

567
00:37:45,000 --> 00:37:48,000
So, we run the algorithm.
For the initialization,

568
00:37:48,000 --> 00:37:51,000
we set the distance estimate
for our source to be zero

569
00:37:51,000 --> 00:37:54,000
because, in fact,
there's only one path from A to

570
00:37:54,000 --> 00:37:57,000
A, and that to do nothing,
the empty path.

571
00:37:57,000 --> 00:38:00,000
So, I'm going to put the key of
zero over here.

572
00:38:00,000 --> 00:38:03,000
And, for everyone else,
we're just going to put

573
00:38:03,000 --> 00:38:08,000
infinity because we don't know
any better at this point.

574
00:38:08,000 --> 00:38:11,000
So, I'll put keys of infinity
for everyone else.

575
00:38:11,000 --> 00:38:15,000
OK, so now you can see what the
algorithm does is extract the

576
00:38:15,000 --> 00:38:18,000
minimum from the queue.
And, given our setup,

577
00:38:18,000 --> 00:38:21,000
we'll definitely choose s,
or in this case,

578
00:38:21,000 --> 00:38:23,000
A.
So, it has a weight of zero.

579
00:38:23,000 --> 00:38:26,000
Everyone else has quite a bit
larger weight.

580
00:38:26,000 --> 00:38:30,000
OK, so we look at s,
or I'll use A here.

581
00:38:30,000 --> 00:38:32,000
So, we look at A.
We add A to our set,

582
00:38:32,000 --> 00:38:33,000
S.
So, it's now removed from the

583
00:38:33,000 --> 00:38:35,000
queue.
It will never go back in

584
00:38:35,000 --> 00:38:38,000
because we never add anything to
the queue, start with all the

585
00:38:38,000 --> 00:38:40,000
vertices, and extract,
and decrease keys.

586
00:38:40,000 --> 00:38:42,000
But we never insert.
So, A is gone.

587
00:38:42,000 --> 00:38:45,000
OK, and now I want to update
the keys of all of the other

588
00:38:45,000 --> 00:38:48,000
vertices.
And the claim is I only need to

589
00:38:48,000 --> 00:38:50,000
look at the vertices that have
edges from A.

590
00:38:50,000 --> 00:38:53,000
So, there's an edge from A to
B, and that has weight ten.

591
00:38:53,000 --> 00:38:56,000
And so, I compare:
well, is it a good idea to go

592
00:38:56,000 --> 00:38:58,000
from A to A, which costs
nothing, and then to go along

593
00:38:58,000 --> 00:39:02,000
this edge, AB,
which costs ten?

594
00:39:02,000 --> 00:39:06,000
Well, it seems like a pretty
good idea because that has a

595
00:39:06,000 --> 00:39:09,000
total weight of zero plus ten,
which is ten,

596
00:39:09,000 --> 00:39:11,000
which is much smaller than
infinity.

597
00:39:11,000 --> 00:39:15,000
So, I'm going to erase this
infinity; write ten,

598
00:39:15,000 --> 00:39:19,000
and over in the queue as well.
That's the decreased key

599
00:39:19,000 --> 00:39:22,000
operation.
So now, I know a path from A to

600
00:39:22,000 --> 00:39:22,000
B.
Good.
A to C is the only other edge.
Zero plus three is less than

601
00:39:27,000 --> 00:39:28,000
infinity, so,
cool.

602
00:39:28,000 --> 00:39:33,000
I'll put three here for C,
and C is there.

603
00:39:33,000 --> 00:39:35,000
OK, the other vertices I don't
touch.

604
00:39:35,000 --> 00:39:38,000
I'm going to rewrite them here,
but the algorithm doesn't have

605
00:39:38,000 --> 00:39:41,000
to copy them.
Those keys were already there.

606
00:39:41,000 --> 00:39:44,000
It's just touching these two.
OK, that was pretty boring.

607
00:39:44,000 --> 00:39:47,000
Now we look at our queue,
and we extract the minimum

608
00:39:47,000 --> 00:39:49,000
element.
So, A is no longer in there,

609
00:39:49,000 --> 00:39:51,000
so the minimum key here is
three.

610
00:39:51,000 --> 00:39:54,000
So, the claim is that this is a
shortest path;

611
00:39:54,000 --> 00:39:56,000
from A to C,
here is the shortest path from

612
00:39:56,000 --> 00:39:58,000
A to C.
There's no other shorter way.

613
00:39:58,000 --> 00:40:03,000
You could check that,
and we'll prove it in a moment.

614
00:40:03,000 --> 00:40:05,000
Cool, so we'll remove C from
the list.

615
00:40:05,000 --> 00:40:07,000
It's gone.
Then we look at all of the

616
00:40:07,000 --> 00:40:10,000
outgoing edges from C.
So, there's one that goes up to

617
00:40:10,000 --> 00:40:13,000
B, which has weight four,
four plus three,

618
00:40:13,000 --> 00:40:15,000
which is the shortest path
weight from A to C.

619
00:40:15,000 --> 00:40:18,000
So, going from A to C,
and C to B should cost three

620
00:40:18,000 --> 00:40:21,000
plus four, which is seven,
which is less than ten.

621
00:40:21,000 --> 00:40:24,000
So, we found an even better
path to get to B.

622
00:40:24,000 --> 00:40:27,000
It's better to go like this
than it is to go like that.

623
00:40:27,000 --> 00:40:30,000
So, we write seven for B,
and there's an outgoing edge

624
00:40:30,000 --> 00:40:35,000
from C to d which costs eight.
Three plus eight is 11.

625
00:40:35,000 --> 00:40:38,000
11 is less than infinity last
time I checked.

626
00:40:38,000 --> 00:40:41,000
So, we write 11 for d.
Then we look at E.

627
00:40:41,000 --> 00:40:45,000
We have three plus two is five,
which is less than infinity.

628
00:40:45,000 --> 00:40:48,000
So, we write five for the new
key for E.

629
00:40:48,000 --> 00:40:51,000
At this point,
we have finite shortest paths

630
00:40:51,000 --> 00:40:54,000
to everywhere,
but they may not be the best

631
00:40:54,000 --> 00:40:56,000
ones.
So, we have to keep looking.

632
00:40:56,000 --> 00:40:59,000
OK, next round of the
algorithm, we extract the

633
00:40:59,000 --> 00:41:04,000
minimum key among all these.
OK, it's not B,

634
00:41:04,000 --> 00:41:07,000
which we've seen though
probably know the answer to.

635
00:41:07,000 --> 00:41:09,000
But it's E.
E has the smallest key.

636
00:41:09,000 --> 00:41:12,000
So, we now declare this to be a
shortest path.

637
00:41:12,000 --> 00:41:16,000
The way we got to E was along
this path: A to C,

638
00:41:16,000 --> 00:41:18,000
C to E, declare that to be
shortest.

639
00:41:18,000 --> 00:41:22,000
We claim we're done with E.
But we still have to update.

640
00:41:22,000 --> 00:41:25,000
What about all the outgoing
edges from E?

641
00:41:25,000 --> 00:41:28,000
There's only one here.
It costs five plus nine,

642
00:41:28,000 --> 00:41:32,000
which is 14,
which is bigger than 11.

643
00:41:32,000 --> 00:41:34,000
So, no go.
That's not an interesting path.

644
00:41:34,000 --> 00:41:38,000
Our previous path,
which went like this at a cost

645
00:41:38,000 --> 00:41:42,000
of the 11, is better than the
one we are considering now.

646
00:41:42,000 --> 00:41:45,000
I'm drawing the whole path,
but the algorithm is only

647
00:41:45,000 --> 00:41:48,000
adding these two numbers.
OK, good.

648
00:41:48,000 --> 00:41:52,000
So, I don't change anything.
Seven, 11, and five is removed,

649
00:41:52,000 --> 00:41:55,000
or E is removed.
Our new keys are seven and 11.

650
00:41:55,000 --> 00:41:57,000
So, we take the key,
seven, here,

651
00:41:57,000 --> 00:42:01,000
which is for element B,
vertex B.

652
00:42:01,000 --> 00:42:04,000
We declare the path we
currently have in our hands from

653
00:42:04,000 --> 00:42:06,000
A to B, which happens to be this
one.

654
00:42:06,000 --> 00:42:09,000
Algorithm can't actually tell
this, by the way,

655
00:42:09,000 --> 00:42:11,000
but we're drawing it anyway.
This path, A,

656
00:42:11,000 --> 00:42:14,000
C, B, is the candidate shortest
path.

657
00:42:14,000 --> 00:42:16,000
The claim is it is indeed
shortest.

658
00:42:16,000 --> 00:42:18,000
Now, we look at all the
outgoing edges.

659
00:42:18,000 --> 00:42:22,000
There's one that goes back to C
at a cost of seven plus one,

660
00:42:22,000 --> 00:42:24,000
which is eight,
which is bigger than three,

661
00:42:24,000 --> 00:42:27,000
which is good.
We already declared C to be

662
00:42:27,000 --> 00:42:29,000
done.
But the algorithm checks this

663
00:42:29,000 --> 00:42:31,000
path and says,
oh, that's no better.

664
00:42:31,000 --> 00:42:34,000
And then we look at this other
edge from B to d.

665
00:42:34,000 --> 00:42:36,000
That costs seven plus two,
which is nine,

666
00:42:36,000 --> 00:42:41,000
which is better than 11.
So, we, in fact,

667
00:42:41,000 --> 00:42:46,000
found an even shorter path.
So, the shortest path weight,

668
00:42:46,000 --> 00:42:51,000
now, for d, is nine because
there is this path that goes A,

669
00:42:51,000 --> 00:42:55,000
C, B, d for a total cost of
three plus four plus two is

670
00:42:55,000 --> 00:42:58,000
nine.
Cool, now there's only one

671
00:42:58,000 --> 00:43:02,000
element in the queue.
We remove it.

672
00:43:02,000 --> 00:43:04,000
d: we look at the outgoing
edges.

673
00:43:04,000 --> 00:43:07,000
There's one going here which
costs nine plus seven,

674
00:43:07,000 --> 00:43:09,000
which is 16,
which is way bigger than five.

675
00:43:09,000 --> 00:43:12,000
So, we're done.
Don't do anything.

676
00:43:12,000 --> 00:43:14,000
At this point,
the queue is empty.

677
00:43:14,000 --> 00:43:18,000
And the claim is that all these
numbers that are written here,

678
00:43:18,000 --> 00:43:21,000
the final values are the
shortest path weights.

679
00:43:21,000 --> 00:43:24,000
This looks an awful lot like a
five, but it's an s.

680
00:43:24,000 --> 00:43:27,000
It has a weight of zero.
I've also drawn in here all the

681
00:43:27,000 --> 00:43:31,000
shortest paths.
And, this is not hard to do.

682
00:43:31,000 --> 00:43:34,000
We're not going to talk about
it too much in this class,

683
00:43:34,000 --> 00:43:37,000
but it's mentioned in a little
bit more detail at the end of

684
00:43:37,000 --> 00:43:40,000
the textbook.
And it's something called the

685
00:43:40,000 --> 00:43:42,000
shortest path tree.
It's just something good to

686
00:43:42,000 --> 00:43:46,000
know about if you actually want
to compute shortest paths.

687
00:43:46,000 --> 00:43:48,000
In this class,
we mainly worry about the

688
00:43:48,000 --> 00:43:50,000
weights because it's pretty much
the same problem.

689
00:43:50,000 --> 00:43:55,000
The shortest path tree is the
union of all shortest paths.

690
00:43:55,000 --> 00:43:58,000
And in particular,
if you look at each vertex in

691
00:43:58,000 --> 00:44:04,000
your graph, if you consider the
last edge into that vertex that

692
00:44:04,000 --> 00:44:08,000
was relaxed among all vertices,
u, you look at the edges,

693
00:44:08,000 --> 00:44:12,000
(u,v), say, was that last one
to relax?

694
00:44:12,000 --> 00:44:15,000
So, just look at the last edges
we relaxed here.

695
00:44:15,000 --> 00:44:20,000
You put them all together:
that's called a shortest path

696
00:44:20,000 --> 00:44:23,000
tree.
And, it has the property that

697
00:44:23,000 --> 00:44:28,000
from S to everywhere else,
there is a unique path down the

698
00:44:28,000 --> 00:44:31,000
tree.
And it's the shortest path.

699
00:44:31,000 --> 00:44:34,000
It's the shortest path that we
found.

700
00:44:34,000 --> 00:44:37,000
OK, so you actually get
shortest paths out of this

701
00:44:37,000 --> 00:44:40,000
algorithm even though it's not
explicitly described.

702
00:44:40,000 --> 00:44:44,000
All we are mainly talking about
are the shortest path weights.

703
00:44:44,000 --> 00:44:48,000
Algorithm clear at this point?
Feels like it's doing the right

704
00:44:48,000 --> 00:44:50,000
thing?
You can check all those numbers

705
00:44:50,000 --> 00:44:53,000
are the best paths.
And now we're going to prove

706
00:44:53,000 --> 00:44:55,000
that.

707
00:45:12,000 --> 00:45:15,000
So: correctness.

708
00:45:25,000 --> 00:45:31,000
So the first thing I want to
prove is that relaxation never

709
00:45:31,000 --> 00:45:35,000
makes a mistake.
If it ever sets d of v to be

710
00:45:35,000 --> 00:45:42,000
something, I want to prove that
d of v is always an upper bound

711
00:45:42,000 --> 00:45:45,000
on delta.
So, we have this variant.

712
00:45:45,000 --> 00:45:51,000
It's greater than or equal to
delta of s, v for all v.

713
00:45:51,000 --> 00:45:55,000
And, this invariant holds at
all times.

714
00:45:55,000 --> 00:45:59,000
So, after initialization,
it doesn't hold before

715
00:45:59,000 --> 00:46:06,000
initialization because d isn't
defined then.

716
00:46:06,000 --> 00:46:09,000
But if you do this
initialization where you set S

717
00:46:09,000 --> 00:46:14,000
to zero, and everyone else to
infinity, and you take any

718
00:46:14,000 --> 00:46:19,000
sequence of relaxation steps,
then this variant will hold

719
00:46:19,000 --> 00:46:22,000
after each relaxation step you
apply.

720
00:46:22,000 --> 00:46:25,000
This is actually a very general
lemma.

721
00:46:25,000 --> 00:46:29,000
It's also pretty easy to prove.
It holds not only for

722
00:46:29,000 --> 00:46:33,000
Dijkstra's algorithm,
but for a lot of other

723
00:46:33,000 --> 00:46:38,000
algorithms we'll see.
Pretty much every algorithm we

724
00:46:38,000 --> 00:46:43,000
see will involve relaxation.
And, this is saying no matter

725
00:46:43,000 --> 00:46:47,000
what relaxations you do,
you always have a reasonable

726
00:46:47,000 --> 00:46:51,000
estimate in the sense that it's
greater than or equal to the

727
00:46:51,000 --> 00:46:56,000
true shortest path weight.
So, it should be converging

728
00:46:56,000 --> 00:46:58,000
from above.
So, that's the lemma.

729
00:46:58,000 --> 00:47:02,000
Let's prove it.
Any suggestions on how we

730
00:47:02,000 --> 00:47:07,000
should prove this lemma?
What technique might we use?

731
00:47:07,000 --> 00:47:09,000
What's that?
Cut and paste?

732
00:47:09,000 --> 00:47:12,000
It would be good for optimal
substructure.

733
00:47:12,000 --> 00:47:16,000
Cut and paste:
maybe sort of what's going on

734
00:47:16,000 --> 00:47:20,000
here but not exactly.
Something a little more

735
00:47:20,000 --> 00:47:22,000
general.
It's just intuition here;

736
00:47:22,000 --> 00:47:26,000
it doesn't have to be the right
answer.

737
00:47:26,000 --> 00:47:32,000
In fact, many answers are
correct, have plausible proofs.

738
00:47:32,000 --> 00:47:34,000
Induction, yeah.
So, I'm not going to write

739
00:47:34,000 --> 00:47:36,000
induction here,
but effectively we are using

740
00:47:36,000 --> 00:47:38,000
induction.
That's the answer I was

741
00:47:38,000 --> 00:47:40,000
expecting.
So, there is sort of an

742
00:47:40,000 --> 00:47:42,000
induction already in time going
on here.

743
00:47:42,000 --> 00:47:45,000
We say after initialization it
should be true.

744
00:47:45,000 --> 00:47:47,000
That's our base case.
And then, every relaxation we

745
00:47:47,000 --> 00:47:50,000
do, it should still be true.
So, we're going to assume by

746
00:47:50,000 --> 00:47:53,000
induction that all the previous
relaxations worked,

747
00:47:53,000 --> 00:47:56,000
and then we're going to prove
that the last relaxation,

748
00:47:56,000 --> 00:47:59,000
whatever it is,
works.

749
00:47:59,000 --> 00:48:01,000
So, first let's do the base
case.

750
00:48:01,000 --> 00:48:04,000
So, this is after an
initialization,

751
00:48:04,000 --> 00:48:09,000
let's say, initially.
So, initially we have d of s

752
00:48:09,000 --> 00:48:12,000
equal to zero.
And we have d of v equal to

753
00:48:12,000 --> 00:48:17,000
infinity for all other vertices,
for all vertices,

754
00:48:17,000 --> 00:48:21,000
v, not equal to little s.
OK, now we have to check that

755
00:48:21,000 --> 00:48:26,000
this inequality holds.
Well, we have delta of s,

756
00:48:26,000 --> 00:48:28,000
s.
We've already argued that

757
00:48:28,000 --> 00:48:33,000
that's zero.
You can't get negative when

758
00:48:33,000 --> 00:48:36,000
there are only nonnegative edge
weights.

759
00:48:36,000 --> 00:48:40,000
So, that's the best.
So, certainly zero is greater

760
00:48:40,000 --> 00:48:44,000
than or equal to zero.
And, we have everything else,

761
00:48:44,000 --> 00:48:47,000
well, I mean,
delta of S, v is certainly less

762
00:48:47,000 --> 00:48:50,000
than or equal to infinity.
So this holds.

763
00:48:50,000 --> 00:48:54,000
Everything is less than or
equal to infinity.

764
00:48:54,000 --> 00:48:59,000
So: base case is done.
So, now we do an induction.

765
00:48:59,000 --> 00:49:06,000
And, I'm going to write it as a
proof by contradiction.

766
00:49:06,000 --> 00:49:12,000
So, let's say,
suppose that this fails to hold

767
00:49:12,000 --> 00:49:18,000
at some point.
So, suppose for contradiction

768
00:49:18,000 --> 00:49:26,000
that the invariant is violated.
So, we'd like to sue the

769
00:49:26,000 --> 00:49:32,000
violator and find a
contradiction.

770
00:49:32,000 --> 00:49:36,000
So, it's going to be violated.
So, let's look at the first

771
00:49:36,000 --> 00:49:39,000
violation, the first time it's
violated.

772
00:49:39,000 --> 00:49:41,000
So, this is,
essentially,

773
00:49:41,000 --> 00:49:45,000
again, a proof by induction.
So, let's say we have some

774
00:49:45,000 --> 00:49:48,000
violation, d of v is less than
delta of s, v.

775
00:49:48,000 --> 00:49:53,000
That would be bad if we somehow
got an estimate smaller than the

776
00:49:53,000 --> 00:49:56,000
shortest path.
Well, then I think about

777
00:49:56,000 --> 00:50:01,000
looking at the first violation
is we know sort of by induction

778
00:50:01,000 --> 00:50:05,000
that all other values are
correct.

779
00:50:05,000 --> 00:50:09,000
OK, d of v is the first one
where we've screwed up.

780
00:50:09,000 --> 00:50:12,000
So, the invariant holds
everywhere else.

781
00:50:12,000 --> 00:50:17,000
Well, what caused this to fail,
this invariant to be violated,

782
00:50:17,000 --> 00:50:20,000
is some relaxation,
OK, on d of v.

783
00:50:20,000 --> 00:50:24,000
So, we had some d of v,
and we replaced it with some

784
00:50:24,000 --> 00:50:30,000
other d of u plus the weight of
the edge from u to v.

785
00:50:30,000 --> 00:50:34,000
And somehow,
this made it invalid.

786
00:50:34,000 --> 00:50:39,000
So, d of v is somehow less than
that.

787
00:50:39,000 --> 00:50:46,000
We just set d of v to this.
So, this must be less than

788
00:50:46,000 --> 00:50:51,000
delta of s, v.
The claim is that that's not

789
00:50:51,000 --> 00:50:59,000
possible because,
let me rewrite a little bit.

790
00:50:59,000 --> 00:51:07,000
We have d of u plus w of (u,v).
And, we have our induction

791
00:51:07,000 --> 00:51:15,000
hypothesis, which holds on u,
u of some other vertex.

792
00:51:15,000 --> 00:51:22,000
We know that d of u is at least
delta of s, u.

793
00:51:22,000 --> 00:51:30,000
So, this has to be at least
delta of s, u plus w of u,

794
00:51:30,000 --> 00:51:34,000
v.
Now, what about this w of u,

795
00:51:34,000 --> 00:51:37,000
v?
Well, that's some path from u

796
00:51:37,000 --> 00:51:40,000
to v.
So, it's got to be bigger than

797
00:51:40,000 --> 00:51:44,000
the shortest path or equal.
So certainly,

798
00:51:44,000 --> 00:51:48,000
this is greater than or equal
to delta of u,

799
00:51:48,000 --> 00:51:50,000
v.
OK, it could be larger if

800
00:51:50,000 --> 00:51:56,000
there's some multi-edged path
that has a smaller total weight,

801
00:51:56,000 --> 00:52:00,000
but it's certainly no smaller
than delta of u,

802
00:52:00,000 --> 00:52:04,000
v.
And, this looks like a good

803
00:52:04,000 --> 00:52:08,000
summation, delta of S to u,
and u to v is a triangle

804
00:52:08,000 --> 00:52:11,000
inequality, yeah.
So, that is,

805
00:52:11,000 --> 00:52:15,000
it's upside down here.
But, the triangle S,

806
00:52:15,000 --> 00:52:19,000
u, u to v, so this is only
longer than S to v.

807
00:52:19,000 --> 00:52:24,000
OK, so we have this thing,
which is simultaneously greater

808
00:52:24,000 --> 00:52:29,000
than or equal to the shortest
path weight from S to v,

809
00:52:29,000 --> 00:52:34,000
and also strictly less than the
shortest path weight from S to

810
00:52:34,000 --> 00:52:37,000
v.
So, that's a contradiction.

811
00:52:37,000 --> 00:52:41,000
Maybe contradiction is the most
intuitive way isn't the most

812
00:52:41,000 --> 00:52:43,000
intuitive way to proceed.
The intuition,

813
00:52:43,000 --> 00:52:47,000
here, is whatever you assign d
of v, you have a path in mind.

814
00:52:47,000 --> 00:52:50,000
You inductively had a path from
s to u.

815
00:52:50,000 --> 00:52:53,000
Then you added this edge.
So, that was a real path.

816
00:52:53,000 --> 00:52:57,000
We always know that every path
has weight greater than or equal

817
00:52:57,000 --> 00:53:00,000
to the shortest path.
So, it should be true,

818
00:53:00,000 --> 00:53:06,000
and here's the inductive proof.
All right, moving right along,

819
00:53:06,000 --> 00:53:14,000
so this was an easy warm-up.
We have greater than or equal

820
00:53:14,000 --> 00:53:18,000
to.
Now we have to prove less than

821
00:53:18,000 --> 00:53:23,000
or equal to at the end of the
algorithm.

822
00:53:23,000 --> 00:53:31,000
This is true all the time;
less than or equal to will only

823
00:53:31,000 --> 00:53:37,000
be true at the end.
So, we are not going to prove

824
00:53:37,000 --> 00:53:41,000
less than or equal to quite yet.
We're going to prove another

825
00:53:41,000 --> 00:53:44,000
lemma, which again,
so both of these lemmas are

826
00:53:44,000 --> 00:53:46,000
useful for other algorithms,
too.

827
00:53:46,000 --> 00:53:51,000
So, we're sort of building some
shortest path theory that we can

828
00:53:51,000 --> 00:53:54,000
apply later.
This one will give you some

829
00:53:54,000 --> 00:53:58,000
intuition about why relaxation,
not only is it not bad,

830
00:53:58,000 --> 00:54:01,000
it's actually good.
Not only does it not screw up

831
00:54:01,000 --> 00:54:07,000
anything, but it also makes
progress in the following sense.

832
00:54:07,000 --> 00:54:12,000
So, suppose you knew the
shortest path from s to some

833
00:54:12,000 --> 00:54:16,000
vertex.
OK, so you go from s to some

834
00:54:16,000 --> 00:54:19,000
other vertices.
Then you go to u.

835
00:54:19,000 --> 00:54:25,000
Then you go to v.
Suppose that is a shortest path

836
00:54:25,000 --> 00:54:29,000
from s to v.
OK, and also suppose that we

837
00:54:29,000 --> 00:54:35,000
already know in d of u the
shortest path weight from s to

838
00:54:35,000 --> 00:54:39,000
u.
So, suppose we have this

839
00:54:39,000 --> 00:54:43,000
equality.
We now know that we always have

840
00:54:43,000 --> 00:54:49,000
a greater than or equal to.
Suppose they are equal for u,

841
00:54:49,000 --> 00:54:53,000
OK, the vertex just before v in
the shortest path.

842
00:54:53,000 --> 00:54:58,000
OK, and suppose we relax that
edge, (u,v), OK,

843
00:54:58,000 --> 00:55:05,000
which is exactly this step.
This is relaxing the edge,

844
00:55:05,000 --> 00:55:08,000
(u,v).
But we'll just call it

845
00:55:08,000 --> 00:55:13,000
relaxation here.
After that relaxation,

846
00:55:13,000 --> 00:55:19,000
d of v equals delta of (s,v).
So, if we had the correct

847
00:55:19,000 --> 00:55:23,000
answer for u,
and we relax (u,v),

848
00:55:23,000 --> 00:55:29,000
then we get the correct answer
for v.

849
00:55:29,000 --> 00:55:32,000
OK, this is good news.
It means, if inductively we can

850
00:55:32,000 --> 00:55:36,000
somehow get the right answer for
u, now we know how to get the

851
00:55:36,000 --> 00:55:38,000
right answer for v.
In the algorithm,

852
00:55:38,000 --> 00:55:42,000
we don't actually know what the
vertex just before v in the

853
00:55:42,000 --> 00:55:45,000
shortest path is,
but in the analysis we can

854
00:55:45,000 --> 00:55:48,000
pretty much know that.
So, we have to prove this

855
00:55:48,000 --> 00:55:50,000
lemma.
This is actually even easier

856
00:55:50,000 --> 00:55:53,000
than the previous one:
don't even need induction

857
00:55:53,000 --> 00:55:57,000
because you just work through
what's going on in relaxation,

858
00:55:57,000 --> 00:56:01,000
and it's true.
So, here we go.

859
00:56:01,000 --> 00:56:04,000
So, we're interested in this
value, delta of Ss v.

860
00:56:04,000 --> 00:56:07,000
And we know what the shortest
path is.

861
00:56:07,000 --> 00:56:12,000
So, the shortest path weight is
the weight of this path.

862
00:56:12,000 --> 00:56:15,000
OK, so we can write down some
equality here.

863
00:56:15,000 --> 00:56:20,000
Well, I'm going to split out
the first part of the path and

864
00:56:20,000 --> 00:56:22,000
the last part of the path.
So, we have,

865
00:56:22,000 --> 00:56:27,000
I'll say, the weight from s,
so, this part of the path from

866
00:56:27,000 --> 00:56:32,000
s to u, plus the weight of this
edge, u, v.

867
00:56:32,000 --> 00:56:40,000
Remember, we could write w of a
path, and that was the total

868
00:56:40,000 --> 00:56:46,000
weight of all those edges.
So, what is this,

869
00:56:46,000 --> 00:56:51,000
the weight of this path from S
to u?

870
00:56:51,000 --> 00:56:59,000
Or, what property should I use
to figure out what that value

871
00:56:59,000 --> 00:57:02,000
is?
Yeah?

872
00:57:02,000 --> 00:57:05,000
s to v is the shortest path,
right?

873
00:57:05,000 --> 00:57:11,000
So, by optimal substructure,
from s to u is also a shortest

874
00:57:11,000 --> 00:57:13,000
path.
So, this is delta of s,

875
00:57:13,000 --> 00:57:14,000
u.
Cool.

876
00:57:14,000 --> 00:57:19,000
We'll hold on for now.
That's all we're going to say.

877
00:57:19,000 --> 00:57:24,000
On the other hand,
we know from this lemma that

878
00:57:24,000 --> 00:57:28,000
matter what we do,
d of v is greater than or equal

879
00:57:28,000 --> 00:57:32,000
to delta of s,
v.

880
00:57:32,000 --> 00:57:36,000
So, let's write that down.
So, there's a few cases,

881
00:57:36,000 --> 00:57:39,000
and this will eliminate some of
the cases.

882
00:57:39,000 --> 00:57:44,000
By that lemma correctness one,
we know that d of v is greater

883
00:57:44,000 --> 00:57:46,000
than or equal to delta of s,
v.

884
00:57:46,000 --> 00:57:50,000
So, it's either equal or
greater than at all times.

885
00:57:50,000 --> 00:57:55,000
So, I'm thinking about the time
before we do the relaxation,

886
00:57:55,000 --> 00:57:58,000
this (u,v).
So, at that point,

887
00:57:58,000 --> 00:58:02,000
this is certainly true.
So, either they're equal before

888
00:58:02,000 --> 00:58:06,000
relaxation or it's greater.

889
00:58:16,000 --> 00:58:19,000
OK, if they are equal before
relaxation, we're happy because

890
00:58:19,000 --> 00:58:23,000
relaxation only decreases values
by correctness one.

891
00:58:23,000 --> 00:58:26,000
It can't get any smaller than
this, so after relaxation it

892
00:58:26,000 --> 00:58:29,000
will also be equal.
OK, so in this case we're done.

893
00:58:29,000 --> 00:58:35,000
So, that's a trivial case.
So let's now suppose that d of

894
00:58:35,000 --> 00:58:41,000
v is greater than delta of s,
v before relaxation.

895
00:58:41,000 --> 00:58:46,000
That's perfectly valid.
Hopefully now we fix it.

896
00:58:46,000 --> 00:58:51,000
OK, well the point is,
we know this delta s,

897
00:58:51,000 --> 00:58:53,000
v.
It is this sum.

898
00:58:53,000 --> 00:58:58,000
OK, we also know this.
So, delta of s,

899
00:58:58,000 --> 00:59:03,000
u we know is d of u.
And, we have this w u,

900
00:59:03,000 --> 00:59:04,000
v.
So, delta of s,

901
00:59:04,000 --> 00:59:07,000
v is d of u plus w of (u,v)
because we are assuming we have

902
00:59:07,000 --> 00:59:11,000
this shortest path structure
where you go from s to u,

903
00:59:11,000 --> 00:59:13,000
and then you follow the edge,
(u,v).

904
00:59:13,000 --> 00:59:16,000
So, we know this.
So, we know d of v is greater

905
00:59:16,000 --> 00:59:19,000
than d of u plus w of (u,v).
By golly, that's this condition

906
00:59:19,000 --> 00:59:22,000
in relaxation.
So, we're just checking,

907
00:59:22,000 --> 00:59:24,000
relaxation actually does
something here.

908
00:59:24,000 --> 00:59:26,000
OK, if you had the wrong
distance estimate,

909
00:59:26,000 --> 00:59:33,000
this if condition is satisfied.
Therefore, we do this.

910
00:59:33,000 --> 00:59:36,000
So, in this case,
we relax.

911
00:59:36,000 --> 00:59:44,000
So, I'm just relaxing.
Then, we set d of v to d of u

912
00:59:44,000 --> 00:59:52,000
plus WUV, which is what we want.
OK, so we set d of v to d of u

913
00:59:52,000 --> 00:59:57,000
plus w of (u,v).
And, this equals,

914
00:59:57,000 --> 01:00:01,761
as we said here,
delta of S, v,

915
01:00:01,761 --> 01:00:08,000
which is what we wanted to
prove.

916
01:00:08,000 --> 01:00:12,662
Done.
OK, I'm getting more and more

917
01:00:12,662 --> 01:00:18,834
excited as we get into the meat
of this proof.

918
01:00:18,834 --> 01:00:22,400
Any questions so far?
Good.

919
01:00:22,400 --> 01:00:28,982
Now comes the hard part.
These are both very easy

920
01:00:28,982 --> 01:00:35,935
lemmas, right?
I'll use these two boards.

921
01:00:35,935 --> 01:00:41,081
We don't need these proofs
anymore.

922
01:00:41,081 --> 01:00:48,043
We just need these statements:
correctness one,

923
01:00:48,043 --> 01:00:52,583
correctness lemma;
great names.

924
01:00:52,583 --> 01:01:00,000
So, now finally we get to
correctness two.

925
01:01:00,000 --> 01:01:03,771
So, we had one and one and a
half.

926
01:01:03,771 --> 01:01:09,485
So, I guess correctness is,
itself, a mini-trilogy,

927
01:01:09,485 --> 01:01:14,400
the mini-series.
OK, so correctness two says

928
01:01:14,400 --> 01:01:20,342
when the algorithm is done,
we have the right answer.

929
01:01:20,342 --> 01:01:26,514
This is really correctness.
But, it's going to build on

930
01:01:26,514 --> 01:01:32,000
correctness one and correctness
lemma.

931
01:01:32,000 --> 01:01:35,699
So, we want d of v to equal
delta of s, v for all vertices,

932
01:01:35,699 --> 01:01:39,207
v at the end of the algorithm.
That is clearly our goal.

933
01:01:39,207 --> 01:01:42,779
Now, this theorem is assuming
that all of the weights are

934
01:01:42,779 --> 01:01:46,223
nonnegative, just to repeat.
It doesn't assume anything

935
01:01:46,223 --> 01:01:48,136
else.
So, it's going to get the

936
01:01:48,136 --> 01:01:50,687
infinities right.
But, if there are minus

937
01:01:50,687 --> 01:01:54,004
infinities, all bets are off.
OK, even if there's any

938
01:01:54,004 --> 01:01:57,831
negative weight edge anywhere,
it's not going to do the right

939
01:01:57,831 --> 01:02:02,573
thing necessarily.
But, assuming all the weights

940
01:02:02,573 --> 01:02:06,656
are nonnegative,
which is reasonable if they're

941
01:02:06,656 --> 01:02:10,207
measuring time.
Usually it costs money to

942
01:02:10,207 --> 01:02:14,378
travel along edges.
They don't pay you to do it.

943
01:02:14,378 --> 01:02:18,106
But who knows?
So, I need just to say a few

944
01:02:18,106 --> 01:02:20,769
things.
One of the things we've

945
01:02:20,769 --> 01:02:26,094
mentioned somewhere along the
way is when you add a vertex to

946
01:02:26,094 --> 01:02:32,363
S, you never change its weight.
OK, that actually requires

947
01:02:32,363 --> 01:02:35,272
proof.
I'm just going to state it

948
01:02:35,272 --> 01:02:37,636
here.
It's not hard to see.

949
01:02:37,636 --> 01:02:42,000
d of v doesn't change.
OK, this is essentially an

950
01:02:42,000 --> 01:02:47,636
induction once v is added to S.
OK, this will actually followed

951
01:02:47,636 --> 01:02:50,727
by something we'll say in a
moment.

952
01:02:50,727 --> 01:02:56,272
OK, so all I really care about
is when a vertex is added to S,

953
01:02:56,272 --> 01:03:01,000
we better have the right
estimate because after that,

954
01:03:01,000 --> 01:03:06,000
we're not going to change it,
let's say.

955
01:03:06,000 --> 01:03:10,097
OK, we could define the
algorithm that way.

956
01:03:10,097 --> 01:03:15,365
We are not, but we could.
I'll say more about this in a

957
01:03:15,365 --> 01:03:18,390
second.
So, all we care about is

958
01:03:18,390 --> 01:03:21,804
whether d of v equals delta of
s, v.

959
01:03:21,804 --> 01:03:26,780
That's what we want to prove.
So, it's clearly that.

960
01:03:26,780 --> 01:03:32,536
It should be true at the end.
But, it suffices to prove that

961
01:03:32,536 --> 01:03:38,000
it holds when v is added to S,
to capital S.

962
01:03:38,000 --> 01:03:40,327
OK, this actually implies the
first statement.

963
01:03:40,327 --> 01:03:42,086
It has sort of a funny
implication.

964
01:03:42,086 --> 01:03:44,982
But, if we can prove this,
that d of v equals delta of s,

965
01:03:44,982 --> 01:03:47,258
v, when you add to S,
we know relaxation only

966
01:03:47,258 --> 01:03:49,586
decreases value.
So, it can't get any smaller.

967
01:03:49,586 --> 01:03:51,241
It would be from correctness
one.

968
01:03:51,241 --> 01:03:54,137
Correctness one says we can't
get any smaller than delta.

969
01:03:54,137 --> 01:03:57,396
So, if we get a quality at that
point, we'll have a quality from

970
01:03:57,396 --> 01:03:59,362
then on.
So, that actually implies d of

971
01:03:59,362 --> 01:04:02,000
v never changes after that
point.

972
01:04:02,000 --> 01:04:06,085
OK, so we're going to prove
this.

973
01:04:06,085 --> 01:04:10,297
Good.
Well, suppose it isn't true.

974
01:04:10,297 --> 01:04:15,787
So this would be a proof by a
contradiction.

975
01:04:15,787 --> 01:04:22,042
Suppose for contradiction that
this fails to hold.

976
01:04:22,042 --> 01:04:26,638
And, let's look at the first
failure.

977
01:04:26,638 --> 01:04:32,000
Suppose u is the first vertex
--

978
01:04:41,000 --> 01:04:45,053
-- that's about to be added to
S.

979
01:04:45,053 --> 01:04:52,146
I want to consider the time
right before it's added to S,

980
01:04:52,146 --> 01:04:56,706
for which we don't have what we
want.

981
01:04:56,706 --> 01:05:04,166
These are not equal.
d of u does not equal delta of

982
01:05:04,166 --> 01:05:08,611
s, u.
Well, if they're not equal,

983
01:05:08,611 --> 01:05:16,805
we know from correctness one
that d of E is strictly greater

984
01:05:16,805 --> 01:05:20,971
than delta of s,
u, so, d of u.

985
01:05:20,971 --> 01:05:28,471
So, we have d of u is strictly
greater than delta of s,

986
01:05:28,471 --> 01:05:32,852
u.
OK, that's the beginning of the

987
01:05:32,852 --> 01:05:37,453
proof, nothing too exciting yet,
just some warm-up.

988
01:05:37,453 --> 01:05:41,319
OK, but this,
used already correctness one.

989
01:05:41,319 --> 01:05:46,564
I think that's the only time
that we use it in this proof.

990
01:05:46,564 --> 01:05:52,177
OK, so I sort of just want to
draw picture of what's going on.

991
01:05:52,177 --> 01:05:55,674
But I need a little bit of
description.

992
01:05:55,674 --> 01:06:00,000
So, let's look at the shortest
path.

993
01:06:00,000 --> 01:06:04,176
Somehow, d of u is greater than
the shortest path.

994
01:06:04,176 --> 01:06:08,352
So, consider the shortest path
or a shortest path.

995
01:06:08,352 --> 01:06:12,784
Let p be a shortest path,
not just any shortest path,

996
01:06:12,784 --> 01:06:15,596
but the shortest path from s to
u.

997
01:06:15,596 --> 01:06:20,028
OK, so that means that the
weight of this path is the

998
01:06:20,028 --> 01:06:24,375
shortest path weight.
So, we have some equations for

999
01:06:24,375 --> 01:06:28,636
what's going on here.
So, we care about delta of s,

1000
01:06:28,636 --> 01:06:32,791
u.
Here's a path with that weight.

1001
01:06:32,791 --> 01:06:37,473
It's got to be one because
shortest paths exist here;

1002
01:06:37,473 --> 01:06:41,796
slight exceptional cases if
it's a plus infinity,

1003
01:06:41,796 --> 01:06:45,127
but I'm not going to worry
about that.

1004
01:06:45,127 --> 01:06:49,000
So, let me draw a picture
somewhere.

1005
01:07:03,000 --> 01:07:06,283
So, we have s.
We have u.

1006
01:07:06,283 --> 01:07:11,344
Here is the shortest path from
s to u.

1007
01:07:11,344 --> 01:07:16,542
That's p.
No idea what it looks like so

1008
01:07:16,542 --> 01:07:21,056
far.
Now, what we also have is the

1009
01:07:21,056 --> 01:07:27,759
notion of capital S.
So, I'm going to draw capital

1010
01:07:27,759 --> 01:07:32,000
S.
So, this is big S.

1011
01:07:32,000 --> 01:07:34,600
We know that little s is in big
S.

1012
01:07:34,600 --> 01:07:37,279
We know that u is not yet in
big S.

1013
01:07:37,279 --> 01:07:40,195
So, I haven't screwed up
anything yet,

1014
01:07:40,195 --> 01:07:42,638
right?
This path starts in S and

1015
01:07:42,638 --> 01:07:47,445
leaves it at some point because
until we are about to add u to

1016
01:07:47,445 --> 01:07:51,070
S, so it hasn't happened yet,
so u is not in S.

1017
01:07:51,070 --> 01:07:53,671
Fine.
What I want to do is look at

1018
01:07:53,671 --> 01:07:57,375
the first place here where the
path, p, exits S.

1019
01:07:57,375 --> 01:08:02,526
So, there is some vertex here.
Let's call it x.

1020
01:08:02,526 --> 01:08:06,436
There's some vertex here.
We'll call it y.

1021
01:08:06,436 --> 01:08:10,632
OK, possibly x equals S.
Possibly y equals u.

1022
01:08:10,632 --> 01:08:16,260
But it's got to exit somewhere,
because it starts inside and

1023
01:08:16,260 --> 01:08:19,979
ends up outside.
And it's a finite path.

1024
01:08:19,979 --> 01:08:25,893
OK, so consider the first time
it happens; not the second time,

1025
01:08:25,893 --> 01:08:29,803
the first.
OK, so consider the first edge,

1026
01:08:29,803 --> 01:08:37,140
(x,y), where p exits capital S.
The shortest path from s to u

1027
01:08:37,140 --> 01:08:42,014
exits capital S.
It's got to happen somewhere.

1028
01:08:42,014 --> 01:08:46,779
Cool, now, what do we know?
Little x is in S.

1029
01:08:46,779 --> 01:08:53,060
So, it has the right answer
because u, we were about to add

1030
01:08:53,060 --> 01:08:59,776
u to S, and that was the first
violation of something in S that

1031
01:08:59,776 --> 01:09:06,206
has the wrong d of x estimate.
So, d of x equals delta of s,

1032
01:09:06,206 --> 01:09:08,565
x.
Because we are looking at the

1033
01:09:08,565 --> 01:09:11,988
first violation,
x is something that got added

1034
01:09:11,988 --> 01:09:14,423
before.
So, by induction on time,

1035
01:09:14,423 --> 01:09:18,608
or because we had the first
violation, d of x equals the

1036
01:09:18,608 --> 01:09:21,042
shortest path weight from S to
x.

1037
01:09:21,042 --> 01:09:25,000
So, that's good news.
Now we are trying to apply this

1038
01:09:25,000 --> 01:09:27,815
lemma.
It's the only thing left to do.

1039
01:09:27,815 --> 01:09:32,000
We haven't used this lemma for
anything.

1040
01:09:32,000 --> 01:09:35,358
So, we have the setup.
If we already know that one of

1041
01:09:35,358 --> 01:09:38,974
the d values is the right
answer, and we relaxed the edge

1042
01:09:38,974 --> 01:09:42,009
that goes out from it,
then we get another right

1043
01:09:42,009 --> 01:09:44,399
answer.
So that's what I want to argue

1044
01:09:44,399 --> 01:09:46,982
over here.
We know that the d of x equals

1045
01:09:46,982 --> 01:09:50,017
this weight because,
again, subpaths of shortest

1046
01:09:50,017 --> 01:09:53,504
paths are shortest paths.
We have optimal substructure,

1047
01:09:53,504 --> 01:09:56,023
so this is a shortest path,
from S to x.

1048
01:09:56,023 --> 01:09:58,800
It might not be the only one,
but it is one.

1049
01:09:58,800 --> 01:10:02,029
So we know that matches.
Now, I want to think about

1050
01:10:02,029 --> 01:10:05,000
relaxing this edge,
(x,y).

1051
01:10:05,000 --> 01:10:08,536
Well, x is in capital S.
And, the algorithm says,

1052
01:10:08,536 --> 01:10:11,778
whenever you add a vertex,
u, to the big set,

1053
01:10:11,778 --> 01:10:15,389
S, you relax all the edges that
go out from there.

1054
01:10:15,389 --> 01:10:19,368
OK, so when we added x to S,
and we now look far in the

1055
01:10:19,368 --> 01:10:22,610
future, we're about to add some
other vertex.

1056
01:10:22,610 --> 01:10:26,073
Right after we added x to S,
we relax this edge,

1057
01:10:26,073 --> 01:10:30,273
(x,y), because we relaxed every
edge that goes out from x,

1058
01:10:30,273 --> 01:10:36,443
OK, whatever they were.
Some of them went into S.

1059
01:10:36,443 --> 01:10:42,091
Some of them went out.
Here's one of them.

1060
01:10:42,091 --> 01:10:46,913
So, when we added x to S,
we got XS.

1061
01:10:46,913 --> 01:10:52,561
When we added x to S,
we relaxed the edge,

1062
01:10:52,561 --> 01:10:57,382
(x,y).
OK, so now we're going to use

1063
01:10:57,382 --> 01:11:04,000
the lemma.
So, by the correctness lemma --

1064
01:11:15,000 --> 01:11:18,863
What do you get?
Well, we add this correct

1065
01:11:18,863 --> 01:11:23,387
shortest path weight to x now.
We relax the edge,

1066
01:11:23,387 --> 01:11:26,403
(x,y).
So, now we should have the

1067
01:11:26,403 --> 01:11:29,607
correct shortest path weight for
y.

1068
01:11:29,607 --> 01:11:33,000
d of y equals delta of s,
y.

1069
01:11:33,000 --> 01:11:35,509
OK, this is sometime in the
past.

1070
01:11:35,509 --> 01:11:38,803
In particular,
now, it should still be true

1071
01:11:38,803 --> 01:11:43,588
because once you get down to the
right answer you never change

1072
01:11:43,588 --> 01:11:45,549
it.
OK, we should be done.

1073
01:11:45,549 --> 01:11:49,156
OK, why are we done?
Well, what else do we know

1074
01:11:49,156 --> 01:11:51,431
here?
We assumed something for

1075
01:11:51,431 --> 01:11:54,803
contradiction,
so we better contradict that.

1076
01:11:54,803 --> 01:11:58,647
We assume somehow,
d of u is strictly greater than

1077
01:11:58,647 --> 01:12:03,142
delta of s, u.
So, d of u here is strictly

1078
01:12:03,142 --> 01:12:06,476
greater than the length of this
whole path.

1079
01:12:06,476 --> 01:12:10,047
Well, we don't really know
whether u equals y.

1080
01:12:10,047 --> 01:12:14,015
It could, could not.
And, but what do we know about

1081
01:12:14,015 --> 01:12:18,857
this shortest path from S to y?
Well, it could only be shorter

1082
01:12:18,857 --> 01:12:21,952
than from S to u because it's a
subpath.

1083
01:12:21,952 --> 01:12:26,476
And it's the shortest path
because it's the subpath of the

1084
01:12:26,476 --> 01:12:29,888
shortest path.
The shortest path from S to y

1085
01:12:29,888 --> 01:12:36,000
has to be less than or equal to
the shortest path from S to u.

1086
01:12:36,000 --> 01:12:44,627
OK, S to y: less than or equal
to s, u, OK, just because the

1087
01:12:44,627 --> 01:12:47,405
subpath.
I'm closer.

1088
01:12:47,405 --> 01:12:51,207
I've got delta of s,
u now.

1089
01:12:51,207 --> 01:12:56,033
Somehow, I want to involve d of
u.

1090
01:12:56,033 --> 01:13:01,443
So, I want to relate d of y to
d of u.

1091
01:13:01,443 --> 01:13:07,389
What do I know about d of u?
Yeah?

1092
01:13:07,389 --> 01:13:10,814
d of u is smaller because we
have a Min heap,

1093
01:13:10,814 --> 01:13:12,449
yeah.
We always chose,

1094
01:13:12,449 --> 01:13:14,862
let's erase,
it's way down here.

1095
01:13:14,862 --> 01:13:17,664
We chose u.
This is the middle of the

1096
01:13:17,664 --> 01:13:20,778
algorithm.
It's the reason I kept this to

1097
01:13:20,778 --> 01:13:23,736
be the minimum key.
This is keyed on d.

1098
01:13:23,736 --> 01:13:28,095
So, we know that at this
moment, when we're trying to add

1099
01:13:28,095 --> 01:13:30,275
u to S, right,
y is not in S,

1100
01:13:30,275 --> 01:13:35,412
and u is not in S.
They might actually be the same

1101
01:13:35,412 --> 01:13:38,057
vertex.
But both of these vertices,

1102
01:13:38,057 --> 01:13:40,080
same or not,
are outside S.

1103
01:13:40,080 --> 01:13:44,204
We chose u because d of u has
the smallest d estimate.

1104
01:13:44,204 --> 01:13:48,250
So, d of y has to be greater
than or equal to d of u.

1105
01:13:48,250 --> 01:13:51,674
It might be equal if they're
the same vertex,

1106
01:13:51,674 --> 01:13:55,020
but it's got to be greater than
or equal to.

1107
01:13:55,020 --> 01:14:00,000
So, d of y here is greater than
or equal to d of u.

1108
01:14:00,000 --> 01:14:03,791
So, here we're using the fact
that we actually made a greedy

1109
01:14:03,791 --> 01:14:06,168
choice.
It's the one place we're using

1110
01:14:06,168 --> 01:14:09,317
the greedy choice.
Better use it somewhere because

1111
01:14:09,317 --> 01:14:13,108
you can't just take an arbitrary
vertex and declare it to be

1112
01:14:13,108 --> 01:14:15,293
done.
You've got to take the greedy

1113
01:14:15,293 --> 01:14:17,477
one.
OK, now we have d of u is less

1114
01:14:17,477 --> 01:14:20,947
than or equal to delta of s,
u, which contradicts this.

1115
01:14:20,947 --> 01:14:24,096
OK, sort of magical that that
all just worked out.

1116
01:14:24,096 --> 01:14:27,566
But sort of like the previous
proofs, you just see what

1117
01:14:27,566 --> 01:14:32,000
happens and it works.
OK, that's the approximation.

1118
01:14:32,000 --> 01:14:35,005
The only real idea here is to
look at this edge.

1119
01:14:35,005 --> 01:14:37,563
In fact, you could look at this
edge too.

1120
01:14:37,563 --> 01:14:41,209
But let's look at some edge
that comes from S and goes out

1121
01:14:41,209 --> 01:14:44,151
of S, and argue that while x has
to be correct,

1122
01:14:44,151 --> 01:14:47,156
and what we made x correct,
y had to be correct,

1123
01:14:47,156 --> 01:14:49,779
and now, why the hell are we
looking at u?

1124
01:14:49,779 --> 01:14:52,337
y is the thing you should have
looked at.

1125
01:14:52,337 --> 01:14:55,598
And, there you get a
contradiction because y had the

1126
01:14:55,598 --> 01:14:57,325
right answer.
If u equals y,

1127
01:14:57,325 --> 01:14:59,755
that's fine,
or if u and y were sort of

1128
01:14:59,755 --> 01:15:02,441
equally good,
that's also fine if all these

1129
01:15:02,441 --> 01:15:07,604
weights were zero.
So, the picture might actually

1130
01:15:07,604 --> 01:15:10,469
look like this.
But, in that case,

1131
01:15:10,469 --> 01:15:14,376
d of u is the correct answer.
It was delta SU.

1132
01:15:14,376 --> 01:15:19,064
We assumed that it wasn't.
That's where we're getting a

1133
01:15:19,064 --> 01:15:21,408
contradiction.
Pretty clear?

1134
01:15:21,408 --> 01:15:25,054
Go over this proof.
It's a bit complicated,

1135
01:15:25,054 --> 01:15:28,440
naturally.
OK, we have a little bit more

1136
01:15:28,440 --> 01:15:34,168
to cover, some easier stuff.
OK, the first thing is what's

1137
01:15:34,168 --> 01:15:36,710
the running time of this
algorithm?

1138
01:15:36,710 --> 01:15:41,196
I'll do this very quick because
we're actually seen this many

1139
01:15:41,196 --> 01:15:45,233
times before last class.
There was some initialization.

1140
01:15:45,233 --> 01:15:48,448
The initialization,
which is no longer here,

1141
01:15:48,448 --> 01:15:50,467
is linear time.
No big deal.

1142
01:15:50,467 --> 01:15:53,308
OK, extract Min.
Well, that's some data

1143
01:15:53,308 --> 01:15:56,373
structure.
So, we have something like size

1144
01:15:56,373 --> 01:15:59,065
of V.
Every vertex we extract the Min

1145
01:15:59,065 --> 01:16:01,607
once, and that's it.
So, size of V,

1146
01:16:01,607 --> 01:16:06,634
extract mins.
OK, so that's pretty simple.

1147
01:16:06,634 --> 01:16:12,466
OK, then we had this main loop.
This is a completely conceptual

1148
01:16:12,466 --> 01:16:16,135
operation.
S is not actually used in the

1149
01:16:16,135 --> 01:16:19,240
algorithm.
It's just for thinking.

1150
01:16:19,240 --> 01:16:23,285
OK, so this takes zero time.
Got to love it.

1151
01:16:23,285 --> 01:16:28,742
OK, and now the heart is here.
So, how many times does this

1152
01:16:28,742 --> 01:16:33,324
loop iterate?
That's the degree of u.

1153
01:16:33,324 --> 01:16:38,780
So, what is the total number of
times that we execute a

1154
01:16:38,780 --> 01:16:43,428
relaxation step?
It doesn't necessarily mean we

1155
01:16:43,428 --> 01:16:47,672
do this, but we at least execute
this body.

1156
01:16:47,672 --> 01:16:53,128
Over the whole algorithm,
how many times do we do this?

1157
01:16:53,128 --> 01:16:57,170
Every vertex,
we look at all the outgoing

1158
01:16:57,170 --> 01:17:02,000
edges from there.
So, the total would be?

1159
01:17:11,000 --> 01:17:12,615
Number of edges,
yeah.

1160
01:17:12,615 --> 01:17:15,307
So, this number of edges
iterations.

1161
01:17:15,307 --> 01:17:19,615
OK, this is essentially the
handshaking lemma we saw last

1162
01:17:19,615 --> 01:17:24,230
time, but for directed graphs.
And we are only looking at the

1163
01:17:24,230 --> 01:17:27,538
outgoing edges.
So, it's not a factor of two

1164
01:17:27,538 --> 01:17:32,000
here because you're only
outgoing from one side.

1165
01:17:32,000 --> 01:17:36,573
So, we have number of
reiterations.

1166
01:17:36,573 --> 01:17:42,358
In the worst case,
we do a decreased key for

1167
01:17:42,358 --> 01:17:45,183
everyone.
So, at most:

1168
01:17:45,183 --> 01:17:50,026
E decreased keys.
OK, so the time is,

1169
01:17:50,026 --> 01:17:57,695
well, we have v extract Mins,
so the time to do an extract

1170
01:17:57,695 --> 01:18:05,771
Min, whatever that is.
And we have E decreased keys,

1171
01:18:05,771 --> 01:18:12,013
whatever that is,
and this is exactly the running

1172
01:18:12,013 --> 01:18:20,336
time we had for Prim's algorithm
for a minimum spanning tree last

1173
01:18:20,336 --> 01:18:24,237
time.
And, it depends what data

1174
01:18:24,237 --> 01:18:31,000
structure you use,
what running time you get.

1175
01:18:31,000 --> 01:18:34,688
So, I'm going to skip the whole
table here.

1176
01:18:34,688 --> 01:18:39,518
But, if you use an array,
the final running time will be

1177
01:18:39,518 --> 01:18:44,436
V^2 because you have order of v
extract Min, and you have

1178
01:18:44,436 --> 01:18:49,090
constant time decreased key.
If you use a binary heap,

1179
01:18:49,090 --> 01:18:53,568
which we know and love,
then we have order log v for

1180
01:18:53,568 --> 01:18:57,609
each operation.
And so, this is V plus E log V.

1181
01:18:57,609 --> 01:19:02,000
And, so that's what we know how
to do.

1182
01:19:02,000 --> 01:19:06,830
And, if you use this fancy data
structure called a Fibonacci

1183
01:19:06,830 --> 01:19:11,005
heap, you get constant time
decreased key amortized.

1184
01:19:11,005 --> 01:19:16,000
And, you get an E plus v log v
worst case bound on the running

1185
01:19:16,000 --> 01:19:18,701
time.
So, this is the best we know

1186
01:19:18,701 --> 01:19:23,368
how to solve shortest paths
without any extra assumptions,

1187
01:19:23,368 --> 01:19:28,116
single source shortest paths
with non-negative edge weights

1188
01:19:28,116 --> 01:19:32,040
in general.
OK, this is almost as good and

1189
01:19:32,040 --> 01:19:34,353
this is sometimes better than
that.

1190
01:19:34,353 --> 01:19:38,163
But these are essentially
irrelevant except that you know

1191
01:19:38,163 --> 01:19:41,020
how to do these.
You don't know how to do a

1192
01:19:41,020 --> 01:19:45,238
Fibonacci heap unless you read
that in the chapter of the book.

1193
01:19:45,238 --> 01:19:48,434
That's why we mention the top
two running times.

1194
01:19:48,434 --> 01:19:51,632
OK, I want to talk briefly
about a simpler case,

1195
01:19:51,632 --> 01:19:55,510
which you may have seen before.
And so it's sort of fun to

1196
01:19:55,510 --> 01:20:00,000
connect this up to breadth first
search in a graph.

1197
01:20:00,000 --> 01:20:03,302
So, I mean that ends Dijkstra,
so to speak.

1198
01:20:03,302 --> 01:20:08,175
But now I want to think about a
special case where the graph is

1199
01:20:08,175 --> 01:20:12,737
unweighted, meaning w of (u,v)
equals one for all vertices,

1200
01:20:12,737 --> 01:20:15,174
u and v.
OK, suppose we had that

1201
01:20:15,174 --> 01:20:17,847
property.
Can we do any better than

1202
01:20:17,847 --> 01:20:20,599
Dijkstra?
Can we do better than this

1203
01:20:20,599 --> 01:20:23,980
running time?
Well, we probably have to look

1204
01:20:23,980 --> 01:20:26,889
at all the edges and all the
vertices.

1205
01:20:26,889 --> 01:20:32,000
So, the only thing I'm
questioning is this log v.

1206
01:20:32,000 --> 01:20:36,067
Can I avoid that?
I gave away the answer a little

1207
01:20:36,067 --> 01:20:38,779
bit.
The answer is called breadth

1208
01:20:38,779 --> 01:20:42,508
first search,
or BFS, which you have probably

1209
01:20:42,508 --> 01:20:45,813
seen before.
Next to depth first search,

1210
01:20:45,813 --> 01:20:50,050
it's one of the standard ways
to look at the graph.

1211
01:20:50,050 --> 01:20:54,627
But we can say a little bit
more than you may have seen

1212
01:20:54,627 --> 01:20:57,762
before.
Breadth for search is actually

1213
01:20:57,762 --> 01:21:02,000
Dijkstra's algorithm:
kind of nifty.

1214
01:21:02,000 --> 01:21:08,607
There are two changes.
First change is that breadth

1215
01:21:08,607 --> 01:21:13,894
for search does not use a
priority queue.

1216
01:21:13,894 --> 01:21:19,048
I'll just tell you what it uses
instead.

1217
01:21:19,048 --> 01:21:26,449
You can use a queue first in
first out honest-to-goodness

1218
01:21:26,449 --> 01:21:32,000
queue instead of a priority
queue.

1219
01:21:32,000 --> 01:21:36,728
OK, it turns out that works.
Instead of doing extract Min,

1220
01:21:36,728 --> 01:21:40,295
you just take the first thing
off the queue.

1221
01:21:40,295 --> 01:21:44,692
Instead of doing decreased key,
OK, here's a subtlety.

1222
01:21:44,692 --> 01:21:48,260
But, this if statement changes
a little bit.

1223
01:21:48,260 --> 01:21:50,831
So, here is the relaxation
step.

1224
01:21:50,831 --> 01:21:54,730
So, in order to relax,
you say this much simpler

1225
01:21:54,730 --> 01:21:57,550
thing.
If we haven't visited v yet,

1226
01:21:57,550 --> 01:22:01,782
then we declare it to have the
shortest path weight,

1227
01:22:01,782 --> 01:22:06,511
say, d of v is d of u plus one,
which is the weight of the

1228
01:22:06,511 --> 01:22:12,423
edge, (u,v).
And we add v to the end of the

1229
01:22:12,423 --> 01:22:16,076
queue.
So, now, we start with the

1230
01:22:16,076 --> 01:22:20,869
queue empty.
Actually, it will just contain

1231
01:22:20,869 --> 01:22:25,777
the vertex, S,
because that's the only thing

1232
01:22:25,777 --> 01:22:31,347
we know the shortest path for.
So, the queue is just for,

1233
01:22:31,347 --> 01:22:33,316
I know the shortest path of
this thing.

1234
01:22:33,316 --> 01:22:36,217
Just deal with it when you
can't look at all the outgoing

1235
01:22:36,217 --> 01:22:38,652
edges when you can.
So, initially that's just S.

1236
01:22:38,652 --> 01:22:40,777
You say, well,
for all the outgoing edges,

1237
01:22:40,777 --> 01:22:42,746
S has zero.
All the outgoing edges from

1238
01:22:42,746 --> 01:22:45,388
there have weight one.
The shortest path weight from

1239
01:22:45,388 --> 01:22:47,668
the source is one.
You certainly can't do any

1240
01:22:47,668 --> 01:22:49,896
better than that if all the
weights are one.

1241
01:22:49,896 --> 01:22:52,434
OK, so we add all those
vertices to the end of the

1242
01:22:52,434 --> 01:22:54,093
queue.
Then, we process things in

1243
01:22:54,093 --> 01:22:55,958
order, and we just keep
incrementing,

1244
01:22:55,958 --> 01:22:57,979
if their value is d of u,
add one to it.

1245
01:22:57,979 --> 01:23:02,543
That's d of v.
And then we are going to add v

1246
01:23:02,543 --> 01:23:05,510
to S what we get to it in the
queue.

1247
01:23:05,510 --> 01:23:09,156
OK, that is breadth for search,
very simple.

1248
01:23:09,156 --> 01:23:13,989
And, you can look at the text
for the algorithm and for an

1249
01:23:13,989 --> 01:23:17,973
example because I don't have
time to cover that.

1250
01:23:17,973 --> 01:23:21,704
But the key thing is that the
time is faster.

1251
01:23:21,704 --> 01:23:25,434
The time is order V plus E
because as before,

1252
01:23:25,434 --> 01:23:30,267
we only look at each edge once
we look at all the outgoing

1253
01:23:30,267 --> 01:23:34,930
edges from all the vertices.
As soon as we set d of v to

1254
01:23:34,930 --> 01:23:40,079
something, it will remain that.
We never touch it.

1255
01:23:40,079 --> 01:23:43,136
We are going to add it to S.
That only happens once.

1256
01:23:43,136 --> 01:23:46,193
So, this if statement,
and so on, in the in-queuing,

1257
01:23:46,193 --> 01:23:48,711
is done order E times,
or actually E times,

1258
01:23:48,711 --> 01:23:50,689
exactly.
An in-queuing to a queue,

1259
01:23:50,689 --> 01:23:54,106
and de-queuing from a queue,
that's what we use instead of

1260
01:23:54,106 --> 01:23:57,583
extract Min, take constant time,
so the total running time,

1261
01:23:57,583 --> 01:24:01,000
number of vertices plus the
number of edges.

1262
01:24:01,000 --> 01:24:04,580
OK, not so obvious that this
works, but you can prove that it

1263
01:24:04,580 --> 01:24:06,550
works using the Dijkstra
analysis.

1264
01:24:06,550 --> 01:24:09,891
All you have to do is prove
that the FIFO priority queue.

1265
01:24:09,891 --> 01:24:12,816
Once you know that,
by the correctness of Dijkstra

1266
01:24:12,816 --> 01:24:15,501
you get the correctness of
breadth for search.

1267
01:24:15,501 --> 01:24:19,022
So, not only is breadth for
search finding all the vertices,

1268
01:24:19,022 --> 01:24:21,588
which is maybe what you
normally use it for,

1269
01:24:21,588 --> 01:24:25,109
but it finds the shortest path
weights from S to every other

1270
01:24:25,109 --> 01:24:27,198
vertex when the weights are all
one.

1271
01:24:27,198 --> 01:24:30,003
So, there we go:
introduction to shortest paths.

1272
01:24:30,003 --> 01:24:33,000
Next time we'll deal with
negative weights.