1
00:00:07,000 --> 00:00:12,000
-- valuable experience.
OK, today we're going to start

2
00:00:12,000 --> 00:00:18,000
talking about a particular class
of algorithms called greedy

3
00:00:18,000 --> 00:00:22,000
algorithms.
But we're going to do it in the

4
00:00:22,000 --> 00:00:27,000
context of graphs.
So, I want to review a little

5
00:00:27,000 --> 00:00:32,000
bit about graphs,
which mostly you can find in

6
00:00:32,000 --> 00:00:39,000
the textbook in appendix B.
And so, if you haven't reviewed

7
00:00:39,000 --> 00:00:45,000
in appendix B recently,
please sit down and review

8
00:00:45,000 --> 00:00:50,000
appendix B.
It will pay off especially

9
00:00:50,000 --> 00:00:55,000
during our take-home quiz.
So, just reminder,

10
00:00:55,000 --> 00:01:01,000
a digraph, what's a digraph?
What's that short for?

11
00:01:01,000 --> 00:01:04,000
Directed graph,
OK?

12
00:01:04,000 --> 00:01:07,000
Directed graph,
G equals (V,E),

13
00:01:07,000 --> 00:01:13,000
OK, has a set,
V, of vertices.

14
00:01:13,000 --> 00:01:17,000
And, I always get people
telling me that I have one

15
00:01:17,000 --> 00:01:20,000
vertice.
The singular is not vertice;

16
00:01:20,000 --> 00:01:21,000
it is vertex,
OK?

17
00:01:21,000 --> 00:01:25,000
The plural is vertices.
The singular is vertex.

18
00:01:25,000 --> 00:01:29,000
It's one of those weird English
words.

19
00:01:29,000 --> 00:01:33,000
It's probably originally like
French or something,

20
00:01:33,000 --> 00:01:37,000
right?
I don't know.

21
00:01:37,000 --> 00:01:47,000
OK, anyway, and we have a set,
E, which is a subset of V cross

22
00:01:47,000 --> 00:01:52,000
V of edges.
So that's a digraph.

23
00:01:52,000 --> 00:02:02,000
And undirected graph,
E contains unordered pairs.

24
00:02:12,000 --> 00:02:15,000
OK, and, sorry?
It's Latin, OK,

25
00:02:15,000 --> 00:02:21,000
so it's probably pretty old,
then, in English.

26
00:02:21,000 --> 00:02:28,000
I guess the vertex would be a
little bit of a giveaway that

27
00:02:28,000 --> 00:02:34,000
maybe it wasn't French.
It started to be used in 1570,

28
00:02:34,000 --> 00:02:39,000
OK.
OK, good, OK,

29
00:02:39,000 --> 00:02:52,000
so the number of edges is,
whether it's directed or

30
00:02:52,000 --> 00:03:02,000
undirected, is O of what?
V^2, good.

31
00:03:02,000 --> 00:03:05,000
OK, and one of the conventions
that will have when we're

32
00:03:05,000 --> 00:03:09,000
dealing, once we get into
graphs, we deal a lot with sets.

33
00:03:09,000 --> 00:03:13,000
We generally drop the vertical
bar notation within O's just

34
00:03:13,000 --> 00:03:16,000
because it's applied.
It just makes it messier.

35
00:03:16,000 --> 00:03:18,000
So, once again,
another abuse of notation.

36
00:03:18,000 --> 00:03:22,000
It really should be order the
size of V^2, but it just messes

37
00:03:22,000 --> 00:03:25,000
up, I mean, it's just more stuff
to write down.

38
00:03:25,000 --> 00:03:29,000
And, you're multiplying these
things, and all those vertical

39
00:03:29,000 --> 00:03:33,000
bars.
Since they don't even have a

40
00:03:33,000 --> 00:03:36,000
sense to the vertical bar,
it gets messy.

41
00:03:36,000 --> 00:03:40,000
So, we just drop the vertical
bars there when it's in

42
00:03:40,000 --> 00:03:44,000
asymptotic notation.
So, E is order V^2 when it's a

43
00:03:44,000 --> 00:03:48,000
set of pairs,
because if it's a set of pairs,

44
00:03:48,000 --> 00:03:52,000
it's at most n choose two,
which is where it's at most n^2

45
00:03:52,000 --> 00:03:56,000
over 2, here it could be,
at most, sorry,

46
00:03:56,000 --> 00:04:00,000
V^2 over 2, here it's at most
V^2.

47
00:04:00,000 --> 00:04:07,000
And then, another property that
sometimes comes up is if the G

48
00:04:07,000 --> 00:04:11,000
is connected,
we have another bound,

49
00:04:11,000 --> 00:04:18,000
implies that the size of E is
at least the size of V minus

50
00:04:18,000 --> 00:04:22,000
one.
OK, so if it's connected,

51
00:04:22,000 --> 00:04:31,000
meaning, what does it mean to
have a graph that's connected?

52
00:04:31,000 --> 00:04:37,000
Yeah, there's a path from any
vertex to any other vertex in

53
00:04:37,000 --> 00:04:40,000
the graph.
That's what it means to be

54
00:04:40,000 --> 00:04:44,000
connected.
So if that's the case,

55
00:04:44,000 --> 00:04:50,000
that a number of edges is at
least the number of vertices

56
00:04:50,000 --> 00:04:53,000
minus one, OK?
And so, what that says,

57
00:04:53,000 --> 00:05:00,000
so one of the things we'll get
into, a fact that I just wanted

58
00:05:00,000 --> 00:05:04,000
to remind you,
is that in that case,

59
00:05:04,000 --> 00:05:09,000
if I look at log E,
OK, log of the number of edges,

60
00:05:09,000 --> 00:05:14,000
that is O of log V.
And by this,

61
00:05:14,000 --> 00:05:18,000
is omega of log V.
So, it's equal to theta of log

62
00:05:18,000 --> 00:05:21,000
V.
OK, so basically the number of,

63
00:05:21,000 --> 00:05:25,000
in the case of a connected
graph, the number of edges,

64
00:05:25,000 --> 00:05:30,000
and the number of vertices are
polynomially related.

65
00:05:30,000 --> 00:05:36,000
So, their logs are comparable.
OK, so that's helpful just to

66
00:05:36,000 --> 00:05:41,000
know because sometimes I just
get questions later on where

67
00:05:41,000 --> 00:05:45,000
people will say,
oh, you showed it was log E but

68
00:05:45,000 --> 00:05:51,000
you didn't show it was log V.
And I could point out that it's

69
00:05:51,000 --> 00:05:55,000
the same thing.
OK, so there's various ways of

70
00:05:55,000 --> 00:05:59,000
representing graphs in
computers, and I'm just going to

71
00:05:59,000 --> 00:06:04,000
cover a couple of the important
ones.

72
00:06:04,000 --> 00:06:11,000
There's actually more.
We'll see some more.

73
00:06:11,000 --> 00:06:20,000
So, the simplest one is what's
called an adjacency matrix.

74
00:06:20,000 --> 00:06:30,000
An adjacency matrix of the
graph, G, equals (V,E),

75
00:06:30,000 --> 00:06:40,000
where, for simplicity,
I'll let V be the set of

76
00:06:40,000 --> 00:06:52,000
integers from one up to n,
OK, is the n by n matrix A

77
00:06:52,000 --> 00:07:04,000
given by the ij-th at the entry
is simply one if the edge,

78
00:07:04,000 --> 00:07:20,000
ij, is in the edge set and zero
if ij is not in the edge set.

79
00:07:20,000 --> 00:07:22,000
OK, so it's simply the matrix
where you say,

80
00:07:22,000 --> 00:07:25,000
the ij entry is one if it's in
the matrix.

81
00:07:25,000 --> 00:07:27,000
So, this is,
in some sense,

82
00:07:27,000 --> 00:07:32,000
giving you the predicate for,
is there an edge from i to j?

83
00:07:32,000 --> 00:07:35,000
OK, remember,
predicate is Boolean formula

84
00:07:35,000 --> 00:07:39,000
that is either zero or one,
and in this case,

85
00:07:39,000 --> 00:07:45,000
you're saying it's one if there
is an edge from i to j and zero

86
00:07:45,000 --> 00:07:48,000
otherwise.
OK, sometimes you have edge

87
00:07:48,000 --> 00:07:52,000
weighted graphs,
and then sometimes what people

88
00:07:52,000 --> 00:07:56,000
will do is replace this by edge
weights.

89
00:07:56,000 --> 00:08:02,000
OK, it will be the weight of
the edge from i to j.

90
00:08:02,000 --> 00:08:12,000
So, let's just do an example of
that just to make sure that our

91
00:08:12,000 --> 00:08:22,000
intuition corresponds to our
mathematical definitions.

92
00:08:22,000 --> 00:08:33,000
So, here's an example graph.
Let's say that's our graph.

93
00:08:33,000 --> 00:08:37,000
So let's just draw the
adjacency the matrix.

94
00:08:37,000 --> 00:08:42,000
OK, so what this says:
is there's an edge from one to

95
00:08:42,000 --> 00:08:44,000
one?
And the answer is no.

96
00:08:44,000 --> 00:08:47,000
Is there an edge from one to
two?

97
00:08:47,000 --> 00:08:50,000
Yes.
Is there an edge from one to

98
00:08:50,000 --> 00:08:51,000
three here?
Yep.

99
00:08:51,000 --> 00:08:54,000
Is there an edge for one to
four?

100
00:08:54,000 --> 00:08:58,000
No.
Is there an edge from two until

101
00:08:58,000 --> 00:09:00,000
one?
No.

102
00:09:00,000 --> 00:09:02,000
Two to two?
No.

103
00:09:02,000 --> 00:09:04,000
Two to three?
Yes.

104
00:09:04,000 --> 00:09:06,000
Two to four?
No.

105
00:09:06,000 --> 00:09:13,000
No edges going out of three.
Edge from four to three,

106
00:09:13,000 --> 00:09:19,000
and that's it.
That's the adjacency matrix for

107
00:09:19,000 --> 00:09:23,000
this particular graph,
OK?

108
00:09:23,000 --> 00:09:33,000
And so, I can represent a graph
as this adjacency matrix.

109
00:09:33,000 --> 00:09:43,000
OK, when I represent it in this
way, how much storage do I need?

110
00:09:43,000 --> 00:09:52,000
OK, n^2 or V^2 because the size
is the same thing for V^2

111
00:09:52,000 --> 00:10:03,000
storage, OK, and that's what we
call a dense representation.

112
00:10:03,000 --> 00:10:06,000
OK, it works well when the
graph is dense.

113
00:10:06,000 --> 00:10:11,000
So, the graph is dense if the
number of edges is close to all

114
00:10:11,000 --> 00:10:14,000
of the edges possible.
OK, then this is a good

115
00:10:14,000 --> 00:10:18,000
representation.
But for many types of graphs,

116
00:10:18,000 --> 00:10:23,000
the number of edges is much
less than the possible number of

117
00:10:23,000 --> 00:10:26,000
edges, in which case we say the
graph is sparse.

118
00:10:26,000 --> 00:10:32,000
Can somebody give me an example
of a sparse graph?

119
00:10:32,000 --> 00:10:35,000
A class of graphs:
so, I want a class of graphs

120
00:10:35,000 --> 00:10:38,000
that as n grows,
the number of edges in the

121
00:10:38,000 --> 00:10:42,000
graph doesn't grow as the
square, but grows rather as

122
00:10:42,000 --> 00:10:45,000
something much smaller.
A linked list,

123
00:10:45,000 --> 00:10:48,000
so, a chain,
OK, if you look at it from a

124
00:10:48,000 --> 00:10:52,000
graph theoretically,
is a perfectly good example:

125
00:10:52,000 --> 00:10:56,000
only n edges in the chain for a
chain of length n.

126
00:10:56,000 --> 00:10:59,000
So therefore,
the number of edges would be

127
00:10:59,000 --> 00:11:02,000
order V.
And in particular,

128
00:11:02,000 --> 00:11:07,000
you'd only have one edge per
row here.

129
00:11:07,000 --> 00:11:10,000
What other graphs are sparse?
Yeah?

130
00:11:10,000 --> 00:11:16,000
Good, a planar graph,
a graph that can be drawn in a

131
00:11:16,000 --> 00:11:21,000
plane turns out that if it has V
vertices has,

132
00:11:21,000 --> 00:11:25,000
and V is at least three,
then it has,

133
00:11:25,000 --> 00:11:30,000
at most, three V minus six
edges.

134
00:11:30,000 --> 00:11:34,000
So, it turns out that's order V
edges again.

135
00:11:34,000 --> 00:11:38,000
What's another example of a
common graph?

136
00:11:38,000 --> 00:11:42,000
Yeah, binary tree,
or even actually any tree,

137
00:11:42,000 --> 00:11:49,000
you know, what's called a free
tree if you read the appendix,

138
00:11:49,000 --> 00:11:54,000
OK, a tree that just is a
connected graph that has no

139
00:11:54,000 --> 00:12:00,000
cycles, OK, is another example.
What's an example of a graph

140
00:12:00,000 --> 00:12:04,000
that's dense?
A complete graph,

141
00:12:04,000 --> 00:12:07,000
OK: it's all ones,
OK, or if you have edge

142
00:12:07,000 --> 00:12:11,000
weights, it would be a
completely filled in matrix.

143
00:12:11,000 --> 00:12:14,000
OK, good.
So, this is good for dense

144
00:12:14,000 --> 00:12:17,000
representation.
But sometimes you want to have

145
00:12:17,000 --> 00:12:22,000
a sparse representation so we
don't have to spend V^2 space to

146
00:12:22,000 --> 00:12:26,000
deal with all of the,
where most of it's going to be

147
00:12:26,000 --> 00:12:28,000
zeroes.
OK, it's sort of like,

148
00:12:28,000 --> 00:12:33,000
if we know why it's zero,
why bother representing it as

149
00:12:33,000 --> 00:12:40,000
zero?
So, one such representation is

150
00:12:40,000 --> 00:12:46,000
an adjacency list
representation.

151
00:12:46,000 --> 00:12:56,000
Actually, adjacency list of a
given vertex is the list,

152
00:12:56,000 --> 00:13:08,000
which we denote by Adj of V,
of vertices adjacent to V.

153
00:13:08,000 --> 00:13:12,000
OK, just in terms by their
terminology, vertices are

154
00:13:12,000 --> 00:13:17,000
adjacent, but edges are incident
on vertices.

155
00:13:17,000 --> 00:13:22,000
OK, so the incidence is a
relation between a vertex and an

156
00:13:22,000 --> 00:13:25,000
edge.
An adjacency is a relation

157
00:13:25,000 --> 00:13:31,000
between two vertices.
OK, that's just the language.

158
00:13:31,000 --> 00:13:36,000
Why they use to different
terms, I don't know,

159
00:13:36,000 --> 00:13:40,000
but that's what they do.
So, in the graph,

160
00:13:40,000 --> 00:13:45,000
for example,
the adjacency list for vertex

161
00:13:45,000 --> 00:13:52,000
one is just the list or the set
of two three because one has

162
00:13:52,000 --> 00:13:57,000
going out of one are edges to
two and three.

163
00:13:57,000 --> 00:14:02,000
The adjacency list for two is
just three, four,

164
00:14:02,000 --> 00:14:07,000
three.
It's the empty set,

165
00:14:07,000 --> 00:14:10,000
and for four,
it is three.

166
00:14:10,000 --> 00:14:13,000
OK, so that's the
representation.

167
00:14:13,000 --> 00:14:21,000
Now, if we want to figure out
how much storage is required for

168
00:14:21,000 --> 00:14:26,000
this representation,
OK, we need to understand how

169
00:14:26,000 --> 00:14:35,000
long the adjacency list is.
So, what is the length of an

170
00:14:35,000 --> 00:14:40,000
adjacency list of a vertex,
V?

171
00:14:40,000 --> 00:14:48,000
What name do we give to that?
It's the degree.

172
00:14:48,000 --> 00:14:57,000
So, in an undirected graph,
we call it the degree of the

173
00:14:57,000 --> 00:15:02,000
vertex.
This is undirected.

174
00:15:02,000 --> 00:15:07,000
OK, about here,
OK.

175
00:15:07,000 --> 00:15:12,000
So that's an undirected case.
In the directed case,

176
00:15:12,000 --> 00:15:18,000
OK, actually I guess the way we
should do this is say this.

177
00:15:18,000 --> 00:15:22,000
If the degree,
we call it the out degree for a

178
00:15:22,000 --> 00:15:25,000
digraph.
OK, so in a digraph,

179
00:15:25,000 --> 00:15:32,000
we have an out degree and an in
degree for each vertex.

180
00:15:32,000 --> 00:15:36,000
So here, the in degree is
three.

181
00:15:36,000 --> 00:15:41,000
Here, the out degree is two,
OK?

182
00:15:41,000 --> 00:15:50,000
So, one of the important lemma
that comes up is what's called

183
00:15:50,000 --> 00:15:56,000
the handshaking lemma.
OK, it's one of these

184
00:15:56,000 --> 00:16:01,000
mathematical lemmas.

185
00:16:11,000 --> 00:16:16,000
And so, it comes from a story.
Go to a dinner party,

186
00:16:16,000 --> 00:16:21,000
and everybody at the dinner
party shakes other people's

187
00:16:21,000 --> 00:16:24,000
hands.
Some people may not shake

188
00:16:24,000 --> 00:16:29,000
anybody's hand.
Some people may shake several

189
00:16:29,000 --> 00:16:32,000
people's hands.
Nobody shakes hands with

190
00:16:32,000 --> 00:16:37,000
themselves.
And at some point during the

191
00:16:37,000 --> 00:16:41,000
dinner party,
the host goes around and counts

192
00:16:41,000 --> 00:16:45,000
up how many, the sum,
of the number of hands that

193
00:16:45,000 --> 00:16:48,000
each person has shaken.
OK, so he says,

194
00:16:48,000 --> 00:16:52,000
how many did you shake?
How many did you shake?

195
00:16:52,000 --> 00:16:55,000
How many did you shake?
He adds them up,

196
00:16:55,000 --> 00:17:00,000
OK, and that number is
guaranteed to be even.

197
00:17:00,000 --> 00:17:03,000
OK, that's the handshaking
lemma.

198
00:17:03,000 --> 00:17:08,000
Or, stated a little bit more
precisely, if I take for any

199
00:17:08,000 --> 00:17:13,000
graph the degree of the vertex,
and sum them all up,

200
00:17:13,000 --> 00:17:20,000
that's how many hands everybody
shook, OK, that's actually equal

201
00:17:20,000 --> 00:17:23,000
to always twice the number of
edges.

202
00:17:23,000 --> 00:17:26,000
So, why is that going to be
true?

203
00:17:26,000 --> 00:17:33,000
Why is that going to be twice
the number of edges?

204
00:17:33,000 --> 00:17:33,000
Yeah?
Yeah.
Every time you put in an edge,
you add one to the degree of

205
00:17:39,000 --> 00:17:44,000
each person on each end.
So, it's just two different

206
00:17:44,000 --> 00:17:48,000
ways of counting up the same
number of edges.

207
00:17:48,000 --> 00:17:52,000
OK, I can go around,
and if you imagine that,

208
00:17:52,000 --> 00:17:56,000
that every time I count the
degree of the node,

209
00:17:56,000 --> 00:18:01,000
I put a mark on every edge.
Then, when I'm done,

210
00:18:01,000 --> 00:18:07,000
every edge has two marks on it,
one for each end.

211
00:18:07,000 --> 00:18:17,000
OK: a pretty simple theorem.
So, what that says is that for

212
00:18:17,000 --> 00:18:26,000
undirected graphs,
that implies that the adjacency

213
00:18:26,000 --> 00:18:35,000
list representation,
uses how much storage?

214
00:18:35,000 --> 00:18:39,000
OK, at most,
2E, so order E because that's

215
00:18:39,000 --> 00:18:42,000
not all.
Yeah, so you have to have the

216
00:18:42,000 --> 00:18:47,000
number of vertices plus order
the number of edges,

217
00:18:47,000 --> 00:18:53,000
OK, whether it's directed or
undirected because I may have a

218
00:18:53,000 --> 00:18:59,000
graph, say it has a whole bunch
of vertices and no edges,

219
00:18:59,000 --> 00:19:05,000
that's still going to cost me
order V, OK?

220
00:19:05,000 --> 00:19:09,000
So, it uses theta of V plus E
storage.

221
00:19:09,000 --> 00:19:15,000
And, it's basically the same
thing asymptotically.

222
00:19:15,000 --> 00:19:22,000
In fact, it's easier to see in
some sense for digraphs because

223
00:19:22,000 --> 00:19:27,000
for digraphs,
what I do is I just add up the

224
00:19:27,000 --> 00:19:33,000
out degrees, and that equal to
E, OK, if I add up the out

225
00:19:33,000 --> 00:19:39,000
degrees as equally.
In fact, this is kind of like

226
00:19:39,000 --> 00:19:41,000
it amortized analysis,
if you will,

227
00:19:41,000 --> 00:19:45,000
a book keeping analysis,
that if I'm adding up the total

228
00:19:45,000 --> 00:19:48,000
number of edges,
one way of doing it is

229
00:19:48,000 --> 00:19:50,000
accounting for a vertex by
vertex.

230
00:19:50,000 --> 00:19:54,000
OK, so for each vertex,
I basically can take each

231
00:19:54,000 --> 00:19:58,000
degree, and basically each
vertex, look at the degree,

232
00:19:58,000 --> 00:20:02,000
and that allocating of account
per edge, and then ending up

233
00:20:02,000 --> 00:20:06,000
with twice the number of edges,
that's exactly accounting type

234
00:20:06,000 --> 00:20:12,000
of analysis that we might do for
amortized analysis.

235
00:20:12,000 --> 00:20:17,000
OK, so we'll see that.
So, this is a sparse

236
00:20:17,000 --> 00:20:22,000
representation,
and it's often better than an

237
00:20:22,000 --> 00:20:25,000
adjacency matrix.
For example,

238
00:20:25,000 --> 00:20:32,000
you can imagine if the World
Wide Web were done with an

239
00:20:32,000 --> 00:20:37,000
adjacency matrix as opposed to,
essentially,

240
00:20:37,000 --> 00:20:44,000
with an adjacency list type of
representation.

241
00:20:44,000 --> 00:20:47,000
Every link on the World Wide
Web, I had to say,

242
00:20:47,000 --> 00:20:50,000
here are the ones that I'm
connected to,

243
00:20:50,000 --> 00:20:53,000
and here are all the ones I'm
not connected to.

244
00:20:53,000 --> 00:20:57,000
OK, that list of things you're
not connected to for a given

245
00:20:57,000 --> 00:20:59,000
page would be pretty
dramatically,

246
00:20:59,000 --> 00:21:03,000
show you that there is an
advantage to sparse

247
00:21:03,000 --> 00:21:06,000
representation.
On the other hand,

248
00:21:06,000 --> 00:21:13,000
one of the nice things about an
adjacency matrix representation

249
00:21:13,000 --> 00:21:19,000
is that each edge can be
represented with a single bit,

250
00:21:19,000 --> 00:21:24,000
whereas typical when I'm
representing things with an

251
00:21:24,000 --> 00:21:30,000
adjacency list representation,
how many bits am I going to

252
00:21:30,000 --> 00:21:35,000
need to represent each
adjacency?

253
00:21:35,000 --> 00:21:39,000
You'll need order log of V to
be able to name each different

254
00:21:39,000 --> 00:21:41,000
vertex.
OK, the log of the number is

255
00:21:41,000 --> 00:21:46,000
the number of bits that I need.
So, there are places where this

256
00:21:46,000 --> 00:21:50,000
is actually a far more efficient
representation.

257
00:21:50,000 --> 00:21:53,000
In particular,
if you have a very dense graph,

258
00:21:53,000 --> 00:21:56,000
OK, this may be a better way of
representing it.

259
00:21:56,000 --> 00:22:01,000
OK, the other thing I want you
to get, and we're going to see

260
00:22:01,000 --> 00:22:05,000
more of this in particular next
week, is that a matrix and a

261
00:22:05,000 --> 00:22:11,000
graph, there are two ways of
looking at the same thing.

262
00:22:11,000 --> 00:22:14,000
OK, and in fact,
there's a lot of graph theory

263
00:22:14,000 --> 00:22:17,000
that when you do things like
multiply the adjacency matrix,

264
00:22:17,000 --> 00:22:20,000
OK, and so forth.
So, there's a lot of

265
00:22:20,000 --> 00:22:24,000
commonality between graphs and
matrices, a lot of mathematics

266
00:22:24,000 --> 00:22:28,000
that if it applies for one,
it applies to the other.

267
00:22:28,000 --> 00:22:31,000
Do you have a question,
or just holding your finger in

268
00:22:31,000 --> 00:22:33,000
the air?
OK, good.

269
00:22:33,000 --> 00:22:37,000
OK, so that's all just review.
Now I want to get onto today's

270
00:22:37,000 --> 00:22:40,000
lecture.
OK, so any questions about

271
00:22:40,000 --> 00:22:42,000
graphs?
So, this is a good time to

272
00:22:42,000 --> 00:22:45,000
review appendix B.
there are a lot of great

273
00:22:45,000 --> 00:22:48,000
properties in there,
and in particular,

274
00:22:48,000 --> 00:22:52,000
there is a theorem that we're
going to cover today that we're

275
00:22:52,000 --> 00:22:56,000
going to talk about today,
which is properties of trees.

276
00:22:56,000 --> 00:23:00,000
Trees are very special kinds of
graphs, so I really want you to

277
00:23:00,000 --> 00:23:05,000
go and look to see what the
properties are.

278
00:23:05,000 --> 00:23:08,000
There is, I think,
something like six different

279
00:23:08,000 --> 00:23:11,000
definitions of trees that are
all equivalent,

280
00:23:11,000 --> 00:23:16,000
OK, and so, I think a very good
idea to go through and read

281
00:23:16,000 --> 00:23:20,000
through that theorem.
We're not going to prove it in

282
00:23:20,000 --> 00:23:23,000
class, but really,
provides a very good basis for

283
00:23:23,000 --> 00:23:27,000
the thinking that we're going to
be doing today.

284
00:23:27,000 --> 00:23:30,000
And we'll see more of that in
the future.

285
00:23:30,000 --> 00:23:33,000
OK, so today,
we're going to talk about

286
00:23:33,000 --> 00:23:38,000
minimum spanning trees.
OK, this is one of the world's

287
00:23:38,000 --> 00:23:42,000
most important algorithms.
OK, it is important in

288
00:23:42,000 --> 00:23:46,000
distributed systems.
It's one of the first things

289
00:23:46,000 --> 00:23:50,000
that almost any distributed
system tries to find is a

290
00:23:50,000 --> 00:23:55,000
minimum spanning tree of the
nodes that happened to be alive

291
00:23:55,000 --> 00:23:56,000
at any point,
OK?

292
00:23:56,000 --> 00:24:01,000
And one of the people who
developed an algorithm for this,

293
00:24:01,000 --> 00:24:04,000
we'll talk about this a little
bit later, OK,

294
00:24:04,000 --> 00:24:09,000
it was the basis of the billing
system for AT&T for many years

295
00:24:09,000 --> 00:24:16,000
while it was a monopoly.
OK, so very important kind of

296
00:24:16,000 --> 00:24:21,000
thing.
It's got a huge number of

297
00:24:21,000 --> 00:24:25,000
applications.
So the problem is the

298
00:24:25,000 --> 00:24:31,000
following.
You have a connected undirected

299
00:24:31,000 --> 00:24:35,000
graph,
G equals (V,E),

300
00:24:35,000 --> 00:24:42,000
with an edge weight function,
w, which maps the edges into

301
00:24:42,000 --> 00:24:50,000
weights that are real numbers.
And for today's lecture,

302
00:24:50,000 --> 00:24:56,000
we're going to make an
important assumption,

303
00:24:56,000 --> 00:25:02,000
OK, for simplicity.
The book does not make this

304
00:25:02,000 --> 00:25:08,000
assumption.
And so, I encourage you to look

305
00:25:08,000 --> 00:25:16,000
at the alternative presentation
or, because what they do in the

306
00:25:16,000 --> 00:25:22,000
book is much more general,
but for simplicity and

307
00:25:22,000 --> 00:25:29,000
intuition, I'm going to make
this a little bit easier.

308
00:25:29,000 --> 00:25:37,000
We're going to assume that all
edge weights are distinct.

309
00:25:37,000 --> 00:25:39,000
OK, all edge weights are
distinct.

310
00:25:39,000 --> 00:25:42,000
So what does that mean?
What does that mean that this

311
00:25:42,000 --> 00:25:46,000
function, w, what property does
the function,

312
00:25:46,000 --> 00:25:48,000
w, have if all edge weights are
distinct?

313
00:25:48,000 --> 00:25:51,000
Who remembers their discreet
math?

314
00:25:51,000 --> 00:25:53,000
It's injective.
OK, it's one to one.

315
00:25:53,000 --> 00:25:56,000
OK, it's not one to one and
onto necessarily.

316
00:25:56,000 --> 00:26:00,000
In fact, it would be kind of
hard to do that because that's a

317
00:26:00,000 --> 00:26:05,000
pretty big set.
OK, but it's one to one.

318
00:26:05,000 --> 00:26:09,000
It's injective.
OK, so that's what we're going

319
00:26:09,000 --> 00:26:14,000
to assume for simplicity.
OK, and the book,

320
00:26:14,000 --> 00:26:19,000
they don't assume that.
It just means that the way you

321
00:26:19,000 --> 00:26:24,000
have to state things is just a
little more precise.

322
00:26:24,000 --> 00:26:28,000
It has to be more technically
precise.

323
00:26:28,000 --> 00:26:33,000
So, that's the input.
The output is--

324
00:26:33,000 --> 00:26:44,000
The output is a spanning tree,
T, and by spanning tree,

325
00:26:44,000 --> 00:26:52,000
we mean it connects all the
vertices.

326
00:26:52,000 --> 00:27:02,000
OK, and it's got to have
minimum weight.

327
00:27:02,000 --> 00:27:08,000
OK, so we can write the weight
of the tree is going to be,

328
00:27:08,000 --> 00:27:14,000
by that, we meet the sum over
all edges that are in the tree

329
00:27:14,000 --> 00:27:18,000
of the weight of the individual
edges.

330
00:27:18,000 --> 00:27:24,000
OK, so here I'(V,E) done a
little bit of abusive notation,

331
00:27:24,000 --> 00:27:31,000
which is that what I should be
writing is w of the edge (u,v)

332
00:27:31,000 --> 00:27:37,000
because this is a mapping from
edges, which would give me a

333
00:27:37,000 --> 00:27:42,000
double parentheses.
And, you know,

334
00:27:42,000 --> 00:27:45,000
as you know,
I love to abuse notation.

335
00:27:45,000 --> 00:27:48,000
So, I'm going to drop that
extra parentheses,

336
00:27:48,000 --> 00:27:54,000
because we understand that it's
really the weight of the edge,

337
00:27:54,000 --> 00:27:57,000
OK, not the weight of the
ordered pair.

338
00:27:57,000 --> 00:28:02,000
So, that's just a little
notational convenience.

339
00:28:02,000 --> 00:28:05,000
OK, so one of the things,
when we do the take-home exam,

340
00:28:05,000 --> 00:28:09,000
notational convenience can make
the difference between having a

341
00:28:09,000 --> 00:28:12,000
horrible time writing up a
problem, and an easy time.

342
00:28:12,000 --> 00:28:16,000
So, it's worth thinking about
what kinds of notation you'll

343
00:28:16,000 --> 00:28:19,000
use in writing up solutions to
problems, and so forth.

344
00:28:19,000 --> 00:28:22,000
OK, and just in general,
a technical communication,

345
00:28:22,000 --> 00:28:25,000
you adopt good notation people
understand you.

346
00:28:25,000 --> 00:28:29,000
You adopt a poor notation:
nobody pays attention to what

347
00:28:29,000 --> 00:28:34,000
you're doing because they don't
understand what you're saying.

348
00:28:34,000 --> 00:28:38,000
OK, so let's do an example.

349
00:28:45,000 --> 00:28:52,000
OK, so here's a graph.
I think for this,

350
00:28:52,000 --> 00:29:02,000
somebody asked once if I was
inspired by biochemistry or

351
00:29:02,000 --> 00:29:09,000
something, OK,
but I wasn't.

352
00:29:09,000 --> 00:29:11,000
I was just writing these things
down, OK?

353
00:29:11,000 --> 00:29:15,000
So, here's a graph.
And let's give us some edge

354
00:29:15,000 --> 00:29:16,000
weights.

355
00:29:31,000 --> 00:29:34,000
OK, so there are some edge
weights.

356
00:29:34,000 --> 00:29:39,000
And now, what we want is we
want to find a tree.

357
00:29:39,000 --> 00:29:45,000
So a connected acyclic graph
such that every vertex is part

358
00:29:45,000 --> 00:29:49,000
of the tree.
But it's got to have the

359
00:29:49,000 --> 00:29:55,000
minimum weight possible.
OK, so can somebody suggest to

360
00:29:55,000 --> 00:30:03,000
me some edges that have to be in
this minimum spanning tree?

361
00:30:03,000 --> 00:30:05,000
Yeah, so nine,
good.

362
00:30:05,000 --> 00:30:09,000
Nine has to be in there
because, why?

363
00:30:09,000 --> 00:30:14,000
It's the only one connecting it
to this vertex,

364
00:30:14,000 --> 00:30:16,000
OK?
And likewise,

365
00:30:16,000 --> 00:30:22,000
15 has to be in there.
So those both have to be in.

366
00:30:22,000 --> 00:30:26,000
What other edges have to be in?
Which one?

367
00:30:26,000 --> 00:30:33,000
14 has to be it.
Why does 14 have to be in?

368
00:30:33,000 --> 00:30:40,000
Well, one of 14 and three has
to be in there.

369
00:30:40,000 --> 00:30:50,000
I want the minimum weight.
The one that has the overall

370
00:30:50,000 --> 00:30:57,000
smallest weight.
So, can somebody argue to me

371
00:30:57,000 --> 00:31:04,000
that three has to be in there?
Yeah?

372
00:31:04,000 --> 00:31:09,000
That's the minimum of two,
which means that if I had a,

373
00:31:09,000 --> 00:31:14,000
if you add something you said
was a minimum spanning tree that

374
00:31:14,000 --> 00:31:19,000
didn't include three,
right, and so therefore it had

375
00:31:19,000 --> 00:31:23,000
to include 14,
then I could just delete this

376
00:31:23,000 --> 00:31:28,000
edge, 14, and put in edge three.
And, I have something of lower

377
00:31:28,000 --> 00:31:34,000
weight, right?
So, three has to be in there.

378
00:31:34,000 --> 00:31:37,000
What other edges have to be in
there?

379
00:31:37,000 --> 00:31:43,000
Do a little puzzle logic.
Six and five have to be in

380
00:31:43,000 --> 00:31:46,000
there.
Why do they have to be in

381
00:31:46,000 --> 00:31:48,000
there?

382
00:32:02,000 --> 00:32:05,000
Yeah, well, I mean,
it could be connected through

383
00:32:05,000 --> 00:32:08,000
this or something.
It doesn't necessarily have to

384
00:32:08,000 --> 00:32:11,000
go this way.
Six definitely has to be in

385
00:32:11,000 --> 00:32:14,000
there for the same reason that
three had to be,

386
00:32:14,000 --> 00:32:16,000
right?
Because we got two choices to

387
00:32:16,000 --> 00:32:19,000
connect up this guy.
And so, if everything were

388
00:32:19,000 --> 00:32:22,000
connected but it weren't,
12, I mean, and 12 was in

389
00:32:22,000 --> 00:32:24,000
there.
I could always,

390
00:32:24,000 --> 00:32:27,000
then, say, well,
let's connect them up this way

391
00:32:27,000 --> 00:32:31,000
instead.
OK, so definitely that's in

392
00:32:31,000 --> 00:32:35,000
there.
I still don't have everything

393
00:32:35,000 --> 00:32:37,000
connected up.

394
00:32:50,000 --> 00:33:03,000
What else has to be in there
for minimum spanning tree?

395
00:33:03,000 --> 00:33:11,000
Seven, five,
and eight, why seven,

396
00:33:11,000 --> 00:33:22,000
five, and eight?
OK, so can we argue those one

397
00:33:22,000 --> 00:33:32,000
at a time?
Why does five have to be in

398
00:33:32,000 --> 00:33:37,000
there?
Yeah?

399
00:33:37,000 --> 00:33:41,000
OK, so we have four connected
components because we have this

400
00:33:41,000 --> 00:33:43,000
one, this one,
we actually have,

401
00:33:43,000 --> 00:33:46,000
yeah, this one here,
and this one,

402
00:33:46,000 --> 00:33:49,000
good.
We need at least three edges to

403
00:33:49,000 --> 00:33:53,000
connect them because each edge
is going to reduce the connected

404
00:33:53,000 --> 00:33:57,000
components by one.
OK, so we need three edges,

405
00:33:57,000 --> 00:33:59,000
and those are the three
cheapest ones.

406
00:33:59,000 --> 00:34:04,000
And they work.
That works, right?

407
00:34:04,000 --> 00:34:11,000
Any other edges are going to be
bigger, so that works.

408
00:34:11,000 --> 00:34:15,000
Good.
OK, and so, now do we have a

409
00:34:15,000 --> 00:34:19,000
spanning tree?
Everything is,

410
00:34:19,000 --> 00:34:24,000
we have one big connected graph
here, right?

411
00:34:24,000 --> 00:34:31,000
Is that what I got?
Hey, that's the same as what I

412
00:34:31,000 --> 00:34:35,000
got.
Life is predictable.

413
00:34:35,000 --> 00:34:41,000
OK, so, so everybody had the
idea of what a minimum spanning

414
00:34:41,000 --> 00:34:44,000
tree is, then,
out of this,

415
00:34:44,000 --> 00:34:49,000
OK, what's going on there?
So, let's first of all make

416
00:34:49,000 --> 00:34:53,000
some observations about this
puzzle.

417
00:34:53,000 --> 00:34:59,000
And what I want to do is remind
you about the optimal

418
00:34:59,000 --> 00:35:06,000
substructure property because it
turns out minimum spanning tree

419
00:35:06,000 --> 00:35:12,000
has a great optimal substructure
property.

420
00:35:12,000 --> 00:35:17,000
OK, so the setup is going to
be, we're going to have some

421
00:35:17,000 --> 00:35:22,000
minimum spanning tree.
Let's call it T.

422
00:35:22,000 --> 00:35:27,000
And, I'm going to show that
with the other edges in the

423
00:35:27,000 --> 00:35:32,000
graph, are not going to be
shown.

424
00:35:32,000 --> 00:35:37,000
OK, so here's a graph.

425
00:35:54,000 --> 00:35:58,000
OK, so here's a graph.
It looks like the one I have my

426
00:35:58,000 --> 00:36:01,000
piece of paper here.
OK, so the idea is,

427
00:36:01,000 --> 00:36:05,000
this is some minimum spanning
tree.

428
00:36:05,000 --> 00:36:09,000
Now, we want to look at a
property of optimal

429
00:36:09,000 --> 00:36:13,000
substructure.
And the way I'm going to get

430
00:36:13,000 --> 00:36:17,000
that, is, I'm going to remove
some edge, (u,v),

431
00:36:17,000 --> 00:36:22,000
move an arbitrary edge,
(u,v), in the minimum spanning

432
00:36:22,000 --> 00:36:26,000
tree.
So, let's call this u and this

433
00:36:26,000 --> 00:36:29,000
V.
And so, we're removing this

434
00:36:29,000 --> 00:36:33,000
edge.
OK, so when I remove an edge in

435
00:36:33,000 --> 00:36:36,000
a tree, what happens to the
tree?

436
00:36:36,000 --> 00:36:39,000
What's left?
I have two trees left,

437
00:36:39,000 --> 00:36:41,000
OK?
I have two trees left.

438
00:36:41,000 --> 00:36:45,000
Now, proving that,
that's basically one of the

439
00:36:45,000 --> 00:36:50,000
properties in that appendix,
and the properties of trees

440
00:36:50,000 --> 00:36:55,000
that I want you to read,
OK, because you can actually

441
00:36:55,000 --> 00:37:00,000
prove that kind of thing rather
than it just being obvious,

442
00:37:00,000 --> 00:37:05,000
which is, OK?
OK, so we remove that.

443
00:37:05,000 --> 00:37:11,000
Then, T is partitioned into two
subtrees.

444
00:37:11,000 --> 00:37:15,000
And, we'll call them T_1 and
T_2.

445
00:37:15,000 --> 00:37:22,000
So, here's one subtree,
and here's another subtree.

446
00:37:22,000 --> 00:37:29,000
We'(V,E) partitioned it.
No matter what edge I picked,

447
00:37:29,000 --> 00:37:38,000
there would be two subtrees
that it's partitioned into.

448
00:37:38,000 --> 00:37:40,000
Even if the sub tree is a
trivial subtree,

449
00:37:40,000 --> 00:37:43,000
for example,
it just has a single node in it

450
00:37:43,000 --> 00:37:45,000
and no edges.

451
00:37:58,000 --> 00:38:11,000
So, the theorem that we'll
prove demonstrates a property of

452
00:38:11,000 --> 00:38:24,000
optimal substructure.
T_1 is a minimum spanning tree

453
00:38:24,000 --> 00:38:31,000
for the graph,
G_1, E_1,

454
00:38:31,000 --> 00:38:43,000
a subgraph of G induced by the
vertices in T_1.

455
00:38:43,000 --> 00:38:55,000
OK, that is,
V_1 is just the vertices in T_1

456
00:38:55,000 --> 00:39:09,000
is what it means to be induced.
OK, so V_1 is the vertices in

457
00:39:09,000 --> 00:39:12,000
T_1.
So, in this picture,

458
00:39:12,000 --> 00:39:16,000
I didn't label it.
This is T_1.

459
00:39:16,000 --> 00:39:20,000
This is T_2.
In this picture,

460
00:39:20,000 --> 00:39:27,000
these are the vertices of T_1.
So, that's V_1,

461
00:39:27,000 --> 00:39:32,000
OK?
And, E_1 is the set of pairs of

462
00:39:32,000 --> 00:39:39,000
vertices, x and y,
that are the edges that are in

463
00:39:39,000 --> 00:39:47,000
E_1 such that both x and y
belong to V_1.

464
00:39:47,000 --> 00:39:49,000
OK, so I haven't shown the
edges of G here.

465
00:39:49,000 --> 00:39:52,000
But basically,
if an edge went from here to

466
00:39:52,000 --> 00:39:57,000
here, that would be in the E_1.
If it went from here to here,

467
00:39:57,000 --> 00:40:00,000
it would not.
And if it went from here to

468
00:40:00,000 --> 00:40:04,000
here, it would not.
OK, so the vertices,

469
00:40:04,000 --> 00:40:10,000
the subgraph induced by the
vertices of T_1 are just those

470
00:40:10,000 --> 00:40:17,000
that connect up things in T_1,
and similarly for T_2.

471
00:40:27,000 --> 00:40:33,000
So, the theorem says that if I
look at just the edges within

472
00:40:33,000 --> 00:40:38,000
the graph here,
G_1, those that are induced by

473
00:40:38,000 --> 00:40:41,000
these vertices,
T_1 is, in fact,

474
00:40:41,000 --> 00:40:46,000
a minimum spanning tree for
that subgraph.

475
00:40:46,000 --> 00:40:51,000
That's what the theorem says.
OK, if I look over here

476
00:40:51,000 --> 00:40:58,000
conversely, or correspondingly,
if I look at the set of edges

477
00:40:58,000 --> 00:41:05,000
that are induced by this set of
vertices, the vertices in T_2,

478
00:41:05,000 --> 00:41:13,000
in fact, T_2 is a minimum
spanning tree on that subgraph.

479
00:41:13,000 --> 00:41:17,000
OK, OK, we can even do it over
here.

480
00:41:17,000 --> 00:41:21,000
If I took a look,
for example,

481
00:41:21,000 --> 00:41:27,000
at these, let's see,
let's say we cut out five,

482
00:41:27,000 --> 00:41:35,000
and if I cut out edge five,
that T_1 would be these four

483
00:41:35,000 --> 00:41:40,000
vertices here.
And, the point is that if I

484
00:41:40,000 --> 00:41:44,000
look at the subgraph induced on
that, that these edges here.

485
00:41:44,000 --> 00:41:48,000
In fact, the six,
eight, and three are all edges

486
00:41:48,000 --> 00:41:52,000
in a minimum spanning tree for
that subgraph.

487
00:41:52,000 --> 00:41:54,000
OK, so that's what the theorem
says.

488
00:41:54,000 --> 00:41:57,000
So let's prove it.

489
00:42:09,000 --> 00:42:19,000
OK, and so what technique are
we going to use to prove it?

490
00:42:19,000 --> 00:42:28,000
OK, we learned this technique
last time: hint,

491
00:42:28,000 --> 00:42:33,000
hint.
It's something you do it in

492
00:42:33,000 --> 00:42:39,000
your text editor all the time:
cut and paste,

493
00:42:39,000 --> 00:42:45,000
good, cut and paste.
OK, so the weight of T I can

494
00:42:45,000 --> 00:42:51,000
express as the weight of the
edge I removed,

495
00:42:51,000 --> 00:42:57,000
plus the weight of T_1,
plus the weight of T_2.

496
00:42:57,000 --> 00:43:07,000
OK, so that's the total weight.
So, the argument is pretty

497
00:43:07,000 --> 00:43:13,000
simple.
Suppose that there were some

498
00:43:13,000 --> 00:43:20,000
T_1 prime that was better than
T_1 for G_1.

499
00:43:20,000 --> 00:43:31,000
Suppose I had some better way
of forming a spanning tree.

500
00:43:31,000 --> 00:43:42,000
OK, then I would make up a T
prime, which just contained the

501
00:43:42,000 --> 00:43:48,000
edges, (u,v),
and T_1 prime,

502
00:43:48,000 --> 00:43:53,000
union T_2.
So, I would take,

503
00:43:53,000 --> 00:44:05,000
if I had a better spanning
tree, a spanning tree of lower

504
00:44:05,000 --> 00:44:12,000
weight for T_1.
And I call that T_1 prime.

505
00:44:12,000 --> 00:44:17,000
I just substitute that and make
up a spanning tree that

506
00:44:17,000 --> 00:44:22,000
consisted of my edge,
(u,v), whatever works well for

507
00:44:22,000 --> 00:44:26,000
T_1 prime and whatever works
well for T.

508
00:44:26,000 --> 00:44:30,000
And, that would be a spanning
tree.

509
00:44:30,000 --> 00:44:36,000
And it would be better than T
itself was for G,

510
00:44:36,000 --> 00:44:44,000
OK, because the weight of these
is just as the weight for this,

511
00:44:44,000 --> 00:44:50,000
I now just get to use the
weight of T_1 prime,

512
00:44:50,000 --> 00:44:54,000
and that's less.
And so, therefore,

513
00:44:54,000 --> 00:45:02,000
the assumption that T was a
minimum spanning tree would be

514
00:45:02,000 --> 00:45:11,000
violated if I could find a
better one for the subpiece.

515
00:45:11,000 --> 00:45:16,000
So, we have this nice property
of optimal substructure.

516
00:45:16,000 --> 00:45:20,000
OK, I have subproblems that
exhibit optimal,

517
00:45:20,000 --> 00:45:25,000
if I have a globally optimal
solution to the whole problem

518
00:45:25,000 --> 00:45:31,000
within it, I can find optimal
solutions to subproblems.

519
00:45:31,000 --> 00:45:36,000
So, now the question is,
that's one hallmark.

520
00:45:36,000 --> 00:45:41,000
That's one hallmark of dynamic
programming.

521
00:45:41,000 --> 00:45:45,000
What about overlapping
subproblems?

522
00:45:45,000 --> 00:45:51,000
Do I have that property?
Do I have overlapping

523
00:45:51,000 --> 00:45:58,000
subproblems over here for this
type of problem?

524
00:46:19,000 --> 00:46:20,000
So, imagine,
for example,

525
00:46:20,000 --> 00:46:22,000
that I'm removing different
edges.

526
00:46:22,000 --> 00:46:26,000
I look at the space of taking a
given edge, and removing it.

527
00:46:26,000 --> 00:46:30,000
It partitions it into two
pieces, and now I have another

528
00:46:30,000 --> 00:46:32,000
piece.
And I remove it,

529
00:46:32,000 --> 00:46:35,000
etc.
Am I going to end up getting a

530
00:46:35,000 --> 00:46:38,000
bunch of subproblems that are
similar in there?

531
00:46:38,000 --> 00:46:41,000
Yeah, I am.
OK, if I take out this one,

532
00:46:41,000 --> 00:46:43,000
then I take out,
say, this one here,

533
00:46:43,000 --> 00:46:46,000
and then I'll have another tree
here and here.

534
00:46:46,000 --> 00:46:51,000
OK, that would be the same as
if I had originally taken this

535
00:46:51,000 --> 00:46:53,000
out, and then taken that one
out.

536
00:46:53,000 --> 00:46:57,000
If I look at simple ordering of
taking out the edges,

537
00:46:57,000 --> 00:47:00,000
I'm going to end up with a
whole bunch of overlapping

538
00:47:00,000 --> 00:47:04,000
subproblems.
Yeah, OK.

539
00:47:04,000 --> 00:47:14,000
So then, what does that suggest
we use as an approach?

540
00:47:14,000 --> 00:47:18,000
Dynamic programming,
good.

541
00:47:18,000 --> 00:47:26,000
What a surprise!
Yes, OK, you could use dynamic

542
00:47:26,000 --> 00:47:33,000
programming.
But it turns out that minimum

543
00:47:33,000 --> 00:47:41,000
spanning tree exhibits an even
more powerful property.

544
00:47:41,000 --> 00:47:48,000
OK, so we'(V,E) got all the
clues for dynamic programming,

545
00:47:48,000 --> 00:47:57,000
but it turns out that there's
an even bigger clue that's going

546
00:47:57,000 --> 00:48:05,000
to help us to use an even more
powerful technique.

547
00:48:05,000 --> 00:48:11,000
And that, we call,
the hallmark for greedy

548
00:48:11,000 --> 00:48:13,000
algorithms.

549
00:48:32,000 --> 00:48:41,000
And that is,
we have a thing called the

550
00:48:41,000 --> 00:48:53,000
greedy choice property,
which says that a locally

551
00:48:53,000 --> 00:49:03,000
optimal choice is globally
optimal.

552
00:49:03,000 --> 00:49:05,000
And, of course,
as all these hallmarks is the

553
00:49:05,000 --> 00:49:09,000
kind of thing you want to box,
OK, because these are the clues

554
00:49:09,000 --> 00:49:12,000
that you're going to be able to
do that.

555
00:49:12,000 --> 00:49:15,000
So, we have this property that
we call the greedy choice

556
00:49:15,000 --> 00:49:18,000
property.
I'm going to show you how it

557
00:49:18,000 --> 00:49:21,000
works in this case.
And when you have a greedy

558
00:49:21,000 --> 00:49:24,000
choice property,
it turns out you can do even

559
00:49:24,000 --> 00:49:29,000
better that dynamic programming.
OK, so when you see the two

560
00:49:29,000 --> 00:49:33,000
dynamic programming properties,
there is a clue that says

561
00:49:33,000 --> 00:49:36,000
dynamic programming,
yes, but also it says,

562
00:49:36,000 --> 00:49:41,000
let me see whether it also has
this greedy property because if

563
00:49:41,000 --> 00:49:46,000
it does, you're going to come up
with something that's even

564
00:49:46,000 --> 00:49:49,000
better than dynamic programming,
OK?

565
00:49:49,000 --> 00:49:53,000
So, if you just have the two,
you can usually do dynamic

566
00:49:53,000 --> 00:49:56,000
programming, but if you have
this third one,

567
00:49:56,000 --> 00:50:00,000
it's like, whoa!
Jackpot!

568
00:50:00,000 --> 00:50:04,000
OK, so here's the theorem we'll
prove to illustrate this idea.

569
00:50:04,000 --> 00:50:08,000
Once again, these are not,
all these hallmarks are not

570
00:50:08,000 --> 00:50:09,000
things.
They are heuristics.

571
00:50:09,000 --> 00:50:14,000
I can't give you an algorithm
to say, here's where dynamic

572
00:50:14,000 --> 00:50:16,000
programming works,
or here's where greedy

573
00:50:16,000 --> 00:50:20,000
algorithms work.
But I can sort of indicate when

574
00:50:20,000 --> 00:50:23,000
they work, the kind of structure
they have.

575
00:50:23,000 --> 00:50:32,000
OK, so here's the theorem.
So let's let T be the MST of

576
00:50:32,000 --> 00:50:40,000
our graph.
And, let's let A be any subset

577
00:50:40,000 --> 00:50:49,000
of V, so, some subset of
vertices.

578
00:50:49,000 --> 00:51:04,000
And now, let's suppose that
edge, (u,v), is the least weight

579
00:51:04,000 --> 00:51:17,000
edge connecting our set A to A
complement, that is,

580
00:51:17,000 --> 00:51:27,000
V minus A.
Then the theorem says that

581
00:51:27,000 --> 00:51:39,000
(u,v) is in the minimum spanning
tree.

582
00:51:39,000 --> 00:51:43,000
So let's just take a look at
our graph over here and see if

583
00:51:43,000 --> 00:51:45,000
that's, in fact,
the case.

584
00:51:45,000 --> 00:51:49,000
OK, so let's take,
so one thing I could do for A

585
00:51:49,000 --> 00:51:53,000
is just take a singleton node.
So, I take a singleton node,

586
00:51:53,000 --> 00:51:56,000
let's say this guy here,
that can be my A,

587
00:51:56,000 --> 00:52:00,000
and everything else is V minus
A.

588
00:52:00,000 --> 00:52:04,000
And I look at the least weight
edge connecting this to

589
00:52:04,000 --> 00:52:07,000
everything else.
Well, there are only two edges

590
00:52:07,000 --> 00:52:10,000
that connect it to everything
else.

591
00:52:10,000 --> 00:52:15,000
And the theorem says that the
lighter one is in the minimum

592
00:52:15,000 --> 00:52:17,000
spanning tree.
Hey, I win.

593
00:52:17,000 --> 00:52:21,000
OK, if you take a look,
every vertex that I pick,

594
00:52:21,000 --> 00:52:25,000
the latest edge coming out of
that vertex is in the minimum

595
00:52:25,000 --> 00:52:29,000
spanning tree.
OK, the lightest weight edge

596
00:52:29,000 --> 00:52:35,000
coming out, but that's not all
the edges that are in here.

597
00:52:35,000 --> 00:52:39,000
OK, or let's just imagine,
let's take a look at these

598
00:52:39,000 --> 00:52:43,000
three vertices connected to this
set of vertices.

599
00:52:43,000 --> 00:52:46,000
I have three edges is going
across.

600
00:52:46,000 --> 00:52:50,000
The least weight one is five.
That's the minimum spanning

601
00:52:50,000 --> 00:52:53,000
tree.
Or, I can cut it this way.

602
00:52:53,000 --> 00:52:57,000
OK, the ones above one,
the edges going down are seven,

603
00:52:57,000 --> 00:53:02,000
eight, and 14.
Seven is the least weight.

604
00:53:02,000 --> 00:53:04,000
It's in the minimum spanning
tree.

605
00:53:04,000 --> 00:53:08,000
So, no matter how I choose,
I could make this one in,

606
00:53:08,000 --> 00:53:10,000
this one out,
this one in,

607
00:53:10,000 --> 00:53:12,000
this one out,
this one in,

608
00:53:12,000 --> 00:53:14,000
this one out,
this one in,

609
00:53:14,000 --> 00:53:18,000
this one out,
take a look at what all the

610
00:53:18,000 --> 00:53:21,000
edges are.
Which ever one to the least

611
00:53:21,000 --> 00:53:24,000
weight: it's in the minimum
spanning tree.

612
00:53:24,000 --> 00:53:28,000
So, in some sense,
that's a local property because

613
00:53:28,000 --> 00:53:34,000
I don't have to look at what the
rest of the tree is.

614
00:53:34,000 --> 00:53:38,000
I'm just looking at some small
set of vertices if I wish,

615
00:53:38,000 --> 00:53:42,000
and I say, well,
if I wanted to connect that set

616
00:53:42,000 --> 00:53:46,000
of vertices to the rest of the
world, what would I pick?

617
00:53:46,000 --> 00:53:50,000
I'd pick the cheapest one.
That's the greedy approach.

618
00:53:50,000 --> 00:53:53,000
It turns out,
that wins, OK,

619
00:53:53,000 --> 00:53:57,000
that picking that thing that's
locally good for that subset,

620
00:53:57,000 --> 00:54:04,000
A, OK, is also globally good.
OK, it optimizes the overall

621
00:54:04,000 --> 00:54:09,000
function.
That's what the theorem says,

622
00:54:09,000 --> 00:54:13,000
OK?
So, let's prove this theorem.

623
00:54:13,000 --> 00:54:20,000
Any questions about this?
OK, let's prove this theorem.

624
00:54:20,000 --> 00:54:27,000
So, we have (u,v) is the least
weight edge connecting A to D

625
00:54:27,000 --> 00:54:32,000
minus A.
So, let's suppose that this

626
00:54:32,000 --> 00:54:40,000
edge, (u,v), is not in the
minimum spanning tree.

627
00:54:40,000 --> 00:54:45,000
OK, let's suppose that somehow
there is a minimum spanning tree

628
00:54:45,000 --> 00:54:50,000
that doesn't include this least
weight edge.

629
00:54:50,000 --> 00:54:55,000
OK, so what technique you think
will use to prove to get a

630
00:54:55,000 --> 00:54:58,000
contradiction here?
Cut and paste,

631
00:54:58,000 --> 00:55:04,000
good.
Yeah, we're going to cut paste.

632
00:55:04,000 --> 00:55:08,000
OK, we're going to cut and
paste.

633
00:55:08,000 --> 00:55:14,000
So here, I did an example.
OK, so --

634
00:55:40,000 --> 00:55:42,000
OK, and so I'm going to use the
notation.

635
00:55:42,000 --> 00:55:44,000
I'm going to color some of
these in.

636
00:56:05,000 --> 00:56:10,000
OK, and so my notation here is
this is an element of A,

637
00:56:10,000 --> 00:56:14,000
and color it in.
It's an element of V minus A.

638
00:56:14,000 --> 00:56:18,000
OK, so if it's not colored it,
that's an A.

639
00:56:18,000 --> 00:56:21,000
This is my minimum spanning
tree.

640
00:56:21,000 --> 00:56:27,000
Once again, I'm not showing the
overall edges of all the graphs,

641
00:56:27,000 --> 00:56:30,000
but they're there,
OK?

642
00:56:30,000 --> 00:56:33,000
So, my edge,
(u,v), which is not my minimum

643
00:56:33,000 --> 00:56:38,000
spanning tree I say,
let's say is this edge here.

644
00:56:38,000 --> 00:56:42,000
It's an edge from u,
u as in A, v as in V minus A.

645
00:56:42,000 --> 00:56:48,000
OK, so everybody see the setup?
So, I want to prove that this

646
00:56:48,000 --> 00:56:52,000
edge should have been in the
minimum spanning tree,

647
00:56:52,000 --> 00:56:58,000
OK, that the contention that
this is a minimum spanning tree,

648
00:56:58,000 --> 00:57:02,000
and does include (u,v) is
wrong.

649
00:57:02,000 --> 00:57:05,000
So, what I want to do,
that, is I have a tree here,

650
00:57:05,000 --> 00:57:08,000
T, and I have two vertices,
u and v, and in a tree,

651
00:57:08,000 --> 00:57:12,000
between any two vertices there
is a unique, simple path:

652
00:57:12,000 --> 00:57:16,000
simple path meaning it doesn't
go back and forth and repeat

653
00:57:16,000 --> 00:57:18,000
edges or vertices.
OK, there's a unique,

654
00:57:18,000 --> 00:57:23,000
simple path from u to v.
So, let's consider that path.

655
00:57:42,000 --> 00:57:46,000
OK, and the way that I know
that that path exists is because

656
00:57:46,000 --> 00:57:51,000
I'(V,E) read appendix B of the
textbook, section B.5.1,

657
00:57:51,000 --> 00:57:56,000
OK, which has this nice theorem
about properties of trees.

658
00:57:56,000 --> 00:58:00,000
OK, so that's how I know that
there exists a unique,

659
00:58:00,000 --> 00:58:05,000
simple path.
OK, so now we're going to do is

660
00:58:05,000 --> 00:58:09,000
take a look at that path.
So in this case,

661
00:58:09,000 --> 00:58:13,000
it goes from here,
to here, to here,

662
00:58:13,000 --> 00:58:16,000
to here.
And along that path,

663
00:58:16,000 --> 00:58:22,000
there must be a point where I
connect from a vertex in A to a

664
00:58:22,000 --> 00:58:25,000
vertex in V minus A.
Why?

665
00:58:25,000 --> 00:58:32,000
Well, because this is in A.
This is in V minus A.

666
00:58:32,000 --> 00:58:42,000
So, along the path somewhere,
there must be a transition.

667
00:58:42,000 --> 00:58:52,000
OK, they are not all in A,
OK, because in particular,

668
00:58:52,000 --> 00:58:58,000
V isn't.
OK, so we're going to do is

669
00:58:58,000 --> 00:59:09,000
swap (u,v) with the first edge
on this path that connects a

670
00:59:09,000 --> 00:59:18,000
vertex in A to a vertex in V
minus A.

671
00:59:18,000 --> 00:59:20,000
So in this case,
it's this edge here.

672
00:59:20,000 --> 00:59:24,000
I go from A to V minus A.
In general, I might be

673
00:59:24,000 --> 00:59:28,000
alternating many times,
OK, and I just picked the first

674
00:59:28,000 --> 00:59:32,000
one that I encounter.
OK, that this guy here.

675
00:59:32,000 --> 00:59:36,000
And what I do is I put this
edge in.

676
00:59:36,000 --> 00:59:38,000
OK, so then,
what happens?

677
00:59:38,000 --> 00:59:42,000
Well, the edge,
(u,v), is the lightest thing

678
00:59:42,000 --> 00:59:46,000
connecting something in A to
something in V minus A.

679
00:59:46,000 --> 00:59:49,000
So that means,
in particular,

680
00:59:49,000 --> 00:59:53,000
it's lighter than this edge,
has lower weight.

681
00:59:53,000 --> 00:59:57,000
So, by swapping this,
I'(V,E) created a tree with

682
00:59:57,000 --> 01:00:02,198
lower overall weight,
contradicting the assumption

683
01:00:02,198 --> 01:00:08,000
that this other thing was a
minimum spanning tree.

684
01:00:08,000 --> 01:00:14,219
OK: so, a lower weight spanning
tree than T results,

685
01:00:14,219 --> 01:00:18,000
and that's a contradiction --

686
01:00:25,000 --> 01:00:33,010
-- than T results.
And that's a contradiction,

687
01:00:33,010 --> 01:00:36,570
OK?
How are we doing?

688
01:00:36,570 --> 01:00:44,225
Everybody with me?
OK, now we get to do some

689
01:00:44,225 --> 01:00:46,895
algorithms.
Yea!

690
01:00:46,895 --> 01:00:55,439
So, we are going to do an
algorithm called Prim's

691
01:00:55,439 --> 01:01:01,853
algorithm.
Prim eventually became a very

692
01:01:01,853 --> 01:01:07,069
high-up at AT&T because he
invented this algorithm for

693
01:01:07,069 --> 01:01:12,187
minimum spanning trees,
and it was used in all of the

694
01:01:12,187 --> 01:01:15,730
billing code for AT&T for many
years.

695
01:01:15,730 --> 01:01:21,438
He was very high up at Bell
Labs back in the heyday of Bell

696
01:01:21,438 --> 01:01:24,784
Laboratories.
OK, so it just shows,

697
01:01:24,784 --> 01:01:30,000
all you have to do is invent an
algorithm.

698
01:01:30,000 --> 01:01:36,702
You too can be a president of a
corporate monopoly.

699
01:01:36,702 --> 01:01:43,807
Of course, the government can
do things to monopolies,

700
01:01:43,807 --> 01:01:49,438
but anyway, if that's your
mission in life,

701
01:01:49,438 --> 01:01:55,202
invent an algorithm.
OK, so here's the idea.

702
01:01:55,202 --> 01:02:03,648
What we're going to do is we're
going to maintain V minus A as a

703
01:02:03,648 --> 01:02:11,923
priority queue.
We'll call it Q.

704
01:02:11,923 --> 01:02:26,076
And each vertex,
we're going to key each vertex

705
01:02:26,076 --> 01:02:39,923
in Q with the weight of the
least weight edge,

706
01:02:39,923 --> 01:02:53,280
connecting it to a vertex in A.
So here's the code.

707
01:02:53,280 --> 01:03:00,000
So, we're going to start out
with Q being all vertices.

708
01:03:00,000 --> 01:03:03,873
So, we start out with A being,
if you will,

709
01:03:03,873 --> 01:03:07,930
the empty set.
OK, and what we're going to do

710
01:03:07,930 --> 01:03:13,095
it is the least weight edge,
therefore, for everything in

711
01:03:13,095 --> 01:03:18,536
the priority queue is basically
going to be infinity because

712
01:03:18,536 --> 01:03:23,700
none of them have any edges.
The least weight edge to the

713
01:03:23,700 --> 01:03:29,325
empty set is going to be empty.
And then, we're going to start

714
01:03:29,325 --> 01:03:33,958
out with one guy.
We'll call him S,

715
01:03:33,958 --> 01:03:39,489
which will set to zero for some
arbitrary S in V.

716
01:03:39,489 --> 01:03:45,135
And then, the main part of the
algorithm kicks in.

717
01:03:45,135 --> 01:03:51,703
So that's our initialization.
OK, when we do the analysis,

718
01:03:51,703 --> 01:03:58,271
I'm going to write some stuff
on the left hand side of the

719
01:03:58,271 --> 01:04:04,406
board.
So if you're taking notes,

720
01:04:04,406 --> 01:04:14,406
you may want to also leave a
little bit of space on the left

721
01:04:14,406 --> 01:04:22,711
hand side of your notes.
So, while Q is not empty,

722
01:04:22,711 --> 01:04:30,000
we get the smallest element out
of it.

723
01:04:41,000 --> 01:04:43,000
And then we do some stuff.

724
01:05:19,000 --> 01:05:21,970
That's it.
And the only thing I should

725
01:05:21,970 --> 01:05:25,503
mention here is,
OK, so let's just see what's

726
01:05:25,503 --> 01:05:28,875
going on here.
And then we'll run it on the

727
01:05:28,875 --> 01:05:32,256
example.
OK, so what we do is we take

728
01:05:32,256 --> 01:05:36,609
out the smallest element out of
the queue at each step.

729
01:05:36,609 --> 01:05:40,156
And then for each step in the
adjacency list,

730
01:05:40,156 --> 01:05:43,783
in other words,
everything for which I have an

731
01:05:43,783 --> 01:05:46,846
edge going from v to u,
we take a look,

732
01:05:46,846 --> 01:05:51,440
and if v is still in our set V
minus A, so things we'(V,E)

733
01:05:51,440 --> 01:05:54,261
taken out are going to be part
of A.

734
01:05:54,261 --> 01:05:57,163
OK, every time we take
something out,

735
01:05:57,163 --> 01:06:02,000
that's going to be a new A that
we construct.

736
01:06:02,000 --> 01:06:04,258
At every step,
we want to find,

737
01:06:04,258 --> 01:06:08,400
what's the cheapest edge
connecting that A to everything

738
01:06:08,400 --> 01:06:11,035
else?
We basically are going to take

739
01:06:11,035 --> 01:06:15,025
whatever that cheapest thing is,
OK, add that edge in,

740
01:06:15,025 --> 01:06:19,242
and now bring that into A and
find the next cheapest one.

741
01:06:19,242 --> 01:06:22,103
And we just keep repeating the
process.

742
01:06:22,103 --> 01:06:25,567
OK, we'll do it on the example.
And what we do,

743
01:06:25,567 --> 01:06:28,955
is every time we bring it in,
I keep track of,

744
01:06:28,955 --> 01:06:34,000
what was the vertex responsible
for bringing me in.

745
01:06:34,000 --> 01:06:43,947
And what I claim is that at the
end, if I look at the set of

746
01:06:43,947 --> 01:06:52,209
these pairs that I'(V,E) made
here, V and pi of V,

747
01:06:52,209 --> 01:06:58,279
that forms the minimum spanning
tree.

748
01:06:58,279 --> 01:07:05,441
So let's just do this.
And, what's that?

749
01:07:05,441 --> 01:07:12,191
We're all set up.
So let's get rid of these guys

750
01:07:12,191 --> 01:07:20,234
here because we are going to
recompute them from scratch.

751
01:07:20,234 --> 01:07:30,000
OK, so you may want to copy the
graph over again in your notes.

752
01:07:30,000 --> 01:07:34,840
I was going to do it,
but it turned out,

753
01:07:34,840 --> 01:07:40,797
this is exactly the board is
going to erase this.

754
01:07:40,797 --> 01:07:47,127
OK, well let me just modify it.
OK, so we start out.

755
01:07:47,127 --> 01:07:54,574
We make everything be infinity.
OK, so that's where I'm going

756
01:07:54,574 --> 01:08:01,028
to keep the key value.
OK, and then what I'm going to

757
01:08:01,028 --> 01:08:07,952
do is find one vertex.
And I'm going to call him S.

758
01:08:07,952 --> 01:08:11,749
And I'm going to do this vertex
here.

759
01:08:11,749 --> 01:08:15,018
We'll call that S.
So basically,

760
01:08:15,018 --> 01:08:19,447
I now make him be zero.
And now, what I do,

761
01:08:19,447 --> 01:08:23,453
is I execute extract min.
So basically,

762
01:08:23,453 --> 01:08:28,198
what I'll do is I'll just shade
him like this,

763
01:08:28,198 --> 01:08:34,000
indicating that he has now
joined the set A.

764
01:08:34,000 --> 01:08:40,931
So, this is going to be A.
And this is element of V minus

765
01:08:40,931 --> 01:08:44,644
A.
OK, so then what we do is we

766
01:08:44,644 --> 01:08:47,986
take a look.
We extract him,

767
01:08:47,986 --> 01:08:53,433
and then for each edge in the
adjacency list,

768
01:08:53,433 --> 01:08:59,002
OK, so for each vertex in the
adjacency lists,

769
01:08:59,002 --> 01:09:05,315
that these guys here,
OK, we're going to look to see

770
01:09:05,315 --> 01:09:12,000
if it's still in Q,
that is, in V minus A.

771
01:09:12,000 --> 01:09:16,795
And if so, and its key value is
less than what the value is at

772
01:09:16,795 --> 01:09:20,254
the edge, there,
we're going to replace it by

773
01:09:20,254 --> 01:09:22,770
the edge value.
So, in this case,

774
01:09:22,770 --> 01:09:25,600
we're going to replace this by
seven.

775
01:09:25,600 --> 01:09:30,317
We're going to replace this by
15, and we're going to replace

776
01:09:30,317 --> 01:09:33,854
this by ten, OK,
because what we're interested

777
01:09:33,854 --> 01:09:39,608
in is, what is the cheapest?
Now, notice that everything in

778
01:09:39,608 --> 01:09:43,782
V minus A, that is,
what's in the priority queue,

779
01:09:43,782 --> 01:09:48,216
everything in there,
OK, now has its cheapest way of

780
01:09:48,216 --> 01:09:53,086
connecting it to the things that
I'(V,E) already removed,

781
01:09:53,086 --> 01:09:57,173
the things that are in A.
OK, and so now I just,

782
01:09:57,173 --> 01:10:01,608
OK, when I actually do that
update, there's actually

783
01:10:01,608 --> 01:10:07,000
something implicit going on in
this priority queue.

784
01:10:07,000 --> 01:10:10,636
And that is that I have to do a
decreased key.

785
01:10:10,636 --> 01:10:14,111
So, there's an implicit
decrease of the key.

786
01:10:14,111 --> 01:10:19,121
So, decreased key is a priority
queue operation that lowers the

787
01:10:19,121 --> 01:10:22,191
value of the key in the priority
queue.

788
01:10:22,191 --> 01:10:26,878
And so, that's implicitly going
on when I look at what data

789
01:10:26,878 --> 01:10:31,646
structure I'm going to use to
implement that priority queue.

790
01:10:31,646 --> 01:10:36,171
OK, so common data structures
for implementing a priority

791
01:10:36,171 --> 01:10:41,376
queue are a min heap.
OK, so I have to make sure that

792
01:10:41,376 --> 01:10:43,905
I'm actually doing this
operation.

793
01:10:43,905 --> 01:10:47,355
I can't just change it and not
affect my heap.

794
01:10:47,355 --> 01:10:51,111
So, there is an implicit
operation going on there.

795
01:10:51,111 --> 01:10:54,407
OK, now I repeat.
I find the cheapest thing,

796
01:10:54,407 --> 01:10:58,547
oh, and I also have to set,
now, a pointer from each of

797
01:10:58,547 --> 01:11:02,931
these guys back to u.
So here, this guy sets a

798
01:11:02,931 --> 01:11:07,114
pointer going this way.
This guy sets a pointer going

799
01:11:07,114 --> 01:11:11,298
this way, and this guy sets a
pointer going this way.

800
01:11:11,298 --> 01:11:16,206
That's my pi thing that's going
to keep track of who caused me

801
01:11:16,206 --> 01:11:20,873
to set my value to what it is.
So now, we go in and we find

802
01:11:20,873 --> 01:11:22,885
the cheapest thing,
again.

803
01:11:22,885 --> 01:11:25,620
And we're going to do it fast,
too.

804
01:11:25,620 --> 01:11:32,361
OK, this is a fast algorithm.
OK, so now we're going to go do

805
01:11:32,361 --> 01:11:36,481
this again.
So now, what's the cheapest

806
01:11:36,481 --> 01:11:39,843
thing to extract?
This guy here,

807
01:11:39,843 --> 01:11:42,987
right?
So, we'll take him out,

808
01:11:42,987 --> 01:11:47,542
OK, and now we update all of
his neighbors.

809
01:11:47,542 --> 01:11:51,771
So this guy gets five.
This guy gets 12.

810
01:11:51,771 --> 01:11:56,542
This guy gets nine.
This guy we don't update.

811
01:11:56,542 --> 01:12:02,722
We don't update him because
he's no longer in the priority

812
01:12:02,722 --> 01:12:07,464
queue.
And all of these guys now,

813
01:12:07,464 --> 01:12:12,297
we make point to where they're
supposed to point to.

814
01:12:12,297 --> 01:12:17,983
And, we're done with that step.
Now we find the cheapest one.

815
01:12:17,983 --> 01:12:22,437
What's the cheapest one now?
The five over here.

816
01:12:22,437 --> 01:12:24,807
Good.
So, we take him out.

817
01:12:24,807 --> 01:12:30,019
OK, we update the neighbors.
Here, yep, that goes to six

818
01:12:30,019 --> 01:12:34,000
now.
And, we have that pointer.

819
01:12:34,000 --> 01:12:39,684
And, this guy we don't do,
because he's not in there.

820
01:12:39,684 --> 01:12:44,604
This guy becomes 14,
and this guy here becomes

821
01:12:44,604 --> 01:12:47,774
eight.
So, we update that guy,

822
01:12:47,774 --> 01:12:52,803
make him be eight.
Did I do this the right way?

823
01:12:52,803 --> 01:12:57,395
Yeah, because pi is a function
of this guy.

824
01:12:57,395 --> 01:13:00,675
So basically,
this thing, then,

825
01:13:00,675 --> 01:13:04,938
disappears.
Yeah, did I have another one

826
01:13:04,938 --> 01:13:09,258
that I missed?
12, yes, good,

827
01:13:09,258 --> 01:13:12,584
it's removed,
OK, because pi is just a

828
01:13:12,584 --> 01:13:14,741
function.
And now I'm OK.

829
01:13:14,741 --> 01:13:18,516
OK, so now what do I do?
OK, so now my set,

830
01:13:18,516 --> 01:13:23,191
A, consists of these three
things, and now I want the

831
01:13:23,191 --> 01:13:26,786
cheapest edge.
I know it's in the minimum

832
01:13:26,786 --> 01:13:30,561
spanning tree.
So let me just greedily pick

833
01:13:30,561 --> 01:13:34,554
it.
OK, so what's the cheapest

834
01:13:34,554 --> 01:13:37,108
thing now?
This guy appear?

835
01:13:37,108 --> 01:13:39,466
Yeah, six.
So we take it.

836
01:13:39,466 --> 01:13:44,771
We go to update these things,
and nothing matters here.

837
01:13:44,771 --> 01:13:50,175
OK, nothing changes because
these guys are already in A.

838
01:13:50,175 --> 01:13:54,202
OK, so now the cheapest one is
eight here.

839
01:13:54,202 --> 01:13:56,856
Good.
So, we take eight out.

840
01:13:56,856 --> 01:14:01,656
OK, we update this.
Nothing to be done.

841
01:14:01,656 --> 01:14:04,970
This: nothing to be done.
This: oh, no,

842
01:14:04,970 --> 01:14:09,242
this one, instead of 14 we can
make this be three.

843
01:14:09,242 --> 01:14:14,212
So, we get rid of that pointer
and make it point that way.

844
01:14:14,212 --> 01:14:16,915
Now three is the cheapest
thing.

845
01:14:16,915 --> 01:14:21,100
So, we take it out,
and of course there's nothing

846
01:14:21,100 --> 01:14:24,239
to be done over there.
And now, last,

847
01:14:24,239 --> 01:14:26,506
I take nine.
And it's done.

848
01:14:26,506 --> 01:14:32,000
And 15: it's done.
And the algorithm terminates.

849
01:14:32,000 --> 01:14:36,972
OK, and as I look at,
now, all the edges that I

850
01:14:36,972 --> 01:14:43,135
picked, those are exactly all
the edges that we had at the

851
01:14:43,135 --> 01:14:48,000
beginning.
OK, let's do an analysis here.

852
01:14:58,000 --> 01:15:06,316
OK, so let's see,
this part here costs me order

853
01:15:06,316 --> 01:15:11,197
V, right?
OK, and this part,

854
01:15:11,197 --> 01:15:16,983
let's see what we are doing
here.

855
01:15:16,983 --> 01:15:27,107
Well, we're going to go through
this loop how many times?

856
01:15:27,107 --> 01:15:32,711
V times.
It's V elements we put into the

857
01:15:32,711 --> 01:15:35,860
queue.
We are not inserting anything.

858
01:15:35,860 --> 01:15:39,795
We're just taking them out.
This goes V times,

859
01:15:39,795 --> 01:15:43,818
OK, and we do a certain number
of extract Mins.

860
01:15:43,818 --> 01:15:47,492
So, we're going to do order V
extract Mins.

861
01:15:47,492 --> 01:15:52,915
And then we go to the adjacency
list, and we have some constant

862
01:15:52,915 --> 01:15:55,801
things.
But we have these implicit

863
01:15:55,801 --> 01:16:00,000
decreased keys for this stuff
here.

864
01:16:00,000 --> 01:16:07,412
That's this thing here.
OK, and so how many implicit

865
01:16:07,412 --> 01:16:14,389
decreased keys do we have?
That's going to be the

866
01:16:14,389 --> 01:16:18,459
expensive thing.
OK, we have,

867
01:16:18,459 --> 01:16:25,000
in this case,
the degree of u of those.

868
01:16:25,000 --> 01:16:31,309
OK, so overall,
how many implicit decreased

869
01:16:31,309 --> 01:16:38,218
keys do we have?
Well, we have V times through.

870
01:16:38,218 --> 01:16:43,025
How big could the degree of u
be?

871
01:16:43,025 --> 01:16:48,433
OK, it could be as big as V,
order V.

872
01:16:48,433 --> 01:16:56,995
So, that's V^2 decreased use.
But we can do a better bound

873
01:16:56,995 --> 01:17:04,189
than that.
How many do we really have?

874
01:17:04,189 --> 01:17:11,948
Yeah, at most order E,
OK, because what am I doing?

875
01:17:11,948 --> 01:17:19,086
I'm summing up the degrees of
all the vertices.

876
01:17:19,086 --> 01:17:27,000
That's how many times I
actually execute that.

877
01:17:27,000 --> 01:17:34,322
So, I have order E,
implicit decreased keys.

878
01:17:34,322 --> 01:17:44,028
So the time overall is order V
times time for whatever the

879
01:17:44,028 --> 01:17:53,224
extract Min is plus E times the
time for decreased key.

880
01:17:53,224 --> 01:18:02,931
So now, let's look at data
structures, and we can evaluate

881
01:18:02,931 --> 01:18:14,000
for different data structures
what this formula gives us.

882
01:18:14,000 --> 01:18:21,492
So, we have different ways of
implementing a data structure.

883
01:18:21,492 --> 01:18:28,222
We have the cost of extract
Min, and of decreased key,

884
01:18:28,222 --> 01:18:32,636
and total.
So, the simplest way of

885
01:18:32,636 --> 01:18:38,369
implementing a data structure is
an unsorted array.

886
01:18:38,369 --> 01:18:44,904
If I have an unsorted array,
how much time does it take me

887
01:18:44,904 --> 01:18:51,668
to extract the minimum element?
If I have an unsorted array?

888
01:18:51,668 --> 01:18:58,433
Right, order V in this case
because it's an array of size V.

889
01:18:58,433 --> 01:19:06,000
And, to do a decreased key,
OK, I can do it in order one.

890
01:19:06,000 --> 01:19:14,245
So, the total is V^2,
good, order V^2 algorithm.

891
01:19:14,245 --> 01:19:22,666
Or, as people suggested,
how about a binary heap?

892
01:19:22,666 --> 01:19:33,017
OK, to do an extract Min in a
binary heap will cost me what?

893
01:19:33,017 --> 01:19:38,905
O of log V.
Decreased key will cost me,

894
01:19:38,905 --> 01:19:44,932
yeah, it turns out you can do
that in order log V because

895
01:19:44,932 --> 01:19:49,668
basically you just have to
shuffle the value,

896
01:19:49,668 --> 01:19:54,295
actually shuffle it up towards
the root, OK?

897
01:19:54,295 --> 01:19:58,708
Or at log V.
And, the total cost therefore

898
01:19:58,708 --> 01:20:01,717
is?
E log V, good.

899
01:20:01,717 --> 01:20:06,869
Which of these is better?
It depends, good.

900
01:20:06,869 --> 01:20:12,758
When is one better,
and when is the other better?

901
01:20:12,758 --> 01:20:18,401
Yeah, if it's a dense graph,
E is close to V^2,

902
01:20:18,401 --> 01:20:24,166
the array is better.
But if it's a sparse graph,

903
01:20:24,166 --> 01:20:33,000
and E is much smaller than V^2,
then the binary heap is better.

904
01:20:33,000 --> 01:20:37,824
So that motivated the invention
of a data structure,

905
01:20:37,824 --> 01:20:43,216
OK, called a Fibonacci Heap.
So, Fibonacci Heap is covered

906
01:20:43,216 --> 01:20:47,851
in Chapter 20 of CLRS.
We're not going to hold you

907
01:20:47,851 --> 01:20:53,148
responsible for the content,
but it's an interesting data

908
01:20:53,148 --> 01:20:57,878
structure because it's an
amortized data structure.

909
01:20:57,878 --> 01:21:01,851
And it turns out that it is
data structure,

910
01:21:01,851 --> 01:21:08,000
you can do extract Min in order
log V amortized time.

911
01:21:08,000 --> 01:21:12,747
And remarkably,
you can do decreased key in

912
01:21:12,747 --> 01:21:17,834
order one amortized.
So, when I plug those in,

913
01:21:17,834 --> 01:21:21,000
what do I get over here?

914
01:21:34,000 --> 01:21:42,088
What's that going to be?
Plug that it here.

915
01:21:42,088 --> 01:21:52,296
It's going to be V times log V
plus E: E plus V log V.

916
01:21:52,296 --> 01:22:00,000
These are amortized,
so what's this?

917
01:22:00,000 --> 01:22:02,317
Trick question.
It's worst-case.

918
01:22:02,317 --> 01:22:05,979
It's not amortized over here.
These are amortized,

919
01:22:05,979 --> 01:22:08,745
but that's the beauty of
amortization.

920
01:22:08,745 --> 01:22:13,006
I can say it's going to be
worst case: E plus V log V over

921
01:22:13,006 --> 01:22:17,715
here, because when I add up the
amortized cost of my operations,

922
01:22:17,715 --> 01:22:20,480
it's an upper bound on the true
costs.

923
01:22:20,480 --> 01:22:24,291
OK, so that's why I say,
one of the beauties of this

924
01:22:24,291 --> 01:22:27,058
amortized analysis,
and in particular,

925
01:22:27,058 --> 01:22:31,692
being able to assign different
costs to different operations is

926
01:22:31,692 --> 01:22:37,000
I can just add them up and I get
my worst-case costs.

927
01:22:37,000 --> 01:22:40,565
So this is already V log V.
There are a couple other

928
01:22:40,565 --> 01:22:43,012
algorithms just before I let you
go.

929
01:22:43,012 --> 01:22:47,066
Kruskal's Algorithm in the book
uses another amortized data

930
01:22:47,066 --> 01:22:50,282
structure called a disjoint set
data structure,

931
01:22:50,282 --> 01:22:53,498
which also runs in E log V,
that is, this time:

932
01:22:53,498 --> 01:22:56,574
runs in this time,
the same as using a binary

933
01:22:56,574 --> 01:23:00,000
heap.
So, I'll refer you to the book.

934
01:23:00,000 --> 01:23:04,934
The best algorithm to date with
this problem is done by our own

935
01:23:04,934 --> 01:23:09,233
David Karger on the faculty here
with one of our former

936
01:23:09,233 --> 01:23:12,975
graduates, Phil Kline,
who is now a professor at

937
01:23:12,975 --> 01:23:17,353
Brown, and Robert Tarjan,
who is sort of like the master

938
01:23:17,353 --> 01:23:22,368
of all data structures who was a
professor at Princeton in 1993.

939
01:23:22,368 --> 01:23:26,189
OK, it's a randomized
algorithm, and it gives you

940
01:23:26,189 --> 01:23:32,000
order V plus E expected time.
OK, so that's the best to date.

941
01:23:32,000 --> 01:23:36,300
It's still open as to whether
there is a deterministic,

942
01:23:36,300 --> 01:23:40,679
there is worst-case bound,
whether there is a worst-case

943
01:23:40,679 --> 01:23:45,059
bound that is linear time.
OK, but there is a randomized

944
01:23:45,059 --> 01:23:47,369
to linear time,
and otherwise,

945
01:23:47,369 --> 01:23:51,509
this is essentially the best
bound without additional

946
01:23:51,509 --> 01:23:54,058
assumptions.
OK, very cool stuff.

947
01:23:54,058 --> 01:23:58,675
Next, we're going to see a lot
of these ideas of greedy and

948
01:23:58,675 --> 01:24:01,000
dynamic programming in practice.