1 00:00:00,050 --> 00:00:01,770 The following content is provided 2 00:00:01,770 --> 00:00:04,010 under a Creative Commons license. 3 00:00:04,010 --> 00:00:06,860 Your support will help MIT OpenCourseWare continue 4 00:00:06,860 --> 00:00:10,720 to offer high quality educational resources for free. 5 00:00:10,720 --> 00:00:13,320 To make a donation or view additional materials 6 00:00:13,320 --> 00:00:17,207 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:17,207 --> 00:00:17,832 at ocw.mit.edu. 8 00:00:22,120 --> 00:00:25,770 PROFESSOR: [INAUDIBLE] audio. 9 00:00:25,770 --> 00:00:32,310 We're going to have a series of lectures on shortest paths. 10 00:00:32,310 --> 00:00:36,380 And one of the big differences between this module 11 00:00:36,380 --> 00:00:39,940 and the previous one, at least from a mathematical standpoint, 12 00:00:39,940 --> 00:00:41,460 is that we're going to be looking 13 00:00:41,460 --> 00:00:44,780 at graphs that have weights on their edges. 14 00:00:44,780 --> 00:00:47,690 So when Eric talked about depth first search and breadth 15 00:00:47,690 --> 00:00:50,140 first search in the last couple of lectures, 16 00:00:50,140 --> 00:00:54,160 we had directed graphs and undirected graphs. 17 00:00:54,160 --> 00:00:57,270 But we didn't really have attributes on the edges. 18 00:00:57,270 --> 00:01:00,690 In particular, you have a much richer class 19 00:01:00,690 --> 00:01:03,190 of problems and applications if you 20 00:01:03,190 --> 00:01:07,700 allow for weights on graph edges. 21 00:01:07,700 --> 00:01:13,020 And these weights can be integers. 22 00:01:13,020 --> 00:01:15,270 They could be real numbers, irrationals. 23 00:01:15,270 --> 00:01:17,560 They could be negative, what have you. 24 00:01:17,560 --> 00:01:22,030 And for different classes of graphs 25 00:01:22,030 --> 00:01:25,790 and different restrictions on weights, 26 00:01:25,790 --> 00:01:29,460 there's really a set of shortest path algorithms, 27 00:01:29,460 --> 00:01:33,150 that we'll look at in the next few lectures, which 28 00:01:33,150 --> 00:01:37,050 are kind of optimized for a particular application. 29 00:01:37,050 --> 00:01:41,900 So we won't do specific algorithms today. 30 00:01:41,900 --> 00:01:43,770 But we'll set up the problem. 31 00:01:43,770 --> 00:01:46,050 We'll talk about the general approach 32 00:01:46,050 --> 00:01:48,970 that most shortest path algorithms take 33 00:01:48,970 --> 00:01:52,010 to solve a particular instance of a problem. 34 00:01:52,010 --> 00:01:56,810 And then we'll close with talking 35 00:01:56,810 --> 00:01:59,390 about a particular property that's pretty important. 36 00:01:59,390 --> 00:02:03,290 That's called the optimum or optimal substructure property. 37 00:02:03,290 --> 00:02:06,904 That is a technique that most shortest path algorithms, 38 00:02:06,904 --> 00:02:08,570 or actually all shortest path algorithms 39 00:02:08,570 --> 00:02:14,690 use to get efficient complexity. 40 00:02:14,690 --> 00:02:18,890 So asymptotic complexity is important, obviously. 41 00:02:18,890 --> 00:02:22,580 And we're always looking for the best algorithm with the best 42 00:02:22,580 --> 00:02:23,930 asymptotic complexity. 43 00:02:23,930 --> 00:02:26,710 And optimal substructure is a hammer 44 00:02:26,710 --> 00:02:30,990 that we're going to use to get that. 45 00:02:30,990 --> 00:02:35,850 So the canonical motivation, of course, for shortest paths 46 00:02:35,850 --> 00:02:38,010 is-- now, if you want to steal, or I 47 00:02:38,010 --> 00:02:40,640 guess borrow, a cannon from Caltech 48 00:02:40,640 --> 00:02:43,860 and bring it over to MIT, then you 49 00:02:43,860 --> 00:02:52,510 want the fastest way of getting here with your illegal goods. 50 00:02:52,510 --> 00:02:56,230 And you want to find the shortest way or the fastest 51 00:02:56,230 --> 00:02:59,080 way of getting from one location to another. 52 00:02:59,080 --> 00:03:03,500 So Google Maps go from point A to point 53 00:03:03,500 --> 00:03:09,210 B. That's a classic application of the shortest path problem. 54 00:03:09,210 --> 00:03:13,000 In this case, you could imagine that distance 55 00:03:13,000 --> 00:03:16,110 would be something that would be a very simple metric that you 56 00:03:16,110 --> 00:03:19,120 could use for the weights on the edges. 57 00:03:19,120 --> 00:03:23,360 So for this entire module, we're going 58 00:03:23,360 --> 00:03:31,390 to be looking at a graph G V, E, W. 59 00:03:31,390 --> 00:03:32,660 And you know what V and E are. 60 00:03:32,660 --> 00:03:38,360 They're the vertices and the edges. 61 00:03:38,360 --> 00:03:47,020 And W is a weight function that maps edges to weights. 62 00:03:47,020 --> 00:03:50,510 And so we're adding that in here. 63 00:03:50,510 --> 00:03:59,140 And so W would be E to R. So the set of real numbers. 64 00:03:59,140 --> 00:04:02,990 We're going to be looking at two different algorithms 65 00:04:02,990 --> 00:04:04,630 in subsequent lectures. 66 00:04:04,630 --> 00:04:07,085 And you'll implement one of them in your problem set. 67 00:04:10,380 --> 00:04:15,030 The simpler algorithm, which we'll look at first, 68 00:04:15,030 --> 00:04:18,510 is called Dijkstra, after Edsger Dijkstra, who 69 00:04:18,510 --> 00:04:21,970 did some similar work in concurrent programming, 70 00:04:21,970 --> 00:04:23,320 won the Turing Award. 71 00:04:23,320 --> 00:04:26,790 But on the side invented this cool algorithm-- 72 00:04:26,790 --> 00:04:30,330 or at least gets credit for it-- called Dijkstra's algorithm 73 00:04:30,330 --> 00:04:36,530 that assumes non-negative weight edges. 74 00:04:36,530 --> 00:04:38,030 So I should really say non-negative. 75 00:04:38,030 --> 00:04:40,690 So read that as non-negative. 76 00:04:40,690 --> 00:04:51,510 And that has a complexity of order V log V plus E. 77 00:04:51,510 --> 00:04:52,290 All right. 78 00:04:52,290 --> 00:04:58,330 So this is practically linear time. 79 00:04:58,330 --> 00:05:01,910 And typically, you're going to be dominated in many cases 80 00:05:01,910 --> 00:05:10,060 by E. In general, if you talk about a simple graph, what's 81 00:05:10,060 --> 00:05:13,690 the asymptotic relationship between E and V? 82 00:05:19,430 --> 00:05:21,040 Can you relate E to V? 83 00:05:21,040 --> 00:05:22,874 Can you give me a bound? 84 00:05:22,874 --> 00:05:23,942 AUDIENCE: V squared. 85 00:05:23,942 --> 00:05:24,650 PROFESSOR: Sorry? 86 00:05:24,650 --> 00:05:25,970 V squared, thanks. 87 00:05:25,970 --> 00:05:27,700 That's good. 88 00:05:27,700 --> 00:05:33,280 So you can think of E as being order V square. 89 00:05:33,280 --> 00:05:36,470 And you can certainly have-- that's worth recursion. 90 00:05:41,310 --> 00:05:44,540 So now, you can kind of imagine a complete graph. 91 00:05:44,540 --> 00:05:46,390 And a complete graph is something 92 00:05:46,390 --> 00:05:52,980 that has an edge between each pair of vertices. 93 00:05:52,980 --> 00:05:57,710 And that's where you'll get E being k w squared. 94 00:05:57,710 --> 00:05:59,370 So when you say simple graph, you're 95 00:05:59,370 --> 00:06:03,630 saying you have at most one edge between any pair of vertices. 96 00:06:03,630 --> 00:06:05,160 A multigraph is something that could 97 00:06:05,160 --> 00:06:08,140 have multiple edges between pairs of vertices. 98 00:06:08,140 --> 00:06:10,480 We won't really be talking about multigraphs 99 00:06:10,480 --> 00:06:12,840 in this sequence of lectures. 100 00:06:12,840 --> 00:06:19,420 But something to think about or keep in the back of your mind 101 00:06:19,420 --> 00:06:21,130 as we go through these algorithms. 102 00:06:21,130 --> 00:06:25,810 And so the dominating factor here, and in many cases really, 103 00:06:25,810 --> 00:06:30,390 is E. And Dijkstra is a nice algorithm, because it's 104 00:06:30,390 --> 00:06:34,080 linear in the number of edges. 105 00:06:34,080 --> 00:06:35,900 So that's Dijkstra. 106 00:06:35,900 --> 00:06:38,090 And that's the first of the algorithms 107 00:06:38,090 --> 00:06:40,750 that we'll look at next time. 108 00:06:40,750 --> 00:06:44,070 But we'll see the general structure of Dijkstra today. 109 00:06:44,070 --> 00:06:50,880 And then there's the Bellman-Ford algorithm 110 00:06:50,880 --> 00:06:56,890 that works on positive and negative edges, weight edges. 111 00:06:56,890 --> 00:07:03,630 And this has a complexity order V E. 112 00:07:03,630 --> 00:07:06,240 So you could imagine a particular implementation 113 00:07:06,240 --> 00:07:10,060 of Bellman-Ford running in order V cubed time. 114 00:07:10,060 --> 00:07:12,360 Because E could be V square. 115 00:07:12,360 --> 00:07:14,610 And you've got this additional E factor. 116 00:07:14,610 --> 00:07:17,800 So it's order V cubed versus order V log V. 117 00:07:17,800 --> 00:07:21,000 So when you have a chance, use Dijkstra. 118 00:07:21,000 --> 00:07:25,670 When you're stuck, you'd want to do Bellman-Ford, 119 00:07:25,670 --> 00:07:27,665 because you have these negative weight edges. 120 00:07:27,665 --> 00:07:29,790 And one of the challenges in negative weight edges, 121 00:07:29,790 --> 00:07:32,550 and I'll say a little bit more as we go along, 122 00:07:32,550 --> 00:07:37,050 is that you end up having to have 123 00:07:37,050 --> 00:07:41,720 to find cycles that are of a negative weight, 124 00:07:41,720 --> 00:07:44,840 because they kind of throw off your shortest path algorithm 125 00:07:44,840 --> 00:07:49,590 if you were just assuming that shortest path lengths are only 126 00:07:49,590 --> 00:07:51,180 going to decrease. 127 00:07:51,180 --> 00:07:55,460 But when you have negative weights, you might take a step 128 00:07:55,460 --> 00:07:57,900 and the overall weight might decrease. 129 00:07:57,900 --> 00:08:01,150 So it's kind of a longer path in terms of the number of edges. 130 00:08:01,150 --> 00:08:02,600 But the weight is smaller. 131 00:08:02,600 --> 00:08:05,970 And that kind of makes the algorithm more complicated. 132 00:08:05,970 --> 00:08:07,362 And it has to do more work. 133 00:08:07,362 --> 00:08:09,820 So that's really why there's a difference between these two 134 00:08:09,820 --> 00:08:10,760 complexities. 135 00:08:10,760 --> 00:08:14,200 And I guarantee you, you'll understand this much better 136 00:08:14,200 --> 00:08:16,700 after we're done with the lectures on Dijkstra 137 00:08:16,700 --> 00:08:20,019 and the lectures on Bellman-Ford. 138 00:08:20,019 --> 00:08:21,560 So that's the set up for the problem. 139 00:08:21,560 --> 00:08:24,030 That's what we're going to be looking at. 140 00:08:24,030 --> 00:08:27,830 Let's look at a couple more definitions 141 00:08:27,830 --> 00:08:34,549 beyond what I have here with respect to just the notation. 142 00:08:34,549 --> 00:08:46,883 And you can think of path p as a sequence of vertices-- V0, V1, 143 00:08:46,883 --> 00:08:49,880 et cetera, to Vk. 144 00:08:49,880 --> 00:08:57,650 And this is the path if Vi, Vi plus 1 145 00:08:57,650 --> 00:09:03,320 belongs to E for 0 less than or equal to i less than 146 00:09:03,320 --> 00:09:05,210 or equal to k. 147 00:09:05,210 --> 00:09:08,290 So a path is a sequence of edges. 148 00:09:08,290 --> 00:09:10,670 And each of those edges has to be in the graph, 149 00:09:10,670 --> 00:09:15,530 has to be in the set of edges E. 150 00:09:15,530 --> 00:09:20,090 And W of p, which is the weight of the path, 151 00:09:20,090 --> 00:09:22,860 we know that by the weight of edges, those are easy. 152 00:09:22,860 --> 00:09:24,970 They're given by the W function. 153 00:09:24,970 --> 00:09:27,840 The weight of the path is simply the summation 154 00:09:27,840 --> 00:09:29,620 of the weights of the edges. 155 00:09:33,100 --> 00:09:33,620 All right. 156 00:09:33,620 --> 00:09:35,610 So fairly obvious definitions. 157 00:09:35,610 --> 00:09:37,730 But obviously, we have to get these right in order 158 00:09:37,730 --> 00:09:40,350 to actually solve the problem correctly. 159 00:09:43,030 --> 00:09:49,760 And the shortest path problem is, as you can imagine, 160 00:09:49,760 --> 00:10:00,050 something that tries to find a path p that has minimum weight. 161 00:10:00,050 --> 00:10:03,840 So in general, you have some set up for the problem. 162 00:10:03,840 --> 00:10:07,350 But it comes down to find p with-- 163 00:10:13,260 --> 00:10:16,599 And there are many, many possible paths. 164 00:10:16,599 --> 00:10:18,640 You have to understand that there are potentially 165 00:10:18,640 --> 00:10:22,082 an exponential number of paths in the graphs 166 00:10:22,082 --> 00:10:23,040 that we would consider. 167 00:10:23,040 --> 00:10:26,610 And here's a real simple example where 168 00:10:26,610 --> 00:10:33,360 you would have an exponential number of paths. 169 00:10:33,360 --> 00:10:38,540 And we'll come back to this example later in the lecture. 170 00:10:38,540 --> 00:10:41,390 But let's assume that all the directions go this way. 171 00:10:44,050 --> 00:10:46,500 And it's a directed graph. 172 00:10:46,500 --> 00:10:49,245 Well, you could have the path that goes all the way here. 173 00:10:49,245 --> 00:10:52,220 But you could have the path that goes on top 174 00:10:52,220 --> 00:10:53,700 and all the way this way. 175 00:10:53,700 --> 00:10:57,510 You have basically two choices on getting to this vertex. 176 00:10:57,510 --> 00:10:59,190 Then you've got, given the two ways 177 00:10:59,190 --> 00:11:01,320 you have of getting to this vertex. 178 00:11:01,320 --> 00:11:03,580 You've got four ways of getting here. 179 00:11:03,580 --> 00:11:06,240 And then, you have eight ways of getting there. 180 00:11:06,240 --> 00:11:07,870 So on and so forth. 181 00:11:07,870 --> 00:11:10,750 So there's an exponential number of paths potentially. 182 00:11:10,750 --> 00:11:12,970 The other thing that's interesting here, 183 00:11:12,970 --> 00:11:15,900 which is important in terms of this complexity 184 00:11:15,900 --> 00:11:19,580 is, what's interesting about what you see here 185 00:11:19,580 --> 00:11:23,380 with respect to the complexity and what you see here. 186 00:11:23,380 --> 00:11:25,590 Anybody want to point that out? 187 00:11:25,590 --> 00:11:30,910 So I have this complexity here and order VE out there. 188 00:11:30,910 --> 00:11:34,762 What's an interesting observation 189 00:11:34,762 --> 00:11:37,095 if you look at this board here and the two complexities? 190 00:11:40,630 --> 00:11:41,810 Anybody? 191 00:11:41,810 --> 00:11:42,580 Yeah, back there. 192 00:11:42,580 --> 00:11:44,330 AUDIENCE: It's not a function of weight. 193 00:11:44,330 --> 00:11:45,690 PROFESSOR: It's not a function of weight. 194 00:11:45,690 --> 00:11:46,190 Great. 195 00:11:46,190 --> 00:11:48,650 That's definitely worth recursion. 196 00:11:48,650 --> 00:11:50,180 And I'll let you throw this one. 197 00:11:53,120 --> 00:11:55,990 All the way back there, all right? 198 00:11:55,990 --> 00:11:57,900 Right. 199 00:11:57,900 --> 00:11:58,580 All right, good. 200 00:11:58,580 --> 00:11:59,537 Good, good. 201 00:11:59,537 --> 00:12:00,120 That was good. 202 00:12:00,120 --> 00:12:01,450 That was better than what I could do. 203 00:12:01,450 --> 00:12:02,180 No, not really. 204 00:12:02,180 --> 00:12:09,390 But I would've been right in his hands right there. 205 00:12:09,390 --> 00:12:12,940 Anyway, so that's a great observation, actually. 206 00:12:12,940 --> 00:12:14,940 And I should have pointed it out right up front. 207 00:12:14,940 --> 00:12:17,880 But I'm glad I got it out of you. 208 00:12:17,880 --> 00:12:22,160 W doesn't exist in the complexity. 209 00:12:22,160 --> 00:12:23,650 This is pretty important. 210 00:12:23,650 --> 00:12:26,410 W could be a large number. 211 00:12:26,410 --> 00:12:29,030 I mean, it could be 2 raised to 64. 212 00:12:29,030 --> 00:12:34,640 The fact is that there's only E square different values 213 00:12:34,640 --> 00:12:37,170 possible for a weight, right. 214 00:12:37,170 --> 00:12:38,730 I mean, roughly speaking. 215 00:12:38,730 --> 00:12:41,390 If you have a complete graph, it's a simple graph, 216 00:12:41,390 --> 00:12:45,160 there's order E square possible weights. 217 00:12:45,160 --> 00:12:49,830 But the range of the weights could be exponential. 218 00:12:49,830 --> 00:12:54,480 I could have an edge weight of 0.0001 219 00:12:54,480 --> 00:12:58,090 and a different edge weight of 10 raised to 98. 220 00:12:58,090 --> 00:13:01,190 There's nothing that's stopping me 221 00:13:01,190 --> 00:13:04,040 from doing that or putting a specification like that. 222 00:13:04,040 --> 00:13:06,830 But the nice thing about Dijkstra, and Bellman-Ford, 223 00:13:06,830 --> 00:13:09,030 and virtually all of the algorithms 224 00:13:09,030 --> 00:13:10,960 that are useful in practice is that they 225 00:13:10,960 --> 00:13:15,450 don't depend on the dynamic range of the weights. 226 00:13:15,450 --> 00:13:17,720 And so keep that in mind as you think 227 00:13:17,720 --> 00:13:19,622 of shortest path algorithms. 228 00:13:19,622 --> 00:13:22,080 And we'll talk a little bit about this in section tomorrow, 229 00:13:22,080 --> 00:13:26,390 or the TAs will, as to why breadth first search and depth 230 00:13:26,390 --> 00:13:30,380 first search aren't directly applicable to the shortest path 231 00:13:30,380 --> 00:13:31,610 problem. 232 00:13:31,610 --> 00:13:36,480 And the hint really is the dynamic range of the weights. 233 00:13:36,480 --> 00:13:37,770 So keep that in mind. 234 00:13:37,770 --> 00:13:40,970 So a couple things why this is an interesting algorithm, 235 00:13:40,970 --> 00:13:44,310 or interesting problem to solve, and harder than the problems 236 00:13:44,310 --> 00:13:47,070 we've looked at so far like sorting and search, 237 00:13:47,070 --> 00:13:49,760 is that you have an exponential number of paths. 238 00:13:49,760 --> 00:13:51,540 And then the dynamic range of the weights 239 00:13:51,540 --> 00:13:53,700 can be very, very large. 240 00:13:53,700 --> 00:13:55,540 And it's not linear by any means. 241 00:13:55,540 --> 00:13:56,040 All right. 242 00:13:56,040 --> 00:13:59,020 So these algorithms are going to have to have some smarts. 243 00:13:59,020 --> 00:14:00,810 And the optimal substructure property 244 00:14:00,810 --> 00:14:03,670 that we'll look at towards the end of today's lecture 245 00:14:03,670 --> 00:14:06,070 will give you a sense of how these algorithms actually 246 00:14:06,070 --> 00:14:08,980 work in basically linear time. 247 00:14:08,980 --> 00:14:12,310 Or VE, you could think of that as being 248 00:14:12,310 --> 00:14:17,330 cubic time in terms of the vertices. 249 00:14:17,330 --> 00:14:19,530 So keep that in mind. 250 00:14:19,530 --> 00:14:23,470 Let's talk a little bit more about weighted graphs. 251 00:14:23,470 --> 00:14:24,990 I want a little bit more notation. 252 00:14:31,940 --> 00:14:38,170 And what I have is V0 using path p to Vk. 253 00:14:38,170 --> 00:14:40,470 So I'm going to write that to say that there's 254 00:14:40,470 --> 00:14:43,810 a particular path of V0 to Vk. 255 00:14:43,810 --> 00:14:48,900 Sometimes I'm searching for the path p with a minimum weight. 256 00:14:48,900 --> 00:14:51,190 And that's how I'm going to represent that. 257 00:14:51,190 --> 00:14:56,120 V0, which is a single vertex path, 258 00:14:56,120 --> 00:15:00,580 is the path from V0 to V0. 259 00:15:00,580 --> 00:15:03,680 So it's really a 0 length path. 260 00:15:03,680 --> 00:15:07,630 And it has weight 0. 261 00:15:07,630 --> 00:15:09,410 So that's one condition. 262 00:15:09,410 --> 00:15:12,670 The other condition that we need to look at, 263 00:15:12,670 --> 00:15:18,120 which is the other case, is what if there isn't a path? 264 00:15:18,120 --> 00:15:20,820 So I want to put those two things together, 265 00:15:20,820 --> 00:15:23,610 the two extremes, and of course all of the cases 266 00:15:23,610 --> 00:15:29,810 in between, in this definition of the shortest pathway. 267 00:15:29,810 --> 00:15:33,990 And so I'm going to talk about the shortest path 268 00:15:33,990 --> 00:15:38,990 value of the weight of the shortest path between u and v 269 00:15:38,990 --> 00:15:43,610 as delta, u, v. And my goal is to find delta. 270 00:15:43,610 --> 00:15:45,560 It's also to find the path. 271 00:15:45,560 --> 00:15:48,890 It doesn't help you very much if you 272 00:15:48,890 --> 00:15:53,270 know that there's a way of getting from here to Lexington 273 00:15:53,270 --> 00:15:56,830 within 14 miles if you don't know what that path is, right. 274 00:15:56,830 --> 00:15:59,690 So that's one aspect of it, which 275 00:15:59,690 --> 00:16:01,110 is you want to get the weight. 276 00:16:01,110 --> 00:16:02,970 But you want to get the path as well. 277 00:16:02,970 --> 00:16:06,230 And these algorithms will do that for you. 278 00:16:06,230 --> 00:16:10,150 And in particular, what we want is 279 00:16:10,150 --> 00:16:24,100 delta u, v to be the minimum over all the paths W p, 280 00:16:24,100 --> 00:16:30,730 such that p is in fact the path from u to v. 281 00:16:30,730 --> 00:16:40,490 And this is the case where if there exists any such path, 282 00:16:40,490 --> 00:16:44,920 and the last thing is you want this to be infinity, 283 00:16:44,920 --> 00:16:47,185 the weight to be infinity otherwise. 284 00:16:51,290 --> 00:16:56,490 So if you're only talking about roads going from here to Tokyo, 285 00:16:56,490 --> 00:16:59,540 should have length infinity. 286 00:16:59,540 --> 00:17:04,099 A little matter of the Pacific Ocean in between. 287 00:17:04,099 --> 00:17:08,970 So that's the set up in terms of the numbers 288 00:17:08,970 --> 00:17:11,040 that we want to see. 289 00:17:11,040 --> 00:17:13,800 If you're starting from a particular point, 290 00:17:13,800 --> 00:17:18,720 you can think of the shortest path length from your source 291 00:17:18,720 --> 00:17:20,240 as being a 0. 292 00:17:20,240 --> 00:17:22,150 Initially, everything is infinity 293 00:17:22,150 --> 00:17:24,790 because you haven't found any paths yet. 294 00:17:24,790 --> 00:17:28,450 And what you're going to do is try and reduce these infinities 295 00:17:28,450 --> 00:17:31,970 down for all of the vertices that 296 00:17:31,970 --> 00:17:34,040 are reachable from the source vertex. 297 00:17:34,040 --> 00:17:37,580 And it's quite possible that you may be given a graph where 298 00:17:37,580 --> 00:17:40,740 a particular vertices, or in your set 299 00:17:40,740 --> 00:17:44,030 V, that can't be reached from the particular source 300 00:17:44,030 --> 00:17:45,290 that you started with. 301 00:17:45,290 --> 00:17:50,650 And for those vertices, you're going to have your delta u, 302 00:17:50,650 --> 00:17:56,650 v. If v is unreachable from you, it will stay at infinity. 303 00:17:56,650 --> 00:17:58,390 So let's look at an example. 304 00:18:03,710 --> 00:18:12,980 Let's take-- it's going to be an iterative process here 305 00:18:12,980 --> 00:18:18,150 of finding the shortest paths. 306 00:18:18,150 --> 00:18:22,100 And so let's take an example that 307 00:18:22,100 --> 00:18:27,770 corresponds to a fairly complex graph, 308 00:18:27,770 --> 00:18:31,330 or at least a nontrivial one, where that's my source vertex. 309 00:18:31,330 --> 00:18:35,420 And I've labeled these other vertices A through F. 310 00:18:35,420 --> 00:18:36,725 And I have a bunch of edges. 311 00:18:44,780 --> 00:18:45,280 5. 312 00:19:06,130 --> 00:19:07,140 I got one more here. 313 00:19:10,230 --> 00:19:11,640 So that's what's given to me. 314 00:19:11,640 --> 00:19:18,720 And I want to find delta S plugged in for u. 315 00:19:18,720 --> 00:19:25,390 And A, B, D, et cetera plugged in for V for this graph. 316 00:19:25,390 --> 00:19:30,850 And let's just do this manually if you will. 317 00:19:30,850 --> 00:19:35,111 And just trying to do some kind of breadth first search. 318 00:19:35,111 --> 00:19:36,610 And we do know breadth first search. 319 00:19:36,610 --> 00:19:38,060 We know depth first search. 320 00:19:38,060 --> 00:19:45,580 You can imagine trying to use those notions to try and find 321 00:19:45,580 --> 00:19:47,020 the shortest paths here. 322 00:19:47,020 --> 00:19:48,829 So now we have to prove afterwards 323 00:19:48,829 --> 00:19:51,370 when we are done that these are, in fact, the shortest paths. 324 00:19:51,370 --> 00:19:52,720 And that's the hard part of it. 325 00:19:52,720 --> 00:19:56,860 But we can certainly try and fill in some numbers associated 326 00:19:56,860 --> 00:19:58,620 with paths that we do know about. 327 00:19:58,620 --> 00:20:02,635 So I'm going to say that the numbers that 328 00:20:02,635 --> 00:20:08,210 are inside each of these vertices-- d of u 329 00:20:08,210 --> 00:20:11,365 is the current weight. 330 00:20:14,090 --> 00:20:17,890 And so initially, I'm going to start with D of S being 0, 331 00:20:17,890 --> 00:20:19,699 because that's a source. 332 00:20:19,699 --> 00:20:21,240 And all of these other ones are going 333 00:20:21,240 --> 00:20:23,073 to have-- I'm not going to write this down-- 334 00:20:23,073 --> 00:20:26,940 but they're going to have infinity for their D of Vs. 335 00:20:26,940 --> 00:20:28,350 So D of A is infinity. 336 00:20:28,350 --> 00:20:31,210 Do of B is infinity, et cetera. 337 00:20:31,210 --> 00:20:34,500 And what I want to do is decrease this D number 338 00:20:34,500 --> 00:20:38,080 to the point where I'm confident that all of the D numbers 339 00:20:38,080 --> 00:20:41,230 that are inside these vertices, these are the current weights, 340 00:20:41,230 --> 00:20:44,310 or end up being the delta numbers. 341 00:20:44,310 --> 00:20:48,520 So my algorithm is done when my d numbers shrink down. 342 00:20:48,520 --> 00:20:52,810 And I got the delta values, the correct delta values. 343 00:20:52,810 --> 00:20:57,500 But if I wanted to do this, sort of a seat of the pants 344 00:20:57,500 --> 00:21:00,920 approach, just go off and try and iteratively 345 00:21:00,920 --> 00:21:03,860 reduce these numbers, you say, well, this one was infinity. 346 00:21:03,860 --> 00:21:08,690 But clearly, if I start from S and I follow the edges in S, 347 00:21:08,690 --> 00:21:11,980 I'm going to be able to mark this as a one. 348 00:21:11,980 --> 00:21:15,630 And similarly here, I'm going to be able to mark this as a 2. 349 00:21:15,630 --> 00:21:19,860 Now, I could arbitrarily pick this one 350 00:21:19,860 --> 00:21:23,870 here and this A vertex, and then start 351 00:21:23,870 --> 00:21:28,090 looking at the edges that emanate from the A vertex. 352 00:21:28,090 --> 00:21:36,470 And I could go off and mark this as 6, for example. 353 00:21:36,470 --> 00:21:42,380 And if I start from here, I'd mark this as 3. 354 00:21:42,380 --> 00:21:48,570 Now, is it in fact true that 6 equals 355 00:21:48,570 --> 00:21:54,670 delta S comma C equals 6? 356 00:21:54,670 --> 00:21:56,100 No. 357 00:21:56,100 --> 00:22:00,550 What is in fact-- is there a better way of getting to C? 358 00:22:00,550 --> 00:22:02,780 And what is the weight of that? 359 00:22:02,780 --> 00:22:06,594 What vertex do I have to go through? 360 00:22:06,594 --> 00:22:19,180 I mean, one way is to go from S to B to D to C, right? 361 00:22:19,180 --> 00:22:21,520 And that would give me 5 right? 362 00:22:21,520 --> 00:22:23,280 So that's 5. 363 00:22:23,280 --> 00:22:26,530 Can I do better than 5? 364 00:22:26,530 --> 00:22:27,770 Not in this graph. 365 00:22:27,770 --> 00:22:28,930 OK 366 00:22:28,930 --> 00:22:36,680 So it's not the case that the shortest length path gave you 367 00:22:36,680 --> 00:22:40,980 the smallest weight. 368 00:22:40,980 --> 00:22:42,600 I mean, that was one example of that. 369 00:22:42,600 --> 00:22:44,600 And I can go on an and bore you with filling 370 00:22:44,600 --> 00:22:45,930 in all of these numbers. 371 00:22:45,930 --> 00:22:47,980 But you can do that on your own. 372 00:22:47,980 --> 00:22:53,020 And it's really not particularly edifying to do that. 373 00:22:53,020 --> 00:22:55,660 But you get a sense of what you need 374 00:22:55,660 --> 00:22:59,430 to be able to do in order to converge on the delta. 375 00:22:59,430 --> 00:23:01,600 And it might take some doing. 376 00:23:01,600 --> 00:23:06,002 Because you have to somehow enumerate in an implicit way-- 377 00:23:06,002 --> 00:23:07,460 you can't do it in an explicit way, 378 00:23:07,460 --> 00:23:10,370 because then there'd be an exponential number of paths. 379 00:23:10,370 --> 00:23:12,670 But you'd have to implicitly enumerate 380 00:23:12,670 --> 00:23:16,480 all the different ways that you can possibly get to a vertex 381 00:23:16,480 --> 00:23:21,790 and discover the shortest path through that process, 382 00:23:21,790 --> 00:23:22,490 all right. 383 00:23:22,490 --> 00:23:25,080 And so we have to be able to do that in these shortest path 384 00:23:25,080 --> 00:23:26,510 algorithms. 385 00:23:26,510 --> 00:23:31,290 And this is a simple graph that has positive weights, 386 00:23:31,290 --> 00:23:34,160 non-negative weights with edges. 387 00:23:34,160 --> 00:23:38,599 It gets more complicated when you have negative weights. 388 00:23:38,599 --> 00:23:40,640 But before I get to that, there's one other thing 389 00:23:40,640 --> 00:23:43,490 that I want to talk about here with respect 390 00:23:43,490 --> 00:23:46,900 to discovering the actual path. 391 00:23:46,900 --> 00:23:49,935 So what we did here was we had delta u, 392 00:23:49,935 --> 00:23:55,890 v that corresponded to the weight of the shortest path. 393 00:23:55,890 --> 00:24:00,080 But if you want the path itself, we 394 00:24:00,080 --> 00:24:04,090 need to have a way of finding the sequence of vertices that 395 00:24:04,090 --> 00:24:08,310 corresponds to the minimum weight path. 396 00:24:08,310 --> 00:24:10,240 And in particular, we're going to have 397 00:24:10,240 --> 00:24:16,290 to define what we call the predecessor relationship. 398 00:24:16,290 --> 00:24:25,270 And so what I have is d of V is the value 399 00:24:25,270 --> 00:24:31,230 inside the circle, which is the current weight. 400 00:24:34,890 --> 00:24:38,400 And as d is something you're very interested in, 401 00:24:38,400 --> 00:24:40,270 eventually you want it to go to delta. 402 00:24:40,270 --> 00:24:42,330 The other thing that you're very interested in-- 403 00:24:42,330 --> 00:24:45,950 and this is really a fairly straightforward data structure 404 00:24:45,950 --> 00:24:50,050 corresponding to just the d number and this predecessor 405 00:24:50,050 --> 00:24:50,910 number. 406 00:24:50,910 --> 00:25:06,820 And pi of V is the predecessor vertex on the best path to V. 407 00:25:06,820 --> 00:25:10,070 And you said, pi of S equals NIL. 408 00:25:13,490 --> 00:25:15,950 And then you can think of this as this is eventually 409 00:25:15,950 --> 00:25:20,600 what we want, and this gets modified as well. 410 00:25:20,600 --> 00:25:24,720 So right now, when you're working and trying 411 00:25:24,720 --> 00:25:27,320 to find the path, you have some particular path 412 00:25:27,320 --> 00:25:29,500 that happens to be the current best path. 413 00:25:29,500 --> 00:25:31,930 And that's a sequence of vertices 414 00:25:31,930 --> 00:25:34,550 that you can get by following the predecessors. 415 00:25:34,550 --> 00:25:40,250 So once you're at a particular vertex E, you say all right, 416 00:25:40,250 --> 00:25:46,700 right now I can look at pi of E. And if that points me to C, 417 00:25:46,700 --> 00:25:49,190 then that's good. 418 00:25:49,190 --> 00:25:52,240 I'm going to look at pi of C. And that might point me 419 00:25:52,240 --> 00:25:54,320 to A, and so on and so forth. 420 00:25:54,320 --> 00:25:57,560 In this particular instance, pi of E 421 00:25:57,560 --> 00:26:02,470 is going to, when you're finally done, is going to point to A. 422 00:26:02,470 --> 00:26:06,150 And pi of A is going to point to S, all right, because that's 423 00:26:06,150 --> 00:26:10,010 the path that is the best path is this one. 424 00:26:10,010 --> 00:26:12,660 Like so and like that. 425 00:26:12,660 --> 00:26:14,380 And so those are the two data structures 426 00:26:14,380 --> 00:26:18,200 you need to keep in mind that you need to iterate on, 427 00:26:18,200 --> 00:26:24,970 this predecessor relationship and the current distance. 428 00:26:24,970 --> 00:26:29,340 And then this ends up being delta. 429 00:26:29,340 --> 00:26:31,370 You're done. 430 00:26:31,370 --> 00:26:33,401 And at that point, your predecessor relationship 431 00:26:33,401 --> 00:26:33,900 is correct. 432 00:26:37,140 --> 00:26:39,842 So that's the set up. 433 00:26:39,842 --> 00:26:41,800 The last complication I want to talk about here 434 00:26:41,800 --> 00:26:44,284 is negative weights. 435 00:26:44,284 --> 00:26:46,450 And it's, I think, appropriate to talk about it when 436 00:26:46,450 --> 00:26:48,370 we have Bellman-Ford up here. 437 00:26:53,467 --> 00:26:55,050 Which is really the general algorithm. 438 00:27:00,410 --> 00:27:09,190 So let's talk about-- so the first question is why. 439 00:27:09,190 --> 00:27:11,780 Why do these things exist, other than making 440 00:27:11,780 --> 00:27:14,380 our lives more difficult? 441 00:27:14,380 --> 00:27:15,780 So give me an example. 442 00:27:15,780 --> 00:27:18,855 What is the motivation for a graph with negative weights? 443 00:27:22,330 --> 00:27:23,990 I mean, I really would like to know. 444 00:27:23,990 --> 00:27:26,450 The best motivation is definitely worth recursion. 445 00:27:26,450 --> 00:27:28,980 Then I can use it next time. 446 00:27:28,980 --> 00:27:29,863 Yeah, go ahead. 447 00:27:29,863 --> 00:27:35,286 AUDIENCE: I'm just thinking like if your goal, 448 00:27:35,286 --> 00:27:37,258 if your goal [INAUDIBLE]. 449 00:27:40,709 --> 00:27:42,188 And some of them cost too much. 450 00:27:42,188 --> 00:27:45,392 Some of them get you money, and you 451 00:27:45,392 --> 00:27:49,090 want to know what-- you're trying to find [INAUDIBLE]. 452 00:27:55,499 --> 00:27:56,485 PROFESSOR: Sure. 453 00:27:56,485 --> 00:27:58,465 Yeah, I mean, I think that's a good motivation. 454 00:28:02,750 --> 00:28:04,980 I think driving, when you think about distances 455 00:28:04,980 --> 00:28:07,530 and so on, there's no notion of a negative distance, 456 00:28:07,530 --> 00:28:09,820 at least physically. 457 00:28:09,820 --> 00:28:13,652 But you can imagine that you could have a case where you're 458 00:28:13,652 --> 00:28:15,110 getting paid to drive or something, 459 00:28:15,110 --> 00:28:18,325 or it costs you to drive, and that would be one. 460 00:28:18,325 --> 00:28:18,950 Yeah, go ahead. 461 00:28:18,950 --> 00:28:20,366 AUDIENCE: It sounds like Monopoly. 462 00:28:20,366 --> 00:28:24,014 So the vertices are supposed to be [INAUDIBLE]. 463 00:28:24,014 --> 00:28:25,680 PROFESSOR: Oh, if you land on something, 464 00:28:25,680 --> 00:28:26,570 you have to pay rent. 465 00:28:26,570 --> 00:28:28,060 Or sometimes you land on something 466 00:28:28,060 --> 00:28:29,010 and you actually get money. 467 00:28:29,010 --> 00:28:29,926 AUDIENCE: [INAUDIBLE]. 468 00:28:32,792 --> 00:28:34,750 PROFESSOR: Takes you forward, backwards, right. 469 00:28:34,750 --> 00:28:35,458 Yeah go ahead. 470 00:28:35,458 --> 00:28:36,374 AUDIENCE: [INAUDIBLE]. 471 00:28:43,710 --> 00:28:46,670 PROFESSOR: So that is such an interesting notion. 472 00:28:46,670 --> 00:28:49,230 Sometimes you may want to go. 473 00:28:49,230 --> 00:28:52,080 And maybe in this case, you're saying 474 00:28:52,080 --> 00:28:55,490 it's better to take your distance metric 475 00:28:55,490 --> 00:28:59,940 and go further away in order to get the best way of getting 476 00:28:59,940 --> 00:29:01,190 there, or something like that. 477 00:29:01,190 --> 00:29:02,106 AUDIENCE: [INAUDIBLE]. 478 00:29:08,380 --> 00:29:09,160 PROFESSOR: Right. 479 00:29:09,160 --> 00:29:09,659 Sure. 480 00:29:09,659 --> 00:29:10,290 That'd be good. 481 00:29:10,290 --> 00:29:10,820 Right. 482 00:29:10,820 --> 00:29:12,103 Victor, you had your hand up. 483 00:29:12,103 --> 00:29:12,728 AUDIENCE: Yeah. 484 00:29:12,728 --> 00:29:15,223 I'm going to give [INAUDIBLE] on the highway, 485 00:29:15,223 --> 00:29:18,217 you can't [INAUDIBLE] distances [INAUDIBLE] negative. 486 00:29:18,217 --> 00:29:22,209 Well, if a government uses [INAUDIBLE] police 487 00:29:22,209 --> 00:29:26,201 to regulate traffic, then you might have a negative distance. 488 00:29:26,201 --> 00:29:28,696 Because obviously, you could go a certain way 489 00:29:28,696 --> 00:29:31,582 minus the [INAUDIBLE]. 490 00:29:31,582 --> 00:29:32,290 PROFESSOR: Right. 491 00:29:32,290 --> 00:29:35,120 Yeah, that's a good example. 492 00:29:35,120 --> 00:29:37,050 One of the things that we have to think about 493 00:29:37,050 --> 00:29:39,310 is-- and this is something that might come up, 494 00:29:39,310 --> 00:29:43,620 by the way, in a problem set or a quiz-- which is, is there 495 00:29:43,620 --> 00:29:48,540 a way of shifting these weights to make them all positive? 496 00:29:48,540 --> 00:29:52,250 So the examples we've talked about, not clear to me 497 00:29:52,250 --> 00:29:56,100 that in the particular settings that we talked about 498 00:29:56,100 --> 00:29:59,130 that you can somehow create the base 499 00:29:59,130 --> 00:30:03,120 case to be 0 rather than being negative. 500 00:30:03,120 --> 00:30:07,370 So it may not be possible in a particular scenario. 501 00:30:07,370 --> 00:30:09,630 But if you can do that-- and the reason 502 00:30:09,630 --> 00:30:12,200 I bring this up is if you can do that, suddenly 503 00:30:12,200 --> 00:30:14,760 instead of using an order V, E algorithm, 504 00:30:14,760 --> 00:30:18,120 if you can prove correctness of the final solution 505 00:30:18,120 --> 00:30:20,630 is exactly what you'd have gotten for the initial problem 506 00:30:20,630 --> 00:30:23,360 certification, you've gone from an order V, E algorithm 507 00:30:23,360 --> 00:30:24,894 to an order V log V algorithm. 508 00:30:24,894 --> 00:30:26,310 So that's a wonderful thing to do. 509 00:30:26,310 --> 00:30:28,160 So keep that in mind. 510 00:30:28,160 --> 00:30:30,720 Try and get rid of negative weight edges if you can 511 00:30:30,720 --> 00:30:32,730 without changing the problem certification. 512 00:30:32,730 --> 00:30:34,080 I saw a hand back there. 513 00:30:34,080 --> 00:30:35,056 AUDIENCE: Oh, no. 514 00:30:35,056 --> 00:30:37,008 I thought you were just asking the question, 515 00:30:37,008 --> 00:30:37,984 if we could do that? 516 00:30:37,984 --> 00:30:41,400 So I was just gettin ready to answer. 517 00:30:41,400 --> 00:30:47,540 PROFESSOR: OK, yeah, so that's something to keep in mind. 518 00:30:47,540 --> 00:30:49,956 One example that I think has come up here, which came up, 519 00:30:49,956 --> 00:30:51,330 I think, the last time I lectured 520 00:30:51,330 --> 00:30:54,620 was imagine that you're driving and there 521 00:30:54,620 --> 00:30:56,240 are all these advertisements. 522 00:30:56,240 --> 00:30:58,750 And you get paid to drive on a freeway. 523 00:30:58,750 --> 00:31:00,197 So the reverse toll. 524 00:31:00,197 --> 00:31:02,530 I mean, it's a reverse toll, because you get to go there 525 00:31:02,530 --> 00:31:04,940 and you have to see all these ads. 526 00:31:04,940 --> 00:31:09,200 And then I guess you drive pretty fast through those ads. 527 00:31:09,200 --> 00:31:11,380 But you have to go through. 528 00:31:11,380 --> 00:31:14,690 And so you get paid to go through those particular roads. 529 00:31:14,690 --> 00:31:16,810 And then what about social networks? 530 00:31:16,810 --> 00:31:19,550 I mean, there's liking people and disliking people. 531 00:31:19,550 --> 00:31:22,037 I mean, that sounds pretty-- that's negative and positive. 532 00:31:22,037 --> 00:31:23,870 One could imagine that social networks would 533 00:31:23,870 --> 00:31:26,190 have positive weights and negative weights. 534 00:31:26,190 --> 00:31:27,750 I'm surprised one of you-- I mean, 535 00:31:27,750 --> 00:31:29,250 I don't have an account on Facebook. 536 00:31:29,250 --> 00:31:32,240 But presumably, you guys do. 537 00:31:32,240 --> 00:31:36,154 So think of what's the-- yeah, that's right. 538 00:31:36,154 --> 00:31:37,570 Well, I'm not sure how this works. 539 00:31:37,570 --> 00:31:40,410 But you guys figure it out. 540 00:31:40,410 --> 00:31:41,240 So why? 541 00:31:41,240 --> 00:31:43,060 Reverse tolls, social networks. 542 00:31:47,080 --> 00:31:47,770 Lots of things. 543 00:31:51,040 --> 00:31:53,140 Even if you're not convinced by the motivation, 544 00:31:53,140 --> 00:31:56,589 I will spend a whole lecture talking about Bellman-Ford. 545 00:31:56,589 --> 00:31:57,880 So that's just so that's clear. 546 00:32:01,020 --> 00:32:04,470 So the issue with the negative weight cycles 547 00:32:04,470 --> 00:32:07,780 is something that is worth spending a minute on. 548 00:32:07,780 --> 00:32:09,780 And I talked about the fact that you 549 00:32:09,780 --> 00:32:13,100 had an exponential number of paths. 550 00:32:13,100 --> 00:32:15,820 And that causes a bit of a problem, 551 00:32:15,820 --> 00:32:18,610 even in the case where you have positive weights. 552 00:32:18,610 --> 00:32:20,420 And I will revisit that example. 553 00:32:20,420 --> 00:32:22,970 But here's an even worse problem that 554 00:32:22,970 --> 00:32:26,955 corresponds to negative cycles. 555 00:32:29,449 --> 00:32:30,990 So eventually, you want to terminate. 556 00:32:30,990 --> 00:32:32,860 The faster you terminate, and if you 557 00:32:32,860 --> 00:32:35,850 can talk about asymptotic complexity, 558 00:32:35,850 --> 00:32:38,100 obviously that means that you've terminated 559 00:32:38,100 --> 00:32:41,160 within a worst case bound of time. 560 00:32:41,160 --> 00:32:43,680 And if that's exponential, that's bad. 561 00:32:43,680 --> 00:32:45,040 You'd want it to be small. 562 00:32:45,040 --> 00:32:47,310 But what if you didn't even terminate? 563 00:32:47,310 --> 00:32:52,610 So suppose you have something like this 564 00:32:52,610 --> 00:33:02,400 where you have a graph that has negative weights on some 565 00:33:02,400 --> 00:33:04,140 of the edges. 566 00:33:04,140 --> 00:33:05,840 But others are positive. 567 00:33:05,840 --> 00:33:07,025 So this one has a minus 6. 568 00:33:17,380 --> 00:33:20,060 I think I got those right. 569 00:33:20,060 --> 00:33:26,520 So 2 for minus 6 over here, 3, 2, 1, and minus 2. 570 00:33:26,520 --> 00:33:28,500 So one thing that you notice from this graph 571 00:33:28,500 --> 00:33:33,700 is that you got this annoying cycle here. 572 00:33:33,700 --> 00:33:35,710 That's a negative weight cycle. 573 00:33:35,710 --> 00:33:38,130 And that's why I've picked this particular example. 574 00:33:38,130 --> 00:33:39,860 Minus 6 plus 2 is minus 4. 575 00:33:39,860 --> 00:33:41,910 Minus 4 plus 3 is minus 1. 576 00:33:41,910 --> 00:33:45,670 So if you had something where you can't depend on the fact 577 00:33:45,670 --> 00:33:50,870 that the D's are going to keep reducing. 578 00:33:50,870 --> 00:33:53,120 And that eventually, they'll stop reducing. 579 00:33:53,120 --> 00:33:53,870 Well, that's true. 580 00:33:53,870 --> 00:33:55,286 Eventually, they'll stop reducing. 581 00:33:55,286 --> 00:33:57,350 Because they're lower bounded by 0 582 00:33:57,350 --> 00:34:00,470 when you have positive weight edges or non-negative weight 583 00:34:00,470 --> 00:34:01,290 edges. 584 00:34:01,290 --> 00:34:05,940 But if you have a graph with a negative cycle-- and I mean, 585 00:34:05,940 --> 00:34:08,969 this is a recipe for an infinite loop, right-- 586 00:34:08,969 --> 00:34:10,980 in your program, potentially a bug. 587 00:34:10,980 --> 00:34:14,380 But maybe not even a bug, not a bug in implementation, 588 00:34:14,380 --> 00:34:15,750 but a bug in the algorithm. 589 00:34:15,750 --> 00:34:19,415 Because this determination condition isn't set properly. 590 00:34:19,415 --> 00:34:20,790 So you can imagine that you would 591 00:34:20,790 --> 00:34:24,100 get to B and the first time-- whoops, 592 00:34:24,100 --> 00:34:26,860 I'm missing a weight here. 593 00:34:26,860 --> 00:34:32,909 So you get to B. And you say, well, I'm done. 594 00:34:32,909 --> 00:34:36,739 Delta of SB is 4. 595 00:34:36,739 --> 00:34:37,576 But that's not true. 596 00:34:37,576 --> 00:34:39,659 Because you could get to B. And then you could get 597 00:34:39,659 --> 00:34:41,854 back to B with the weight of 3. 598 00:34:41,854 --> 00:34:44,020 And then you could do it again with the weight of 2, 599 00:34:44,020 --> 00:34:45,540 and so on and so forth. 600 00:34:45,540 --> 00:34:46,989 So that's a problem. 601 00:34:46,989 --> 00:34:49,050 So what would you like an algorithm to do? 602 00:34:49,050 --> 00:34:52,719 What would you like the Bellman-Ford to do here? 603 00:34:52,719 --> 00:34:56,855 It's not the case that all of the delta values, 604 00:34:56,855 --> 00:34:58,230 that is the shortest path values, 605 00:34:58,230 --> 00:35:00,210 are undefined for this graph. 606 00:35:00,210 --> 00:35:03,410 Some of them are well defined. 607 00:35:03,410 --> 00:35:05,440 This one, you can't ever get back to it. 608 00:35:05,440 --> 00:35:10,040 So clearly, delta S, S is 0. 609 00:35:10,040 --> 00:35:12,150 Everybody buy that? 610 00:35:12,150 --> 00:35:14,610 What about this one? 611 00:35:14,610 --> 00:35:16,640 It's 2, right. 612 00:35:16,640 --> 00:35:18,600 Delta S, A is 2. 613 00:35:21,132 --> 00:35:23,340 And everybody buys that, because there's just no way. 614 00:35:23,340 --> 00:35:25,560 You can't, you don't touch a negative weight cycle. 615 00:35:25,560 --> 00:35:27,750 You, in fact, don't touch a negative weight edge. 616 00:35:27,750 --> 00:35:31,400 But more importantly, you don't touch a negative weight cycle 617 00:35:31,400 --> 00:35:32,830 in order to get to A. And there's 618 00:35:32,830 --> 00:35:34,960 no way of touching that. 619 00:35:34,960 --> 00:35:38,970 On the other hand, anything that's in here 620 00:35:38,970 --> 00:35:40,870 you could run many times. 621 00:35:40,870 --> 00:35:44,430 And you could end up with whatever weight you wanted. 622 00:35:44,430 --> 00:35:46,940 There'd be a minus infinity weight. 623 00:35:46,940 --> 00:35:48,680 So what you want an algorithm that 624 00:35:48,680 --> 00:35:51,770 handles in particular negative cycles, 625 00:35:51,770 --> 00:35:53,400 which are the hard part here. 626 00:35:53,400 --> 00:35:55,590 Negative weights aren't the hard part 627 00:35:55,590 --> 00:35:58,960 if you can't run through these edges more than once. 628 00:35:58,960 --> 00:36:01,450 It's actually the negative cycles that are hard. 629 00:36:01,450 --> 00:36:03,500 And the negative cycles are going 630 00:36:03,500 --> 00:36:08,330 to make shortest path lengths indeterminate, but not 631 00:36:08,330 --> 00:36:12,940 necessarily for every node in the graph, 632 00:36:12,940 --> 00:36:14,800 like this example shows. 633 00:36:14,800 --> 00:36:17,600 So what you want your Bellman-Ford algorithm to do, 634 00:36:17,600 --> 00:36:20,730 or your shortest path algorithm that 635 00:36:20,730 --> 00:36:23,390 handles negative cycles to do, is 636 00:36:23,390 --> 00:36:25,810 to finish in reasonable amounts of time. 637 00:36:25,810 --> 00:36:28,930 Order V, E will take and give you 638 00:36:28,930 --> 00:36:33,540 the delta numbers for all of the vertices 639 00:36:33,540 --> 00:36:37,290 that actually have finite numbers 640 00:36:37,290 --> 00:36:39,150 and then mark all of these other vertices 641 00:36:39,150 --> 00:36:43,140 as being indeterminate, or essentially minus infinity. 642 00:36:43,140 --> 00:36:43,720 OK 643 00:36:43,720 --> 00:36:45,560 So that's your termination condition. 644 00:36:45,560 --> 00:36:47,770 It's different from the termination condition 645 00:36:47,770 --> 00:36:50,920 if you simply had non-negative edge weights. 646 00:36:50,920 --> 00:36:51,890 All right. 647 00:36:51,890 --> 00:36:55,750 So remember, it's cycles that cause a problem, not just 648 00:36:55,750 --> 00:36:57,230 the edges. 649 00:36:57,230 --> 00:37:00,250 And you have to do something about the cycles. 650 00:37:00,250 --> 00:37:03,210 But they may not affect the entire part of the computation. 651 00:37:03,210 --> 00:37:06,890 So if you don't know that you have a cycle or not, 652 00:37:06,890 --> 00:37:12,180 then you end up with having to use Bellman-Ford. 653 00:37:12,180 --> 00:37:14,750 And so that also tells you something which is interesting, 654 00:37:14,750 --> 00:37:18,400 which is Bellman-Ford has to detect negative cycles. 655 00:37:18,400 --> 00:37:20,880 If Bellman-Ford couldn't detect negative cycles, 656 00:37:20,880 --> 00:37:23,310 then how could it possibly be a correct algorithm 657 00:37:23,310 --> 00:37:25,090 for the arbitrary case? 658 00:37:25,090 --> 00:37:27,380 So Dijkstra doesn't have to do that. 659 00:37:27,380 --> 00:37:29,450 And that's why Dijkstra is simpler. 660 00:37:29,450 --> 00:37:30,720 All right. 661 00:37:30,720 --> 00:37:36,820 So let me talk about the general structure of shortest path 662 00:37:36,820 --> 00:37:37,600 algorithms. 663 00:37:37,600 --> 00:37:42,750 And the 2 important notions that I want to talk about here 664 00:37:42,750 --> 00:37:49,030 are the notion of relaxation, which we sort of did already 665 00:37:49,030 --> 00:37:51,590 when we ran through this example. 666 00:37:51,590 --> 00:37:53,460 But I need to formalize that. 667 00:37:53,460 --> 00:37:59,465 And then we'll go back and revisit this exponential graph 668 00:37:59,465 --> 00:37:59,965 example. 669 00:38:03,980 --> 00:38:06,650 So the general structural of shortest path algorithms 670 00:38:06,650 --> 00:38:09,260 are as follows. 671 00:38:09,260 --> 00:38:16,470 We're going to initialize for all u belonging to the vertex 672 00:38:16,470 --> 00:38:20,380 set, we set d v to be infinity. 673 00:38:20,380 --> 00:38:26,260 And we set the predecessor to be NIL. 674 00:38:26,260 --> 00:38:31,650 And then we'll set d of S to be 0. 675 00:38:31,650 --> 00:38:33,400 We're talking about a single source, here. 676 00:38:33,400 --> 00:38:35,800 We'll set that to be 0. 677 00:38:35,800 --> 00:38:39,740 And what we're going to do is essentially repeat. 678 00:38:45,110 --> 00:38:54,240 Select some edge u comma v. And I'm not specifying how. 679 00:38:54,240 --> 00:38:56,560 This is going to result in a different algorithm 680 00:38:56,560 --> 00:38:58,640 depending on the specifics of how. 681 00:38:58,640 --> 00:39:07,430 But the important notion is that we're going to relax edge u, 682 00:39:07,430 --> 00:39:11,980 v. And what the notion of relaxation is 683 00:39:11,980 --> 00:39:14,140 is that you're going to look at it. 684 00:39:14,140 --> 00:39:21,880 And you'll say, well, if d of v is greater than d of u 685 00:39:21,880 --> 00:39:29,000 plus w u, v, then I've discovered a better 686 00:39:29,000 --> 00:39:33,640 way of getting to v then I currently know. 687 00:39:33,640 --> 00:39:36,100 So d of v would currently be infinity, 688 00:39:36,100 --> 00:39:39,020 which means I haven't found a way of getting to v yet. 689 00:39:39,020 --> 00:39:43,110 But I know that d of u, for example, is a finite number. 690 00:39:43,110 --> 00:39:45,840 And I do know that this edge exists from u to v, which 691 00:39:45,840 --> 00:39:48,910 means that I can update the value of d of v. 692 00:39:48,910 --> 00:39:53,550 And that's what we call relaxation of the edge u, v. 693 00:39:53,550 --> 00:39:57,710 And so what you do here is if the if is true, 694 00:39:57,710 --> 00:40:05,770 then you set d, v to be d, u plus w u, v. 695 00:40:05,770 --> 00:40:09,570 And you'll also update the predecessor relationship, 696 00:40:09,570 --> 00:40:16,430 because the current best predecessor for v 697 00:40:16,430 --> 00:40:17,826 is going to be u. 698 00:40:17,826 --> 00:40:20,900 So that's the notion of relaxation. 699 00:40:20,900 --> 00:40:23,880 And I kind of ran out of room here. 700 00:40:23,880 --> 00:40:26,930 But you keep doing this. 701 00:40:26,930 --> 00:40:28,010 This repeat. 702 00:40:28,010 --> 00:40:30,525 So what is the repeat? 703 00:40:30,525 --> 00:40:42,110 Well, the repeat is until all edges 704 00:40:42,110 --> 00:40:56,860 have d of v less than or equal to d of u plus w u, v. 705 00:40:56,860 --> 00:41:01,940 And the assumption here is that you have no negative cycles. 706 00:41:01,940 --> 00:41:05,020 We need a different structure. 707 00:41:05,020 --> 00:41:07,840 The notion of relaxation is still going to be relevant. 708 00:41:07,840 --> 00:41:11,650 But don't think of this structure 709 00:41:11,650 --> 00:41:14,940 as being the structure that Bellman-Ford uses, 710 00:41:14,940 --> 00:41:19,580 or algorithms that can handle negative cycles use. 711 00:41:19,580 --> 00:41:23,370 So hopefully, you got the notion of relaxation, 712 00:41:23,370 --> 00:41:26,580 which is from a pictorial standpoint, 713 00:41:26,580 --> 00:41:29,420 it's simply something that we did 714 00:41:29,420 --> 00:41:35,150 when we looked at updating the value of 6 to 5, for example. 715 00:41:35,150 --> 00:41:39,840 So we said through this process, if I 716 00:41:39,840 --> 00:41:42,970 relax this particular edge and d was already set up-- 717 00:41:42,970 --> 00:41:47,230 let's say d, the vertex here had 3. 718 00:41:47,230 --> 00:41:49,450 And this was originally 6. 719 00:41:49,450 --> 00:41:53,620 And I look at it and I say, D of C is 6. 720 00:41:53,620 --> 00:41:58,530 On other hand, 6 is greater than d 721 00:41:58,530 --> 00:42:03,060 of the vertex D, which happens to be 3 plus 2. 722 00:42:03,060 --> 00:42:06,370 And since 5 is less than 6, I can relax this edge 723 00:42:06,370 --> 00:42:09,610 and update the value of 6 to 5. 724 00:42:09,610 --> 00:42:12,690 And then I update the predecessor relationship 725 00:42:12,690 --> 00:42:18,300 to have a pi of C to be D. That's 726 00:42:18,300 --> 00:42:19,460 the notion of relaxation. 727 00:42:19,460 --> 00:42:20,610 Fundamental notion. 728 00:42:20,610 --> 00:42:24,860 Going to use it in every algorithm that we talk about. 729 00:42:24,860 --> 00:42:26,530 When do you stop? 730 00:42:26,530 --> 00:42:28,480 Well, when you don't have negative cycles, 731 00:42:28,480 --> 00:42:32,740 there's a fairly clean termination condition, 732 00:42:32,740 --> 00:42:35,940 which says that you can't relax any of the edges any more. 733 00:42:35,940 --> 00:42:39,040 OK You get to the point where you 734 00:42:39,040 --> 00:42:41,430 have values that are associated with each 735 00:42:41,430 --> 00:42:43,170 of these vertices inside. 736 00:42:43,170 --> 00:42:45,150 And it doesn't matter what edge you pick, 737 00:42:45,150 --> 00:42:47,580 you can't improve them. 738 00:42:47,580 --> 00:42:50,820 So this termination condition, it 739 00:42:50,820 --> 00:42:54,271 could involve an order E check. 740 00:42:54,271 --> 00:42:55,770 So we're not talking complexity here 741 00:42:55,770 --> 00:42:59,190 yet in terms of being efficient. 742 00:42:59,190 --> 00:43:03,040 But you can imagine when I say until all edges cannot be 743 00:43:03,040 --> 00:43:05,720 relaxed, that you'd have to look at all the edges. 744 00:43:05,720 --> 00:43:07,740 And if any one of them can be relaxed, 745 00:43:07,740 --> 00:43:11,480 it's possible that another one can now be relaxed. 746 00:43:11,480 --> 00:43:13,800 So you've got to keep going until you get to the point 747 00:43:13,800 --> 00:43:17,610 where none of the edges can be relaxed. 748 00:43:17,610 --> 00:43:20,040 So this is a brute force algorithm. 749 00:43:20,040 --> 00:43:21,420 And it'll work. 750 00:43:21,420 --> 00:43:23,720 It'll just be slow. 751 00:43:23,720 --> 00:43:26,110 It'll work for known negative cycles. 752 00:43:26,110 --> 00:43:29,290 And if you just kind of randomly select these edges 753 00:43:29,290 --> 00:43:31,340 and just keep going, I'll give you 754 00:43:31,340 --> 00:43:36,880 an example where it works pretty badly in a minute. 755 00:43:36,880 --> 00:43:38,340 But this is an algorithm. 756 00:43:38,340 --> 00:43:40,030 So I guess I lied when I said we weren't 757 00:43:40,030 --> 00:43:42,270 going to give you an algorithm. 758 00:43:42,270 --> 00:43:43,150 It is an algorithm. 759 00:43:43,150 --> 00:43:47,150 It's just an algorithm that you never want to implement. 760 00:43:47,150 --> 00:43:49,940 You do want to implement the relaxation condition. 761 00:43:49,940 --> 00:43:55,130 But not this random way of selecting edges and having 762 00:43:55,130 --> 00:43:58,470 this termination condition that, in of itself, 763 00:43:58,470 --> 00:44:01,770 is an order E check. 764 00:44:01,770 --> 00:44:03,490 And one of the reasons why you don't 765 00:44:03,490 --> 00:44:06,080 want to implement this algorithm is coming up shortly 766 00:44:06,080 --> 00:44:08,710 in our exponential graph example. 767 00:44:08,710 --> 00:44:12,530 But let me make sure that people aren't bored. 768 00:44:12,530 --> 00:44:15,040 Any questions about the general structure, 769 00:44:15,040 --> 00:44:18,156 relaxation, anything? 770 00:44:18,156 --> 00:44:19,870 Are we good? 771 00:44:19,870 --> 00:44:20,550 OK. 772 00:44:20,550 --> 00:44:24,050 So you guys, I walk away from lecture thinking 773 00:44:24,050 --> 00:44:26,320 I've given this spectacular lecture and everybody 774 00:44:26,320 --> 00:44:27,170 understands. 775 00:44:27,170 --> 00:44:29,320 And then Victor tells me when he shows up 776 00:44:29,320 --> 00:44:32,260 in section in the morning, he says 777 00:44:32,260 --> 00:44:34,050 did people understand graphs? 778 00:44:34,050 --> 00:44:36,130 And everyone says no. 779 00:44:36,130 --> 00:44:37,810 Or did people understand x? 780 00:44:37,810 --> 00:44:38,850 And people say no. 781 00:44:38,850 --> 00:44:43,460 So at least tomorrow, tell Victor that you understood. 782 00:44:43,460 --> 00:44:45,840 Whether you did or not. 783 00:44:45,840 --> 00:44:47,090 So then I feel better. 784 00:44:47,090 --> 00:44:49,774 AUDIENCE: That's going to make my life real easy. 785 00:44:49,774 --> 00:44:51,680 PROFESSOR: Yeah, right. 786 00:44:51,680 --> 00:44:53,030 So good. 787 00:44:53,030 --> 00:44:55,540 Well, you probably like hearing stuff from Victor 788 00:44:55,540 --> 00:44:56,470 better than me anyway. 789 00:44:56,470 --> 00:44:59,100 That's the secret here, right? 790 00:45:02,770 --> 00:45:05,010 All right, so one of the reasons why 791 00:45:05,010 --> 00:45:09,380 you don't want to implement this algorithm 792 00:45:09,380 --> 00:45:12,850 is precisely this example that I put up. 793 00:45:12,850 --> 00:45:16,120 And this is a really neat example that I like a lot, 794 00:45:16,120 --> 00:45:20,540 because it points out two different things. 795 00:45:20,540 --> 00:45:24,670 It points out that exponential number 796 00:45:24,670 --> 00:45:27,840 of paths, an exponential number of paths in a graph, 797 00:45:27,840 --> 00:45:29,864 could cause a problem with this algorithm. 798 00:45:29,864 --> 00:45:31,280 The other thing that it points out 799 00:45:31,280 --> 00:45:35,590 is that we got issues with the weights of edges. 800 00:45:35,590 --> 00:45:39,480 One of the nice observations one of you made earlier on 801 00:45:39,480 --> 00:45:42,560 is that we had these neat algorithms that did not 802 00:45:42,560 --> 00:45:45,760 depend on the dynamic range of the weights. 803 00:45:45,760 --> 00:45:48,480 So let's just say that I in fact had 804 00:45:48,480 --> 00:45:50,970 an exponential range for the weights. 805 00:45:50,970 --> 00:45:55,220 I know 4 isn't exponential, but at some level, 806 00:45:55,220 --> 00:45:57,500 you could imagine that it's exponentially 807 00:45:57,500 --> 00:45:59,980 related to 1 or 2. 808 00:45:59,980 --> 00:46:02,390 And the point here is that if I created 809 00:46:02,390 --> 00:46:13,940 a graph that looked like this, where I have V4, V5, V6, V7, 810 00:46:13,940 --> 00:46:20,390 V8, and it had this structure, then 811 00:46:20,390 --> 00:46:23,270 I'm going to end up having something 812 00:46:23,270 --> 00:46:29,000 like 2 raised to n over 2 weight if I 813 00:46:29,000 --> 00:46:30,700 have n vertices in this graph. 814 00:46:30,700 --> 00:46:33,060 Or at least the dynamic range of these weights 815 00:46:33,060 --> 00:46:35,730 is going to be 2 raised to n divided by 2. 816 00:46:35,730 --> 00:46:36,640 Everybody buy that? 817 00:46:39,720 --> 00:46:43,600 So think of this graph as being a fragment of this large graph, 818 00:46:43,600 --> 00:46:46,380 which where n could be 100 and the weights 819 00:46:46,380 --> 00:46:48,380 could be 2 raised to 50. 820 00:46:48,380 --> 00:46:49,920 And 2 raised to 50 isn't a number 821 00:46:49,920 --> 00:46:51,780 that we can't handle on a computer, right? 822 00:46:51,780 --> 00:46:53,720 It's still less than 64 bits, right? 823 00:46:53,720 --> 00:46:56,250 So it's a pretty reasonable example. 824 00:46:56,250 --> 00:46:59,190 And we talked about multiple precision arithmetic, 825 00:46:59,190 --> 00:47:00,440 infinite precision arithmetic. 826 00:47:00,440 --> 00:47:02,520 So we can handle arbitrary numbers 827 00:47:02,520 --> 00:47:05,030 of an arbitrary position. 828 00:47:05,030 --> 00:47:07,940 So there's nothing that's stopping us 829 00:47:07,940 --> 00:47:11,280 from putting square root of 2 and all sorts of things. 830 00:47:11,280 --> 00:47:12,990 We won't do imaginary numbers. 831 00:47:12,990 --> 00:47:15,790 But you could imagine putting numbers 832 00:47:15,790 --> 00:47:20,140 with a high dynamic range as edges in a particular graph 833 00:47:20,140 --> 00:47:22,870 and expect the Dijkstra, assuming that all of the edges 834 00:47:22,870 --> 00:47:28,090 are non-negative, that Dijkstra should be able to run on it. 835 00:47:28,090 --> 00:47:30,760 So what happens with this example? 836 00:47:30,760 --> 00:47:33,090 Well, with this example, here's what happens. 837 00:47:33,090 --> 00:47:36,340 Let's say that I ran this algorithm. 838 00:47:36,340 --> 00:47:42,210 And initially, I just followed this chain here. 839 00:47:42,210 --> 00:47:45,320 And I get-- this starts with a 0. 840 00:47:45,320 --> 00:47:48,530 And this is a 4, because I get there with 4. 841 00:47:48,530 --> 00:47:50,280 This one is 8. 842 00:47:50,280 --> 00:47:51,650 And this is 10. 843 00:47:51,650 --> 00:47:55,380 And this is 12, 13, 14. 844 00:47:57,930 --> 00:48:00,740 And that's the initial pass. 845 00:48:00,740 --> 00:48:02,990 That's the selection. 846 00:48:02,990 --> 00:48:10,020 What ends up happening is that you could now relax at this-- 847 00:48:10,020 --> 00:48:11,430 you see 14. 848 00:48:11,430 --> 00:48:13,660 And let's say you relax this edge. 849 00:48:13,660 --> 00:48:18,940 You see that 12 and 14, you've turned that into 13. 850 00:48:18,940 --> 00:48:22,500 And then when you relax this edge, this turns into 12. 851 00:48:25,020 --> 00:48:27,640 So you go through that process. 852 00:48:27,640 --> 00:48:30,510 Now, this one stays 12. 853 00:48:30,510 --> 00:48:32,560 But now you relax this edge. 854 00:48:32,560 --> 00:48:35,350 And so this 12 becomes 10. 855 00:48:35,350 --> 00:48:37,310 And then when this changes, you need 856 00:48:37,310 --> 00:48:41,830 to-- if you relax this edge first, 857 00:48:41,830 --> 00:48:45,060 then this 13 is going to become 11. 858 00:48:45,060 --> 00:48:46,440 It doesn't really matter. 859 00:48:46,440 --> 00:48:49,450 This becomes, I guess, 11. 860 00:48:49,450 --> 00:48:51,281 And-- is that right? 861 00:48:51,281 --> 00:48:51,780 Yup. 862 00:48:51,780 --> 00:48:55,690 This is 11 and that's 11 as well. 863 00:48:55,690 --> 00:48:57,440 It might start out being 12 if you 864 00:48:57,440 --> 00:48:59,250 relax this edge and that edge. 865 00:48:59,250 --> 00:49:02,920 So you might go to 12 to 11, and so on and so forth. 866 00:49:02,920 --> 00:49:07,450 So for a pathological ordering, I won't belabor the point. 867 00:49:07,450 --> 00:49:09,700 But you see that you're going 14, 13, 12, 868 00:49:09,700 --> 00:49:14,480 11 with a bad ordering that corresponds 869 00:49:14,480 --> 00:49:16,990 to the selection of the edges. 870 00:49:16,990 --> 00:49:21,266 And so if the overall weight here and overall weight here, 871 00:49:21,266 --> 00:49:22,640 when you start out with, is going 872 00:49:22,640 --> 00:49:25,390 to be order 2 raised to n over 2. 873 00:49:25,390 --> 00:49:29,160 OK And you could be, in this particular graph, 874 00:49:29,160 --> 00:49:32,640 relaxing edges an exponential number of times 875 00:49:32,640 --> 00:49:36,940 in order to finish. 876 00:49:36,940 --> 00:49:39,840 And so the number of times you relax an edge 877 00:49:39,840 --> 00:49:42,440 could be of the order of the weights 878 00:49:42,440 --> 00:49:43,820 that you start out with. 879 00:49:43,820 --> 00:49:47,220 And that makes this algorithm an exponential time algorithm. 880 00:49:47,220 --> 00:49:49,050 So clearly, we have to do better than that 881 00:49:49,050 --> 00:49:52,540 when it comes to Dijkstra or Bellman-Ford. 882 00:49:52,540 --> 00:49:54,840 So how are we going to do better than that? 883 00:49:54,840 --> 00:49:56,260 Yeah, question back there. 884 00:49:56,260 --> 00:49:57,718 AUDIENCE: Is it an issue that we're 885 00:49:57,718 --> 00:49:59,176 starting at the [INAUDIBLE]? 886 00:50:02,807 --> 00:50:04,140 PROFESSOR: You're exactly right. 887 00:50:04,140 --> 00:50:07,410 There's an issue with the ordering that we've chosen. 888 00:50:07,410 --> 00:50:10,920 But what you have to show is that for any graph, 889 00:50:10,920 --> 00:50:12,790 the particular ordering that you choose 890 00:50:12,790 --> 00:50:16,910 will result in V log V plus E and so on and so forth. 891 00:50:16,910 --> 00:50:17,910 So you're exactly right. 892 00:50:17,910 --> 00:50:20,118 I mean, it's an issue with the ordering we've chosen. 893 00:50:20,118 --> 00:50:22,050 This is a pathological ordering. 894 00:50:22,050 --> 00:50:23,850 It's just meaning to say that we have 895 00:50:23,850 --> 00:50:27,080 to be careful about how we select. 896 00:50:27,080 --> 00:50:30,040 If you selected wrong, you've got problems. 897 00:50:30,040 --> 00:50:32,110 And so the purpose of next week is 898 00:50:32,110 --> 00:50:35,360 going to be how do we select these edges properly. 899 00:50:35,360 --> 00:50:40,070 And so I leave you with this notion of, 900 00:50:40,070 --> 00:50:44,200 very simple notion of, optimal substructure using 901 00:50:44,200 --> 00:50:46,430 two very simple terms that you can 902 00:50:46,430 --> 00:50:50,200 prove in literally a line of text. 903 00:50:50,200 --> 00:50:59,410 And the first one says as subpaths of a shortest path 904 00:50:59,410 --> 00:51:00,580 are shortest paths. 905 00:51:05,520 --> 00:51:11,920 And all that means is if I had V0, and I went to V1, 906 00:51:11,920 --> 00:51:15,500 and I went to V2, and these are paths here. 907 00:51:15,500 --> 00:51:22,930 So this could be p01, p02, p03. 908 00:51:22,930 --> 00:51:26,240 And so there are many vertices potentially between V0 and V1. 909 00:51:26,240 --> 00:51:30,010 And if you tell me that V0 through V3, 910 00:51:30,010 --> 00:51:34,460 the concatenation of p01, p02, and, sorry, p03 911 00:51:34,460 --> 00:51:35,485 are a shortest path. 912 00:51:40,740 --> 00:51:44,110 If this is an SP, shortest path, then that 913 00:51:44,110 --> 00:51:50,030 implies that each of these are shortest paths as well. 914 00:51:50,030 --> 00:51:52,650 And that makes sense, because if in fact there 915 00:51:52,650 --> 00:51:55,740 was a better way of getting from V0 to V1 916 00:51:55,740 --> 00:52:00,200 that was better than p01, why would you ever put p01 in here? 917 00:52:00,200 --> 00:52:02,390 You would use that better way. 918 00:52:02,390 --> 00:52:03,580 So very simple. 919 00:52:03,580 --> 00:52:06,810 That's what's called the optimum substructure property. 920 00:52:06,810 --> 00:52:12,190 And this notion of the triangle inequality 921 00:52:12,190 --> 00:52:14,440 is also related to that. 922 00:52:14,440 --> 00:52:21,700 And that simply says that if I have something like this, 923 00:52:21,700 --> 00:52:28,110 that I have V0, V1, and V2, then when 924 00:52:28,110 --> 00:52:32,810 I look at the delta value of V0, V1, 925 00:52:32,810 --> 00:52:38,500 and I compare that with the delta values of V0, V2, and V2, 926 00:52:38,500 --> 00:52:46,290 V1, then this has got to be smaller than or equal to this 927 00:52:46,290 --> 00:52:48,096 plus that. 928 00:52:48,096 --> 00:52:49,220 And that again makes sense. 929 00:52:49,220 --> 00:52:54,090 Because if this plus this was smaller than that, 930 00:52:54,090 --> 00:52:59,030 well remember I'm talking about paths here, not edges. 931 00:52:59,030 --> 00:53:00,890 And the better way of getting to V1 932 00:53:00,890 --> 00:53:03,140 would be to follow-- go through V2 rather 933 00:53:03,140 --> 00:53:05,640 than following this path up on top. 934 00:53:05,640 --> 00:53:07,480 Amazingly, these two notions are going 935 00:53:07,480 --> 00:53:09,670 to be enough to take this algorithm 936 00:53:09,670 --> 00:53:12,980 and turn it into essentially a linear time algorithm. 937 00:53:12,980 --> 00:53:15,530 And we'll do that next time.