1 00:00:00,080 --> 00:00:01,770 The following content is provided 2 00:00:01,770 --> 00:00:04,000 under a Creative Commons license. 3 00:00:04,000 --> 00:00:06,860 Your support will help MIT OpenCourseWare continue 4 00:00:06,860 --> 00:00:10,720 to offer high quality educational resources for free. 5 00:00:10,720 --> 00:00:13,320 To make a donation or view additional materials 6 00:00:13,320 --> 00:00:17,207 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:17,207 --> 00:00:17,832 at ocw.mit.edu. 8 00:00:21,404 --> 00:00:22,820 PROFESSOR: Good morning, everyone. 9 00:00:25,410 --> 00:00:32,150 So lecture three of four in the shortest path module and today 10 00:00:32,150 --> 00:00:37,390 we'll finally confront our nemesis, 11 00:00:37,390 --> 00:00:40,220 which are negative cycles and negative edges. 12 00:00:40,220 --> 00:00:44,240 And we will describe an algorithm 13 00:00:44,240 --> 00:00:46,470 that is due to two different people. 14 00:00:46,470 --> 00:00:49,360 They didn't collaborate to produce this algorithm. 15 00:00:49,360 --> 00:00:52,570 Bellman and Ford. 16 00:00:52,570 --> 00:00:58,720 This computes shortest paths in a graph with negative edges. 17 00:00:58,720 --> 00:01:01,060 And not only that, even in the graph 18 00:01:01,060 --> 00:01:03,890 has negative cycles in it, the algorithm 19 00:01:03,890 --> 00:01:06,800 will be correct in the sense that it 20 00:01:06,800 --> 00:01:09,830 will report the existence of a negative cycle 21 00:01:09,830 --> 00:01:13,490 and, essentially, abort the computation of shortest paths 22 00:01:13,490 --> 00:01:15,290 that are undefined. 23 00:01:15,290 --> 00:01:18,730 And for the few vertices that do not 24 00:01:18,730 --> 00:01:23,020 have negative cycles in between them and the source, 25 00:01:23,020 --> 00:01:25,980 the algorithm will report correct shortest paths. 26 00:01:25,980 --> 00:01:29,260 So it is a polynomial time algorithm. 27 00:01:29,260 --> 00:01:30,900 It's fairly easy to describe. 28 00:01:30,900 --> 00:01:35,080 And what we'll do is describe it, analyze its complexity 29 00:01:35,080 --> 00:01:40,800 and, for once, we'll do a formal proof of its correctness 30 00:01:40,800 --> 00:01:43,940 to show that it reports the existence of negative cycles 31 00:01:43,940 --> 00:01:45,140 if they do exist. 32 00:01:45,140 --> 00:01:47,150 And if they don't exist, it correctly 33 00:01:47,150 --> 00:01:49,270 computes shortest path weights. 34 00:01:52,720 --> 00:01:59,530 So recall that when we look at the general case 35 00:01:59,530 --> 00:02:02,300 of the shortest path problem. 36 00:02:02,300 --> 00:02:07,150 We're going to have, let's say, a vertex u that, in this case, 37 00:02:07,150 --> 00:02:09,490 happens to be our source. 38 00:02:09,490 --> 00:02:11,640 And let's say for argument's sake 39 00:02:11,640 --> 00:02:17,312 that we have a negative weight cycle like so. 40 00:02:17,312 --> 00:02:19,010 So let me to draw this in bold. 41 00:02:21,810 --> 00:02:24,560 And this happens to be a negative rate cycle. 42 00:02:24,560 --> 00:02:30,930 Let's assume that all of these edges have positive weights. 43 00:02:30,930 --> 00:02:37,640 Then, if you have an algorithm that 44 00:02:37,640 --> 00:02:41,600 needs to work on this type of graph, what you want 45 00:02:41,600 --> 00:02:45,590 to be able to do is to detect that this negative cycle 46 00:02:45,590 --> 00:02:46,770 exists. 47 00:02:46,770 --> 00:02:49,190 And you're going to, essentially, 48 00:02:49,190 --> 00:02:53,170 say if this vertex is v1, for example, 49 00:02:53,170 --> 00:02:59,720 you want to be able to say delta u v1 is undefined and similarly 50 00:02:59,720 --> 00:03:03,317 for v2, v3, et cetera. 51 00:03:03,317 --> 00:03:05,400 For all of these things, the shortest path lengths 52 00:03:05,400 --> 00:03:09,010 are undefined because you can essentially 53 00:03:09,010 --> 00:03:11,330 run through this negative cycle any number of times 54 00:03:11,330 --> 00:03:16,180 and get whatever shortest path weight you want. 55 00:03:16,180 --> 00:03:22,400 For this node, let's call that v0, we have delta u v0 56 00:03:22,400 --> 00:03:25,240 equals 2. 57 00:03:25,240 --> 00:03:29,070 And there's a simple path of length 1 58 00:03:29,070 --> 00:03:32,390 in this case that gets you from u to v0. 59 00:03:32,390 --> 00:03:36,790 You don't encounter a cycle or negative cycle in between. 60 00:03:36,790 --> 00:03:38,480 So that's cool. 61 00:03:38,480 --> 00:03:39,850 All right? 62 00:03:39,850 --> 00:03:44,140 And of course, if you have a vertex over here, z, 63 00:03:44,140 --> 00:03:47,360 that can't be reached from u then we're 64 00:03:47,360 --> 00:03:51,630 going to have delta uz being infinity. 65 00:03:51,630 --> 00:03:55,140 And you can assume at the beginning of these algorithms 66 00:03:55,140 --> 00:03:58,840 that the source-- in this case, I call the source u-- 67 00:03:58,840 --> 00:04:03,850 but the shortest path to u would be 0. 68 00:04:03,850 --> 00:04:06,140 And all of the other ones are infinity. 69 00:04:06,140 --> 00:04:08,250 And some of them may stay infinity. 70 00:04:08,250 --> 00:04:13,090 Some of them may obtain finite shortest path weights. 71 00:04:13,090 --> 00:04:14,810 And some of them will be undefined 72 00:04:14,810 --> 00:04:17,820 if you have a graph with negative cycles in it. 73 00:04:17,820 --> 00:04:20,839 So that's sort of the specification, if you will, 74 00:04:20,839 --> 00:04:23,530 of the requirements on the Bellman-Ford algorithm. 75 00:04:23,530 --> 00:04:27,270 We want it to be able to do all of the things I just described. 76 00:04:27,270 --> 00:04:29,300 OK? 77 00:04:29,300 --> 00:04:40,540 So let's take a second look at our generic shortest path 78 00:04:40,540 --> 00:04:47,510 algorithm that I put up, I think, about a week ago. 79 00:04:47,510 --> 00:04:52,400 And this is a good review of our notation. 80 00:04:52,400 --> 00:04:53,840 But there are a couple more things 81 00:04:53,840 --> 00:04:56,100 I want to say about this algorithm 82 00:04:56,100 --> 00:04:58,490 that I didn't get to last time. 83 00:04:58,490 --> 00:05:02,910 So you're given a graph and you set all of the vertices 84 00:05:02,910 --> 00:05:06,760 in the graph to have infinite shortest path 85 00:05:06,760 --> 00:05:10,020 weights, initially. 86 00:05:10,020 --> 00:05:13,220 Set the predecessors to be null. 87 00:05:13,220 --> 00:05:18,710 And then we'll set d of s to be 0. 88 00:05:18,710 --> 00:05:20,850 That's your source. 89 00:05:20,850 --> 00:05:30,920 And the main loop would be something like repeat, 90 00:05:30,920 --> 00:05:33,910 select, and edge. 91 00:05:33,910 --> 00:05:37,800 And we have a particular way of selecting this edge. 92 00:05:37,800 --> 00:05:39,810 And we have positive edge weights 93 00:05:39,810 --> 00:05:42,860 that corresponds to the minimum priority. 94 00:05:42,860 --> 00:05:45,172 And we talked about Dijkstra but we have, maybe, 95 00:05:45,172 --> 00:05:46,380 different ways of doing that. 96 00:05:46,380 --> 00:05:48,530 We have to select an edge somehow. 97 00:05:48,530 --> 00:05:52,045 And then, we relaxed that edge. 98 00:05:55,370 --> 00:05:58,080 u, v, w. 99 00:05:58,080 --> 00:06:00,040 And you know about the relaxation step. 100 00:06:00,040 --> 00:06:03,000 I won't bother writing it out right now. 101 00:06:03,000 --> 00:06:05,920 But it's basically something where 102 00:06:05,920 --> 00:06:09,340 you look at the value of d v. And if d v 103 00:06:09,340 --> 00:06:14,280 is greater than d u plus the weight, you relax the edge. 104 00:06:14,280 --> 00:06:17,350 And you keep doing this. 105 00:06:17,350 --> 00:06:20,250 The other thing that you do in the relaxation 106 00:06:20,250 --> 00:06:23,260 is to set the predecessor pointers to be correct. 107 00:06:23,260 --> 00:06:25,630 And that's part of the relax routine. 108 00:06:25,630 --> 00:06:33,905 And you keep doing this until you can't relax anymore. 109 00:06:36,791 --> 00:06:37,290 All right? 110 00:06:37,290 --> 00:06:40,830 So that's our generic shortest path algorithm. 111 00:06:40,830 --> 00:06:45,050 There are two problems with this algorithm. 112 00:06:45,050 --> 00:06:48,700 The first, which we talked about and both of these 113 00:06:48,700 --> 00:06:52,880 have to do with the complexity but the first one 114 00:06:52,880 --> 00:07:00,120 is that the complexity could be exponential time, 115 00:07:00,120 --> 00:07:01,840 even for positive edge weights. 116 00:07:08,090 --> 00:07:10,450 And the particular example we talked about 117 00:07:10,450 --> 00:07:16,900 was something where you had an exponential number of paths. 118 00:07:16,900 --> 00:07:21,370 And if you had a graph that looks like this, 119 00:07:21,370 --> 00:07:25,950 then it's possible that a pathological selection of edges 120 00:07:25,950 --> 00:07:31,630 is going to make you relax edges an exponential number of times. 121 00:07:31,630 --> 00:07:35,384 And in particular, if you have n nodes in this graph, 122 00:07:35,384 --> 00:07:37,050 it's plausible that you'd end up getting 123 00:07:37,050 --> 00:07:41,390 the complexity of order 2 raised to n over 2. 124 00:07:41,390 --> 00:07:42,360 OK? 125 00:07:42,360 --> 00:07:44,809 So that's one problem. 126 00:07:44,809 --> 00:07:46,350 The second problem, which is actually 127 00:07:46,350 --> 00:07:52,740 a more obvious problem, is that this algorithm might not even 128 00:07:52,740 --> 00:08:04,800 terminate if this-- actually will not 129 00:08:04,800 --> 00:08:14,950 terminate the way it's written if there's a negative weight 130 00:08:14,950 --> 00:08:18,335 cycle reachable from the source. 131 00:08:29,180 --> 00:08:31,380 All right, so there's two problems. 132 00:08:31,380 --> 00:08:33,720 We fixed the first one. 133 00:08:33,720 --> 00:08:38,549 In the case of positive edges are non-negative edges. 134 00:08:38,549 --> 00:08:41,600 We have a neat algorithm that is an efficient algorithm called 135 00:08:41,600 --> 00:08:43,990 Dijkstra that we talked about last time that fixed 136 00:08:43,990 --> 00:08:45,200 the first part. 137 00:08:45,200 --> 00:08:47,460 But we don't know yet how we're going 138 00:08:47,460 --> 00:08:51,371 to handle negative cycles in the general case. 139 00:08:51,371 --> 00:08:52,870 We know how to handle negative edges 140 00:08:52,870 --> 00:08:55,650 in the case of a DAG-- a directed acyclic graph-- 141 00:08:55,650 --> 00:08:57,760 but not in the general case. 142 00:08:57,760 --> 00:08:58,260 OK? 143 00:09:01,930 --> 00:09:07,150 So there's this great little skit from Saturday Night Live 144 00:09:07,150 --> 00:09:11,680 from the 1980s-- so way before your time-- called The Five 145 00:09:11,680 --> 00:09:13,130 Minute University. 146 00:09:13,130 --> 00:09:15,100 Anybody seen this? 147 00:09:15,100 --> 00:09:15,600 All right. 148 00:09:15,600 --> 00:09:16,450 Look it up on YouTube. 149 00:09:16,450 --> 00:09:18,408 Don't look it up during lecture but afterwards. 150 00:09:20,920 --> 00:09:23,940 So the character here is a person 151 00:09:23,940 --> 00:09:26,620 by the name of-- I forget his real name 152 00:09:26,620 --> 00:09:30,200 but his fake name is Father Guido Sarducci. 153 00:09:30,200 --> 00:09:31,470 All right? 154 00:09:31,470 --> 00:09:33,990 So what's this Five Minute University about? 155 00:09:33,990 --> 00:09:37,940 Five Minute University, he's selling this notion 156 00:09:37,940 --> 00:09:42,040 and he says, look, five years after you graduate 157 00:09:42,040 --> 00:09:44,340 you, essentially, are going to remember nothing. 158 00:09:44,340 --> 00:09:44,910 OK? 159 00:09:44,910 --> 00:09:46,800 I mean, you're not going to remember anything 160 00:09:46,800 --> 00:09:49,580 about all the courses you took, et cetera. 161 00:09:49,580 --> 00:09:53,590 So why waste your time on a college education or waste 162 00:09:53,590 --> 00:09:55,810 money-- $100,000-- on a college education? 163 00:09:55,810 --> 00:10:00,850 You know, for $20 I'll teach you in five minutes what you're 164 00:10:00,850 --> 00:10:04,320 going to remember five years after you graduate. 165 00:10:04,320 --> 00:10:05,630 All right? 166 00:10:05,630 --> 00:10:07,960 So let's take it to an extreme. 167 00:10:07,960 --> 00:10:11,550 Here's a 30 second version up 6006. 168 00:10:11,550 --> 00:10:15,700 And this is what I want you to remember five years or 10 years 169 00:10:15,700 --> 00:10:17,180 or whatever after you graduate. 170 00:10:17,180 --> 00:10:18,080 All right? 171 00:10:18,080 --> 00:10:23,220 And maybe the 10 second version as polynomial time is great. 172 00:10:23,220 --> 00:10:23,720 OK? 173 00:10:23,720 --> 00:10:25,800 Exponential time is bad. 174 00:10:25,800 --> 00:10:28,200 And infinite time gets you fired. 175 00:10:28,200 --> 00:10:29,720 OK? 176 00:10:29,720 --> 00:10:33,267 So that's all you need to remember. 177 00:10:33,267 --> 00:10:35,350 No, that's all you need to remember for the final. 178 00:10:35,350 --> 00:10:38,352 This happens, you know, five years after you graduate. 179 00:10:38,352 --> 00:10:40,060 So you need to remember a lot more if you 180 00:10:40,060 --> 00:10:42,850 want to take your quiz next week and the final exam. 181 00:10:42,850 --> 00:10:44,450 But I think that summarized over here. 182 00:10:44,450 --> 00:10:47,480 You have a generic shortest path algorithm. 183 00:10:47,480 --> 00:10:50,530 And you realize that if you do this wrong 184 00:10:50,530 --> 00:10:54,450 you could very easily get into a situation 185 00:10:54,450 --> 00:10:56,220 where a polynomial time algorithm, and we 186 00:10:56,220 --> 00:10:59,850 know one for Dijkstra, turns into exponential time 187 00:10:59,850 --> 00:11:03,010 in the worst case, you know, for a graph like that 188 00:11:03,010 --> 00:11:05,890 because you're selecting edges wrongly. 189 00:11:05,890 --> 00:11:09,900 And in particular, that's problem number one. 190 00:11:09,900 --> 00:11:13,180 And problem number two is if you have 191 00:11:13,180 --> 00:11:16,250 a graph that isn't what you expect. 192 00:11:16,250 --> 00:11:19,850 In this case, let's say you expected that a graph 193 00:11:19,850 --> 00:11:23,300 with no negative cycles or maybe not even negative edges in it. 194 00:11:23,300 --> 00:11:25,190 You could easily get into a situation 195 00:11:25,190 --> 00:11:26,920 where your termination condition is such 196 00:11:26,920 --> 00:11:29,520 that your algorithm never completes. 197 00:11:29,520 --> 00:11:32,910 So we need to fix problem number two today 198 00:11:32,910 --> 00:11:36,830 using this algorithm called Bellman-Ford. 199 00:11:36,830 --> 00:11:40,760 And as it turns out, this algorithm 200 00:11:40,760 --> 00:11:43,300 is incredibly straightforward. 201 00:11:43,300 --> 00:11:45,640 I mean, its complexity we'll have to look at. 202 00:11:45,640 --> 00:11:47,850 But from a description standpoint, 203 00:11:47,850 --> 00:11:50,440 it's four lines of code. 204 00:11:50,440 --> 00:11:54,670 And let me put that up. 205 00:11:54,670 --> 00:12:04,270 So Bellman-Ford takes a graph, weights, and a source s. 206 00:12:04,270 --> 00:12:12,990 And you can assume an adjacency list specification of the graph 207 00:12:12,990 --> 00:12:14,840 or the representation of the graph. 208 00:12:14,840 --> 00:12:17,050 And we do some initialization. 209 00:12:17,050 --> 00:12:20,330 It's exactly the same as in the generic case 210 00:12:20,330 --> 00:12:23,750 except the d values will still be looking at the d values 211 00:12:23,750 --> 00:12:26,500 and talking about the relaxation operation. 212 00:12:26,500 --> 00:12:28,710 So we do an initialization. 213 00:12:28,710 --> 00:12:33,210 And then, this algorithm has multiple passes 214 00:12:33,210 --> 00:12:38,120 because for I equals 1 to v minus 1. 215 00:12:38,120 --> 00:12:42,650 So it does v minus 1 passes roughly order v passes 216 00:12:42,650 --> 00:12:45,040 where v is the number of vertices. 217 00:12:45,040 --> 00:12:53,410 And in each of these passes for each edge u v belonging to e 218 00:12:53,410 --> 00:12:54,700 relaxes every edge. 219 00:13:02,850 --> 00:13:14,060 And just so everyone remembers, relax u, v, w is if d of v 220 00:13:14,060 --> 00:13:29,450 is greater than d of u plus w u v then we'll set d v to be-- 221 00:13:29,450 --> 00:13:32,200 and we also set pi v to be u. 222 00:13:35,110 --> 00:13:35,610 OK. 223 00:13:41,080 --> 00:13:45,730 That's relax operation over here. 224 00:13:45,730 --> 00:13:48,460 So that's the algorithm. 225 00:13:48,460 --> 00:13:57,460 And if you know magically that they're 226 00:13:57,460 --> 00:14:01,840 no negative cycles in the graph. 227 00:14:01,840 --> 00:14:04,810 So if they're no negative cycles in the graph, 228 00:14:04,810 --> 00:14:07,360 then after these-- we'll have to prove this. 229 00:14:07,360 --> 00:14:10,310 But after these v minus 1 passes you're 230 00:14:10,310 --> 00:14:12,731 going to get the correct shortest pathways. 231 00:14:12,731 --> 00:14:13,230 OK? 232 00:14:15,760 --> 00:14:17,460 You want to do a little bit more, right? 233 00:14:17,460 --> 00:14:21,950 I motivated what we want Bellman-Ford to do 234 00:14:21,950 --> 00:14:23,700 earlier in the lecture. 235 00:14:23,700 --> 00:14:26,010 So you can also do a check. 236 00:14:26,010 --> 00:14:30,750 So you may not know if they're negative weight cycles or not. 237 00:14:30,750 --> 00:14:32,520 But at this point, you can say I'm 238 00:14:32,520 --> 00:14:37,190 going to do one more pass so the v path-- the v 239 00:14:37,190 --> 00:14:41,440 is the number of vertices-- over the graph. 240 00:14:41,440 --> 00:14:48,940 So for each edge in the graph, if you do one more relaxation 241 00:14:48,940 --> 00:14:55,800 and you see that d v is greater than d u plus w u v. So 242 00:14:55,800 --> 00:14:57,050 you're not doing a relaxation. 243 00:14:57,050 --> 00:15:02,180 You're doing a check to see if you can relax the edge. 244 00:15:02,180 --> 00:15:11,860 Then report minus v negative cycle exists. 245 00:15:11,860 --> 00:15:15,930 So this is the check. 246 00:15:15,930 --> 00:15:17,850 And the first part is the computation. 247 00:15:22,310 --> 00:15:23,290 So that's kind of neat. 248 00:15:23,290 --> 00:15:26,720 I mean, it fit's on a board. 249 00:15:26,720 --> 00:15:29,140 We talk about the correctness. 250 00:15:29,140 --> 00:15:31,085 The functionality, I hope everyone got. 251 00:15:31,085 --> 00:15:33,851 Do people understand what's happening here 252 00:15:33,851 --> 00:15:35,100 with respect to functionality? 253 00:15:35,100 --> 00:15:35,683 Any questions? 254 00:15:37,489 --> 00:15:39,155 Not about correctness but functionality? 255 00:15:39,155 --> 00:15:39,655 Yeah? 256 00:15:39,655 --> 00:15:41,214 AUDIENCE: Where does the [INAUDIBLE] 257 00:15:41,214 --> 00:15:42,567 get used in the formula? 258 00:15:42,567 --> 00:15:43,985 PROFESSOR: Oh, it doesn't. 259 00:15:46,780 --> 00:15:48,380 It's just a counter that makes sure 260 00:15:48,380 --> 00:15:50,335 that you do v minus 1 passes. 261 00:15:54,760 --> 00:15:57,690 So what's that complexity of this algorithm 262 00:15:57,690 --> 00:16:03,490 using the best data structure that we can think of? 263 00:16:03,490 --> 00:16:03,990 Anyone? 264 00:16:07,958 --> 00:16:08,950 Yeah, go ahead. 265 00:16:08,950 --> 00:16:12,918 AUDIENCE: [INAUDIBLE] v plus e if you're using a [INAUDIBLE] 266 00:16:12,918 --> 00:16:14,387 to access [INAUDIBLE]? 267 00:16:14,387 --> 00:16:15,220 PROFESSOR: v plus e? 268 00:16:15,220 --> 00:16:18,412 AUDIENCE: Or v e plus e. 269 00:16:18,412 --> 00:16:20,632 PROFESSOR: So that would be? 270 00:16:20,632 --> 00:16:22,295 AUDIENCE: That's using a dictionary? 271 00:16:22,295 --> 00:16:24,170 PROFESSOR: Yeah, I know. v e plus e would be? 272 00:16:24,170 --> 00:16:25,204 That's correct but. 273 00:16:25,204 --> 00:16:26,120 AUDIENCE: [INAUDIBLE]. 274 00:16:26,120 --> 00:16:26,828 PROFESSOR: Right. 275 00:16:26,828 --> 00:16:31,800 But I mean when do v e plus e you can ignore the e. 276 00:16:31,800 --> 00:16:36,750 So say you have just v times e. 277 00:16:36,750 --> 00:16:37,250 All right. 278 00:16:37,250 --> 00:16:37,580 Good. 279 00:16:37,580 --> 00:16:38,079 Here you go. 280 00:16:41,190 --> 00:16:44,830 So this part here is v times e. 281 00:16:44,830 --> 00:16:46,490 And it doesn't really matter. 282 00:16:46,490 --> 00:16:49,990 I mean, you can use an array structure adjacency list. 283 00:16:49,990 --> 00:16:54,142 It's not like Dijkstra where we have this neat requirement 284 00:16:54,142 --> 00:16:56,100 for a priority queue and there's different ways 285 00:16:56,100 --> 00:16:58,840 of implementing the priority queue. 286 00:16:58,840 --> 00:17:02,280 This part would be order of v e. 287 00:17:02,280 --> 00:17:05,060 And that gives you the overall complexity. 288 00:17:05,060 --> 00:17:07,640 This part here is only one pass through the edges. 289 00:17:07,640 --> 00:17:10,390 So that's order e, like you said. 290 00:17:10,390 --> 00:17:13,329 So the complexities order v e. 291 00:17:13,329 --> 00:17:17,520 And this could be large, as I said before in, I think, 292 00:17:17,520 --> 00:17:18,930 the first lecture. 293 00:17:18,930 --> 00:17:26,710 e is order of v square in a simple graph. 294 00:17:26,710 --> 00:17:29,240 So you might end up with a v cubed complexity 295 00:17:29,240 --> 00:17:30,690 if you run Bellman-Ford. 296 00:17:30,690 --> 00:17:33,410 So there's no question that Bellman-Ford 297 00:17:33,410 --> 00:17:37,170 is, from a practical standpoint, substantially slower 298 00:17:37,170 --> 00:17:38,880 than Dijkstra. 299 00:17:38,880 --> 00:17:46,460 You can get Dijkstra down to linear complexity. 300 00:17:46,460 --> 00:17:49,050 But this would potentially, at least in terms of vertices, 301 00:17:49,050 --> 00:17:52,320 be cubic complexity. 302 00:17:52,320 --> 00:17:54,979 So when you have a chance, you want to use Dijkstra. 303 00:17:54,979 --> 00:17:56,520 And you're forced to use Bellman-Ford 304 00:17:56,520 --> 00:17:58,840 because you could potentially have negative weight 305 00:17:58,840 --> 00:18:01,100 cycles while you're stuck with that. 306 00:18:01,100 --> 00:18:03,100 All right? 307 00:18:03,100 --> 00:18:04,770 OK, so why does this work? 308 00:18:04,770 --> 00:18:06,830 This looks a bit like magic. 309 00:18:06,830 --> 00:18:12,260 It turns out we can actually do a fairly straightforward proof 310 00:18:12,260 --> 00:18:14,480 of correctness of Bellman-Ford. 311 00:18:14,480 --> 00:18:15,985 And we're going to do two things. 312 00:18:15,985 --> 00:18:19,850 We're going to not only show that if negative weight 313 00:18:19,850 --> 00:18:25,540 cycles don't exist that this will correctly 314 00:18:25,540 --> 00:18:27,720 compute shorter stats. 315 00:18:27,720 --> 00:18:31,480 But we also have to show that it will detect negative weight 316 00:18:31,480 --> 00:18:33,900 cycles if they in fact exist. 317 00:18:33,900 --> 00:18:35,670 So there's two parts to this. 318 00:18:35,670 --> 00:18:36,645 And let's start. 319 00:18:39,480 --> 00:18:43,840 So what we have here for this algorithm 320 00:18:43,840 --> 00:18:55,440 is that it can guarantee in a graph g equals 321 00:18:55,440 --> 00:19:13,150 v E. If it contains no negative weight cycles then 322 00:19:13,150 --> 00:19:21,850 after Bellman-Ford finishes execution, 323 00:19:21,850 --> 00:19:33,300 d v equals delta s v for all v belonging to v. All right? 324 00:19:33,300 --> 00:19:35,460 And then there's that. 325 00:19:35,460 --> 00:19:37,290 That's the theorem you want to prove. 326 00:19:37,290 --> 00:19:44,230 And the second piece of it is corollary 327 00:19:44,230 --> 00:19:46,210 that we want to prove. 328 00:19:46,210 --> 00:19:48,260 And that has to do with the check. 329 00:19:48,260 --> 00:19:55,460 And this says if a value of d of v 330 00:19:55,460 --> 00:20:05,530 fails to converge after v minus 1 331 00:20:05,530 --> 00:20:17,416 passes there exists a negative weight cycle reachable from s. 332 00:20:24,120 --> 00:20:29,400 So those are the two things that we need to show. 333 00:20:29,400 --> 00:20:31,670 I'll probably take a few minutes to do each of these. 334 00:20:31,670 --> 00:20:34,370 That theorem is a little more involved. 335 00:20:37,104 --> 00:20:38,520 So one of the first things that we 336 00:20:38,520 --> 00:20:41,940 have to do in order to prove this theorem 337 00:20:41,940 --> 00:20:50,880 is to think about exactly what the shortest path corresponds 338 00:20:50,880 --> 00:20:53,310 to in a generic sense. 339 00:20:53,310 --> 00:20:58,900 So when we have source vertex s and you have 340 00:20:58,900 --> 00:21:02,810 a particular vertex v then there's 341 00:21:02,810 --> 00:21:07,760 the picture that we need to keep in mind as we try 342 00:21:07,760 --> 00:21:09,780 and prove this theorem. 343 00:21:09,780 --> 00:21:18,230 So you have v0, v1, v2, et cetera all the way to vk. 344 00:21:18,230 --> 00:21:21,460 This is my vertex v. This is s. 345 00:21:21,460 --> 00:21:24,860 So s equals v0. 346 00:21:24,860 --> 00:21:26,150 V equals vk. 347 00:21:26,150 --> 00:21:26,870 All right? 348 00:21:26,870 --> 00:21:29,970 So I'm going to have a path p. 349 00:21:29,970 --> 00:21:35,470 That is v0, v1, all the way to vk. 350 00:21:35,470 --> 00:21:35,970 OK? 351 00:21:40,810 --> 00:21:46,200 How big is k in the worst case? 352 00:21:46,200 --> 00:21:47,120 How big is k? 353 00:21:50,940 --> 00:21:51,950 Anybody? 354 00:21:51,950 --> 00:21:52,495 How big is k? 355 00:21:55,150 --> 00:21:56,535 It's up on the black board. 356 00:21:56,535 --> 00:21:58,190 AUDIENCE: [INAUDIBLE]. 357 00:21:58,190 --> 00:21:59,501 PROFESSOR: v minus 1, right? 358 00:21:59,501 --> 00:22:00,000 Why? 359 00:22:03,370 --> 00:22:08,520 What would happen if k is larger than v minus 1? 360 00:22:08,520 --> 00:22:09,550 I'd have a cycle. 361 00:22:09,550 --> 00:22:11,780 I'd be visiting a vertex more than once. 362 00:22:11,780 --> 00:22:13,870 And it wouldn't be a simple path. 363 00:22:13,870 --> 00:22:14,370 Right? 364 00:22:17,300 --> 00:22:24,810 So k is less than or equal to v minus 1 else I'd have a cycle. 365 00:22:24,810 --> 00:22:26,590 OK? 366 00:22:26,590 --> 00:22:28,630 I wouldn't have a simple path. 367 00:22:28,630 --> 00:22:31,520 And we're looking for the shortest, simple paths 368 00:22:31,520 --> 00:22:33,150 because if you ever get to the point 369 00:22:33,150 --> 00:22:35,470 where-- why are we looking for shortest, simple paths? 370 00:22:35,470 --> 00:22:39,446 Well, in this case, we're looking 371 00:22:39,446 --> 00:22:40,570 for shortest, simple paths. 372 00:22:40,570 --> 00:22:43,700 And if there's a negative cycle, we're 373 00:22:43,700 --> 00:22:46,900 in trouble because the shortest path is not 374 00:22:46,900 --> 00:22:49,690 necessarily the simple path because you 375 00:22:49,690 --> 00:22:52,730 could go around the cycle a bunch of times. 376 00:22:52,730 --> 00:22:54,080 I'll get back to that. 377 00:22:54,080 --> 00:22:58,680 But in the case where we're trying to prove the theorem, 378 00:22:58,680 --> 00:23:00,809 we know that no negative cycles exist. 379 00:23:00,809 --> 00:23:02,350 We can assume that no negative cycles 380 00:23:02,350 --> 00:23:04,060 exist for the case of the theorem. 381 00:23:04,060 --> 00:23:07,940 And we want to show that Bellman-Ford correctly 382 00:23:07,940 --> 00:23:11,230 computes each of the shortest path weights. 383 00:23:11,230 --> 00:23:13,970 And in that case, there's no negative weight cycles. 384 00:23:13,970 --> 00:23:16,955 We're guaranteed that k is less than or equal to v minus 1. 385 00:23:16,955 --> 00:23:18,880 All right? 386 00:23:18,880 --> 00:23:21,190 Everybody buy that? 387 00:23:21,190 --> 00:23:21,880 Good. 388 00:23:21,880 --> 00:23:22,380 All right. 389 00:23:22,380 --> 00:23:24,500 So that's the picture I want you keep in mind. 390 00:23:24,500 --> 00:23:30,475 Let's dive in and prove this theorem. 391 00:23:30,475 --> 00:23:33,130 And we prove it using induction. 392 00:23:51,560 --> 00:23:54,830 So let v be any vertex. 393 00:23:54,830 --> 00:23:57,780 And let's say that we're looking at a path. 394 00:23:57,780 --> 00:24:01,780 v0, v1, v2, to vk. 395 00:24:01,780 --> 00:24:06,570 And like I said, from v0 equals s to vk 396 00:24:06,570 --> 00:24:14,270 equals v. And in particular, I'm not 397 00:24:14,270 --> 00:24:22,830 going to say that this path p is a shortest 398 00:24:22,830 --> 00:24:28,835 path with the minimum number of edges. 399 00:24:33,550 --> 00:24:35,770 So there may be many shortest paths. 400 00:24:35,770 --> 00:24:37,230 And I'm going to pick the one that 401 00:24:37,230 --> 00:24:39,040 has the minimum number of edges. 402 00:24:39,040 --> 00:24:42,020 If there's a unique shortest path, then that's a given. 403 00:24:42,020 --> 00:24:46,580 But it may be that I have a path with four edges that 404 00:24:46,580 --> 00:24:48,880 has the same weight as another path with three edges. 405 00:24:48,880 --> 00:24:51,390 I'm going to pick the one that has three edges. 406 00:24:51,390 --> 00:24:51,890 OK? 407 00:24:51,890 --> 00:24:54,406 So it may not be unique with respect 408 00:24:54,406 --> 00:24:56,530 that they're not necessarily unique shortest paths. 409 00:24:56,530 --> 00:24:59,900 But I can certainly pick one. 410 00:24:59,900 --> 00:25:07,370 And no negative weight cycles implies that p is simple. 411 00:25:10,390 --> 00:25:16,660 And that implies that k is less than or equal to v minus 1, 412 00:25:16,660 --> 00:25:19,470 which is what I just argued. 413 00:25:19,470 --> 00:25:24,560 Now keep in mind that picture over there to the right. 414 00:25:24,560 --> 00:25:30,850 And basically, the argument is going to go as follows. 415 00:25:30,850 --> 00:25:33,250 Remember that I'm going to be relaxing 416 00:25:33,250 --> 00:25:38,310 every edge in each pass of the algorithm. 417 00:25:38,310 --> 00:25:39,120 OK? 418 00:25:39,120 --> 00:25:42,450 There's no choices here. 419 00:25:42,450 --> 00:25:45,490 I'm going be relaxing every edge in each pass of the algorithm. 420 00:25:45,490 --> 00:25:48,680 And essentially, the proof goes as follows. 421 00:25:48,680 --> 00:25:53,930 I'm going to be moving closer and closer to vk 422 00:25:53,930 --> 00:26:00,170 and constructing this shortest path at every pass. 423 00:26:00,170 --> 00:26:04,140 So at some point in the first pass, 424 00:26:04,140 --> 00:26:07,620 I'm going to relax this edge v0, v1. 425 00:26:07,620 --> 00:26:08,600 OK? 426 00:26:08,600 --> 00:26:14,640 And at that point, thanks to the optimal substructure property, 427 00:26:14,640 --> 00:26:16,800 given that this is the shortest path, 428 00:26:16,800 --> 00:26:19,380 this has to be a shortest path, as well. 429 00:26:19,380 --> 00:26:22,880 Any subset of the shortest path has to be a shortest path. 430 00:26:22,880 --> 00:26:25,150 I'm going to relax this edge and I'm 431 00:26:25,150 --> 00:26:31,100 going to get the value of delta from s to v1. 432 00:26:31,100 --> 00:26:33,430 And it's going to be this relaxation that's 433 00:26:33,430 --> 00:26:34,861 going to get me that value. 434 00:26:34,861 --> 00:26:37,360 And after the first pass, I'm going to be able to get to v1. 435 00:26:37,360 --> 00:26:39,420 After the second pass, I can get to v2. 436 00:26:39,420 --> 00:26:42,220 And after k passes, I'm going to be able to get to vk. 437 00:26:42,220 --> 00:26:47,100 So I'm just growing this frontier one node every pass. 438 00:26:47,100 --> 00:26:49,380 And that's your induction. 439 00:26:49,380 --> 00:26:51,580 And you can write that out. 440 00:26:51,580 --> 00:26:53,660 And I'll write it out here. 441 00:26:53,660 --> 00:26:55,640 But that's basically it. 442 00:26:55,640 --> 00:27:10,640 So after one pass through all of the edges e, we have d of v1 443 00:27:10,640 --> 00:27:12,560 to be delta s v1. 444 00:27:15,890 --> 00:27:19,380 And the reason for this is because we'll relax. 445 00:27:25,310 --> 00:27:27,300 We're guaranteed to relax all the edges. 446 00:27:27,300 --> 00:27:33,820 And we'll relax the edge v0, v1 during this pass. 447 00:27:36,540 --> 00:27:39,480 And we can't find a shorter path than this path 448 00:27:39,480 --> 00:27:41,950 because, otherwise we'd violate the optimum substructure 449 00:27:41,950 --> 00:27:44,000 property. 450 00:27:44,000 --> 00:27:47,270 And that means that it's a contradiction 451 00:27:47,270 --> 00:27:50,370 that we selected a shortest path in the first place. 452 00:27:50,370 --> 00:27:57,070 So can argue that we have delta s v1 after the first pass. 453 00:27:57,070 --> 00:27:58,246 And this goes on. 454 00:27:58,246 --> 00:27:59,620 I'm going to write out this proof 455 00:27:59,620 --> 00:28:01,590 because I think it's important for you guys 456 00:28:01,590 --> 00:28:06,340 to see the full proof. 457 00:28:06,340 --> 00:28:08,600 But you can probably guess the rest at this point. 458 00:28:16,780 --> 00:28:18,320 After one pass, that's what you get. 459 00:28:18,320 --> 00:28:25,940 After two passes through e we have 460 00:28:25,940 --> 00:28:33,710 d v2 equals delta s v2 because in the second pass 461 00:28:33,710 --> 00:28:38,500 we're going to relax edge v1, v2. 462 00:28:46,949 --> 00:28:48,990 So it' a different edge that needs to be relaxed. 463 00:28:48,990 --> 00:28:51,400 But that's cool because I'm relaxing all the edges. 464 00:28:51,400 --> 00:28:53,740 And I'm going to be able to grow my frontier. 465 00:28:53,740 --> 00:28:57,020 I'm going to be able to compute delta s v2 and the end 466 00:28:57,020 --> 00:29:00,820 of my second pass and so on and so forth. 467 00:29:00,820 --> 00:29:13,450 So after k passes, we have d vk equals delta s vk. 468 00:29:16,100 --> 00:29:26,320 And if I run through v minus 1 passes, which 469 00:29:26,320 --> 00:29:37,070 is what I do in the algorithm, all reachable vertices 470 00:29:37,070 --> 00:29:38,570 have delta values. 471 00:29:41,410 --> 00:29:41,910 All right? 472 00:29:41,910 --> 00:29:44,232 That's basically it. 473 00:29:44,232 --> 00:29:44,815 Any questions? 474 00:29:51,220 --> 00:29:55,710 It's actually a simpler proof than the Dijkstra proof, 475 00:29:55,710 --> 00:29:57,455 which I just sketched last time. 476 00:29:57,455 --> 00:29:58,830 I'll just give you some intuition 477 00:29:58,830 --> 00:29:59,850 of the Dijkstra proof. 478 00:29:59,850 --> 00:30:04,590 It's probably a little too painful to do in a lecture. 479 00:30:04,590 --> 00:30:09,660 But this one is, as you can see, nice and clean and fits 480 00:30:09,660 --> 00:30:14,620 on two boards, which is kind of an important criterion here. 481 00:30:14,620 --> 00:30:16,370 So good. 482 00:30:16,370 --> 00:30:19,940 All right, so that takes care of the theorem. 483 00:30:19,940 --> 00:30:21,940 Hopefully you're all on board with the theorem. 484 00:30:21,940 --> 00:30:26,230 And one thing that we haven't done is talk about the check. 485 00:30:26,230 --> 00:30:31,180 So the argument with respect to the corollary 486 00:30:31,180 --> 00:30:36,490 bootstraps this particular argument for the theorem. 487 00:30:36,490 --> 00:30:44,110 But this requires the insight that if after v 488 00:30:44,110 --> 00:30:49,790 minus 1 passes, if you can find an edge that can be relaxed, 489 00:30:49,790 --> 00:30:50,980 well what does that mean? 490 00:31:03,494 --> 00:31:11,280 So at this point, let's say that I've done my v minus 1 passes 491 00:31:11,280 --> 00:31:19,725 and we find an edge that can be relaxed. 492 00:31:23,310 --> 00:31:39,210 Well, this means that the current shortest path from s 493 00:31:39,210 --> 00:31:47,200 to some vertex that is obviously reachable v 494 00:31:47,200 --> 00:31:56,370 is not simple once I've relaxed this edge because I 495 00:31:56,370 --> 00:32:01,286 have a repeated vertex. 496 00:32:08,000 --> 00:32:11,160 So that means it's not simple to have a repeated vertex that's 497 00:32:11,160 --> 00:32:13,140 the same as I found a cycle. 498 00:32:17,040 --> 00:32:19,940 And it's a negative weight cycle because I 499 00:32:19,940 --> 00:32:24,800 was able to relax the edge and reduce the weight after I 500 00:32:24,800 --> 00:32:28,121 added a vertex that cost a cycle. 501 00:32:28,121 --> 00:32:28,620 All right? 502 00:32:28,620 --> 00:32:32,237 So this cycle has to be negative weight. 503 00:32:32,237 --> 00:32:33,820 Found a cycle that is negative weight. 504 00:32:39,930 --> 00:32:40,794 All right. 505 00:32:40,794 --> 00:32:41,710 That's pretty much it. 506 00:32:46,200 --> 00:32:49,920 So it's, I guess, a painful algorithm 507 00:32:49,920 --> 00:32:52,780 from a standpoint of it's not particularly smart. 508 00:32:52,780 --> 00:32:54,340 It's just relaxing all of the edges 509 00:32:54,340 --> 00:32:56,560 a certain fixed number of times. 510 00:32:56,560 --> 00:33:02,940 And it just works out because you will find these cycles. 511 00:33:02,940 --> 00:33:06,302 And if you keep going, it's like this termination condition. 512 00:33:06,302 --> 00:33:08,760 What is neat is that I don't have the generic shortest path 513 00:33:08,760 --> 00:33:10,450 algorithm up there anymore. 514 00:33:10,450 --> 00:33:12,150 But in effect, what you're saying 515 00:33:12,150 --> 00:33:14,760 is after a certain number of passes, 516 00:33:14,760 --> 00:33:16,750 if you haven't finished, you can quit 517 00:33:16,750 --> 00:33:19,600 because you have found a negative cycle. 518 00:33:19,600 --> 00:33:22,710 So it's very similar to the generic shortest path 519 00:33:22,710 --> 00:33:23,990 algorithm. 520 00:33:23,990 --> 00:33:25,600 You're not really selecting the edges. 521 00:33:25,600 --> 00:33:28,080 You're selecting all of them, in this case. 522 00:33:28,080 --> 00:33:31,310 And you're running through a bunch of different passes. 523 00:33:31,310 --> 00:33:34,260 All right? 524 00:33:34,260 --> 00:33:36,770 So that's it with respect to Bellman-Ford. 525 00:33:36,770 --> 00:33:38,800 I want to do a couple of special cases 526 00:33:38,800 --> 00:33:41,740 and revisit the directed acyclic graph. 527 00:33:41,740 --> 00:33:45,340 But stop me here if you have any questions about Bellman-Ford. 528 00:33:48,270 --> 00:33:49,989 You first and then back there. 529 00:33:49,989 --> 00:33:50,489 Yeah? 530 00:33:50,489 --> 00:33:52,132 AUDIENCE: Maybe I'm just confused 531 00:33:52,132 --> 00:33:54,749 about the definition of a cycle. 532 00:33:54,749 --> 00:33:57,123 But if you had, like, a tree, which had a negative weight 533 00:33:57,123 --> 00:33:58,956 edge, wouldn't it produce the same situation 534 00:33:58,956 --> 00:34:01,360 where you relaxed that edge. 535 00:34:01,360 --> 00:34:03,840 PROFESSOR: But you would have relaxed that edge previously. 536 00:34:03,840 --> 00:34:04,810 AUDIENCE: But it wouldn't be a cycle, right? 537 00:34:04,810 --> 00:34:06,750 PROFESSOR: Yeah, it wouldn't be a cycle. 538 00:34:06,750 --> 00:34:07,666 So let's look at that. 539 00:34:07,666 --> 00:34:08,794 That's a fine question. 540 00:34:08,794 --> 00:34:10,418 AUDIENCE: Doesn't that make assumptions 541 00:34:10,418 --> 00:34:12,529 about this structure? 542 00:34:12,529 --> 00:34:15,239 PROFESSOR: Well if you had a tree-- I mean, 543 00:34:15,239 --> 00:34:16,909 a tree is a really simple case. 544 00:34:16,909 --> 00:34:20,340 But if you had something like this 545 00:34:20,340 --> 00:34:25,610 and if you did have a minus 1 edge here, 546 00:34:25,610 --> 00:34:27,770 right-- we'll do a more complicated example. 547 00:34:27,770 --> 00:34:29,520 But let's say you had something like this. 548 00:34:29,520 --> 00:34:31,179 2 3 minus 1. 549 00:34:31,179 --> 00:34:35,070 And what will happen is if this happens to be your s vertex 550 00:34:35,070 --> 00:34:38,949 and in the first step you relax all the edges. 551 00:34:38,949 --> 00:34:41,530 And this one would get two. 552 00:34:41,530 --> 00:34:45,380 And then, depending on the order in which you relaxed, 553 00:34:45,380 --> 00:34:50,199 it's quite possible that if you relax this edge first-- 554 00:34:50,199 --> 00:34:56,070 let's say in the first pass the ordering of the relaxation 555 00:34:56,070 --> 00:34:59,570 is 1, 2, and 3. 556 00:34:59,570 --> 00:35:02,934 So the edges are ordered in a certain way each time, 557 00:35:02,934 --> 00:35:05,100 and you're going to be relaxing the edges in exactly 558 00:35:05,100 --> 00:35:06,990 the same order each time. 559 00:35:06,990 --> 00:35:07,940 All right? 560 00:35:07,940 --> 00:35:08,829 It doesn't matter. 561 00:35:08,829 --> 00:35:10,745 The beauty of Bellman-Ford is that-- let's say 562 00:35:10,745 --> 00:35:11,950 you relax this edge. 563 00:35:11,950 --> 00:35:14,010 Initially, this is at infinity. 564 00:35:14,010 --> 00:35:15,120 So this is at 0. 565 00:35:15,120 --> 00:35:16,030 This is at infinity. 566 00:35:16,030 --> 00:35:16,930 This is at infinity. 567 00:35:16,930 --> 00:35:18,800 This is at infinity. 568 00:35:18,800 --> 00:35:21,420 If you relax this edge, nothing happens. 569 00:35:21,420 --> 00:35:22,920 All right? 570 00:35:22,920 --> 00:35:25,830 Then you relax, let's say, this edge because that's number two. 571 00:35:25,830 --> 00:35:27,820 This gets set to two. 572 00:35:27,820 --> 00:35:30,190 You relax this edge because that's 3. 573 00:35:30,190 --> 00:35:31,910 And this is infinity so nothing happens. 574 00:35:31,910 --> 00:35:34,480 Of course, this is already at two so nothing would happen. 575 00:35:34,480 --> 00:35:38,420 So the end of the first pass, what you have is this is 0. 576 00:35:38,420 --> 00:35:39,330 That's 2. 577 00:35:39,330 --> 00:35:41,060 This is still infinity. 578 00:35:41,060 --> 00:35:42,770 That's still infinity. 579 00:35:42,770 --> 00:35:43,270 OK? 580 00:35:43,270 --> 00:35:45,603 That's going to stay infinity because you can't reach it 581 00:35:45,603 --> 00:35:46,180 from s. 582 00:35:46,180 --> 00:35:47,800 So we can, sort of, ignore that. 583 00:35:47,800 --> 00:35:51,220 And then, of the second pass, what you have is 584 00:35:51,220 --> 00:35:55,420 you start with this edge again because that's the ordering. 585 00:35:55,420 --> 00:36:01,240 And this 2 minus 1 would give this a 1. 586 00:36:01,240 --> 00:36:04,720 And then you relax this edge or try to relax this edge. 587 00:36:04,720 --> 00:36:06,062 Nothing happens. 588 00:36:06,062 --> 00:36:07,020 Try to relax this edge. 589 00:36:07,020 --> 00:36:08,460 Nothing happens. 590 00:36:08,460 --> 00:36:10,320 And at this point, you have one more pass 591 00:36:10,320 --> 00:36:12,110 to go because you got 4 vertices. 592 00:36:12,110 --> 00:36:15,529 And in that past, nothing changes again. 593 00:36:15,529 --> 00:36:16,820 So that's what you end up with. 594 00:36:16,820 --> 00:36:21,000 You end up with 2 for this and 1 for that. 595 00:36:21,000 --> 00:36:22,260 OK? 596 00:36:22,260 --> 00:36:25,200 That makes sense? 597 00:36:25,200 --> 00:36:27,330 So the important thing to understand 598 00:36:27,330 --> 00:36:28,980 is that you are actually relaxing 599 00:36:28,980 --> 00:36:31,872 all of the edges in every pass. 600 00:36:31,872 --> 00:36:33,830 And there's a slightly more complicated example 601 00:36:33,830 --> 00:36:35,910 than this that is in the notes. 602 00:36:35,910 --> 00:36:38,210 And you can take a look at that offline. 603 00:36:38,210 --> 00:36:40,120 There's another question in the back. 604 00:36:40,120 --> 00:36:42,050 Did you have a question? 605 00:36:42,050 --> 00:36:43,330 Someone raised their hand. 606 00:36:43,330 --> 00:36:43,520 Yeah? 607 00:36:43,520 --> 00:36:44,999 AUDIENCE: Yes, I'm just curious-- 608 00:36:44,999 --> 00:36:47,628 is there a unknown better algorithm that 609 00:36:47,628 --> 00:36:49,350 can do the same thing? 610 00:36:49,350 --> 00:36:51,350 PROFESSOR: No, there's no known better algorithm 611 00:36:51,350 --> 00:36:53,670 for solving the general case like this. 612 00:36:53,670 --> 00:36:59,160 There are a couple of algorithms that assume weights 613 00:36:59,160 --> 00:37:00,400 within a certain range. 614 00:37:00,400 --> 00:37:04,130 And then there complexities include both v and e, as well 615 00:37:04,130 --> 00:37:08,250 as w where w is the dynamic range of the weights. 616 00:37:08,250 --> 00:37:12,125 And depending on what w is, you could 617 00:37:12,125 --> 00:37:13,750 argue that they have better complexity. 618 00:37:13,750 --> 00:37:15,800 But they're kind of incomparable in the sense 619 00:37:15,800 --> 00:37:17,670 that they have this extra parameter, which 620 00:37:17,670 --> 00:37:18,919 is the dynamic range of the w. 621 00:37:18,919 --> 00:37:20,650 OK? 622 00:37:20,650 --> 00:37:22,430 Now there's lots of special cases, 623 00:37:22,430 --> 00:37:25,150 like I said, and well take a look at the DAG special case 624 00:37:25,150 --> 00:37:28,490 in a second where you could imagine doing better 625 00:37:28,490 --> 00:37:32,040 but not for the case where you have an arbitrary graph that 626 00:37:32,040 --> 00:37:34,820 could have negative cycles in it because it's got negative rate 627 00:37:34,820 --> 00:37:36,950 edges. 628 00:37:36,950 --> 00:37:37,747 Yeah? 629 00:37:37,747 --> 00:37:39,580 AUDIENCE: In the corollary, does that assume 630 00:37:39,580 --> 00:37:42,041 you have a connected graph because, you know, 631 00:37:42,041 --> 00:37:44,447 you could have a negative weight edge 632 00:37:44,447 --> 00:37:47,158 in a separate part of the graph, which 633 00:37:47,158 --> 00:37:48,646 isn't reachable from this. 634 00:37:51,622 --> 00:37:52,540 PROFESSOR: Yeah. 635 00:37:52,540 --> 00:38:03,420 So you're going to start when you have an undefined weight. 636 00:38:03,420 --> 00:38:06,180 Remember your initialization condition. 637 00:38:06,180 --> 00:38:08,290 What is affected by s? 638 00:38:08,290 --> 00:38:10,760 Initialize is affected by s. 639 00:38:10,760 --> 00:38:12,450 The rest of it isn't affected by s 640 00:38:12,450 --> 00:38:14,810 because you're just relaxing the edges. 641 00:38:14,810 --> 00:38:17,310 Initialize is affected by s because d of s 642 00:38:17,310 --> 00:38:20,740 starts out being 0, like I put over here, and the rest of them 643 00:38:20,740 --> 00:38:22,910 are infinity. 644 00:38:22,910 --> 00:38:26,500 So there is an effect of the choice of the starting vertex. 645 00:38:26,500 --> 00:38:30,240 And the rest of it follows that you 646 00:38:30,240 --> 00:38:33,820 will get an undefined value, or you 647 00:38:33,820 --> 00:38:37,220 will find that negative cycle exists 648 00:38:37,220 --> 00:38:39,500 based on whether you can reach it from s or not. 649 00:38:39,500 --> 00:38:42,800 So if you happen to have s over here, 650 00:38:42,800 --> 00:38:45,090 and it's just the one node, and then 651 00:38:45,090 --> 00:38:49,870 it has no edges going out of it, this algorithm 652 00:38:49,870 --> 00:38:51,660 would just be trivial. 653 00:38:51,660 --> 00:38:54,070 But it wouldn't detect any negative cycles 654 00:38:54,070 --> 00:38:55,960 that aren't reachable from s. 655 00:38:55,960 --> 00:38:56,725 That make sense? 656 00:38:56,725 --> 00:38:58,600 AUDIENCE: Yeah. 657 00:38:58,600 --> 00:39:01,410 PROFESSOR: So there is this-- it's kind of hidden over there. 658 00:39:01,410 --> 00:39:02,920 So I'm glad you asked that question. 659 00:39:02,920 --> 00:39:05,410 But initialize is setting things up. 660 00:39:05,410 --> 00:39:08,060 And that is something that affects 661 00:39:08,060 --> 00:39:10,190 the rest of the algorithm because d of s is 0 662 00:39:10,190 --> 00:39:12,410 and the rest of them are set to infinity. 663 00:39:12,410 --> 00:39:14,540 All right? 664 00:39:14,540 --> 00:39:19,330 So if there are no other questions, 665 00:39:19,330 --> 00:39:21,870 I'll move on to the case of the DAG 666 00:39:21,870 --> 00:39:24,750 and talk a little bit about shortest paths versus longest 667 00:39:24,750 --> 00:39:25,810 paths. 668 00:39:25,810 --> 00:39:28,710 And this is somewhat of a preview of a lecture 669 00:39:28,710 --> 00:39:34,010 that Eric is going to give a month from now on complexity 670 00:39:34,010 --> 00:39:36,810 and the difference between polynomial time and exponential 671 00:39:36,810 --> 00:39:39,910 time, though I'm not going to go into much depth here. 672 00:39:39,910 --> 00:39:42,630 But there's some interesting relationships 673 00:39:42,630 --> 00:39:46,950 between the shortest path problem and the longest path 674 00:39:46,950 --> 00:39:49,400 problem that I'd like to get to. 675 00:39:49,400 --> 00:39:54,140 But any other questions on this? 676 00:39:54,140 --> 00:39:55,750 OK, so let me ask a question. 677 00:39:58,940 --> 00:40:04,270 Suppose I wanted to find longest paths in a graph 678 00:40:04,270 --> 00:40:09,960 and let's say that this graph had all positive edge weights. 679 00:40:09,960 --> 00:40:12,240 OK. 680 00:40:12,240 --> 00:40:17,320 What if I negated all of the edge weights 681 00:40:17,320 --> 00:40:19,840 and ran a Bellman-Ford? 682 00:40:23,110 --> 00:40:26,930 Would I find the longest path in the graph? 683 00:40:29,174 --> 00:40:30,590 Do people understand the question? 684 00:40:35,230 --> 00:40:35,980 I don't need this. 685 00:40:45,500 --> 00:40:48,810 So maybe we can talk about what a longest path means first. 686 00:40:57,150 --> 00:41:03,180 So if this was s and this v1, v2, v3, fairly straightforward, 687 00:41:03,180 --> 00:41:05,700 you know how to compute shortest paths now. 688 00:41:05,700 --> 00:41:06,900 These are all positive. 689 00:41:06,900 --> 00:41:08,450 Even easier. 690 00:41:08,450 --> 00:41:13,120 The longest path to v3 is of length. 691 00:41:13,120 --> 00:41:16,180 Six because I go here, go there, and go there, right? 692 00:41:16,180 --> 00:41:18,050 So that's my longest path. 693 00:41:18,050 --> 00:41:20,440 OK? 694 00:41:20,440 --> 00:41:23,930 And the shortest path to v3 is of length 4. 695 00:41:23,930 --> 00:41:29,080 So shortest path, longest paths, have these nice duality. 696 00:41:29,080 --> 00:41:32,536 What if I said, well, you know, I 697 00:41:32,536 --> 00:41:34,630 can solve the longest path problem, as well, 698 00:41:34,630 --> 00:41:37,440 given all of what I've learned about shortest paths 699 00:41:37,440 --> 00:41:49,750 simply by negating each of these edges and running Bellman-Ford. 700 00:41:49,750 --> 00:41:52,270 What would happen? 701 00:41:52,270 --> 00:41:52,770 Yeah? 702 00:41:52,770 --> 00:41:59,244 AUDIENCE: [INAUDIBLE] shortest path branch [INAUDIBLE] 703 00:41:59,244 --> 00:42:02,730 values, and if you switched to absolute value, 704 00:42:02,730 --> 00:42:05,220 it will give you the longest path. 705 00:42:05,220 --> 00:42:06,692 PROFESSOR: So you think it works? 706 00:42:06,692 --> 00:42:07,317 AUDIENCE: Yeah. 707 00:42:07,317 --> 00:42:09,435 It will also check the cycles. 708 00:42:09,435 --> 00:42:12,760 So the negative cycles will be the longest path cycles that 709 00:42:12,760 --> 00:42:13,710 [INAUDIBLE]. 710 00:42:13,710 --> 00:42:16,840 PROFESSOR: But I think that's the key question. 711 00:42:16,840 --> 00:42:19,550 What will Bellman-Ford do when it is run on this? 712 00:42:19,550 --> 00:42:22,492 What would it return? 713 00:42:22,492 --> 00:42:24,445 AUDIENCE: [INAUDIBLE]. 714 00:42:24,445 --> 00:42:26,320 PROFESSOR: No, what will Bellman-Ford return? 715 00:42:26,320 --> 00:42:28,390 I'm asking. 716 00:42:28,390 --> 00:42:31,000 Someone else? 717 00:42:31,000 --> 00:42:33,830 What will Bellman-Ford return if I ran this? 718 00:42:33,830 --> 00:42:34,840 Undefined. 719 00:42:34,840 --> 00:42:35,910 Right? 720 00:42:35,910 --> 00:42:39,574 Undefined because you got this negative weight cycle here. 721 00:42:39,574 --> 00:42:40,490 AUDIENCE: [INAUDIBLE]. 722 00:42:40,490 --> 00:42:41,540 PROFESSOR: Sorry? 723 00:42:41,540 --> 00:42:43,420 Oh! 724 00:42:43,420 --> 00:42:45,920 Let's put another one in there. 725 00:42:45,920 --> 00:42:47,110 Oops, sorry. 726 00:42:47,110 --> 00:42:47,610 Now I see. 727 00:42:47,610 --> 00:42:48,151 You're right. 728 00:42:48,151 --> 00:42:49,070 You're right. 729 00:42:49,070 --> 00:42:49,861 I'm wrong. 730 00:42:49,861 --> 00:42:51,110 And why did you say undefined? 731 00:42:51,110 --> 00:42:52,595 AUDIENCE: I was wrong. 732 00:42:52,595 --> 00:42:54,080 PROFESSOR: OK, good. 733 00:42:54,080 --> 00:42:55,560 I got company. 734 00:42:55,560 --> 00:42:56,060 Thank you. 735 00:42:56,060 --> 00:42:58,020 Thank you. 736 00:42:58,020 --> 00:42:58,520 Good. 737 00:42:58,520 --> 00:43:01,020 Let's take it all over again. 738 00:43:01,020 --> 00:43:01,911 All over again. 739 00:43:01,911 --> 00:43:02,410 All right. 740 00:43:10,570 --> 00:43:12,500 All right, start over. 741 00:43:12,500 --> 00:43:15,990 s v1 v2 v3. 742 00:43:18,950 --> 00:43:20,630 Yeah, that is a cycle. 743 00:43:20,630 --> 00:43:23,461 All right, good. 744 00:43:23,461 --> 00:43:23,960 Cycle. 745 00:43:27,640 --> 00:43:33,580 So when you actually negate each of these edges, 746 00:43:33,580 --> 00:43:37,270 you end up with a negative weight cycle. 747 00:43:37,270 --> 00:43:40,250 So it's plausible that you could have a graph like this one 748 00:43:40,250 --> 00:43:44,800 where this strategy won't work because what would happen 749 00:43:44,800 --> 00:43:50,510 is Bellman-Ford would come back with, essentially, an abort 750 00:43:50,510 --> 00:43:53,160 that says I can't compute shortest paths because they're 751 00:43:53,160 --> 00:43:54,500 undefined. 752 00:43:54,500 --> 00:43:55,190 All right? 753 00:43:55,190 --> 00:44:00,060 Now it turns out it's actually more subtle than that. 754 00:44:00,060 --> 00:44:04,340 What we're trying to do in Bellman-Ford is, in the case 755 00:44:04,340 --> 00:44:11,770 where negative weight cycles don't exist, 756 00:44:11,770 --> 00:44:15,350 we report on the shortest simple path. 757 00:44:18,870 --> 00:44:20,860 That's the whole notion of the proof. 758 00:44:20,860 --> 00:44:23,460 We say that the path has a certain length, which 759 00:44:23,460 --> 00:44:27,290 is, at most, v minus 1 and so on and so forth. 760 00:44:27,290 --> 00:44:28,800 We get the shortest simple path. 761 00:44:34,620 --> 00:44:48,310 But if you actually have a problem where you say-- let 762 00:44:48,310 --> 00:44:49,310 me start over again. 763 00:44:49,310 --> 00:44:52,090 Let's say I want to find the shortest simple path 764 00:44:52,090 --> 00:44:56,380 for a different graph and it happens 765 00:44:56,380 --> 00:45:01,280 to have a negative weight cycle in it. 766 00:45:01,280 --> 00:45:02,630 So I have something like this. 767 00:45:02,630 --> 00:45:10,240 2 3 minus 6, 3 over here, 3 over here, and so on. 768 00:45:10,240 --> 00:45:11,750 Maybe 2 here. 769 00:45:11,750 --> 00:45:19,430 And I want to find the shortest simple path that 770 00:45:19,430 --> 00:45:23,250 reaches v from s. 771 00:45:23,250 --> 00:45:24,110 OK? 772 00:45:24,110 --> 00:45:27,670 What is the shortest simple path that reaches v from s? 773 00:45:27,670 --> 00:45:29,890 It's this path that goes horizontally, 774 00:45:29,890 --> 00:45:32,820 which has a weight 3 plus 2, 5, 5 775 00:45:32,820 --> 00:45:38,470 plus 3, 8, 8 plus 3, 11, 11 plus 2, 13. 776 00:45:38,470 --> 00:45:39,240 All right? 777 00:45:39,240 --> 00:45:48,580 So the shortest simple path is 13. 778 00:45:48,580 --> 00:45:52,095 Will Bellman-Ford give you any information about this path? 779 00:45:54,594 --> 00:45:55,510 AUDIENCE: [INAUDIBLE]. 780 00:45:55,510 --> 00:45:57,910 PROFESSOR: No because in [INAUDIBLE]. 781 00:45:57,910 --> 00:46:06,200 After it does its v minus 1 passes, v is reachable from s. 782 00:46:06,200 --> 00:46:11,480 But you potentially go through a negative weight cycle 783 00:46:11,480 --> 00:46:14,700 before you reach v. OK? 784 00:46:14,700 --> 00:46:18,550 So it turns out that if you have a graph with negative weight 785 00:46:18,550 --> 00:46:22,470 cycles, finding the shortest simple path 786 00:46:22,470 --> 00:46:24,520 is an NP-hard problem. 787 00:46:24,520 --> 00:46:26,240 It's a really hard problem. 788 00:46:26,240 --> 00:46:27,680 That's what NP means. 789 00:46:27,680 --> 00:46:30,420 No, it means something else that Eric 790 00:46:30,420 --> 00:46:33,550 will explain to you in a month or so. 791 00:46:33,550 --> 00:46:36,600 But it means that we don't know any algorithm that 792 00:46:36,600 --> 00:46:40,500 is better than exponential time to solve this problem. 793 00:46:40,500 --> 00:46:42,140 OK? 794 00:46:42,140 --> 00:46:48,620 So amazingly, all you've done is taken the shortest path problem 795 00:46:48,620 --> 00:46:51,170 and changed it ever so slightly. 796 00:46:51,170 --> 00:46:53,610 You said I want to look for the shortest simple path 797 00:46:53,610 --> 00:46:56,970 in the general case where I could, potentially, 798 00:46:56,970 --> 00:47:00,770 have negative weight cycles in my graph. 799 00:47:00,770 --> 00:47:03,370 And when you do that, all bets are off. 800 00:47:03,370 --> 00:47:06,470 You're not in the polynomial time complexity domain anymore. 801 00:47:06,470 --> 00:47:08,630 At least, not that we know of. 802 00:47:08,630 --> 00:47:13,045 And the best that you can do is an exponential time algorithm 803 00:47:13,045 --> 00:47:15,430 to find shorter simple paths. 804 00:47:15,430 --> 00:47:17,370 And this problem, as it turns out, 805 00:47:17,370 --> 00:47:24,850 is equivalent to the longest path problem in the sense 806 00:47:24,850 --> 00:47:26,700 that they're both NP-hard. 807 00:47:26,700 --> 00:47:29,840 If you can solve one, you could solve the other. 808 00:47:29,840 --> 00:47:33,050 So to summarize, what happens here simply 809 00:47:33,050 --> 00:47:36,140 is that in the case of Bellman-Ford 810 00:47:36,140 --> 00:47:41,910 running on the original shortest path problem, 811 00:47:41,910 --> 00:47:46,650 you're allowed to abort when you detect the fact that there's 812 00:47:46,650 --> 00:47:48,640 a negative cycle. 813 00:47:48,640 --> 00:47:50,880 So given that you're allowed to abort when there's 814 00:47:50,880 --> 00:47:55,090 a negative cycle, you have a polynomial time solution 815 00:47:55,090 --> 00:47:58,760 using Bellman-Ford that is not necessarily 816 00:47:58,760 --> 00:48:02,980 going to give you shortest path weights but will 817 00:48:02,980 --> 00:48:06,120 in the case of no negative cycles. 818 00:48:06,120 --> 00:48:07,610 All right? 819 00:48:07,610 --> 00:48:11,770 But if you ask for more-- a little bit more-- you said, 820 00:48:11,770 --> 00:48:14,040 you know, it'd be great if you could somehow 821 00:48:14,040 --> 00:48:18,080 process these negative cycles and tell me 822 00:48:18,080 --> 00:48:23,140 that if I had a simple path and I don't go through cycles what 823 00:48:23,140 --> 00:48:25,030 would the shortest weight be, it becomes 824 00:48:25,030 --> 00:48:27,660 a much more difficult problem. 825 00:48:27,660 --> 00:48:29,450 It goes from order of ve complexity 826 00:48:29,450 --> 00:48:34,260 to exponential time complexity to the best of our knowledge. 827 00:48:34,260 --> 00:48:36,770 So that's what I'd like to leave you with. 828 00:48:36,770 --> 00:48:41,160 That there's much more to algorithms than just the ones 829 00:48:41,160 --> 00:48:42,230 that we're looking at. 830 00:48:42,230 --> 00:48:43,730 And we get a little bit of a preview 831 00:48:43,730 --> 00:48:46,120 of this-- so the difference between polynomial time 832 00:48:46,120 --> 00:48:49,510 an exponential time-- later on in the term.