Session 21: Dynamic Connectivity Lower Bound

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality, educational resources for free. To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

ERIK DEMAINE: All right, welcome to my last lecture for the semester. We finish our coverage of dynamic graphs, and also our coverage of lower bounds. We saw one big lower bound in this class in the cell probe model. You may recall cell probe model, you just count how many cells of memory do you touch. You want to prove a lower bound on that.

And today we're going to prove a cell probe from lower bound on dynamic connectivity, which is a problem we've solved a few different times. Our lower bound will apply even when each of the connected components of your graph are just a path. And so in particular, they imply matching lower bounds for dynamic trees.

So here is the theorem we'll be proving today. You want to insert and delete edges, and do connectivity queries between pairs of vertices, vw. I want to know is there a path from v to w, just like we've been considering. These require omega log n, time for operation.

This is the max of updating, query times has to be at least log n time per operation, even if the connected components are paths, and even amortized, and even randomized. Although I'm not going to prove of all of these versions, I won't prove the amortized version. I'm going to prove a worst case log and lower bound, it's just a little more work to prove an amortized lower bound. But same principles.

And so that's going to be today, is proving this theorem. It's not a short proof, but it combines a bunch of relatively simple ideas, and ends up being pretty clean overall, piece-by-piece, but there's just a bunch of pieces, as we will get to. Key concept is an idea introduced in this paper, which is to build a balanced binary tree over time, over your access sequence. And argue about different subtrees within that tree.

This is a paper that maybe came out of this class, in some sense, it was by Mihai Patrascu and myself. Back when Mihai was an undergrad, I think he'd just taken this class. But at that point the class didn't even cover dynamic connectivity, so-- and time here is cell probes.

So this is a very strong model, it implies a lower bound on ram, and implies a lower bound on pointer machine. We know matching upper bounds for trees on a pointer machine link/cut trees in [INAUDIBLE] to our trees. It's kind of fun that this lower bound even applies to paths, because most of the work in link/cut trees is about decomposing your tree into paths.

And so what this is saying is even if that's done for you, and you just need to be able to take paths, and concatenate them together by adding edges, then maintaining the find root property so that you can do connectivity queries, even that requires log n time. So converting a tree into a path is basically free, the hard part is maintaining the paths.

So let's prove a theorem. The lower bound, we get to choose what access sequence we think is bad. And so we're going to come up with a particular style of graph, which looks like the following.

Graph is going to be-- the vertices are going to be a root n by root n grid. And we're going to-- these guys are in, what did I call them? Groups? Columns. Columns is a good name. These are columns of the matrix, or vertices. And what I'd like to have is between consecutive columns, I want to have a perfect matching between these vertices.

So could be, I don't know, this edge, this edge, this edge, and that edge. And I also want a perfect match between these two columns, so maybe this one, this one, this one, this one. And you can have some boring things too. Something like that. So between every pair of columns is a perfect matching, meaning perfect pairing.

OK, this of course results in a collection of paths, square root of n paths. You can start at any vertex on the left, and you'll have a unique way to go to the right. And so that's path 1, this is path 2, this is path 3, and this is path 4. And so if I-- an interesting query. Well, an interesting query is something like I want to know, is this vertex connected to this one? And it's not so easy to figure it out, because you have to sort of walk through this path to figure that out.

We're going to think of each of these perfect matchings as defining a permutation on the vertices, on the column really. So you start with the identity permutation, and then some things get swapped around, that's pi 1. Something gets swapped around again, that's pi 2. Somethings get swapped around here, that's pi 3. And then this position would be pi 3 of pi 2 of pi 1 of vertex 4. We call this vertex 4. Or row, row 4.

So in some sense, we have to compose permutations. I'll call this pi 1, circle pi 2, circle pi 3 of 4. And we're going to show, basically, composing permutations is tough when you can change those permutations dynamically. So what we're going to do is a series of block operations, which change or query entire permutations.

So here, an update is going to be a whole bunch of insertions and deletions of edges. Basically, what we want to do is set pi i equal to pi. So that's what update of i comma pi does. It changes an entire perfect matching to be a specified permutation. So how do you do that? Well, you delete all the edges that are in the existing permutation, then you insert all the new edges. So this can be done in square root of n edge deletions and insertions.

So it's a bulk update of square root of n operations. And so this could only make our problem easier, because we're given square root of n updates that we all need to do at once. So you could amortize over them, you could do lots of different things, but we're sure that won't help. And then we have a query, and the query is going to be a little bit weird, and it's also going to make the proof a little bit more awkward.

But what it asks is if I look at the composition of pi j, from 1 up to i. This is 1. So I want to know is that composition equal to pi? Yes or no? This is what I'll call verify sum, sum meaning composition. But the sum terminology comes from a different problem, which we won't talk about directly here, called partial sums.

Partial sums is basically this problem, you can change numbers in an array, and you can compute the prefix sum from 1 up to i. Here we're not computing it. Why are we not computing it? Because actually figuring out what pi 3, or pi 2 of pi 1 is of something is tricky in this setting. The operations we're given are, given two vertices, are they connected by a path?

So to figure out the other end of this path, that requires-- I mean, it's hard to figure out where the other end is. If I told you is it this one? Then I can answer that question with just a connectivity query. So verify sum can be done with order square root of n connectivity queries. Whereas computing the sum could not be, as far as we know. If I tell you what that composition is supposed to be, I can check does 4 go to 1? Yes or no? Does 3 go to 3? Yes or no? Does 2 go to 4? Yes or no? Does 1 go to 2? Yes or no?

So with 4 queries, I can check whether the permutation is what it is. If any of those fail, then I return no. So the way this proceeds is first we proved a lower bound on partial sums, which is computing this value when you're not told what the answer is. And then we extended that, and we'll do such a proof here today.

First, we're going to prove a lower bound on the sum operation, which is computing this value that's on our outline over here, sum lower bound. And then we'll extend that and make the argument a little bit more complicated, then we'll get an actual connectivity lower bound, a lower bound on verify sum.

OK, but obviously if we can prove that these operations take a long time to do, we can prove that these original operations take a long time to do. So what we claim is that square root of n updates, these block updates plus square of n verify some queries, require root n times root n log n cell probes. I guess this is the amortized claim.

So if I want to do root n updates and root n queries, and I take root n times root n times log n-- funny way of writing n log n-- cell probes, then if I divide through, I want the amortized lower bound, I lose one of these root ns, because I'm doing different operations. I lose another root n, because each of these updates corresponds to root n operations. Each of the verify sums corresponds to root n operations. So overall per operation, per original operation of edge deletion insertion or connectivity query, I'm paying log n per operation.

So if I can prove this claim, then I get this theorem. All clear? So now we've reduced the problem to these bulk operations, we'll just be thinking about the bulk operations, update verify sum. We won't think about edge deletion insertions and connectivity queries anymore.

OK. So this is just sort of the general set up of what the graphs are going to look like. And now I'm going to tell you what sequence of updates and verify sums we're actually going to do that are bad. This is the bad access sequence. And this is actually something we've seen before in lecture 6, I think, the binary search tree stuff.

We're going to look at the bit reversal sequence. So you may recall a bit reversal sequence. You take binary numbers in order, reverse the bits. So this becomes 000, 100, 010, 110, 001, 101, 011, and 111. So those are the reversed strings.

And then you reinterpret those as regular numbers. So this is 0, 4, 2, 6, and then it should be the same thing, but the odd version. So I have 1, 5, 3, 7.

OK, I claimed, I think probably didn't prove, that this bit reversal sequence has a high Wilber lower bound. And so any binary search tree accessing items in this order requires log n per operation. And we want log n per operation here, so it seems like a good choice, why not?

So we're going to follow this access sequence. And sorry, I've changed notation here, we're going to number the permutations from 0 to root n minus 1 now. And assume root n is a power of 2, so the bit reversal sequence is well defined. And then we are going to do two things for each such i. We're going to do a verify sum operation. Actually maybe it is starting at 1, I don't know. It doesn't matter. And then we'll do an update

OK, so let's see. This pi random is just a uniform random permutation, it's computed fresh every time. So we're just re-randomizing pi i in this operation. Before we do that, we're checking that the sum, the composition of all the permutations up to position i, is what it is. So this is the actual value here, and we're verifying that that is indeed the sum. So this will always return yes.

But data structure has to be correct. So it needs to really verify that that is the case. There's the threat that maybe we gave the wrong answer here, and it needs to double check that that is indeed the right answer. It may seem a little weird, but we'll see why it works.

So this is the bad access sequence. Just do a query, do an update in this weird order in i. OK, and big idea is to build a nice balanced binary tree over time. So we have on the ground here 0, 4, 2, 6, 1, 5, 3, 7. And when I write 5, I mean verify sum of 5, and update permutation 5. And then we can build a binary tree on that.

And for each node in this tree, we have the notion of a left subtree, and we have the notion of a right subtree. And cool thing about bit reversal sequence is this nice self-similarity. If you look at the left subtree of any node and the right subtree of any of node, those items interleave.

If you look at the sorted order, it's 1 on the left, 3 on the right, 5 on the left, 7 on the right. They always perfectly interleave, because this thing is designed to interleave at every possible level. So that's the fact we're going to use. We're going to analyze each node separately, and talk about what information has to be carried from the left subtree to the right subtree.

In particular, we're interested in the updates being done on the left subtree, because here we change pi 1, we change pi 5. And the query's being done on the right subtree, because here we query 3, we query 7. When we query 3, that queries everything, all the permutations, up to 3. It's a composition of all permutations up to 3. So in particular it involves 1. So the claim is going to be that the permutation that we set in 1 has to be carried over to this query. And similarly, a changing permutation 5 will affect the query for 7. Also query, the update for 1, will affect the query for 7. So we need to formalize that little bit.

So here is the claim. For every node in the tree, say it has l leaves in its subtree-- This should be a comma and this should be a colon. Here's what we say. During the right subtree of v, so right subtree corresponds to an interval of time. So we're talking about those operations done during the right subtree of v. Claim is we must do omega l root n cell probes-- sorry, expected cell probes.

We are using some randomness here, right? We said we're going to update each permutation to a random value, so we can only make claims about the expected performance. Fine. But that's actually a stronger thing, it implies a lower bound, even for randomized algorithms. So if you can randomize your input set. And then not just any cell probes, but they're cell probes that read cells last written during the left subtree.

So this is what I was saying at a high level before. We're looking at reads over here, to cells that are written over here. Because we claim the updates over here have to store some information that is-- whatever the updates that happen over here influence the queries that happen over here. So these queries have to read the data that was written over here.

And specifically, we're claiming at least l root n cell probes have to be read over here, from cells that were written-- that were basically just written in the left subtree. If we could prove this, then we get our other claim-- this one over here, that root n updates, and root n verifies sums that require this much time. The difference is-- well, here we have an l, for an l leaf tree. And so what I'd like to do is sum this lower bound over every node in the tree. I need to check that that is valid to do. So let's do that.

OK, for every node v, we are claiming there's a certain number of reads that happen over here, that correspond to writes over here. But let's say you look at the parent of v, which is over here. This thing is also in the right subtree, and we're claiming there's some number of reads on the right subtree, that read things that are written over on the left.

The worry would be that the reads we counted here, we also count at the next level up. We don't want to double count in our lower bounds. If we're able to sum them up, we can't be double counting. But the claim is we're not double counting, because if you look at any particular-- any read-- so here's time, and suppose you do a read here. You're reading a cell that was written sometime in the past, if it was never written, it's a not very interesting read, it communicates no information.

So there's some write in the past that changed the cell that's just read. And we are going to count this read at a particular node, namely the lca of those two times. So if you look at the lca of the times of the reads and the writes, that is the single note that we'll think about that read that happened in the right subtree, that was written in the left subtree. So no double counting, because we only count at the lca.

The other thing that we need to be able to do is, because this is an expected lower bound, we need linearity of expectation. But expectation is indeed linear, so we're all set. OK, so all that's left is a little bit of common [INAUDIBLE] if we take l root n, where l is the size of the subtree below a given node, we sum that up over all nodes, and it's a balanced binary search tree-- or a balanced binary tree, I should say, not a search tree.

What do we get? Well, every leaf appears in log n subtrees. So we get the total size of the tree times log n for this, and we get another root n over here. The total size of the tree is root n. So we get this root n log n, that's when you sum up the l part. Then everything gets multiplied by root n, and that becomes our lower bound, and that's exactly what we need over here. So now this claim is done. Maybe I should do a check mark. Provided we can prove this claim.

So now our goal is to prove this thing. And now we're in a more local world, looking at a single node, counting reads over here, the corresponding rights over there. And then you just add up those lower bounds, you get what you want. So this is where the log comes from, because it's a balanced tree. And there's log n levels in a balanced tree, that's where we're getting our lower bound. The root n's are just keeping track of the size of the things we're manipulating. All right. So it remains to prove this claim. Prove that claim, we get that claim, and then we get this theorem.

So proof of claim. We're going to do an information theoretic argument, so let me set it up. It's again, it's making this claim I said before, that the permutations that get written over here somehow have to be communicated to the queries over here, because they matter. Because the permutations that get said over here changed the answers to all the queries over here, because of the interleaving between left and right.

So how are we going to formalize that? Well, left subtree does l/2 updates with l/2 random permutations, uniform random permutations, because every node does an update. And so the information theoretic idea is that if we were to somehow encode those permutations,

That encoding must use omega l log l-- l? No, I'm sorry. It's not right. Off by some root n factors here, l root n log n. OK, each permutation must take root n log root n bits to encode. If you have a random permutation, expected number of bits have a very high probability. Almost every permutation requires root n log root n bits. I'm not going to worry about constant factors, put an omega here, so the root n turns into an n. And then we've got l over two of them, so again, ignoring constant factors, that's l root n log n bits.

And this is just information, theoretic fact, our common [INAUDIBLE] theory fact. And once we know that, the idea is let's find an encoding that's better than this, and get a contradiction. Of course we shouldn't get a contradiction unless this claim is false. So either this claim is true and we're happy, but if somehow the word not enough cell reads on the right, that did things that were written on the left, then we will, from that, get a smaller encoding of the update permutations that happen on the left. If we could somehow do that, then we can get a contradiction, and therefore conclude the claim is in fact true.

So, if the claim fails, we'll find a smaller encoding, which will give us a contradiction. All right, so let's set up this problem a little bit more. I'm going to-- because we're really just interested in this subtree v stuff on the left, stuff on the right, but this of course lives in a much bigger tree, there's stuff that happens over here. This I will call the past. I'm just going to assume we know everything about the past.

Everything to the left of the subtree, we can assume that we know. When I say we know, what do we know? We know all the updates, we know all the queries that happen, and we know, at this moment in particular, what is the state of the data structure. Because this claim has nothing to do with this stuff, it's all about reads here that corresponds to writes here. So we can just assume we know everything up to this point. In our encoding, this is a key point.

One way to say this in a probabilistic sense is we're conditioning on what happened over here on the left, what updates happened. And if we can prove that whatever we need to happen here holds no matter what the condition is, then it will hold overall. So that's probabilistic justification for why we can assume we know the past OK. So then our goal is to encode, this is a little bit different from this goal.

What we really want to do is encode the update permutations on the left. That's a little awkward to think about, because this is a claim about how many probes happen on the right. So instead, what we're going to do is encode the query permutations on the right. So there are updates over here, that's what we want to encode, but we're instead going to encode the queries over here. I claim if you know what the results of the queries were over here, then you know what the updates were over there. Basically because of this interleaving property. So I can write that down a little more formally.

So if we look at time here, over, let's say this is v's subtree. Then what we have are a sequence of updates and a sequence of queries. These are queries, and these are updates. This is what the sequence looks like-- sorry, this is v's subtree, this is the pi is, I should say. I mean, these operations are all happened during time, but now I'm sorting by i. A little confusing.

There are two orders to think about, right? There's the sequence over time, we're now looking at such a left subtree where we do say 1, 5, and 3, 7. What that means-- so you're imagining here, this is 1, this is 5, this is 3, this is 7. Here we're sorted by the value written down there, we're sorting by the i, the pi i that they're changing or querying. And so all the read things are in the right subtree of v. And all the updates are in the left subtree of v. This is the interleaving property that I mentioned earlier.

So I claim that if I encode the results of the queries, namely I encode these permutations, these are like summary-- partial sums. These are prefixed sums of the permutation list. Then I can figure out what the updates were. Why? Because if I figure out what this query, what it's permutation is, that's the sum of all of these permutations.

Now only one of them changed in the left subtree, the rest all are in the past. They were all set before this time over here, and I know everything about the past, I'm assuming. So most of these I already know, the one thing I don't know is this one, but I claim if I know this sum, and I know all the others, then I can figure out what this one is, right?

It's slightly awkward to do, if I give you this, I give you the sum of pi j from j equals 0 to i, or something. I've got to-- I want to strip away all these, strip away all these. So I'm going to multiply by sum of pi j inverses over here, and multiply by sum pi j-- when I say multiply, I mean compose. Sum pi j inverse is here, maybe let's not worry about the exact indices here. But the point is, this is all in the past, and this is all in the past, so I know all these pi js, I know they're inverses. So if I have this total sum, and I right multiply with these inverses, left multiply with these inverses, I get the one that I want. This gives me some particular pi k, if I set the indices right.

OK? So if I know this query, I figure out what this update is. Now once I know what this update is, and I know this query, then in this sum, I know everything except this one thing. And so by using the same trick, I can figure out what this update is. So now I know the first two updates, if I then know the answer to this query, I can figure out what this update is. If I know the answer to this query, I can figure this update. Because they're perfectly interleaved, I only need to reconstruct one update at a time.

So if I'm given-- if I've somehow encoded all of the queries results, all of these prefix sums, and I'm given the past, then I can reconstruct what all the updates were. So that's basically saying these two are the same issue. If I can encode the verified sums in the right subtree, using less than l root n log n bits, then I'll get a contradiction, because it implies that from that same encoding, you can also decode the update permutations in the left subtree. So that's our goal.

OK. So we'd like to prove this for verify sum. But the first thing I'm going to do is consider an easier problem, which is sum. So suppose, basically, this was not an input to the query. Suppose the query was, what is the sum of i? Like this. I just want-- this is the partial sum problem. I'm given an index i, I want to know what is the permutation from pi 0 up to pi i. Now that is not-- that doesn't correspond to dynamic connectivity, it's a new problem. We'll first prove a lower bound for that problem, and then we'll put the verify word back in.

OK, so that's-- we're now here at sum lower bound. Where should I go? Different-- so this is a lower bound on the operation sum, as opposed to here, where we're adding up lower bounds. Sorry for the conflation of terms. Let's go here.

So I'll call this a warm up. Suppose a query is sum of i, which is supposed to give you this prefix sum of pi j again, sum means composition. So this is going to be relatively easy to prove, but it's not the problem we actually want to solve, we'll use it to then solve the real problem. And this is the order in which we actually solve things.

First, we prove a lower bound of partial sums. OK, so let me give you some notation, so we can really get at this claim. Reading on the right, writing on the left. So let r be all the cells that are read during the right subtree, which is an interval of time. And let w be the cells written in the left subtree.

OK, so what we're talking about over here is that r intersects w, those are cells that are read during the right subtree, that were at some point written during the left subtree, should be large. So we want to prove a lower bound on the size of r intersect w. So if the lower bound doesn't hold, that means that r intersect w is relatively small. So imagine a situation where r intersect w is very small, there's not very much information passed from the left subtree to the right subtree. If r intersect w is small, then presumably I can afford to write it down, I can encode it. So that's what we're going to do, and we'll compute-- we'll figure out that this is indeed something we can afford.

I'm going to encode r intersect w explicitly. Meaning-- and this is a set of cells in memory. So for every cell, I'm going to write down what it's address is, and what the contents of the cell are. So write down the addresses and the contents for every such cell.

So how many bits does that take? I'm going to say that it's r intersect w times log n bits. Here's where I need to mention an assumption. I'm assuming that the address space is order log n bits long, that's like saying that the space of your data structure is order-- is polynomial in n. And if you want any hope of having a reasonable update time, you need to have polynomial space at most. So assuming polynomial space, each of those addresses only takes order log n bits to write down. The contents, let's say, also take order log n bits to write down.

OK, so fine. That's-- I mean, yeah. We don't really need to make those assumptions, I don't think, but we will for here to keep things simple. So if r intersect w is small, meaning smaller than this thing, then this will be small, smaller than l root log n. OK.

So on the other hand, we know that every encoding should take l root n log n bits. And so this will be a contradiction, although we haven't quite encoded what we need yet, or we haven't proved that, but we're getting to be at the right point. These log ns are going to cancel in a moment.

So what we need to do is, I claim this is actually enough to encode what we need. And so all that's left is a decoding algorithm for the sum queries in the right subtree. So how are we going to do that? So this is my encoding, these are the bits that I have written down. So now what I know, as a decoder, is I know everything about the past. I don't know what these updates are, that's my whole goal, to figure out what they are. I don't know what the results of the queries are, but magically, I know that r intersect w. Well, not magically. I wrote it down, kept track on a piece of paper. So that's what I know. And so the idea is, well, somebody gave us a data structure, tells you how to do an update, tells you how to do a query.

Let's run the query algorithms over here. Run that query, run that query, or whatever. It's a little hard to run them, because we don't know what happened in this intermediate part. But I claim r intersect w tells us everything we need to know. So the decoding algorithm is just simulate sum queries, simulate that algorithm.

And let's go up here.

How do we simulate that algorithm? Well, the algorithm makes a series of cell reads, and maybe writes, but really we care about the reads. Writes are pretty easy to simulate.

There are three cases for reads. It could be that the thing you're trying to read was written in the right subtree, it could be that it was written in the left subtree, or it could be it was written in the past, before we got to v subtree. Now we don't necessarily know which case we're in, but I claim we'll be able to figure it out. Because any cells that are written in the right subtree, we've just been running the simulation algorithm, so every time we do it right, we just can store it off to the side.

So when we're doing simulations, we don't need that the simulation takes low space. We just need that the input-- these decoding algorithms doesn't have to be low space, we just need that the encoding was small. We've already made the encoding small. And so the decoding algorithm can spend lots of time and space, we just need to show that decoding algorithm can recover what it's supposed to recover. It's like a compression algorithm, to show there's some way to decompress, could take arbitrarily amount of time and space.

So when we're simulating the right subtree, and we simulate not only the sum queries, but also the updates. So whatever gets written during that simulation, we just store it, and so it's easy to reread it. If it was written in the left subtree, well, that is r intersect w. And we've written down r intersect w. So we can detect that this happened, because we look at r intersect w, we see, oh that word was in there, that address was in there, and so we read the contents from the encoding.

If it was in the past, it's also easy. We already know it. OK, so basically what we do-- what the simulation algorithm is doing is it says, OK, let's assume that main memory was whatever it was at this point. That data structure, I mean we know everything about the past, so we know what the data structure looked like at this moment, store that. Update all of the cells that are in r intersect w given by our encoding. And then just run the algorithm.

So we're sort of jumping into this moment in time with a slightly weird data structure. It's not the correct data structure. It's not what the data structure will actually look like at this point, but it's close enough. Because anything that's read here, either was written here, in which case it's correct, or was written here, in which case it's correct because r intersect w had it. Or isn't it written here, in which case-- maybe it's always correct. No, no.

See there could be some writes that happened here, where there's no corresponding read over here. So the data structure may have been changed in ways here that don't matter for this execution of the right subtree. So any rights that happened here to some cell probe, to some cell, where that cell is not read over here, we don't care about, because they don't affect the simulation. So we have a good enough data structure here, it may not be completely accurate, but it's accurate enough to run these queries.

Once we run the queries, the queries output the sums. That's what we're assuming in this warm up, we run the query, we get the sum. Once I have that sum, as I argued before, once you know what the results of these queries were, I can figure out what the arguments to the updates were, by doing that inverse multiplication stuff. So that's actually it.

What this implies is that this is a correct encoding, which means that this order, r intersect w times log n bits that we use to encode, must be at least this big. Because we know any encoding is going to require at least that many bits, l root n log n. And so the log ns cancel, and we're left with r intersect w is at least l root n. And this is exactly the quantity we cared about for this claim. So same thing, r intersect w is at least l root n.

OK, so warm up done. Any questions about the warm up? So in this weird problem, which does not correspond to dynamic connectivity, because it's this other problem, prefix sums computation. We get the intended lower bound, you need log n per operation. Or you need root n log n per block operation.

OK, but this is not what we really want, we really want a lower bound on verify sum. Where you're given as an argument the permutation that we're talking about over here. So this goal is not the right goal for verify sum, in some sense. Well, sort of the right goal. It's a little awkward though, because they're given as inputs to the queries. So what is there to encode? Well, we can still set it up in a useful way. Same goal, slightly restated.

So this is the last step to verify sum lower bound.

So here's the set up.

OK, so slightly different set up here. Here I assumed that we just knew the past. I also basically assumed these two things, that we didn't know what the update permutations were in the left subtree, and we didn't know what the answers to the queries were in the right subtree. Now I'm going to assume we don't even know what we're passing into the queries, because that is the information we're trying to figure out.

These two things are basically the same, if you knew all the update permutations, you could figure out all the query permutations. If you knew all the query permutations, you could figure out all the update permutations. That's what we argued over here, it's enough to figure out query permutations, then we could figure out the update permutations.

It's just a little more awkward, because now there are arguments to queries. And so if we did this simulation, right? We'd simulate-- we don't know how to simulate the query algorithm, because it's supposed to be, given the argument, which is what we're trying to figure out. So we can't simulate the query algorithm. It's kind of annoying, but otherwise the set up is roughly the same.

The one thing we know is that the query is supposed to return yes, because if you look at this bad access sequence, it is designed to always return yes. So that is a thing we know, but we don't know the arguments to the updates on the left, we don't know arguments to the updates on the right. We'll assume we know everything else, basically, up to this time.

Again, this is a probabilistic statement, that conditioned on the past, conditioned on the queries on the left, which probably don't matter, conditioned on the updates on the right, which do matter, but they're sort of irrelevant to this r intersect w issue. Conditioned on all those things will prove that the expected number of operations you need to-- or expected encoding size, for this problem, is at least what it is, l root n log n bits. And from that lower bound, you can then take the sum over all possible setups, over all conditions. And that implies a lower bound on the overall setting without these assumptions. OK?

So all I'm saying is in this set up, it still takes a lot of bits to encode these updates, because we don't have the queries which would tell us the answers. So we get a lower bound on encoding these updates, or a lower bound on encoding these queries, because we assume we don't know them. The rest of the-- all the remaining operations don't tell us enough about this. OK.

So how the heck are we going to do-- prove a lower bound in this setting, when we can't simulate the query algorithm? There's one cool idea to make this work. You may recall our last cell probe lower bound for the predecessor problem. Use this idea of round elimination. The idea with round elimination was-- Alice is sending a message, Bob was sending a response. But that first m-- we set things up, we set up the problem so the first message sent by Alice had, on average, less than 1-bit of information to Bob, or very little information to Bob. And so what Bob could do is basically guess what that message was. And that would be accurate with some probability.

Now here, we're not quite allowed to do that, we're not allowed to change the accuracy of our results, because of our particular setting. So we can't afford to just guess by flipping coins what we were supposed to know. What we're supposed to know here is-- we're trying to simulate a query operation, and so we need to know the argument, that whole permutation to the queries.

It's hard to run it without that permutation. So instead of guessing by flipping coins, we're going to guess in the dynamic programming sense, which is we're going to try all the possibilities. Run the simulation over all possible queries, all possible second arguments to the query. We don't know what the presentation is, so just try them all. Cool thing is, only one argument here should return yes. That's the one we're looking for. So if you try them all, find which one says yes, we'll be done.

So this is called the decoding idea. Simulate verify sum of i comma pi, for all pi. And take the one that returns yes, that is our permutation. And so if we figure out what those query permutations are, then we figure out what the update permutations are, and we get our lower bounds just like before.

OK. This is easier said than done, unfortunately. We'd like to run the simulation just like here, so simulate inquiry algorithm. They said, OK, still the case, that if you're reading a cell that's either in the left subtree, in the right subtree, or in the past. And we said this was easy, this was known. And the hard part is this case, because if we're running this query, and it reads something that was written in the left subtree, it may not be in r intersect w. Why is that? Little puzzle for you. So we're running one of these queries for sum pi. And I claim that when we read something in the left subtree, we don't know if it's in r intersect w, it might not be.

Let's see if we're on the same page. So r is the set of cells read during the right subtree when executing these operations. OK? But what we're doing now is simulating some executions that didn't necessarily happen. We're doing a verify sum of i comma pi, but in the bad access sequence, we did verify sum of i comma something specific, not any pi, but the correct pi. So we only ran the yes verify sums, and that's what r is defined with respect to. r is the set of things that get read during these operations, where the verify sum is always output yes.

If you now run a verify sum where the answer is no, it may read stuff that the other verify sum didn't read maybe. Shouldn't matter, but it's awkward, because now it's not just r intersect w we need to encode. We need to encode some more stuff. It's basically a new r prime that may happen during these reads, and we just can't afford to encode that r prime, because it's not the thing we care about. We care about what happens in the actual access sequence, not in this arbitrary simulation.

So this is the annoying thing. Trouble. If you look at an incorrect query, meaning the wrong pi, this is like a no query, the output's no. Reads some different set of cells, r prime, which isn't the same thing as r. And so if-- we have some good news, which is if we can somehow detect that this happened, that we read something that is in r prime, but not r, then the answer must be no.

So that's our saving hope, is that either we're reading something at r intersect w, in which case it's been written down, we know how to do it. What's not written there, and if it's not written there, then it should be, hopefully, in r prime minus r. So the answer should be no. Maybe.

Slight problem, though, because we used r intersect w to detect what case we were in. If we were in r intersect w, then we knew we should read from those encoded cells. If we weren't, we were either in the past or in the right subtree, these things were easy to detect, because they got written during the simulation. But we need to distinguish between-- did we read something that was in the left subtree, or did we read something that was known? This is a little tricky, because this gets at exactly the issue. Left subtree might write some stuff that didn't get read by verify sum. So now you go to read it, you need to know, am I reading something that was not in r intersect w? And therefore-- Yeah.

Basically the issue is, is it in w? If it's in w, but not in r intersect w, then I know the answer is no, and I should stop. If it's not in w though, that means it was in the known past, and then I should continue. How do I know if I should stop or continue? So this is the tricky part.

We can't tell whether there's the weird notation. We want to know whether r is in w minus r or past minus r intersect w. OK, we can tell whether it's in r intersect w, if it is, we're happy. If it's not in r intersect w, it could be that's because it was just in some past thing we were reading, that didn't get read otherwise. Or it could be we're reading something that was written in the left subtree, but not read in the right subtree.

So in this case, we want to abort. And in this case, it's known, and so we just continue. So that's what the simulation would like to do, if we could distinguish between these two cases. But right now, we can't distinguish between these two cases, because we don't have enough information. So we're going to make our encoding a little bit bigger.

What we're going to do-- this is here-- is encode a separator for r minus w and w minus r. So let's-- over here.

What does this mean? Separators going to call, called S. So I want this picture, r minus w sits inside S. And w minus r sits outside S. This is my universe of cells. These are the things that are read in the right subtree, but not written in the left subtree. Those are the things I care about-- well, no quite this, the other ones. So things that are read in the right subtree and that are not written in the last, this is the past essentially, that's useful over there. Over here, I have w minus r, these are things that are written in the left subtree, but not read in the right subtree. These are the things that I worry about, because those ones I need to detect that that was changed, and say whoops, you must have an answer of no.

OK? So I can't afford to store these sets exactly, so I'm going to approximate them, by saying, well, let's store the separator out here. And if you're in S, then you're definitely not in w minus r. If you're definitely not in w minus r, then you can run-- you can treat it as if it was known. OK, so if you're in s, this would be-- why don't I write it here. For the decoding algorithm, if you want to read a cell that is written, or last written in the right subtree, in the past, these are the two easy case. Sorry-- I don't want to write what's in the past, because the whole point is to figure out what's in the past.

The other easy case is if it's in r intersect w, then it's written down for us. So this is encoded. This is easy, because during the simulation we did those rights, and so we know what they were. r intersect w, we've written down, so it's easy to know. Then the other cases are either you're in S, or you're not in S. OK. I claim if you're in S, you must be in the past, that cell must have been written in the past, and so you know what the value was. And so you can continue writing the simulation, just like in this situation.

The other situation is you're not in S, then you don't know, it could have been written or might not have been. But what you know is that you're definitely not in r. Because if you're not in r minus w, and you're not in r intersect w, then you're not in r. If you're not in r, then we're in this situation. If you read something not in r, that means you're running the wrong query. Because the correct query does r-- only reads from r. So if you're not an S, you must not be in r. And so in this case, you know you can abort and try the next pi.

So we're going to do this for all pi, run the simulation according to this way of reading cells. At the end, the queries are either going to say yes or no, or it may abort early. So if it says no or it aborts early, then we know that was not the right pi. Only one of them can say yes, that tells us what the pi is, that tells us what the queries were. Once we know what the queries were in the right subtree, we can use the same multiplying by inverses trick, figure out what the updates were in the left subtree. But those permutations require l root n log n bits. Which used to be on this board, it's been erased now. That's what we use for this argument.

And so what we get is overall, encoding must use l root n log n bits. OK, but our encoding's a little bit bigger now. The big issue is how do we store the separator? We need to do store this separator with very few bits, otherwise we haven't really proved anything. We want encoding to be small. So we get that the encoding must use omega l root n log n bits in expectation, because this is a valid decoding algorithm, it will figure out what the permutations were. And they require at least this many bits, so encoding must use this many bits in expectation.

Now the question is how many bits does the encoding use? Then we'll get either a contradiction or we'll prove the claim. So let's go over here.

So here's a fun fact about separators. I'm not going to prove it fully, but I'm going to rely on some hashing ability. So given some universe U, in this case it's going to be the cells in our data structure. But speak a little bit more generally of the universe U, I have some number m, which is our set size. And what we're interested in is in defining our separator family. Kind of like a family of hash functions, closely related, in fact. Call it S. And it's going to work for size m sets. And so S is a separator family if, for any two sets, A and B, in the universe of size, at most, m, and disjoint.

So A intersect B is the empty set. So of course what we're thinking about here is r minus w, and w minus r. These are two subsets of the universe. Hopefully they're not too big, because if this one is huge, that means you read a huge amount of data in the right subtree. If this one is huge, it meant you wrote a huge amount of data in the left subtree. And then we get lower bounds in an easier way. Or they're not so big, let's say they're size at most m. They're disjoint for sure, by definition, r minus w's disjoint from w minus r. It removes the intersection. So that's our set up.

Then, what we want, is that there is some set C in the separator family, such that A is contained in C, and B is outside of C. So B is in the universe minus C. So this is exactly our picture from before, we have A. A contains C, and we have B over on the right. And this is the whole universe U, and so B is outside of C, A is entirely inside C. OK.

This is what we want to exist, because if a separator family exists, then we know whatever our r minus w, and w minus r sets were, as long as they're not too big, they're definitely disjoint, we can find one of these separators that encodes what we need to encode, which is the set C. Which is called s over there. Cool. How do we encode it? Well, if the number-- if the size of the separator family is something, then we need log of that bits to write down the separator set. So as long as this is small, we're happy.

So let me tell you what's to know about separators There exists a separator family S, with size of S at most 2 to the order m plus log log U. Now this is getting into an area that we haven't spent a lot of time on, but-- so I'm going to give you a sketch of a proof of this claim. Relying on perfect hash functions. So the idea is the following, we want to know, basically, which elements are in A, which elements are in B. But it's kind of annoying to do that, it can't start that for all universe elements.

So if we could just find a nice perfect hash function that maps the elements of a and the elements would B to different slots in some hash table, then for every slot in the hash table we could say, is it in A, or is it in B? Now if you are not in A union B and you hash somewhere, you'll get some bit, who knows what that that stores. I don't care. For the things outside of A union B, they could be in C or not in C, I don't really care. And so all I care about is if A and B have no collisions between each other, I don't want any A thing to hash to B thing. Then I can store a bit in every cell in the hash table, and that will tell me, in particular, A versus B. And then the rest of the items are somehow categorized, but I don't care how they're categorized.

So we're going to use this fact that there is a set of perfect hash functions of the same size. Sorry, that should be H. This is what's really true, size of H is 2 the order m plus log log U. OK, I'm not going to prove this, but this is about succinct hash functions. It may be hard to find such a hash family, but the claim is that they exist. Or it's hard to find the hash function of the family that has no collisions, but the guarantee is, as long as you have, in total, two items, out of your universe of size U, you can get a collision-free hash function, 2 to the order m plus log log U.

OK. So this is going to-- Yeah. Maps, say A union B, to an order m sized table. And here, there are no collisions. So then what we also store is an A or B bit for each table entry.

So that's our encoding. We store a perfect hash function, that's going to cost log H bits for this part, and log of H is just m plus log log U. And then we're going to store this A or B bit for every table entry. Number of table entries is order m, so this is going to take 2 to the order m bits. Or sorry, not-- sorry, in term of bits, its order m bits, I should say.

In terms of functions, it's 2 to the order m possible choices for this bit vector. And so the easy way is to just sum up these bits, you use log of H bits plus order m bits. This already had an order m term, and so you get this. The log of S is order m plus log log U. So that's the end of proof sketch of the claim. If you believe perfect hash functions can be written down in a small way, then we're done.

Now first with separators, now let's apply this separator theorem claim to this setting. So now we can compute the size of our encoding, our encoding involved writing down r intersect w. That takes r intersect w times log n, just like before. It also involves writing down the separator. Separator takes order m bits, m is r plus w. Things that are-- it's order r plus w. These are all the things-- I'm trying to write down r minus w and w minus r, so that you add up those sizes, basically r plus w. Plus log log U. U is some small thing, size of memory, number of cells in memory. We're assuming that polynomials, so you take log log of a polynomial, that's like log log n.

So let's finish this off. So before this was our equation, r intersect w times log n, that was the size of our encoding. We still have that term. Sorry, r intersect w, size of that, times log n. So we still do that. Now we also pay, for this separator, we're going to pay r plus w, that's the m part. Plus log log n. This is the number of bits in our encoding. And I claim, or what we've proved over here, is that any encoding must use l root n log n bits.

So this thing must be at least this thing. So we have a little bit more work to prove. There are now two cases. It depends-- there's basically-- and log log n is unlikely to dominate. We're doing a block operation on root n things, probably need to use at least log log n steps. So it's not really relevant. What will dominate is either this term, as it used to, or this term. These are two different cases, call them case one, case two.

In case two, r plus w is at least l root n log n. That's the lower bound we want. If we can-- in case two, r plus w is omega l root n log n. What that means is in this subtree, the amount of reading we did in the right subtree, plus the amount of writing we did in the left subtree, is at least l root n log n. That's our goal over here. We want to prove-- sorry, it's a previous claim, that's by now erased. Is the easier claim, we just want to show that the total amount of time spent in v's subtree is at least log n per operation. We're doing l root n things here. So this is a ton of reading and writing. So in that case, we're happy, because we get an actual lower bound on time.

Otherwise, we don't-- I mean, these are actual reads and writes, or total number of reads and writes. Here we're getting-- in the other case, we get r intersect w log n is at least l root n log n, just like before. So again, the log ns cancel. So here we lose the log n factor, but it's OK, because this is only talking about r intersect w. This we use the LCA charging, to say, well, if you look at a particular read, it's only gets charged by the LCA. So then we can afford to sum up large amounts.

So it's a little bit weird. In this situation, we add up all the lower bounds. Each of them doesn't give us a log n, but in aggregate, we get a log n, because every leaf appears in log n levels. In this case, we don't need to aggregate, because we just say, well, the number of operations in the subtree is at least log n per operation. This time spent, cell probe's done, is at least log n per operation. So in that case, we don't need to sum the lower bounds, which is done. So in either case, we're happy.

Little weird, because you could have a mix of cases, one vertex v could be in case two, then you just ignore all the things below it. The rest of the tree might be in case one, but you can mix and match one and two, as long as you don't use a one below a two, you're OK, you won't double count. And so in either case, we're happy, we get a log n lower bound, either on time per operation, or on this kind of time per operation. Add up all those lower bounds, you get log n per operation, or get root n log n per block operation, which implies log n per insert delete edge, or connectivity query. And that proves right there, more or less on time.

You can use the same technique to do a trade off between updates and queries. This is just log n, worst case of the two. I mentioned what the bound was last time. Same trick works, you just do more updates than queries, or more queries than updates. So we get link/cut trees are optimal, other [? tour ?] trees are optimal. And we've got lots of other points on the trade off curve, as you may recall last class. Like our log squared update is optimal for a log over log log query.

And that's the end of dynamic graphs, the end of advanced data structures. Hope you had a fun time, we got to see lots of different topics. And I hope you'll enjoy watching on the videos, and let me know if you have any comments, send an email or whatever. Yay.

[APPLAUSE]