1
00:00:13,000 --> 00:00:20,000
And this is going to use some
of the techniques we learned

2
00:00:20,000 --> 00:00:25,000
last time with respect to
amortized analysis.

3
00:00:25,000 --> 00:00:32,000
And, what's neat about what
we're going to talk about today

4
00:00:32,000 --> 00:00:39,000
is it's a way of comparing
algorithms that are so-called

5
00:00:39,000 --> 00:00:48,000
online algorithms.
And we're going to introduce

6
00:00:48,000 --> 00:01:00,000
this notion with a problem which
is called self organizing lists.

7
00:01:00,000 --> 00:01:10,000
OK, and so the set up for this
problem is that we have a list,

8
00:01:10,000 --> 00:01:17,000
L, of n elements.
And, we have an operation.

9
00:01:17,000 --> 00:01:26,000
Woops, I've got to spell things
right, access of x,

10
00:01:26,000 --> 00:01:33,000
which accesses item x in the
list.

11
00:01:33,000 --> 00:01:40,000
It could be by searching,
or it could be however you want

12
00:01:40,000 --> 00:01:42,000
to do it.
But basically,

13
00:01:42,000 --> 00:01:46,000
it goes and touches that
element.

14
00:01:46,000 --> 00:01:54,000
And we were to say the cost of
that operation is whatever the

15
00:01:54,000 --> 00:02:01,000
rank of x is in the list,
which is just the distance of x

16
00:02:01,000 --> 00:02:10,000
from the head of the list.
And the other thing that we can

17
00:02:10,000 --> 00:02:18,000
do that the algorithm can do,
so this is what the user would

18
00:02:18,000 --> 00:02:22,000
do.
He just simply runs a whole

19
00:02:22,000 --> 00:02:30,000
bunch of accesses on the list,
OK, accessing one element after

20
00:02:30,000 --> 00:02:36,000
another in any order that he or
she cares to.

21
00:02:36,000 --> 00:02:43,000
And then, L can be reordered,
however, by transposing

22
00:02:43,000 --> 00:02:53,000
adjacent elements.
And the cost for that is one.

23
00:02:53,000 --> 00:03:01,000
So, for example,
suppose the list is the

24
00:03:01,000 --> 00:03:04,000
following.

25
00:03:21,000 --> 00:03:29,000
OK, I missed something here.
It doesn't matter.

26
00:03:29,000 --> 00:03:40,000
Well, I'll just make it be what
I have so that it matches the

27
00:03:40,000 --> 00:03:46,000
online video.
OK, so here we have a list.

28
00:03:46,000 --> 00:03:51,000
And so, if I do something like
access element 14 here,

29
00:03:51,000 --> 00:03:56,000
the element of key 14,
OK, that this costs me one,

30
00:03:56,000 --> 00:04:01,000
two, three, four.
So here the cost is four to

31
00:04:01,000 --> 00:04:05,000
access.
And so, we're going to have

32
00:04:05,000 --> 00:04:10,000
some sequence of accesses that
the user is going to do.

33
00:04:10,000 --> 00:04:14,000
And obviously,
if something is accessed more

34
00:04:14,000 --> 00:04:21,000
frequently, we'd like to move it
up to the front of the list so

35
00:04:21,000 --> 00:04:24,000
that you don't have to search as
far.

36
00:04:24,000 --> 00:04:28,000
OK, and to do that,
if I want to transpose

37
00:04:28,000 --> 00:04:31,000
something, so,
for example,

38
00:04:31,000 --> 00:04:38,000
if I transpose three and 50,
that just costs me one.

39
00:04:38,000 --> 00:04:43,000
OK, so then I would make this
be 50, and make this be three.

40
00:04:43,000 --> 00:04:49,000
OK, sorry, normally you just do
it by swapping pointers.

41
00:04:49,000 --> 00:04:52,000
OK, so those are the two
operations.

42
00:04:52,000 --> 00:04:59,000
And, we are going to do this in
what's called an online fashion.

43
00:04:59,000 --> 00:05:06,000
So, let's just define online.
So, a sequence,

44
00:05:06,000 --> 00:05:16,000
S, of operations is provided
one at a time for each

45
00:05:16,000 --> 00:05:24,000
operation.
An online algorithm must

46
00:05:24,000 --> 00:05:34,000
execute the operation
immediately without getting a

47
00:05:34,000 --> 00:05:48,000
chance to look at what else is
coming in the sequence.

48
00:05:48,000 --> 00:05:51,000
So, when you make your decision
for the first element,

49
00:05:51,000 --> 00:05:54,000
you don't get to see ahead as
to what the second,

50
00:05:54,000 --> 00:05:57,000
or third, or whatever is.
And the second one you get,

51
00:05:57,000 --> 00:06:00,000
you get and you have to make
your decision as to what to do

52
00:06:00,000 --> 00:06:06,000
and so forth.
So, that's an online algorithm.

53
00:06:06,000 --> 00:06:11,000
Similarly, an off-line
algorithm, OK,

54
00:06:11,000 --> 00:06:18,000
may see all of S in advance.
OK, so you can see an off-line

55
00:06:18,000 --> 00:06:27,000
algorithm gets to see the whole
sequence, and then decide what

56
00:06:27,000 --> 00:06:33,000
it wants to do about element
one, element two,

57
00:06:33,000 --> 00:06:37,000
or whatever.
OK, so an off-line algorithm

58
00:06:37,000 --> 00:06:41,000
can look at the whole sequence
and say, OK, I can see that item

59
00:06:41,000 --> 00:06:45,000
number 17 is being accessed a
lot, or early on move him up

60
00:06:45,000 --> 00:06:49,000
closer to the front of the list,
and then the accesses cost less

61
00:06:49,000 --> 00:06:53,000
for the off-line algorithm.
The online algorithm doesn't

62
00:06:53,000 --> 00:06:56,000
get to see any of that.
OK, so this is sort of like,

63
00:06:56,000 --> 00:07:00,000
if you're familiar with the
game Tetris.

64
00:07:00,000 --> 00:07:02,000
OK, and Tetris,
you get one shape after another

65
00:07:02,000 --> 00:07:05,000
that starts coming down,
and you have to twiddle it,

66
00:07:05,000 --> 00:07:08,000
and move it to the side,
and drop it into place.

67
00:07:08,000 --> 00:07:11,000
And there, sometimes you get a
one step look-ahead on some of

68
00:07:11,000 --> 00:07:14,000
them so you can see what the
next shape is,

69
00:07:14,000 --> 00:07:17,000
but often it's purely online.
You don't get to see that next

70
00:07:17,000 --> 00:07:20,000
shape or whatever,
and you have to make a decision

71
00:07:20,000 --> 00:07:22,000
for each one.
And you make a decision,

72
00:07:22,000 --> 00:07:26,000
and you realize that the next
shape, ah, if you had made a

73
00:07:26,000 --> 00:07:28,000
different decision it would have
been better.

74
00:07:28,000 --> 00:07:32,000
OK, so that's the kind of
problem.

75
00:07:32,000 --> 00:07:38,000
Off-line Tetris would be,
I get to see the whole sequence

76
00:07:38,000 --> 00:07:42,000
of shapes.
And now let me decide what I'm

77
00:07:42,000 --> 00:07:47,000
going to do with this one.
OK, and so, in this,

78
00:07:47,000 --> 00:07:53,000
the goal is for any of the
algorithms, either online or

79
00:07:53,000 --> 00:08:00,000
off-line is to minimize the
total cost, which we'll denote

80
00:08:00,000 --> 00:08:06,000
by, I forgot to name this.
This is algorithm A here.

81
00:08:06,000 --> 00:08:09,000
The total cost,
C_A of S, OK,

82
00:08:09,000 --> 00:08:13,000
so the cost of algorithm A on
the sequence,

83
00:08:13,000 --> 00:08:16,000
S.
That's just the notation we'll

84
00:08:16,000 --> 00:08:22,000
use for what the total cost is.
So, any questions about the

85
00:08:22,000 --> 00:08:27,000
setup to this problem?
So, we have an online problem.

86
00:08:27,000 --> 00:08:32,000
We're going to get these things
one at a time,

87
00:08:32,000 --> 00:08:37,000
OK, and we have to decide what
to do.

88
00:08:37,000 --> 00:08:41,000
So, let's do a worst-case
analysis for this.

89
00:08:52,000 --> 00:08:56,000
OK, so if we're doing a
worst-case analysis,

90
00:08:56,000 --> 00:09:01,000
we can view that we have an
adversary that we are playing

91
00:09:01,000 --> 00:09:06,000
against who's going to provide
the sequence.

92
00:09:06,000 --> 00:09:10,000
The user is going to be able to
see what we do.

93
00:09:10,000 --> 00:09:14,000
And so, what's the adversary
strategy?

94
00:09:14,000 --> 00:09:18,000
Thwart our plots,
yes, that's his idea.

95
00:09:18,000 --> 00:09:22,000
And how is he going to thwart
them, or she?

96
00:09:22,000 --> 00:09:29,000
Which is what for this problem?
What's he going to do?

97
00:09:29,000 --> 00:09:32,000
Yeah.
No matter how we reorder

98
00:09:32,000 --> 00:09:39,000
elements using the transposes,
he's going to look at every

99
00:09:39,000 --> 00:09:43,000
step and say,
what's the last element?

100
00:09:43,000 --> 00:09:48,000
That's the one I'm going to
access, right?

101
00:09:48,000 --> 00:09:53,000
So, the adversary always,
always accesses the tail

102
00:09:53,000 --> 00:09:57,000
element of L.
No matter what it is,

103
00:09:57,000 --> 00:10:03,000
no matter how we reorder
things, OK, for each one,

104
00:10:03,000 --> 00:10:09,000
adversary just accesses the
tail.

105
00:10:09,000 --> 00:10:13,000
So the cost of this,
of any algorithm,

106
00:10:13,000 --> 00:10:18,000
then, is going to be omega size
of S times n,

107
00:10:18,000 --> 00:10:25,000
OK, because you're always going
to pay for every sequence.

108
00:10:25,000 --> 00:10:31,000
You're going to have to go in
to pay a cost of n,

109
00:10:31,000 --> 00:10:36,000
OK, for every element in the
sequence.

110
00:10:36,000 --> 00:10:42,000
OK, so not terribly in the
worst-case.

111
00:10:42,000 --> 00:10:46,000
Not terribly good.
So, people in studying this

112
00:10:46,000 --> 00:10:50,000
problem: question?
That analysis is for the online

113
00:10:50,000 --> 00:10:54,000
algorithm, right.
The off-line algorithm,

114
00:10:54,000 --> 00:10:59,000
right, if you named those
things, that's right.

115
00:10:59,000 --> 00:11:03,000
OK, so we're looking at trying
to solve this in an off-line

116
00:11:03,000 --> 00:11:06,000
sense, sorry,
in an online sense.

117
00:11:06,000 --> 00:11:10,000
OK, and so the point is that
for the online algorithm,

118
00:11:10,000 --> 00:11:15,000
the adversary can be incredibly
mean, OK, and just always access

119
00:11:15,000 --> 00:11:17,000
the thing at the end,
OK?

120
00:11:17,000 --> 00:11:21,000
So, what sort of the history of
this problem is,

121
00:11:21,000 --> 00:11:24,000
that people said,
well, if I can't do well in the

122
00:11:24,000 --> 00:11:29,000
worst-case, maybe I should be
looking at average case,

123
00:11:29,000 --> 00:11:32,000
OK, and look at,
say, the different elements

124
00:11:32,000 --> 00:11:37,000
having some probability
distribution.

125
00:11:37,000 --> 00:11:44,000
OK, so the average case
analysis, OK,

126
00:11:44,000 --> 00:11:56,000
let's suppose that element x is
accessed with probability,

127
00:11:56,000 --> 00:12:03,000
P of x.
OK, so suppose that we have

128
00:12:03,000 --> 00:12:14,000
some a priori distribution on
the elements.

129
00:12:14,000 --> 00:12:20,000
OK, then the expected cost of
the algorithm on a sequence,

130
00:12:20,000 --> 00:12:26,000
OK, so if I put all the
elements into some order,

131
00:12:26,000 --> 00:12:33,000
OK, and don't try to reorder,
but just simply look at,

132
00:12:33,000 --> 00:12:39,000
is there a static ordering that
would work well for a

133
00:12:39,000 --> 00:12:44,000
distribution?
It's just going to be,

134
00:12:44,000 --> 00:12:49,000
by definition of expectation,
the probability of x times,

135
00:12:49,000 --> 00:12:54,000
in this case,
the cost, which is the rank of

136
00:12:54,000 --> 00:13:01,000
x in whatever that ordering is
that I decide I'm going to use.

137
00:13:01,000 --> 00:13:09,000
OK, and this is minimized when?
So, this is just the definition

138
00:13:09,000 --> 00:13:15,000
of expectations:
the probability that I access x

139
00:13:15,000 --> 00:13:21,000
times the cost summed over all
the elements.

140
00:13:21,000 --> 00:13:28,000
And the cost is just going to
be its position in the list.

141
00:13:28,000 --> 00:13:33,000
So, when is this value,
this summation,

142
00:13:33,000 --> 00:13:41,000
going to be minimized?
When the element is most likely

143
00:13:41,000 --> 00:13:44,000
as the lowest rank,
and then what,

144
00:13:44,000 --> 00:13:49,000
what about the other element?
OK, so what does that mean?

145
00:13:49,000 --> 00:13:54,000
Yeah, sort them,
yeah, sort them on the basis of

146
00:13:54,000 --> 00:13:58,000
decreasing probability,
OK?

147
00:13:58,000 --> 00:14:04,000
So, it's minimized when L is
sorted, OK, in decreasing order

148
00:14:04,000 --> 00:14:10,000
with respect to P.
OK, so just sort them with the

149
00:14:10,000 --> 00:14:16,000
most likely one at the front,
and then just decreasing

150
00:14:16,000 --> 00:14:20,000
probability.
That way, whenever I access

151
00:14:20,000 --> 00:14:27,000
something with some probability,
OK, I'm going to access,

152
00:14:27,000 --> 00:14:33,000
it's more likely that I'm going
to access.

153
00:14:33,000 --> 00:14:37,000
And that's not too difficult to
actually prove.

154
00:14:37,000 --> 00:14:41,000
You just look at,
suppose there were two that

155
00:14:41,000 --> 00:14:46,000
were out of order,
and show that if you swap them,

156
00:14:46,000 --> 00:14:50,000
you would improve this
optimization function.

157
00:14:50,000 --> 00:14:56,000
OK, so if you didn't know it,
this suggests the following

158
00:14:56,000 --> 00:15:05,000
heuristic,
OK, which is simply keep

159
00:15:05,000 --> 00:15:21,000
account of the number of times
each element is accessed,

160
00:15:21,000 --> 00:15:38,000
and maintain the list in order
of decreasing count.

161
00:15:38,000 --> 00:15:42,000
OK, so whenever something is
accessed, increment its count,

162
00:15:42,000 --> 00:15:45,000
OK, and that will move it,
at most, one position,

163
00:15:45,000 --> 00:15:48,000
which only costs me one
transposed to move it,

164
00:15:48,000 --> 00:15:50,000
perhaps, forward.
OK, actually,

165
00:15:50,000 --> 00:15:54,000
I guess it could be more if you
have a whole bunch of ties,

166
00:15:54,000 --> 00:15:55,000
right?
Yeah.

167
00:15:55,000 --> 00:15:58,000
So, it could cost more.
But the idea is,

168
00:15:58,000 --> 00:16:02,000
over time, the law of large
numbers says that this is going

169
00:16:02,000 --> 00:16:06,000
to approach the probability
distribution.

170
00:16:06,000 --> 00:16:11,000
The frequency with which you
access this, divided by the

171
00:16:11,000 --> 00:16:15,000
total number of accesses,
will be the probability.

172
00:16:15,000 --> 00:16:20,000
And so, therefore you will get
things in decreasing

173
00:16:20,000 --> 00:16:24,000
probability, OK,
assuming that there is some

174
00:16:24,000 --> 00:16:29,000
distribution that all of these
elements are chosen according

175
00:16:29,000 --> 00:16:35,000
to, or accessed according to.
So, it doesn't seem like

176
00:16:35,000 --> 00:16:40,000
there's that much more you could
really do here.

177
00:16:40,000 --> 00:16:46,000
And that's why I think this
notion of competitive analysis

178
00:16:46,000 --> 00:16:51,000
is so persuasive,
because it's really amazingly

179
00:16:51,000 --> 00:16:55,000
strong, OK?
And it came about because of

180
00:16:55,000 --> 00:17:00,000
what people were doing in
practice.

181
00:17:00,000 --> 00:17:04,000
So practice,
what people implement it was a

182
00:17:04,000 --> 00:17:07,000
so-called move to front
heuristic.

183
00:17:07,000 --> 00:17:12,000
OK, and the basic idea was,
after you access an element,

184
00:17:12,000 --> 00:17:18,000
just move it up to the front.
OK, that only doubles the cost

185
00:17:18,000 --> 00:17:24,000
of accessing the element because
I go and I access it,

186
00:17:24,000 --> 00:17:29,000
chasing it down paying the
rank, and then I have to do rank

187
00:17:29,000 --> 00:17:36,000
number of transposes to bring it
back to the front.

188
00:17:36,000 --> 00:17:39,000
So, it only cost me a factor of
two, and now,

189
00:17:39,000 --> 00:17:43,000
if it happens to be a
frequently accessed elements,

190
00:17:43,000 --> 00:17:47,000
over time you would hope that
the most likely elements were

191
00:17:47,000 --> 00:17:55,000
near the front of that list.
So, after accessing x,

192
00:17:55,000 --> 00:18:05,000
move x, the head of the list
using transposes,

193
00:18:05,000 --> 00:18:17,000
and the cost is just equal to
twice the rank in L of x,

194
00:18:17,000 --> 00:18:25,000
OK, where the two here has two
parts.

195
00:18:25,000 --> 00:18:34,000
One is the access,
and the other is the

196
00:18:34,000 --> 00:18:41,000
transposes.
OK, so that's sort of what they

197
00:18:41,000 --> 00:18:43,000
did.
And one of the nice properties

198
00:18:43,000 --> 00:18:47,000
of this is that if it turns out
that there is locality in the

199
00:18:47,000 --> 00:18:50,000
access pattern,
if it's not just a static

200
00:18:50,000 --> 00:18:53,000
distribution,
but rather once I've accessed

201
00:18:53,000 --> 00:18:57,000
something, if it's more likely
I'm going to access it again,

202
00:18:57,000 --> 00:19:01,000
which tends to be the case for
many input types of patterns,

203
00:19:01,000 --> 00:19:05,000
this responds well to locality
because it's going to be up near

204
00:19:05,000 --> 00:19:10,000
the front if I access it very
soon after I've accessed.

205
00:19:10,000 --> 00:19:15,000
So, there is what's called
temporal locality,

206
00:19:15,000 --> 00:19:19,000
meaning that in time,
I tend to access,

207
00:19:19,000 --> 00:19:26,000
so it may be that I access some
thing's very hot for awhile;

208
00:19:26,000 --> 00:19:32,000
then it gets very cold.
This type of algorithm responds

209
00:19:32,000 --> 00:19:37,000
very well to the hotness of the
accessing.

210
00:19:37,000 --> 00:19:43,000
OK, so it responds well to
locality in S.

211
00:19:43,000 --> 00:19:48,000
So, this is sort of what was
known up to the point that a

212
00:19:48,000 --> 00:19:54,000
very famous paper was written by
Danny Sleator and Bob Tarjan,

213
00:19:54,000 --> 00:19:59,000
where they took a totally
different approach to looking at

214
00:19:59,000 --> 00:20:04,000
this kind of problem.
OK, and it's an approach that

215
00:20:04,000 --> 00:20:09,000
matter you see everywhere from
analysis of caching and

216
00:20:09,000 --> 00:20:14,000
high-performance processors to
analyses of disk paging to just

217
00:20:14,000 --> 00:20:21,000
a huge number of applications of
this basic technique.

218
00:20:21,000 --> 00:20:30,000
And, that's the technique of
competitive analysis.

219
00:20:30,000 --> 00:20:45,000
OK, so here's the definition.
So, online algorithm A is alpha

220
00:20:45,000 --> 00:20:55,000
competitive.
If there exists a constant,

221
00:20:55,000 --> 00:21:08,000
k, such that for any sequence,
S, of operations,

222
00:21:08,000 --> 00:21:23,000
the cost of S using algorithm A
is bounded by alpha times the

223
00:21:23,000 --> 00:21:39,000
cost of opt, where opt is the
optimal offline algorithm.

224
00:21:39,000 --> 00:21:43,000
OK, so the optimal off-line,
the one that knows the whole

225
00:21:43,000 --> 00:21:47,000
sequence and does the absolute
best it could do on that

226
00:21:47,000 --> 00:21:50,000
sequence, OK,
that's this cost here.

227
00:21:50,000 --> 00:21:53,000
This is sometimes called God's
algorithm, OK,

228
00:21:53,000 --> 00:21:58,000
not to bring religion into the
classroom, or to offend anybody,

229
00:21:58,000 --> 00:22:03,000
but that is what people
sometimes call it.

230
00:22:03,000 --> 00:22:07,000
OK, so the fully omniscient
knows absolutely the best thing

231
00:22:07,000 --> 00:22:10,000
that could be done,
sees into the future,

232
00:22:10,000 --> 00:22:12,000
the whole works,
OK?

233
00:22:12,000 --> 00:22:16,000
It gets to apply that.
That's what opts algorithm is.

234
00:22:16,000 --> 00:22:21,000
And, what we're saying is that
the cost is basically whatever

235
00:22:21,000 --> 00:22:24,000
this alpha factor is.
It could be a function of

236
00:22:24,000 --> 00:22:27,000
things, or it could be a
constant, OK,

237
00:22:27,000 --> 00:22:30,000
times whatever the best
algorithm is.

238
00:22:30,000 --> 00:22:36,000
Plus, there's a potential for a
constant out here.

239
00:22:36,000 --> 00:22:38,000
OK, so for example,
if alpha is two,

240
00:22:38,000 --> 00:22:43,000
and we say it's two
competitive, that means you're

241
00:22:43,000 --> 00:22:47,000
going to do, at worst,
twice the algorithm that has

242
00:22:47,000 --> 00:22:51,000
all the information.
But you're doing it online,

243
00:22:51,000 --> 00:22:54,000
for example.
OK, it's a really pretty

244
00:22:54,000 --> 00:22:57,000
powerful notion.
And what's interesting about

245
00:22:57,000 --> 00:23:04,000
this, it's not even clear these
things should exist to my mind.

246
00:23:04,000 --> 00:23:08,000
OK, what's pretty remarkable
about this, I think,

247
00:23:08,000 --> 00:23:12,000
is that there is no assumption
of distribution,

248
00:23:12,000 --> 00:23:16,000
of probability distribution or
anything.

249
00:23:16,000 --> 00:23:20,000
It's whatever the sequence is
that you give it.

250
00:23:20,000 --> 00:23:24,000
You are within a factor of
alpha, essentially,

251
00:23:24,000 --> 00:23:30,000
of the best algorithm,
OK, which is pretty remarkable.

252
00:23:30,000 --> 00:23:38,000
OK, and so, we're going to
prove the following theorem,

253
00:23:38,000 --> 00:23:45,000
which is the one that Sleator
and Tarjan proved.

254
00:23:45,000 --> 00:23:55,000
And that is that MTF is four
competitive for self organizing

255
00:23:55,000 --> 00:23:58,000
lists.
OK, so the idea here is that

256
00:23:58,000 --> 00:24:02,000
suppose the adversary says,
oh, I'm always going to access

257
00:24:02,000 --> 00:24:06,000
the thing at the end of the list
like we said in the beginning.

258
00:24:06,000 --> 00:24:10,000
So, the adversary says,
I'm always going to access the

259
00:24:10,000 --> 00:24:12,000
thing there.
I'm going to make MTF work

260
00:24:12,000 --> 00:24:16,000
really bad, because you're going
to go and move that thing all

261
00:24:16,000 --> 00:24:20,000
the way up to the front.
And I'm just going to access

262
00:24:20,000 --> 00:24:23,000
the thing way at the end again.
OK, well it turns out,

263
00:24:23,000 --> 00:24:26,000
yeah, that's a bad sequence for
move to front,

264
00:24:26,000 --> 00:24:30,000
OK, and it will take a long
time.

265
00:24:30,000 --> 00:24:34,000
But it turns out God couldn't
have done better,

266
00:24:34,000 --> 00:24:40,000
OK, by more than a factor of
four no matter how long the list

267
00:24:40,000 --> 00:24:43,000
is.
OK, that's pretty amazing.

268
00:24:43,000 --> 00:24:49,000
OK, so that's a bad sequence.
But, if there's a way that the

269
00:24:49,000 --> 00:24:55,000
sequence exhibits any kind of
locality or anything that can be

270
00:24:55,000 --> 00:25:00,000
taken advantage of,
if you could see the whole

271
00:25:00,000 --> 00:25:06,000
thing, MTF takes advantage of it
too, OK, within a factor of

272
00:25:06,000 --> 00:25:10,000
four.
OK, it's a pretty remarkable

273
00:25:10,000 --> 00:25:14,000
theorem, and it's the basis of
many types of analysis of online

274
00:25:14,000 --> 00:25:17,000
algorithms.
Almost all online algorithms

275
00:25:17,000 --> 00:25:21,000
today are analyzed using some
kind of competitive analysis.

276
00:25:21,000 --> 00:25:25,000
OK, not always.
Sometimes you do probabilistic

277
00:25:25,000 --> 00:25:28,000
analysis, or whatever,
but the dominant thing is too

278
00:25:28,000 --> 00:25:33,000
competitive analysis because
then you don't have to make any

279
00:25:33,000 --> 00:25:38,000
statistical assumptions.
OK, just prove that it works

280
00:25:38,000 --> 00:25:40,000
well no matter what.
This is remarkable,

281
00:25:40,000 --> 00:25:42,000
I think.
Isn't it remarkable?

282
00:25:42,000 --> 00:25:47,000
So, let's prove this theorem,
we're just going to spend the

283
00:25:47,000 --> 00:25:49,000
rest of the lecture on this
proof.

284
00:25:49,000 --> 00:25:54,000
OK, and the proof in some ways
is not hard, but it's also not

285
00:25:54,000 --> 00:25:56,000
necessarily completely
intuitive.

286
00:25:56,000 --> 00:26:00,000
So, you will have to pay
attention.

287
00:26:00,000 --> 00:26:09,000
OK, so let's get some notation
down.

288
00:26:09,000 --> 00:26:22,000
Let's let L_i be MTF's list
after the i'th access.

289
00:26:22,000 --> 00:26:38,000
And, let's let L be opt's list
after the i'th access.

290
00:26:38,000 --> 00:26:42,000
OK, so generally what I'll do
is I'll put a star if we are

291
00:26:42,000 --> 00:26:46,000
talking about opt,
and have nothing if we're

292
00:26:46,000 --> 00:26:50,000
talking about MTF.
OK, so that's going to be the

293
00:26:50,000 --> 00:26:51,000
list.
So, we can say,

294
00:26:51,000 --> 00:26:55,000
what's the list?
So, we're going to set it up

295
00:26:55,000 --> 00:27:00,000
where we have one in operation
that transforms list i minus one

296
00:27:00,000 --> 00:27:03,000
into list i.
OK, that's what the i'th

297
00:27:03,000 --> 00:27:09,000
operation does.
OK, and move to front does it

298
00:27:09,000 --> 00:27:16,000
by moving whatever the thing
that was accessed to the front.

299
00:27:16,000 --> 00:27:23,000
And opt does whatever opt
thinks is the best thing to do.

300
00:27:23,000 --> 00:27:28,000
We don't know.
So, we're going to let c_i be

301
00:27:28,000 --> 00:27:33,000
MTF's cost for the I'th
operation.

302
00:27:33,000 --> 00:27:40,000
And that's just twice the rank
in L_i minus one of x if the

303
00:27:40,000 --> 00:27:47,000
operation accesses x,
OK, two times the rank in L_i

304
00:27:47,000 --> 00:27:55,000
minus one because we're going to
be accessing it in L_i minus one

305
00:27:55,000 --> 00:28:01,000
and transforming it into L_i.
And similarly,

306
00:28:01,000 --> 00:28:09,000
we'll let c_i star be opt's
cost for the i'th operation.

307
00:28:09,000 --> 00:28:14,000
And that's just equal to,
well, to access it,

308
00:28:14,000 --> 00:28:20,000
it's going to be the rank in
L_i minus one star,

309
00:28:20,000 --> 00:28:28,000
whatever its list is of x at
that step, because it's got to

310
00:28:28,000 --> 00:28:34,000
access it.
And then, some number of

311
00:28:34,000 --> 00:28:41,000
transposes, t_i if opt forms t_i
transposes.

312
00:28:41,000 --> 00:28:51,000
OK, so we have the setup where
we have two different lists that

313
00:28:51,000 --> 00:28:59,000
are being managed,
and we have different costs in

314
00:28:59,000 --> 00:29:04,000
the list.
And, we're interested in is

315
00:29:04,000 --> 00:29:11,000
comparing in some way MTF's list
with opt's list at any point in

316
00:29:11,000 --> 00:29:14,000
time.
And, how do you think we're

317
00:29:14,000 --> 00:29:19,000
going to do that?
What technique do you think we

318
00:29:19,000 --> 00:29:25,000
should use to compare these two
lists, general technique from

319
00:29:25,000 --> 00:29:28,000
last lecture?
Well, it is going to be

320
00:29:28,000 --> 00:29:35,000
amortized, but what?
How are we going to compare

321
00:29:35,000 --> 00:29:39,000
them?
What technique did we learn

322
00:29:39,000 --> 00:29:43,000
last time?
Potential function,

323
00:29:43,000 --> 00:29:47,000
good.
OK, we're going to define a

324
00:29:47,000 --> 00:29:54,000
potential function,
OK, that measures how far apart

325
00:29:54,000 --> 00:29:59,000
these two lists are.
OK, and the idea is,

326
00:29:59,000 --> 00:30:08,000
if, let's define that and then
we'll take a look at it.

327
00:30:08,000 --> 00:30:21,000
So, we're going to define the
potential function phi mapping

328
00:30:21,000 --> 00:30:37,000
the set of MTF's lists into the
real numbers by the following.

329
00:30:37,000 --> 00:30:46,000
phi of L_i is going to be twice
the cardinality of this set.

330
00:31:04,000 --> 00:31:08,000
OK, so this is the
precedes-operation and list,

331
00:31:08,000 --> 00:31:10,000
i.
So, we can define a

332
00:31:10,000 --> 00:31:15,000
relationship between any two
elements that says that x

333
00:31:15,000 --> 00:31:20,000
precedes y in L_i if,
as I'm accessing it from the

334
00:31:20,000 --> 00:31:25,000
head, I hit x first.
OK, so what I'm interested in,

335
00:31:25,000 --> 00:31:31,000
here, are in some sense the
disagreements between the two

336
00:31:31,000 --> 00:31:35,000
lists.
This is where x precedes y in

337
00:31:35,000 --> 00:31:39,000
MTF's list, but y precedes x in
opt's list.

338
00:31:39,000 --> 00:31:40,000
They disagree,
OK?

339
00:31:40,000 --> 00:31:45,000
And, what we're interested in
is the cardinality of the set.

340
00:31:45,000 --> 00:31:49,000
And we're going to multiply it
by two.

341
00:31:49,000 --> 00:31:53,000
OK, so that's equal to two
times; so there is a name for

342
00:31:53,000 --> 00:31:58,000
this type of thing.
We saw that when we were doing

343
00:31:58,000 --> 00:32:03,000
sorting.
Anybody remember the name?

344
00:32:03,000 --> 00:32:08,000
It was very briefly.
I don't expect anybody to

345
00:32:08,000 --> 00:32:14,000
remember, but somebody might.
Inversions: good,

346
00:32:14,000 --> 00:32:18,000
OK, twice the number of
inversions.

347
00:32:18,000 --> 00:32:25,000
So, let's just do an example.
So, let's say L_i is the list

348
00:32:25,000 --> 00:32:37,000
with five elements.
OK, I'll use characters for the

349
00:32:37,000 --> 00:32:47,000
order just to keep things
simple.

350
00:32:47,000 --> 00:33:01,000
So, in this case phi of L_i is
going to be twice the

351
00:33:01,000 --> 00:33:12,000
cardinality of the set.
So what we want to do is see

352
00:33:12,000 --> 00:33:18,000
which things are out of order.
So here, I look at E and C are

353
00:33:18,000 --> 00:33:21,000
in this order,
but C and E in that order.

354
00:33:21,000 --> 00:33:26,000
So, those are out of order.
So, that counts as one of my

355
00:33:26,000 --> 00:33:29,000
elements, EC,
and then, E and A,

356
00:33:29,000 --> 00:33:36,000
A and E.
OK, so those are out of order,

357
00:33:36,000 --> 00:33:41,000
and then ED,
DE, out of order,

358
00:33:41,000 --> 00:33:48,000
and then EB,
BE, those are out of order.

359
00:33:48,000 --> 00:33:53,000
And now, I go C,
A, C, A.

360
00:33:53,000 --> 00:34:00,000
Those are in order,
so it doesn't count.

361
00:34:00,000 --> 00:34:08,000
CD, CD, CB, CB,
so, nothing with C.

362
00:34:08,000 --> 00:34:12,000
Then, A, D, A,
D, those are in order,

363
00:34:12,000 --> 00:34:16,000
A, B, A, B, those are in order.
So then, DB,

364
00:34:16,000 --> 00:34:20,000
BD, so BD.
And that's the last one.

365
00:34:20,000 --> 00:34:26,000
So, that's my potential
function, which is equal to,

366
00:34:26,000 --> 00:34:31,000
therefore, ten,
because the cardinality of the

367
00:34:31,000 --> 00:34:37,000
set is five.
I have five inversions,

368
00:34:37,000 --> 00:34:44,000
OK, between the two lists.
OK, so let's just check some

369
00:34:44,000 --> 00:34:49,000
properties of this potential
function.

370
00:34:49,000 --> 00:34:57,000
The first one is notice that
phi of L_i is greater than or

371
00:34:57,000 --> 00:35:04,000
equal to zero for all i.
The number of inversions might

372
00:35:04,000 --> 00:35:07,000
be zero, but is never less than
zero.

373
00:35:07,000 --> 00:35:11,000
OK, it's always at least zero.
So, that's one of the

374
00:35:11,000 --> 00:35:16,000
properties that we normally have
we are dealing with potential

375
00:35:16,000 --> 00:35:19,000
functions.
And, the other thing is,

376
00:35:19,000 --> 00:35:23,000
well, what about phi of L0?
Is that equal to zero?

377
00:35:23,000 --> 00:35:27,000
Well, it depends upon what list
they start with.

378
00:35:27,000 --> 00:35:31,000
OK, so what's the initial
ordering?

379
00:35:31,000 --> 00:35:35,000
So, it's zero if they start
with the same list.

380
00:35:35,000 --> 00:35:41,000
Then there are no inversions.
But, they might start with

381
00:35:41,000 --> 00:35:46,000
different lists.
We'll talk about different

382
00:35:46,000 --> 00:35:50,000
lists later on,
but let's say for now that it's

383
00:35:50,000 --> 00:35:55,000
zero because they start with the
same list.

384
00:35:55,000 --> 00:36:00,000
That seems like a fair
comparison.

385
00:36:00,000 --> 00:36:04,000
OK, so we have this potential
function now that's counting up,

386
00:36:04,000 --> 00:36:07,000
how different are these two
lists?

387
00:36:07,000 --> 00:36:10,000
Intuitively,
we're going to do is the more

388
00:36:10,000 --> 00:36:14,000
differences there are in the
list, the more we are going to

389
00:36:14,000 --> 00:36:18,000
be able to have more stored up
work than we can pay for it.

390
00:36:18,000 --> 00:36:22,000
That's the basic idea.
So, the more that opt changes

391
00:36:22,000 --> 00:36:27,000
the list, so it's not the same
as ours, in some sense the more

392
00:36:27,000 --> 00:36:32,000
we are going to be in a position
as MTF to take advantage of that

393
00:36:32,000 --> 00:36:37,000
difference in delivering up work
for us to do.

394
00:36:37,000 --> 00:36:42,000
And we'll see how that plays
out.

395
00:36:42,000 --> 00:36:50,000
So, let's first also make
another observation.

396
00:36:50,000 --> 00:36:59,000
So, how much does phi change
from one transpose?

397
00:36:59,000 --> 00:37:08,000
How much does phi change from
one transpose?

398
00:37:08,000 --> 00:37:14,000
So, basically that's asking,
if you do a transpose,

399
00:37:14,000 --> 00:37:18,000
what happens to the number of
inversions?

400
00:37:18,000 --> 00:37:24,000
So, what happens when a
transposing is done?

401
00:37:24,000 --> 00:37:31,000
What's going to happen to phi?
What's going to happen to the

402
00:37:31,000 --> 00:37:37,000
number of inversions?
So, if I change,

403
00:37:37,000 --> 00:37:43,000
it is less than n minus one,
yes, if n is sufficiently

404
00:37:43,000 --> 00:37:47,000
large, yes.
OK, if I change,

405
00:37:47,000 --> 00:37:54,000
so you can think about it here.
Suppose I switch two of these

406
00:37:54,000 --> 00:37:59,000
elements here.
How much are things going to

407
00:37:59,000 --> 00:38:05,000
change?
Yeah, it's basically one or

408
00:38:05,000 --> 00:38:11,000
minus one, OK,
because a transpose creates or

409
00:38:11,000 --> 00:38:17,000
destroys one inversion.
So, if you think about it,

410
00:38:17,000 --> 00:38:21,000
what if I change,
for example,

411
00:38:21,000 --> 00:38:29,000
C and A, the relationship of C
and A to everything else in the

412
00:38:29,000 --> 00:38:36,000
list is going to stay the same.
The only thing,

413
00:38:36,000 --> 00:38:41,000
possibly, that happens is that
if they are in the same order

414
00:38:41,000 --> 00:38:46,000
when I transpose them,
I've created an inversion.

415
00:38:46,000 --> 00:38:51,000
Or, if they were in the wrong
order when I transpose them,

416
00:38:51,000 --> 00:38:55,000
now they're in the right order.
So therefore,

417
00:38:55,000 --> 00:39:01,000
the change to the potential
function is going to be plus or

418
00:39:01,000 --> 00:39:08,000
minus two because we're counting
twice the number of inversions.

419
00:39:08,000 --> 00:39:14,000
OK, any questions about that?
So, transposes don't change the

420
00:39:14,000 --> 00:39:17,000
potential very much,
just by one.

421
00:39:17,000 --> 00:39:22,000
It either goes up by two or
down by two, just by one

422
00:39:22,000 --> 00:39:26,000
inversion.
So now, let's take a look at

423
00:39:26,000 --> 00:39:31,000
how these two algorithms
operate.

424
00:39:46,000 --> 00:40:03,000
OK, so what happens when op i
accesses x in the two lists?

425
00:40:03,000 --> 00:40:07,000
What's going to be going on?
To do that, let's define the

426
00:40:07,000 --> 00:40:09,000
following sets.

427
00:40:33,000 --> 00:40:34,000
Why do I keep doing that?

428
00:41:54,000 --> 00:41:57,000
OK, so we're going to look at
the, when we access x,

429
00:41:57,000 --> 00:42:01,000
we are going to look at the two
lists, and see what the

430
00:42:01,000 --> 00:42:05,000
relationship is,
so, based on things that come

431
00:42:05,000 --> 00:42:10,000
before and after.
So, I think a picture is very

432
00:42:10,000 --> 00:42:15,000
helpful to understand what's
going on here.

433
00:42:15,000 --> 00:42:19,000
OK, so let's let,
so here's L_i minus one,

434
00:42:19,000 --> 00:42:25,000
and we have our list,
which I'll draw like this.

435
00:42:25,000 --> 00:42:30,000
And somewhere in there,
we have x.

436
00:42:30,000 --> 00:42:40,000
OK, and then we have L_i minus
one star, which is opt's list,

437
00:42:40,000 --> 00:42:48,000
OK, and he's got x somewhere
else, or she.

438
00:42:48,000 --> 00:42:59,000
OK, and so, what is this set?
This is the set of Y that come

439
00:42:59,000 --> 00:43:05,000
before x.
So, that basically sets A and

440
00:43:05,000 --> 00:43:08,000
B.
OK, those things that come

441
00:43:08,000 --> 00:43:13,000
before x in both.
And, some of them,

442
00:43:13,000 --> 00:43:19,000
the A's come before it in x,
but come after it in,

443
00:43:19,000 --> 00:43:24,000
come before it in A,
but come after it in B.

444
00:43:24,000 --> 00:43:31,000
OK, and similarly down here,
what's this set?

445
00:43:40,000 --> 00:43:42,000
That's A union C,
good.

446
00:43:42,000 --> 00:43:44,000
And this one?
Duh.

447
00:43:44,000 --> 00:43:51,000
Yeah, it better be C union D
because I've got A union B over

448
00:43:51,000 --> 00:43:57,000
there, and I've got x.
So that better be everything

449
00:43:57,000 --> 00:44:02,000
else.
OK, and here is B union D.

450
00:44:02,000 --> 00:44:11,000
OK, so those are the four sets
that we're going to care about.

451
00:44:11,000 --> 00:44:19,000
We're actually mostly going to
care about these two sets.

452
00:44:19,000 --> 00:44:26,000
OK, and we also know something
about the r here.

453
00:44:26,000 --> 00:44:35,000
The position of x is going to
be the rank in L_i minus one of

454
00:44:35,000 --> 00:44:39,000
x.
And here, this is our star.

455
00:44:39,000 --> 00:44:44,000
It's just to the rank in L_i
minus one star of x.

456
00:44:44,000 --> 00:44:47,000
So, we know what these ranks
are.

457
00:44:47,000 --> 00:44:51,000
And what we're going to be
interested in is,

458
00:44:51,000 --> 00:44:56,000
in fact, characterizing the
rank in terms of the sets.

459
00:44:56,000 --> 00:45:01,000
OK, so what's the position of
this?

460
00:45:01,000 --> 00:45:09,000
Well, the rank,
we have that r is equal to the

461
00:45:09,000 --> 00:45:17,000
size of A.
What's the size of B plus one?

462
00:45:17,000 --> 00:45:28,000
OK, and r star is equal to the
size of A plus the size of C

463
00:45:28,000 --> 00:45:34,000
plus one.
So, let's take a look at what

464
00:45:34,000 --> 00:45:41,000
happens when these two
algorithms do their thing.

465
00:45:41,000 --> 00:45:48,000
So, when the access to x
occurs, we move x to the front

466
00:45:48,000 --> 00:45:53,000
of the list.
OK, it goes right up to the

467
00:45:53,000 --> 00:45:57,000
front.
So, how many inversions are

468
00:45:57,000 --> 00:46:03,000
created and destroyed?
So, how many are created by

469
00:46:03,000 --> 00:46:05,000
this?
That's probably a,

470
00:46:05,000 --> 00:46:08,000
how many inversions are
created?

471
00:46:33,000 --> 00:46:35,000
How many inversions are
created?

472
00:46:35,000 --> 00:46:39,000
So, we move x to the front.
So what we are concerned about

473
00:46:39,000 --> 00:46:43,000
is that anything that was in one
of these sets that came,

474
00:46:43,000 --> 00:46:47,000
where it's going to change in
order versus down here.

475
00:46:47,000 --> 00:46:51,000
So, if I look in B,
well, let's take a look at A.

476
00:46:51,000 --> 00:46:56,000
OK, so A, those are the things
that are in the same order in

477
00:46:56,000 --> 00:46:58,000
both.
So, everything that's in A,

478
00:46:58,000 --> 00:47:02,000
when I move x to the front,
each thing in A is going to

479
00:47:02,000 --> 00:47:09,000
count for one more inversion.
Does everybody see that?

480
00:47:09,000 --> 00:47:15,000
So, I create a cardinality of A
inversions.

481
00:47:15,000 --> 00:47:23,000
And, we are going to destroy,
well, everything in B came

482
00:47:23,000 --> 00:47:31,000
before x in this list,
and after x in this.

483
00:47:31,000 --> 00:47:36,000
But after we move x,
they're in the right order.

484
00:47:36,000 --> 00:47:43,000
So, I'm going to destroy B
inversions, cardinality of B

485
00:47:43,000 --> 00:47:48,000
inversions.
OK, so that's what happens we

486
00:47:48,000 --> 00:47:52,000
operate with move to front.
We destroy.

487
00:47:52,000 --> 00:47:58,000
We create A inversions and
destroy B inversions,

488
00:47:58,000 --> 00:48:06,000
OK, by doing this movement.
OK, now, let's take a look at

489
00:48:06,000 --> 00:48:09,000
what opt does.
So, each transpose,

490
00:48:09,000 --> 00:48:16,000
we don't know what opt does.
He might move x this way or

491
00:48:16,000 --> 00:48:18,000
that way.
We don't know.

492
00:48:18,000 --> 00:48:22,000
But each transpose,
I opt, well,

493
00:48:22,000 --> 00:48:29,000
we're going to be interested in
how many inversions it creates,

494
00:48:29,000 --> 00:48:34,000
and we already argued that it's
going to create,

495
00:48:34,000 --> 00:48:40,000
at most, one inversion per
transpose.

496
00:48:40,000 --> 00:48:46,000
So, he can go and create more
inversions, OK?

497
00:48:46,000 --> 00:48:53,000
So, let me write it over here.
Thus --

498
00:49:05,000 --> 00:49:16,000
-- the change in potential is
going to be, at most,

499
00:49:16,000 --> 00:49:26,000
twice, A minus B plus t_i.
OK, so t_i, remember,

500
00:49:26,000 --> 00:49:39,000
was the number of transposes
that opt does on the i'th step

501
00:49:39,000 --> 00:49:49,000
for the i'th operation.
OK, so we're going to create

502
00:49:49,000 --> 00:49:56,000
the change in potential is,
at most, twice this function.

503
00:49:56,000 --> 00:50:07,000
So, we are now going to look to
see how we use this fact,

504
00:50:07,000 --> 00:50:16,000
and these two facts,
this fact and this fact,

505
00:50:16,000 --> 00:50:26,000
OK, to show that opt can't be
much better than MTF.

506
00:50:26,000 --> 00:50:33,000
OK, good.
The way we are going to do that

507
00:50:33,000 --> 00:50:38,000
is look at the amortized cost of
the I'th operation.

508
00:50:38,000 --> 00:50:41,000
OK, what's MTF's amortized
cost?

509
00:50:41,000 --> 00:50:47,000
OK, and then we'll make the
argument, which is the one you

510
00:50:47,000 --> 00:50:53,000
always make that the amortized
cost upper bound the true costs,

511
00:50:53,000 --> 00:50:57,000
OK?
But the amortized cost is going

512
00:50:57,000 --> 00:51:04,000
to be easier to calculate.
OK, so amortized cost is just C

513
00:51:04,000 --> 00:51:09,000
hat, actually,
let me make sure I have lots of

514
00:51:09,000 --> 00:51:15,000
room here on the right,
c_i hat, which is equal to the

515
00:51:15,000 --> 00:51:19,000
true cost plus the change in
potential.

516
00:51:19,000 --> 00:51:25,000
OK, that's just the definition
of amortized cost when given

517
00:51:25,000 --> 00:51:29,000
potential functions,
OK?

518
00:51:29,000 --> 00:51:36,000
So, what is the cost of
operation i, OK,

519
00:51:36,000 --> 00:51:45,000
in this context here?
OK, we accessed x there.

520
00:51:45,000 --> 00:51:56,000
What's the cost of operation i?
Two times the rank of x,

521
00:51:56,000 --> 00:52:05,000
which is 2r.
OK, so 2r, that part of it.

522
00:52:05,000 --> 00:52:13,000
OK, well, we have an upper
bound on the change in

523
00:52:13,000 --> 00:52:17,000
potential.
That's this.

524
00:52:17,000 --> 00:52:25,000
OK, so that's two times the
cardinality of A minus

525
00:52:25,000 --> 00:52:33,000
cardinality of B plus t_i.
OK, everybody with me?

526
00:52:33,000 --> 00:52:37,000
Yeah?
OK, I see lots of nods.

527
00:52:37,000 --> 00:52:42,000
That's good.
OK, that's equal to 2r plus two

528
00:52:42,000 --> 00:52:48,000
of size of A minus,
OK, I want to plug in for B,

529
00:52:48,000 --> 00:52:55,000
and it turns out very nicely.
I have an equation involving A,

530
00:52:55,000 --> 00:53:00,000
B, and r.
So, I get rid of the variable

531
00:53:00,000 --> 00:53:06,000
size of B by just plugging that
in.

532
00:53:06,000 --> 00:53:11,000
OK, and so what do I plug in
here?

533
00:53:11,000 --> 00:53:18,000
What's B equal to?
Yeah, r minus size of A minus

534
00:53:18,000 --> 00:53:23,000
one.
I wrote it the other way.

535
00:53:23,000 --> 00:53:31,000
OK, and then plus t_i.
OK, and this is since r is A

536
00:53:31,000 --> 00:53:37,000
plus B plus one.
OK, everybody with me still?

537
00:53:37,000 --> 00:53:44,000
I'm just doing algebra.
We've got to make sure we do

538
00:53:44,000 --> 00:53:50,000
the algebra right.
OK, so that's equal to,

539
00:53:50,000 --> 00:53:57,000
let's just multiply all this
out now and get 2r plus,

540
00:53:57,000 --> 00:54:03,000
I have 2A here minus A.
So, that's 4A.

541
00:54:03,000 --> 00:54:08,000
And then, two times minus r is
minus 2r.

542
00:54:08,000 --> 00:54:12,000
Two times minus one is minus
two.

543
00:54:12,000 --> 00:54:18,000
Oh, but it's minus-minus two,
so it's plus two.

544
00:54:18,000 --> 00:54:24,000
OK, and then I have 2t_i.
So, that's just algebra.

545
00:54:24,000 --> 00:54:31,000
OK, so that's not bad.
We've just got rid of another

546
00:54:31,000 --> 00:54:37,000
variable.
What variable did we get rid

547
00:54:37,000 --> 00:54:38,000
of?
r.

548
00:54:38,000 --> 00:54:46,000
It didn't matter what the rank
was as long as I knew what the

549
00:54:46,000 --> 00:54:54,000
number of inversions was here.
OK, so that's now equal to 4A

550
00:54:54,000 --> 00:55:02,000
plus two plus 2t_i.
And, that's less than or equal

551
00:55:02,000 --> 00:55:11,000
to, I claim, four times r star
plus t_i using our other fact.

552
00:55:11,000 --> 00:55:19,000
Since r star is equal to the
size of A plus the size of C,

553
00:55:19,000 --> 00:55:28,000
plus one, then that's greater
than or equal to the size of A

554
00:55:28,000 --> 00:55:35,000
plus one.
OK, if I look at this,

555
00:55:35,000 --> 00:55:42,000
I'm basically looking at A.
The fact that A,

556
00:55:42,000 --> 00:55:51,000
what did I do here?
If r star is greater than or

557
00:55:51,000 --> 00:55:58,000
equal to A plus one,
right, so therefore,

558
00:55:58,000 --> 00:56:05,000
A plus one, good.
Yeah, so this is basically less

559
00:56:05,000 --> 00:56:09,000
than or equal to 4A plus four,
which is four times A plus one.

560
00:56:09,000 --> 00:56:13,000
I probably should have put in
another algebra step here,

561
00:56:13,000 --> 00:56:16,000
OK, because if I can't verify
it like this,

562
00:56:16,000 --> 00:56:19,000
then I get nervous.
This is basically,

563
00:56:19,000 --> 00:56:23,000
at most, 4A plus four.
That's four times A plus one,

564
00:56:23,000 --> 00:56:26,000
and A plus one is less than or
equal to r star.

565
00:56:26,000 --> 00:56:30,000
And then, 2t_i is,
at most, 4TI.

566
00:56:30,000 --> 00:56:43,000
So, I've got this.
Does everybody see where that

567
00:56:43,000 --> 00:56:54,000
came from?
But what is r star plus t_i?

568
00:56:54,000 --> 00:57:04,000
What is r star plus t_i?
What is it?

569
00:57:04,000 --> 00:57:13,000
It's c_i star.
That's just c_i star.

570
00:57:13,000 --> 00:57:24,000
So, the amortized cost of i'th
operation is,

571
00:57:24,000 --> 00:57:36,000
at most, four times opt's cost.
OK, that's pretty remarkable.

572
00:57:36,000 --> 00:57:44,000
OK, so amortized cost of the
i'th operation is just four

573
00:57:44,000 --> 00:57:48,000
times opt's cost.
Now, of course,

574
00:57:48,000 --> 00:57:55,000
we have to now go through and
analyze the total cost.

575
00:57:55,000 --> 00:58:03,000
But this is now the routine way
that we analyze things with a

576
00:58:03,000 --> 00:58:12,000
potential function.
So, the costs of MTF of S is

577
00:58:12,000 --> 00:58:21,000
just the summation of the
individual costs,

578
00:58:21,000 --> 00:58:30,000
OK, by definition.
And that is just the sum,

579
00:58:30,000 --> 00:58:38,000
i equals one,
to S of the amortized cost

580
00:58:38,000 --> 00:58:45,000
plus, minus the change in
potential.

581
00:58:45,000 --> 00:58:55,000
OK, did I do this right?
No, I put the parentheses in

582
00:58:55,000 --> 00:59:01,000
the wrong place.
Now I've got it right.

583
00:59:01,000 --> 00:59:04,000
Good.
I just missed a parenthesis.

584
00:59:04,000 --> 00:59:08,000
OK, so this is,
so in the past what I did was I

585
00:59:08,000 --> 00:59:13,000
expressed the amortized cost as
being equal to c_i plus the

586
00:59:13,000 --> 00:59:17,000
change in potential.
I'm just throwing these two

587
00:59:17,000 --> 00:59:22,000
terms over to the other side and
saying, what's the true cost in

588
00:59:22,000 --> 00:59:26,000
terms of the amortized cost?
OK, so I get c hat of i plus

589
00:59:26,000 --> 00:59:31,000
phi sub L_i minus one minus phi
of L_i, OK, by making that

590
00:59:31,000 --> 00:59:36,000
substitution.
OK, that's less than or equal

591
00:59:36,000 --> 00:59:41,000
to since this is linear.
Well, I know what the sum of

592
00:59:41,000 --> 00:59:44,000
the amortized cost is.
It's, at most,

593
00:59:44,000 --> 00:59:47,000
4c_i star.
So, the sum of them is,

594
00:59:47,000 --> 00:59:52,000
at most, to that sum,
I equals one to S of 4c_i star.

595
00:59:52,000 --> 00:59:56,000
And then, as happens in all
these things,

596
00:59:56,000 --> 01:00:01,000
you get a telescope with these
terms.

597
01:00:01,000 --> 01:00:08,706
Every term is added in once and
subtracted out once,

598
01:00:08,706 --> 01:00:13,542
except for the ones at the
limit.

599
01:00:13,542 --> 01:00:22,760
So, I get plus phi of L_0 minus
phi of L sub cardinality of S.

600
01:00:22,760 --> 01:00:31,222
And now, this term is zero.
And this term is greater than

601
01:00:31,222 --> 01:00:39,468
or equal to zero.
OK, so therefore this whole

602
01:00:39,468 --> 01:00:47,579
thing is less than or equal to,
well, what's that?

603
01:00:47,579 --> 01:00:53,041
That's just four times opt's
cost.

604
01:00:53,041 --> 01:01:00,591
And so, we're four competitive.
OK, this is amazing,

605
01:01:00,591 --> 01:01:02,739
I think.
It's not that hard,

606
01:01:02,739 --> 01:01:06,956
OK, but it's quite amazing that
just by doing a simple

607
01:01:06,956 --> 01:01:11,650
heuristic, you're nearly as good
as any omniscient algorithm

608
01:01:11,650 --> 01:01:15,151
could possibly be.
OK, you're nearly as good.

609
01:01:15,151 --> 01:01:17,141
And, in fact,
in practice,

610
01:01:17,141 --> 01:01:21,358
this is a great heuristic.
So, if ever you have things

611
01:01:21,358 --> 01:01:25,893
like a hash table that you're
actually seeing by chaining,

612
01:01:25,893 --> 01:01:30,667
OK, often it's the case that if
when you access the elements,

613
01:01:30,667 --> 01:01:35,679
you're just bringing them up to
the front of the list if it's an

614
01:01:35,679 --> 01:01:40,533
unsorted list that you've put
them into, just bring them up to

615
01:01:40,533 --> 01:01:45,448
the front.
You can easily save 30 to 40%

616
01:01:45,448 --> 01:01:50,767
in run time for the accessing to
the hash table because you will

617
01:01:50,767 --> 01:01:54,736
be much more likely to find the
elements inside.

618
01:01:54,736 --> 01:01:59,295
Of course, it depends on the
distribution and so forth,

619
01:01:59,295 --> 01:02:03,601
for empirical matters,
but the point is that you are

620
01:02:03,601 --> 01:02:08,667
not going to be too far off from
the ordering that an optimal

621
01:02:08,667 --> 01:02:12,551
algorithm would do,
optimal off-line algorithm:

622
01:02:12,551 --> 01:02:17,037
I mean, amazing.
OK: optimal off-line.

623
01:02:17,037 --> 01:02:22,276
Now, it turns out that in the
reading that we assigned,

624
01:02:22,276 --> 01:02:28,000
so, we assigned you Sleator and
Tarjan's original paper.

625
01:02:28,000 --> 01:02:39,388
In that reading,
they actually have a slightly

626
01:02:39,388 --> 01:02:55,080
different model where they count
transposes that move in excess

627
01:02:55,080 --> 01:03:09,000
to element x towards the front
of the list as free.

628
01:03:09,000 --> 01:03:14,555
OK, so, and this basically
models, so here's the idea is if

629
01:03:14,555 --> 01:03:19,536
I actually have a linked list,
and when I chase down,

630
01:03:19,536 --> 01:03:23,846
once I find x,
I can actually move x up to the

631
01:03:23,846 --> 01:03:29,306
front with just a constant
number of pointer operations to

632
01:03:29,306 --> 01:03:34,000
splice it out and put it up to
the front.

633
01:03:34,000 --> 01:03:36,785
I don't actually have to
transpose all way back down.

634
01:03:36,785 --> 01:03:39,196
OK, so that's kind of the model
that they use,

635
01:03:39,196 --> 01:03:40,857
which is a more realistic
model.

636
01:03:40,857 --> 01:03:43,750
OK, I presented this argument
because it's a little bit

637
01:03:43,750 --> 01:03:45,732
simpler.
OK, and the model is a little

638
01:03:45,732 --> 01:03:47,285
bit simpler.
But in our model,

639
01:03:47,285 --> 01:03:50,339
they have, when you access
something, you want to bring it

640
01:03:50,339 --> 01:03:52,803
up to the front,
or anything that you happen to

641
01:03:52,803 --> 01:03:55,910
go across during that time,
you could bring up to the front

642
01:03:55,910 --> 01:04:06,000
essentially for free.
This model is the splicing in,

643
01:04:06,000 --> 01:04:17,466
splicing x in and out of L in
constant time.

644
01:04:17,466 --> 01:04:24,400
Then, MTF is,
it turns out,

645
01:04:24,400 --> 01:04:32,288
too competitive.
It's within a factor of two of

646
01:04:32,288 --> 01:04:36,487
optimal, OK, if you use that.
And that's actually a good

647
01:04:36,487 --> 01:04:40,761
exercise to work through.
You could also go read about it

648
01:04:40,761 --> 01:04:44,958
in the reading to understand
this better, to look to see

649
01:04:44,958 --> 01:04:47,401
where you would use those
things.

650
01:04:47,401 --> 01:04:51,675
You have to have another term
representing the number of,

651
01:04:51,675 --> 01:04:55,796
quote, "free" transposes.
But it turns out that all the

652
01:04:55,796 --> 01:04:58,467
math works out pretty much the
same.

653
01:04:58,467 --> 01:05:01,673
OK, let's see,
another thing I promised you

654
01:05:01,673 --> 01:05:06,023
is, what if, to look at the
case, what if they don't start

655
01:05:06,023 --> 01:05:14,480
with the same lists?
OK, what if the two lists are

656
01:05:14,480 --> 01:05:25,251
different when they start?
Then, the potential function at

657
01:05:25,251 --> 01:05:33,000
the beginning might be as big as
what?

658
01:05:33,000 --> 01:05:37,997
How big are the potential
function start out as if the

659
01:05:37,997 --> 01:05:42,805
lists are different?
So, suppose we're starting out,

660
01:05:42,805 --> 01:05:45,538
you have a list,
and opt says,

661
01:05:45,538 --> 01:05:51,008
OK, I'm going to start out by
ordering my list according to

662
01:05:51,008 --> 01:05:56,948
the sequence that I want to use,
OK, and MTF orders it according

663
01:05:56,948 --> 01:06:02,511
to the sequence it must use.
What list is opt going to start

664
01:06:02,511 --> 01:06:09,900
out with as an adversary?
Yeah, it's going to pick the

665
01:06:09,900 --> 01:06:16,447
reverse of what ever MTF starts
out with, right,

666
01:06:16,447 --> 01:06:21,601
because then,
if he picks the reverse,

667
01:06:21,601 --> 01:06:25,920
what's the number of
inversions?

668
01:06:25,920 --> 01:06:34,000
It's how many inversions in a
reverse ordered list?

669
01:06:34,000 --> 01:06:36,465
Yeah, n choose two,
OK.

670
01:06:36,465 --> 01:06:41,508
Is it n choose two,
or n minus one choose two?

671
01:06:41,508 --> 01:06:47,112
n minus one choose two,
OK, inversions that you get

672
01:06:47,112 --> 01:06:53,163
because it's basically a
triangular number when you add

673
01:06:53,163 --> 01:06:55,853
them up.
But in any case,

674
01:06:55,853 --> 01:07:00,000
it's order n^2,
worst case.

675
01:07:00,000 --> 01:07:07,029
So, what does that do to our
analysis here?

676
01:07:07,029 --> 01:07:15,064
It says that the cost of MTF of
S is going to be,

677
01:07:15,064 --> 01:07:22,596
well, this is no longer zero.
This is now n^2.

678
01:07:22,596 --> 01:07:30,630
OK, so we get that costs of MTF
of S is, at most,

679
01:07:30,630 --> 01:07:39,000
four times opt's thing plus
order n^2, OK?

680
01:07:39,000 --> 01:07:51,329
And, if we look at the
definition, did we erase it

681
01:07:51,329 --> 01:07:58,625
already?
OK, this is still for

682
01:07:58,625 --> 01:08:09,696
competitive, OK,
since n^2 is constant as the

683
01:08:09,696 --> 01:08:19,283
size of S goes to infinity.
This is, once again,

684
01:08:19,283 --> 01:08:22,363
sort of your notion of,
what does it mean to be a

685
01:08:22,363 --> 01:08:24,868
constant?
OK, so as the size of the list

686
01:08:24,868 --> 01:08:28,846
gets bigger, all we're doing is
accessing whatever that number,

687
01:08:28,846 --> 01:08:31,863
n, is of elements.
That number doesn't grow with

688
01:08:31,863 --> 01:08:34,752
the problem size,
OK, even if it starts out as

689
01:08:34,752 --> 01:08:37,000
some variable number,
n.

690
01:08:37,000 --> 01:08:42,578
OK, it doesn't grow with the
problem size.

691
01:08:42,578 --> 01:08:47,071
We still end up being
competitive.

692
01:08:47,071 --> 01:08:53,603
This is just the k that was in
that definition of

693
01:08:53,603 --> 01:08:58,229
competitiveness.
OK, any questions?

694
01:08:58,229 --> 01:09:02,761
Yeah?
Well, so you could change the

695
01:09:02,761 --> 01:09:05,523
cost model a little bit.
Yeah.

696
01:09:05,523 --> 01:09:08,666
And that's a good one to work
out.

697
01:09:08,666 --> 01:09:12,666
But if you say the cost of
transposing, so,

698
01:09:12,666 --> 01:09:17,904
the cost of transposing is
probably moving two pointers,

699
01:09:17,904 --> 01:09:21,523
approximately.
No, one, three pointers.

700
01:09:21,523 --> 01:09:26,952
So, suppose that the cost of,
wow, that's a good exercise,

701
01:09:26,952 --> 01:09:29,714
OK?
Suppose the cost was three

702
01:09:29,714 --> 01:09:34,571
times to do a transpose,
was three times the cost of

703
01:09:34,571 --> 01:09:40,000
doing an access,
of following a pointer.

704
01:09:40,000 --> 01:09:43,741
OK, how would that change the
number here?

705
01:09:43,741 --> 01:09:46,752
OK, good exercise,
great exercise.

706
01:09:46,752 --> 01:09:51,863
OK, hmm, good final question.
OK, yes, it will affect the

707
01:09:51,863 --> 01:09:56,517
constant here just as when we do
the free transpose,

708
01:09:56,517 --> 01:10:02,266
when we move things towards the
front, that we consider those as

709
01:10:02,266 --> 01:10:06,090
free, OK.
Those operations end up

710
01:10:06,090 --> 01:10:11,545
reducing the constant as well.
OK, but the point is that this

711
01:10:11,545 --> 01:10:17,000
constant is independent of the
constant having to do with the

712
01:10:17,000 --> 01:10:22,636
number of elements in the list.
So that's a different constant.

713
01:10:22,636 --> 01:10:27,181
So, this is a constant.
OK, and so as with a lot of

714
01:10:27,181 --> 01:10:31,000
these things,
there's two things.

715
01:10:31,000 --> 01:10:34,505
One is, there's the theory.
So, theory here backs up

716
01:10:34,505 --> 01:10:37,048
practice.
OK, those practitioners knew

717
01:10:37,048 --> 01:10:40,485
what they were doing,
OK, without knowing what they

718
01:10:40,485 --> 01:10:43,028
were doing.
OK, so that's really good.

719
01:10:43,028 --> 01:10:46,603
OK, and we have a deeper
understanding that's led to,

720
01:10:46,603 --> 01:10:50,727
as I say, many algorithms for
things like, the important ones

721
01:10:50,727 --> 01:10:53,682
are like paging.
So, what's the comment page

722
01:10:53,682 --> 01:10:57,532
replacement policy that people
study, people have at most

723
01:10:57,532 --> 01:11:02,000
operating systems?
Who's done 6.033 or something?

724
01:11:02,000 --> 01:11:04,272
Yeah, it's Least Recently Used,
LRU.

725
01:11:04,272 --> 01:11:06,155
People have heard of that,
OK.

726
01:11:06,155 --> 01:11:09,597
So, you can analyze LRU
competitive, and show that LRU

727
01:11:09,597 --> 01:11:13,363
is actually competitive with
optimal page replacement under

728
01:11:13,363 --> 01:11:16,480
certain assumptions.
OK, and there are also other

729
01:11:16,480 --> 01:11:18,363
things.
Like, people do random

730
01:11:18,363 --> 01:11:21,805
replacement algorithms,
and there are a whole bunch of

731
01:11:21,805 --> 01:11:25,831
other kinds of things that can
be analyzed with the competitive

732
01:11:25,831 --> 01:11:28,883
analysis framework.
OK, so it's very cool stuff.

733
01:11:28,883 --> 01:11:32,324
And, we are going to see more
in recitation on Friday,

734
01:11:32,324 --> 01:11:36,090
see a couple of other really
good problems that are maybe a

735
01:11:36,090 --> 01:11:40,181
little bit easier than this one,
OK, definitely easier than this

736
01:11:40,181 --> 01:11:45,479
one.
OK, they give you hopefully

737
01:11:45,479 --> 01:11:51,407
some more intuition about
competitive analysis.

738
01:11:51,407 --> 01:11:58,237
I also want to warn you about
next week's problem set.

739
01:11:58,237 --> 01:12:07,000
So, next week's problem set has
a programming assignment on it.

740
01:12:07,000 --> 01:12:10,676
OK, and the programming
assignment is mandatory,

741
01:12:10,676 --> 01:12:13,649
meaning, well,
all the problem sets are

742
01:12:13,649 --> 01:12:17,639
mandatory as you know,
but if you decide not to do a

743
01:12:17,639 --> 01:12:22,568
problem there's a little bit of
a penalty and then the penalties

744
01:12:22,568 --> 01:12:26,401
scale dramatically as you stop
doing problem sets.

745
01:12:26,401 --> 01:12:30,000
But this one is
mandatory-mandatory.

746
01:12:30,000 --> 01:12:33,823
OK, you don't pass the class.
You'll get an incomplete if you

747
01:12:33,823 --> 01:12:36,180
do not do this programming
assignment.

748
01:12:36,180 --> 01:12:39,430
Now, I know that some people
are less practiced with

749
01:12:39,430 --> 01:12:42,169
programming.
And so, what I encourage you to

750
01:12:42,169 --> 01:12:45,865
do over the weekend is spent a
few minutes and work on your

751
01:12:45,865 --> 01:12:49,624
programming skills if you're not
up to snuff in programming.

752
01:12:49,624 --> 01:12:53,129
It's not going to be a long
assignment, but if you don't

753
01:12:53,129 --> 01:12:55,932
know how to read a file and
write out a file,

754
01:12:55,932 --> 01:12:58,608
and be able to write a dozen
lines of code,

755
01:12:58,608 --> 01:13:02,176
OK, if you are weak on that,
this weekend would be a good

756
01:13:02,176 --> 01:13:06,000
idea to practice reading in a
text file.

757
01:13:06,000 --> 01:13:09,571
It's going to be a text file.
Read it in a text file,

758
01:13:09,571 --> 01:13:12,524
decent manipulations,
write out a text file,

759
01:13:12,524 --> 01:13:14,791
OK?
So, I don't want people to get

760
01:13:14,791 --> 01:13:19,186
caught with this being mandatory
and that not have time to finish

761
01:13:19,186 --> 01:13:23,513
it because they are busy trying
to learn how to program in short

762
01:13:23,513 --> 01:13:25,848
order.
I know some people take this

763
01:13:25,848 --> 01:13:31,000
course without quite getting all
the programming prerequisites.

764
01:13:31,000 --> 01:13:33,964
Here's where you need it.
Question?

765
01:13:33,964 --> 01:13:37,712
No language limitations.
Pick your language.

766
01:13:37,712 --> 01:13:41,548
The answer will be written in,
I think, Java,

767
01:13:41,548 --> 01:13:46,082
and Eric has graciously
volunteered to use Python for

768
01:13:46,082 --> 01:13:51,138
his solution to this problem.
We'll see whether he lives up

769
01:13:51,138 --> 01:13:53,928
to that promise.
You did already?

770
01:13:53,928 --> 01:13:57,241
OK, and George wrote the Java
solution.

771
01:13:57,241 --> 01:14:00,117
And so, C is fine.
Matlab is fine.

772
01:14:00,117 --> 01:14:05,000
OK, what else is fine?
Anything is fine.

773
01:14:05,000 --> 01:14:07,600
Scheme is fine.
Scheme is fine.

774
01:14:07,600 --> 01:14:11,587
Scheme is great.
OK, so any such things will be

775
01:14:11,587 --> 01:14:15,141
just fine.
So, we don't care what language

776
01:14:15,141 --> 01:14:18,434
you program in,
but you will have to do

777
01:14:18,434 --> 01:14:21,295
programming to solve this
problem.

778
01:14:21,295 --> 01:14:24,000
OK, so thanks very much.
See you next week.