1
00:00:00,577 --> 00:00:02,410
PROFESSOR: The issue
of the approximate rate

2
00:00:02,410 --> 00:00:06,120
at which things happen is a
common in a lot of sciences,

3
00:00:06,120 --> 00:00:08,370
the rate at which
things fall, or the rate

4
00:00:08,370 --> 00:00:10,350
at which reactions occur.

5
00:00:10,350 --> 00:00:14,410
And in computer science, typical
concern about growth rates

6
00:00:14,410 --> 00:00:17,680
comes up in looking at the
efficiency of algorithms,

7
00:00:17,680 --> 00:00:21,350
and whether they're growing
linearly, or quadratically,

8
00:00:21,350 --> 00:00:22,910
or more.

9
00:00:22,910 --> 00:00:26,600
And we're going to look
today at four notations that

10
00:00:26,600 --> 00:00:30,860
describe relations between
the growth rates of functions.

11
00:00:30,860 --> 00:00:33,000
The first of these relations
is the simplest one.

12
00:00:33,000 --> 00:00:35,100
It's called asymptotic
equivalence,

13
00:00:35,100 --> 00:00:36,850
or asymptotic equality.

14
00:00:36,850 --> 00:00:42,120
So this is tilde symbol is read
as asymptotically equal to.

15
00:00:42,120 --> 00:00:45,970
So f of n is asymptotically
equal to g of n,

16
00:00:45,970 --> 00:00:50,790
if and only if the limit of the
quotient of f of n over g of n

17
00:00:50,790 --> 00:00:52,249
is 1.

18
00:00:52,249 --> 00:00:53,290
Let's look at an example.

19
00:00:59,410 --> 00:01:03,220
N squared is asymptotically
equal to n squared plus n.

20
00:01:03,220 --> 00:01:03,820
Why is that?

21
00:01:03,820 --> 00:01:08,080
Well, it follows trivially
by manipulating the algebra.

22
00:01:08,080 --> 00:01:11,250
The limit of n squared
plus n over n squared,

23
00:01:11,250 --> 00:01:15,140
simplifying, is just the same
as the limit of 1 plus 1 over n.

24
00:01:15,140 --> 00:01:16,880
But as n goes to
infinity, 1 over n

25
00:01:16,880 --> 00:01:20,310
goes to 0, so the
limit is 1 as claimed.

26
00:01:20,310 --> 00:01:23,650
Those two expressions, or
the functions they define,

27
00:01:23,650 --> 00:01:28,750
n squared and n squared plus
1 are asymptotically equal.

28
00:01:28,750 --> 00:01:31,810
So there's some easy properties
of asymptotic equality

29
00:01:31,810 --> 00:01:34,050
that follow immediately
from the definition.

30
00:01:34,050 --> 00:01:37,930
One of them is that
it's symmetric,

31
00:01:37,930 --> 00:01:42,080
namely, suppose that f is
asymptotically equal to g.

32
00:01:42,080 --> 00:01:44,630
I want to prove that g is
asymptotically equal to f.

33
00:01:44,630 --> 00:01:46,420
Well, what's going on here?

34
00:01:46,420 --> 00:01:50,100
Let's look at the limit
of g over f, which

35
00:01:50,100 --> 00:01:51,940
is what I'd like to prove as 1.

36
00:01:51,940 --> 00:01:55,480
Well, the limit of g over
f, by algebra, g over f

37
00:01:55,480 --> 00:02:00,140
is the same as 1 over f over
g, so just moving a limit

38
00:02:00,140 --> 00:02:02,150
across the division,
that's the same as 1

39
00:02:02,150 --> 00:02:05,710
over the limit of f over
g, which is 1 over 1.

40
00:02:05,710 --> 00:02:07,750
And we've proved in
other words, the g

41
00:02:07,750 --> 00:02:11,650
is asymptotically equal to f,
given that f is asymptotically

42
00:02:11,650 --> 00:02:12,820
equal to g.

43
00:02:12,820 --> 00:02:14,630
It's symmetric.

44
00:02:14,630 --> 00:02:16,472
There's a similar
argument for transitivity.

45
00:02:16,472 --> 00:02:18,930
Let's just crank through it
for practice on the definition.

46
00:02:18,930 --> 00:02:21,380
Suppose that f is
asymptotically equal to g,

47
00:02:21,380 --> 00:02:23,240
and g is asymptotically
equal to h,

48
00:02:23,240 --> 00:02:27,790
I'd like to prove that f it's
asymptotically equal to h.

49
00:02:27,790 --> 00:02:32,120
Well, again, we just
plug into the algebra

50
00:02:32,120 --> 00:02:33,190
and distribute limits.

51
00:02:33,190 --> 00:02:34,220
Let's just look at this.

52
00:02:34,220 --> 00:02:36,920
We're given that 1 is
the limit of f over g,

53
00:02:36,920 --> 00:02:39,040
because f is asymptotic to g.

54
00:02:39,040 --> 00:02:43,200
But f over g can be expressed
as f over h divided by g over h.

55
00:02:43,200 --> 00:02:44,430
That's just algebra.

56
00:02:44,430 --> 00:02:46,070
The h's cancel.

57
00:02:46,070 --> 00:02:49,770
So this limit, now, I
can distribute the limits

58
00:02:49,770 --> 00:02:51,380
to the numerator
and denominator,

59
00:02:51,380 --> 00:02:54,730
assuming both exist,
and the numerator limit

60
00:02:54,730 --> 00:02:56,200
is what I'm interested in.

61
00:02:56,200 --> 00:03:01,160
The denominator limit
is going to be 1,

62
00:03:01,160 --> 00:03:04,310
because the limit
of g over h is 1.

63
00:03:04,310 --> 00:03:07,380
Since g is asymptotically
equal to h,

64
00:03:07,380 --> 00:03:10,590
the conclusion is indeed
that the limit of f over h

65
00:03:10,590 --> 00:03:13,120
is equal to 1.

66
00:03:13,120 --> 00:03:15,980
This is not really
very interesting stuff,

67
00:03:15,980 --> 00:03:18,420
and the top level
message is that

68
00:03:18,420 --> 00:03:21,140
many of these
elementary properties

69
00:03:21,140 --> 00:03:25,480
of asymptotic equality and
the other asymptotic relations

70
00:03:25,480 --> 00:03:27,780
that we're going to
see follow by this kind

71
00:03:27,780 --> 00:03:31,090
of elementary algebra, and
distributing the limits

72
00:03:31,090 --> 00:03:33,460
over sub expressions.

73
00:03:33,460 --> 00:03:38,400
Anyway, the corollary of this
is that asymptotic equality

74
00:03:38,400 --> 00:03:40,259
is in fact an
equivalence relation.

75
00:03:40,259 --> 00:03:42,050
We've proved it's
symmetric and transitive,

76
00:03:42,050 --> 00:03:44,240
and it's trivially reflexive.

77
00:03:44,240 --> 00:03:46,930
It's an equivalence relation.

78
00:03:46,930 --> 00:03:49,270
By the way, it's
worth noting that it's

79
00:03:49,270 --> 00:03:53,020
an equivalence relation on
functions of one variable, when

80
00:03:53,020 --> 00:03:56,500
we write sometimes that
f of n is asymptotically

81
00:03:56,500 --> 00:03:58,170
equal to g of n.

82
00:03:58,170 --> 00:04:02,370
But we mean that f of n is the
description of the function f.

83
00:04:02,370 --> 00:04:04,420
We're not talking about
the number that f of n

84
00:04:04,420 --> 00:04:07,100
happens to have for some
particular value of f.

85
00:04:07,100 --> 00:04:09,110
So we'll sometimes
write f of n is

86
00:04:09,110 --> 00:04:10,890
asymptotically equal to g of n.

87
00:04:10,890 --> 00:04:13,130
For descriptive purposes,
the proper thing

88
00:04:13,130 --> 00:04:16,940
we should be writing is that f
is asymptotically equal to g.

89
00:04:16,940 --> 00:04:21,450
Asymptotic equality is a
relation between functions.

90
00:04:21,450 --> 00:04:22,240
OK.

91
00:04:22,240 --> 00:04:25,040
The next asymptotic relation
we're going to look at

92
00:04:25,040 --> 00:04:29,300
is called asymptotically smaller
than, and the notation for it

93
00:04:29,300 --> 00:04:30,800
is this little o notation.

94
00:04:30,800 --> 00:04:35,650
So you'd write that f of n is
equal to little o of g of n,

95
00:04:35,650 --> 00:04:41,930
if and only if the limit of f
of n over g of n goes to 0 as n'

96
00:04:41,930 --> 00:04:44,380
approaches infinity.

97
00:04:44,380 --> 00:04:45,910
So let's look at
an example of that.

98
00:04:45,910 --> 00:04:48,360
N squared is little
o of n cubed,

99
00:04:48,360 --> 00:04:52,570
because trivially, the limit
of n squared over n cubed

100
00:04:52,570 --> 00:04:54,130
is the same as the
limit of 1 over n.

101
00:04:54,130 --> 00:04:55,050
It's equal to zero.

102
00:04:57,610 --> 00:04:59,560
And by similar kind
of reasoning that we

103
00:04:59,560 --> 00:05:02,320
did for asymptotic equality
being an equivalence relation,

104
00:05:02,320 --> 00:05:07,440
it's not very hard to
prove that little o defines

105
00:05:07,440 --> 00:05:10,000
a strict partial
order on functions.

106
00:05:12,820 --> 00:05:17,510
So the third asymptotic
relation is the most complicated

107
00:05:17,510 --> 00:05:19,250
of the three.

108
00:05:19,250 --> 00:05:23,070
It's arguably the most
important in computer science.

109
00:05:23,070 --> 00:05:26,750
It's called the asymptotic
order of growth, o.

110
00:05:26,750 --> 00:05:28,570
And the definition
is that if function

111
00:05:28,570 --> 00:05:32,770
f is big o of a function
g, what it means

112
00:05:32,770 --> 00:05:37,940
is that the limit of
f over g is finite.

113
00:05:37,940 --> 00:05:43,930
So it might be other than
0 or 1, but it's finite.

114
00:05:43,930 --> 00:05:48,230
And that means that
f is big o of g.

115
00:05:48,230 --> 00:05:50,170
Now, there's a
technicality there

116
00:05:50,170 --> 00:05:54,170
where the expression actually
says the limsup of f of n

117
00:05:54,170 --> 00:05:55,420
over g of n.

118
00:05:55,420 --> 00:05:58,850
Let's just ignore that for
now, and we'll look at it

119
00:05:58,850 --> 00:06:02,740
and explain why the limsup
is there a little later.

120
00:06:02,740 --> 00:06:07,960
So as an example,
3n squared is big o

121
00:06:07,960 --> 00:06:11,670
of n squared, because
the quotient of 3n

122
00:06:11,670 --> 00:06:15,350
squared over n squared is 3,
which is less than infinity.

123
00:06:15,350 --> 00:06:18,540
So what big o is doing is kind
of saying that constant factors

124
00:06:18,540 --> 00:06:20,640
don't matter.

125
00:06:20,640 --> 00:06:25,150
And that turns out to be
particularly useful in computer

126
00:06:25,150 --> 00:06:28,110
science, where you can't
really talk about the time

127
00:06:28,110 --> 00:06:30,020
that a procedure takes,
because that's going

128
00:06:30,020 --> 00:06:31,890
to depend on the hardware.

129
00:06:31,890 --> 00:06:35,850
So when you implement
it on faster hardware,

130
00:06:35,850 --> 00:06:38,820
it may grow at the
same rate, but the time

131
00:06:38,820 --> 00:06:40,460
will actually change.

132
00:06:40,460 --> 00:06:45,120
And that's why big o
plays a prominent role.

133
00:06:45,120 --> 00:06:47,890
The final relation
of the four is called

134
00:06:47,890 --> 00:06:51,130
theta, or same order of growth.

135
00:06:51,130 --> 00:06:54,510
The definition of
f is theta of g

136
00:06:54,510 --> 00:06:59,700
is simply that f is o
of g and g is o of f.

137
00:06:59,700 --> 00:07:02,790
And it's easy to show
from the definition

138
00:07:02,790 --> 00:07:06,370
that theta is an
equivalence relation.

139
00:07:06,370 --> 00:07:11,290
So to summarize, there
are these four relations.

140
00:07:11,290 --> 00:07:15,370
F asymptotically equal
to g means informally

141
00:07:15,370 --> 00:07:18,860
that f and g are nearly equal.

142
00:07:18,860 --> 00:07:21,620
F equals little o of
g informally means

143
00:07:21,620 --> 00:07:23,845
that f is much less than g.

144
00:07:23,845 --> 00:07:27,256
F equals o of g means
that f is roughly less

145
00:07:27,256 --> 00:07:28,630
than or equal to
g, where roughly

146
00:07:28,630 --> 00:07:31,530
means that we're not concerned
about constant factors.

147
00:07:31,530 --> 00:07:36,760
And f equals theta of g means
that f is roughly equal to g.

148
00:07:36,760 --> 00:07:39,120
And we'll examine
these properties

149
00:07:39,120 --> 00:07:42,190
in more detail in
the next segment.