1
00:00:00,990 --> 00:00:04,270
So we have just seen
that a clever trick based

2
00:00:04,270 --> 00:00:06,890
on the frequency interpretation
of the transitions

3
00:00:06,890 --> 00:00:10,350
between successive
states, like here,

4
00:00:10,350 --> 00:00:13,530
allows us to write a simple
set of equations which

5
00:00:13,530 --> 00:00:19,800
can be solved recursively,
given here, giving pi i plus 1

6
00:00:19,800 --> 00:00:22,330
as a function of pi of i.

7
00:00:22,330 --> 00:00:25,720
More specifically,
we have pi i plus 1

8
00:00:25,720 --> 00:00:30,570
equals pi of i times p of i.

9
00:00:30,570 --> 00:00:33,330
Divide by q of i plus 1.

10
00:00:33,330 --> 00:00:38,040
And this is true for
i equal 0 up to m.

11
00:00:38,040 --> 00:00:42,620
And to start the recursion,
we need to find pi of 0.

12
00:00:42,620 --> 00:00:49,170
And this can be done using this
normalization condition-- which

13
00:00:49,170 --> 00:01:00,110
leads to pi of 0 times 1 plus
p0 over q1 plus et cetera

14
00:01:00,110 --> 00:01:02,650
equals 1.

15
00:01:02,650 --> 00:01:04,970
Let's illustrate the
details of this procedure

16
00:01:04,970 --> 00:01:06,710
on a special case.

17
00:01:06,710 --> 00:01:09,750
Let's assume that all
the p's are the same

18
00:01:09,750 --> 00:01:12,450
and all the q's are the same.

19
00:01:12,450 --> 00:01:16,390
So this is a special case
in which we are interested.

20
00:01:16,390 --> 00:01:21,030
So at each point in time, if
we are somewhere in the middle,

21
00:01:21,030 --> 00:01:24,110
you have probability
p of moving up,

22
00:01:24,110 --> 00:01:27,770
and probability
q of moving down.

23
00:01:27,770 --> 00:01:32,500
Define rho to be the
ratio of p over q.

24
00:01:32,500 --> 00:01:35,930
Rho can be interpreted as
the frequency of going up

25
00:01:35,930 --> 00:01:38,420
versus the frequency
of going down.

26
00:01:38,420 --> 00:01:41,190
If it's a service system,
you can think of it

27
00:01:41,190 --> 00:01:44,470
as a measure of how
loaded the system is.

28
00:01:44,470 --> 00:01:48,340
If p equals q, that means that
if you are at this state--

29
00:01:48,340 --> 00:01:52,500
you are equally likely
to move left or right.

30
00:01:52,500 --> 00:01:54,720
So the chain does
not have a tendency

31
00:01:54,720 --> 00:01:59,070
to move in that direction
or in that direction.

32
00:01:59,070 --> 00:02:04,880
If rho is bigger than 1,
so that p is bigger than q,

33
00:02:04,880 --> 00:02:08,430
it means that whenever we are
at some state in the middle,

34
00:02:08,430 --> 00:02:12,050
we are more likely to
move right, as opposed

35
00:02:12,050 --> 00:02:14,800
to moving left.

36
00:02:14,800 --> 00:02:17,230
Which means that the
chain has a tendency

37
00:02:17,230 --> 00:02:19,440
to move in that direction.

38
00:02:19,440 --> 00:02:23,240
And if you think of this as a
number of customers in queue,

39
00:02:23,240 --> 00:02:26,450
it means your system has the
tendency to become loaded

40
00:02:26,450 --> 00:02:28,160
and to build up a queue.

41
00:02:28,160 --> 00:02:31,480
So rho being bigger
than 1 corresponds

42
00:02:31,480 --> 00:02:34,390
to a heavy load,
where queues build up.

43
00:02:34,390 --> 00:02:37,079
Rho less than one
corresponds to the system

44
00:02:37,079 --> 00:02:39,370
where queues have the
tendency to drain down.

45
00:02:39,370 --> 00:02:42,690
The system is going to
move in that direction.

46
00:02:42,690 --> 00:02:46,210
Now let us write
down these equations

47
00:02:46,210 --> 00:02:48,350
for that special case.

48
00:02:48,350 --> 00:02:52,090
We end up with that,
which is pi i times rho,

49
00:02:52,090 --> 00:02:54,100
by definition of rho.

50
00:02:54,100 --> 00:02:58,760
Once you look at this equation,
you realize that pi of 1

51
00:02:58,760 --> 00:03:01,860
is pi of 0 times rho.

52
00:03:01,860 --> 00:03:07,280
And pi of 2 is pi of
1 times rho equals

53
00:03:07,280 --> 00:03:10,740
pi of 0 times rho square.

54
00:03:10,740 --> 00:03:12,810
And so on and so forth.

55
00:03:12,810 --> 00:03:16,200
And you find that you
can express pi of i

56
00:03:16,200 --> 00:03:18,750
as pi of 0 times
rho at the power

57
00:03:18,750 --> 00:03:24,600
i for any possible
i between 0 and m.

58
00:03:24,600 --> 00:03:31,780
And now if we use the
normalization condition,

59
00:03:31,780 --> 00:03:36,370
we get that pi of 0
times 1 plus rho plus rho

60
00:03:36,370 --> 00:03:44,050
squared plus rho at the
power m is equal to 1.

61
00:03:44,050 --> 00:03:45,970
Let's now complete
the calculations

62
00:03:45,970 --> 00:03:47,920
for two special cases.

63
00:03:47,920 --> 00:03:51,610
If rho is equal to 1,
that means p equals q.

64
00:03:51,610 --> 00:03:56,860
Then pi i equals
pi of 0 for all i.

65
00:03:56,860 --> 00:04:00,500
It means that all the steady
state probabilities are equal.

66
00:04:00,500 --> 00:04:04,690
This special case is called
a symmetric random walk.

67
00:04:04,690 --> 00:04:07,640
So you start at the
state at a point in time.

68
00:04:07,640 --> 00:04:11,260
Either you stay in place, or
you have an equal probability

69
00:04:11,260 --> 00:04:14,220
of going left or right.

70
00:04:14,220 --> 00:04:16,589
There is no bias in
either direction.

71
00:04:16,589 --> 00:04:18,510
You might think that
in such a process,

72
00:04:18,510 --> 00:04:20,428
you will tend to
get stuck either

73
00:04:20,428 --> 00:04:23,390
near one end or the other end.

74
00:04:23,390 --> 00:04:25,740
It turns out that
no, in the long run,

75
00:04:25,740 --> 00:04:28,250
the symmetric random
walk is equally likely

76
00:04:28,250 --> 00:04:30,660
to be at any of those states.

77
00:04:30,660 --> 00:04:34,659
And for the special case--
this equation here-- is simply

78
00:04:34,659 --> 00:04:40,159
that pi of 0 times
1 plus m equals one.

79
00:04:40,159 --> 00:04:47,520
That means that pi of 0
equals 1 over 1 plus m.

80
00:04:47,520 --> 00:04:49,350
Which is consistent
with the fact

81
00:04:49,350 --> 00:04:51,440
that all steady-state
probabilities are the same.

82
00:04:51,440 --> 00:04:52,890
They are all equally likely.

83
00:04:52,890 --> 00:04:54,370
They are end states.

84
00:04:54,370 --> 00:04:59,850
And so each one of them, pi i
is pi of 0, which is 1 over 1

85
00:04:59,850 --> 00:05:00,690
plus m.

86
00:05:00,690 --> 00:05:02,840
The Markov chain
is equally likely

87
00:05:02,840 --> 00:05:07,590
to be in any of these m plus
1 states in the long run.

88
00:05:07,590 --> 00:05:12,630
Suppose now instead of p equals
q, that m is very, very large,

89
00:05:12,630 --> 00:05:13,970
a very large number.

90
00:05:13,970 --> 00:05:16,410
Let's take m going to infinity.

91
00:05:16,410 --> 00:05:20,050
And suppose that the system
is on the stable side.

92
00:05:20,050 --> 00:05:23,320
That means that
p is less than q,

93
00:05:23,320 --> 00:05:25,960
which means that there's
a tendency for customers

94
00:05:25,960 --> 00:05:28,280
to be served faster
than they arrive.

95
00:05:28,280 --> 00:05:33,840
In other words, the chain is
drifting toward that direction.

96
00:05:33,840 --> 00:05:38,010
So that means that
rho is less than 1

97
00:05:38,010 --> 00:05:41,560
and what it means is that
this infinite series, when

98
00:05:41,560 --> 00:05:45,200
m goes to infinity, is
the geometric series.

99
00:05:45,200 --> 00:05:49,760
And this series is going
to be 1 over 1 minus rho.

100
00:05:49,760 --> 00:05:57,980
That is, this infinite
series is 1 over 1 minus rho.

101
00:05:57,980 --> 00:06:02,110
And since pi of 0 is 1
over this infinite series,

102
00:06:02,110 --> 00:06:06,800
we end up having pi
0 equals 1 minus rho.

103
00:06:06,800 --> 00:06:13,940
And since we have pi of i equals
pi 0 times rho at the power i,

104
00:06:13,940 --> 00:06:19,225
we end up having that pi
of i equals pi of 0, which

105
00:06:19,225 --> 00:06:27,662
is 1 minus rho times
rho at the power i,

106
00:06:27,662 --> 00:06:31,470
for i equal-- this
pi i can be seen

107
00:06:31,470 --> 00:06:34,080
as coming from the
probability distribution.

108
00:06:34,080 --> 00:06:37,640
They tell us that if we observe
that chain at time-- let's

109
00:06:37,640 --> 00:06:40,710
say one billion--
and ask-- where

110
00:06:40,710 --> 00:06:43,580
is the state of
the Markov chain?

111
00:06:43,580 --> 00:06:47,490
The answer will be the
chain is in state zero, that

112
00:06:47,490 --> 00:06:51,659
is, the system is empty with
a probability 1 minus rho,

113
00:06:51,659 --> 00:06:54,040
or there is one
customer in the system.

114
00:06:54,040 --> 00:06:57,240
And that happens with
probability 1 minus rho times

115
00:06:57,240 --> 00:06:58,640
rho.

116
00:06:58,640 --> 00:07:00,420
And so on.

117
00:07:00,420 --> 00:07:04,330
So the distribution
can be drawn like that.

118
00:07:04,330 --> 00:07:11,012
You have here i corresponding to
a state and if you put pi of i

119
00:07:11,012 --> 00:07:19,240
here, 0 here, then 1, 2, 3--
then pi of 0 is 1 minus rho

120
00:07:19,240 --> 00:07:19,740
here.

121
00:07:23,090 --> 00:07:29,590
pi of 1 will be rho times
1 minus rho and pi of 2

122
00:07:29,590 --> 00:07:30,270
and so forth.

123
00:07:33,550 --> 00:07:35,500
So if you look at this
distribution here,

124
00:07:35,500 --> 00:07:37,530
it's pretty much a
geometric distribution,

125
00:07:37,530 --> 00:07:41,640
except that it has shifted so
that it starts at 0 instead

126
00:07:41,640 --> 00:07:43,180
of starting at 1.

127
00:07:43,180 --> 00:07:45,220
So it's a shifted geometric.

128
00:07:48,830 --> 00:07:51,320
This model is the first
and simplest model

129
00:07:51,320 --> 00:07:54,960
that one encounters when
studying queuing theory.

130
00:07:54,960 --> 00:07:57,740
So a final note--
the PMF that we

131
00:07:57,740 --> 00:08:00,750
have here has an expected value.

132
00:08:00,750 --> 00:08:06,090
And the expectation is
given here-- e of x of m

133
00:08:06,090 --> 00:08:12,950
is-- let me rewrite it here--
it's rho over 1 minus rho.

134
00:08:12,950 --> 00:08:16,540
And this formula-- which
is interesting to anyone

135
00:08:16,540 --> 00:08:18,820
who tries to analyze a
system of this kind--

136
00:08:18,820 --> 00:08:25,160
tells you the following-- that
as long as rho is less than 1,

137
00:08:25,160 --> 00:08:28,340
then the expected number
of customers in the system

138
00:08:28,340 --> 00:08:29,890
is finite.

139
00:08:29,890 --> 00:08:34,169
But if rho, this little rho,
becomes very close to 1,

140
00:08:34,169 --> 00:08:36,760
then you're going to have
1 over something that

141
00:08:36,760 --> 00:08:38,400
is very close to 0.

142
00:08:38,400 --> 00:08:41,299
And that number will
be very, very big.

143
00:08:41,299 --> 00:08:44,460
So when rho becomes
very close to 1,

144
00:08:44,460 --> 00:08:47,850
that means the load factor
is something like-- let's say

145
00:08:47,850 --> 00:08:52,100
0.99-- you expect to have a
very large number of customers

146
00:08:52,100 --> 00:08:55,300
in the system at any given time.