1
00:00:00,500 --> 00:00:02,270
In this final
lecture, we will first

2
00:00:02,270 --> 00:00:05,700
review the various properties
of a nice Markov chain, which

3
00:00:05,700 --> 00:00:09,550
ensures steady state behavior,
and go over some of the notions

4
00:00:09,550 --> 00:00:12,090
in more detail
with some examples.

5
00:00:12,090 --> 00:00:15,540
Providing some insights on
how good an approximation

6
00:00:15,540 --> 00:00:18,750
we have when we use
steady state probabilities

7
00:00:18,750 --> 00:00:21,880
to characterize the behavior
of a Markov chain, which

8
00:00:21,880 --> 00:00:26,520
has run for a long time, but
not an infinite amount of time.

9
00:00:26,520 --> 00:00:29,170
We will then consider
a classical application

10
00:00:29,170 --> 00:00:32,420
of Markov chains, which has to
do with the design of a phone

11
00:00:32,420 --> 00:00:33,710
system.

12
00:00:33,710 --> 00:00:38,130
This is a famous problem, which
was posed, analyzed, and solved

13
00:00:38,130 --> 00:00:41,290
by a Danish engineer
by the name Erlang.

14
00:00:41,290 --> 00:00:44,090
It was more than 100
years ago when phones just

15
00:00:44,090 --> 00:00:46,300
started to exist,
but we will see

16
00:00:46,300 --> 00:00:49,580
that this methodology
remains relevant to design

17
00:00:49,580 --> 00:00:52,220
similar systems
in today's world.

18
00:00:52,220 --> 00:00:54,150
We will then make
use of all what

19
00:00:54,150 --> 00:00:56,620
we have learned so far
in order to calculate

20
00:00:56,620 --> 00:00:59,140
some interesting short
term behaviors of Markov

21
00:00:59,140 --> 00:01:03,130
chains having more than
one recurrent classes.

22
00:01:03,130 --> 00:01:06,430
We will introduce the
notion of absorbing states,

23
00:01:06,430 --> 00:01:07,990
and we will show
how to calculate

24
00:01:07,990 --> 00:01:10,950
the probability of ending
up in such a state,

25
00:01:10,950 --> 00:01:14,690
as well as related quantities
such as the expected time it

26
00:01:14,690 --> 00:01:16,600
takes to do so.

27
00:01:16,600 --> 00:01:19,010
As a classical
example, we will look

28
00:01:19,010 --> 00:01:21,310
at the gambler
continuously playing

29
00:01:21,310 --> 00:01:23,855
a simple game of
chance, say a lottery,

30
00:01:23,855 --> 00:01:27,730
until he either accumulates
a given amount of money

31
00:01:27,730 --> 00:01:29,870
or loses all his money.

32
00:01:29,870 --> 00:01:32,280
Both of these states
are absorbing.

33
00:01:32,280 --> 00:01:35,120
What are their
corresponding probabilities?

34
00:01:35,120 --> 00:01:39,090
After this lecture, you will be
able to answer such questions.