1
00:00:00,500 --> 00:00:02,960
In this lecture,
we will concentrate

2
00:00:02,960 --> 00:00:06,000
on the study of Markov
chains in the long run,

3
00:00:06,000 --> 00:00:08,670
and study under what
conditions a Markov chain

4
00:00:08,670 --> 00:00:12,780
exhibits steady-state behavior,
and under what conditions

5
00:00:12,780 --> 00:00:15,620
such steady-state
behavior is independent

6
00:00:15,620 --> 00:00:18,120
of the initial starting state.

7
00:00:18,120 --> 00:00:21,840
More precisely, we will look
at long-term state occupancy

8
00:00:21,840 --> 00:00:26,470
behavior-- that is, in the
n-step transition probabilities

9
00:00:26,470 --> 00:00:28,700
when n is large.

10
00:00:28,700 --> 00:00:32,860
So assume that we have a
Markov chain which is initially

11
00:00:32,860 --> 00:00:36,770
in a given state i, and
consider the probability

12
00:00:36,770 --> 00:00:41,900
that the chain is in a specific
state j after n transitions.

13
00:00:41,900 --> 00:00:45,450
Question-- does that probability
converge to some constant

14
00:00:45,450 --> 00:00:48,010
when n goes to infinity?

15
00:00:48,010 --> 00:00:52,940
And if this is the case-- second
question-- can this constant be

16
00:00:52,940 --> 00:00:56,750
independent of the
initial state i?

17
00:00:56,750 --> 00:00:59,490
We will see that for
nice Markov chains,

18
00:00:59,490 --> 00:01:02,800
the answers to both
questions will be yes.

19
00:01:02,800 --> 00:01:05,690
How to characterize
nice Markov chains?

20
00:01:05,690 --> 00:01:08,780
We will use several new
concepts, one dealing

21
00:01:08,780 --> 00:01:11,880
with a Markov chain
being aperiodic or not,

22
00:01:11,880 --> 00:01:16,230
and the other with the
notion of recurrent classes.

23
00:01:16,230 --> 00:01:18,930
Without going into
details now, let

24
00:01:18,930 --> 00:01:21,380
us simply mention
that we will show

25
00:01:21,380 --> 00:01:23,370
that the existence
of convergence

26
00:01:23,370 --> 00:01:27,100
will be tied to having an
aperiodic Markov chain.

27
00:01:27,100 --> 00:01:30,220
And in case we have
convergence, the independence

28
00:01:30,220 --> 00:01:31,990
from the initial
state will be tied

29
00:01:31,990 --> 00:01:35,160
to having a single
recurrent class.

30
00:01:35,160 --> 00:01:38,580
We will end this lecture
by looking in detail

31
00:01:38,580 --> 00:01:42,080
at the special and important
class of Markov chains usually

32
00:01:42,080 --> 00:01:45,100
known as birth-death processes.