1
00:00:00,800 --> 00:00:03,220
The other thing that we
discussed in the last lecture

2
00:00:03,220 --> 00:00:06,410
was a classification of the
different states of the Markov

3
00:00:06,410 --> 00:00:09,270
chain into different types.

4
00:00:09,270 --> 00:00:11,440
A Markov chain in
general has states

5
00:00:11,440 --> 00:00:15,740
that are recurrent, which means
that from that recurrent state,

6
00:00:15,740 --> 00:00:19,380
you can go somewhere else and
then from that somewhere else

7
00:00:19,380 --> 00:00:22,090
you can always come back to it.

8
00:00:22,090 --> 00:00:25,080
So if you have a Markov
chain of this form

9
00:00:25,080 --> 00:00:27,910
and you start in state
nine, the options for you

10
00:00:27,910 --> 00:00:32,000
is to either go to state
three or to state five.

11
00:00:32,000 --> 00:00:34,630
But no matter what,
if you go to three,

12
00:00:34,630 --> 00:00:37,036
you can come back always,
and if you go to five,

13
00:00:37,036 --> 00:00:38,410
you can always
come back as well.

14
00:00:38,410 --> 00:00:42,830
So clearly nine here would be
a recurrent state, and three

15
00:00:42,830 --> 00:00:45,890
for the same reason,
and five as well.

16
00:00:45,890 --> 00:00:49,120
Now, if you look at
the state six or seven,

17
00:00:49,120 --> 00:00:50,180
it is the same thing.

18
00:00:50,180 --> 00:00:53,600
Starting from six, the
only way that you can go to

19
00:00:53,600 --> 00:00:57,400
is to either stay at six or go
to seven, and then in that case

20
00:00:57,400 --> 00:00:58,360
you always come back.

21
00:00:58,360 --> 00:01:01,620
And same thing from seven,
you can either go to six

22
00:01:01,620 --> 00:01:03,540
and that's it's,
actually, and come back.

23
00:01:03,540 --> 00:01:07,520
So both of these are
recurrent as well.

24
00:01:07,520 --> 00:01:09,930
So in a case the state
is not recurrent,

25
00:01:09,930 --> 00:01:11,030
we will call it transient.

26
00:01:11,030 --> 00:01:14,220
So let's look at for
example state one.

27
00:01:14,220 --> 00:01:18,260
From state one, if you go from
one to two and then go to six,

28
00:01:18,260 --> 00:01:20,080
there is no way to
come back to one.

29
00:01:20,080 --> 00:01:23,810
So the state 1 is transient,
and for the same reason

30
00:01:23,810 --> 00:01:25,520
the state four
will be transient,

31
00:01:25,520 --> 00:01:27,620
and the state 2
will be transient.

32
00:01:27,620 --> 00:01:28,810
What about eight?

33
00:01:28,810 --> 00:01:32,900
Well, the same reason, the
state is transient as well.

34
00:01:32,900 --> 00:01:36,440
So what we have seen also is
the notion of a recurrent class.

35
00:01:36,440 --> 00:01:39,630
A recurrent class is,
again, a collection

36
00:01:39,630 --> 00:01:43,320
of recurrent states that can
communicate between each other.

37
00:01:43,320 --> 00:01:47,550
So here, for this specific
example, we have two classes.

38
00:01:47,550 --> 00:01:52,970
This is one class, right,
so it's a class one.

39
00:01:52,970 --> 00:01:56,140
Let's call it
recurrent class one.

40
00:01:56,140 --> 00:02:02,550
And this is a recurrent
class, recurrent class two.

41
00:02:02,550 --> 00:02:05,610
So here again we have two
classes instead of one,

42
00:02:05,610 --> 00:02:10,788
because if you are in
one of these classes,

43
00:02:10,788 --> 00:02:13,970
there is no way that you can
find a path to go to one state

44
00:02:13,970 --> 00:02:15,710
here and vice versa.

45
00:02:15,710 --> 00:02:18,480
If you are in one of
these states here,

46
00:02:18,480 --> 00:02:22,670
there is no path that would lead
you to that recurrent class.

47
00:02:22,670 --> 00:02:25,950
In the case where you have two
recurrent classes, like here

48
00:02:25,950 --> 00:02:29,980
or more, it is pretty
clear that in the long run,

49
00:02:29,980 --> 00:02:32,200
the steady state behavior
of the Markov chain

50
00:02:32,200 --> 00:02:34,980
will really depend
on where you started.

51
00:02:34,980 --> 00:02:37,960
So for example, if
your Markov chain

52
00:02:37,960 --> 00:02:42,720
started in that recurrent
class, there is no probability

53
00:02:42,720 --> 00:02:46,500
that in the long run it will be
in that class, and vice versa.

54
00:02:46,500 --> 00:02:48,600
If it started here,
the probability

55
00:02:48,600 --> 00:02:52,800
of being in that recurrent
class in the long run is zero.

56
00:02:52,800 --> 00:02:56,390
So the long run behavior
of the Markov chain

57
00:02:56,390 --> 00:03:00,260
will depend on the
initial condition.

58
00:03:00,260 --> 00:03:03,685
In the case where you have
only one recurrent class,

59
00:03:03,685 --> 00:03:06,420
let's forget about that
portion, for example,

60
00:03:06,420 --> 00:03:09,330
and you have only
that portion here.

61
00:03:09,330 --> 00:03:12,470
Then maybe the initial
condition will not

62
00:03:12,470 --> 00:03:14,970
matter in the long
run, but in fact it's

63
00:03:14,970 --> 00:03:17,470
not going to be always
the case, depending

64
00:03:17,470 --> 00:03:19,920
on the recurrent
class being periodic

65
00:03:19,920 --> 00:03:23,380
or not as we will
see in the next clip.