1
00:00:00,500 --> 00:00:03,590
In the last example
of the previous clip,

2
00:00:03,590 --> 00:00:06,240
we have observed that the
initial state of where

3
00:00:06,240 --> 00:00:10,740
the Markov chain starts can
matter, and that in some cases,

4
00:00:10,740 --> 00:00:12,660
the influence of
the initial state

5
00:00:12,660 --> 00:00:15,030
never vanishes in the long run.

6
00:00:15,030 --> 00:00:18,560
So when we wanted to calculate
the probability of ending up

7
00:00:18,560 --> 00:00:24,200
in state one, knowing that
you started in state one,

8
00:00:24,200 --> 00:00:29,430
give you that probability, and
if you started in state three,

9
00:00:29,430 --> 00:00:33,510
you had another long
term probability.

10
00:00:33,510 --> 00:00:37,940
And in case you were
started in state two,

11
00:00:37,940 --> 00:00:43,260
then you were equally likely
to end up being in state one

12
00:00:43,260 --> 00:00:45,930
or in state three.

13
00:00:45,930 --> 00:00:48,660
We have said that this
was due to the fact

14
00:00:48,660 --> 00:00:51,020
that some states
are not accessible

15
00:00:51,020 --> 00:00:54,070
from some other states.

16
00:00:54,070 --> 00:00:57,500
Our goal here is to make
such a statement more

17
00:00:57,500 --> 00:01:01,200
precise and formal.

18
00:01:01,200 --> 00:01:04,819
In order to do so, we will
classify the states of a Markov

19
00:01:04,819 --> 00:01:10,030
chain in a transition diagram
into two types, recurrent

20
00:01:10,030 --> 00:01:11,700
and transient.

21
00:01:11,700 --> 00:01:15,380
A state is said to be
recurrent if starting

22
00:01:15,380 --> 00:01:18,740
from that state, no
matter where you go,

23
00:01:18,740 --> 00:01:22,530
there is always a
path to come back.

24
00:01:22,530 --> 00:01:28,050
A state is said to be transient
if it is not recurrent.

25
00:01:28,050 --> 00:01:31,210
It means that starting
from that state,

26
00:01:31,210 --> 00:01:34,600
there is at least one
place where you can go,

27
00:01:34,600 --> 00:01:38,660
and then there will be
no paths back to it.

28
00:01:38,660 --> 00:01:45,200
For example, starting from state
eight, no matter where you go,

29
00:01:45,200 --> 00:01:51,170
there is always a
path back to eight.

30
00:01:51,170 --> 00:01:55,300
So eight will be said
to be a recurrent state.

31
00:01:55,300 --> 00:01:58,370
For the same reason,
seven will also

32
00:01:58,370 --> 00:02:03,030
be a recurrent state,
as well as six.

33
00:02:03,030 --> 00:02:05,890
Now, what about five?

34
00:02:05,890 --> 00:02:08,788
From five, the only
state where you can go to

35
00:02:08,788 --> 00:02:16,800
is five itself, so five
is also a recurrent state.

36
00:02:16,800 --> 00:02:21,480
Let us look now at a
state like state four.

37
00:02:21,480 --> 00:02:25,320
From four, there
is the possibility

38
00:02:25,320 --> 00:02:30,720
of a path going
through two, then one,

39
00:02:30,720 --> 00:02:34,360
and then going to
five, from which

40
00:02:34,360 --> 00:02:38,870
there is no path back to four.

41
00:02:38,870 --> 00:02:42,670
So four cannot be
a recurrent state.

42
00:02:42,670 --> 00:02:46,350
It is then called
a transient state.

43
00:02:46,350 --> 00:02:49,820
For the same reason,
starting from three,

44
00:02:49,820 --> 00:02:56,620
there is the possibility that
after doing many transitions

45
00:02:56,620 --> 00:03:02,120
around here, eventually, you
will end up being in either two

46
00:03:02,120 --> 00:03:08,460
and then go to six, or you
will eventually be in one

47
00:03:08,460 --> 00:03:10,870
and then go to five.

48
00:03:10,870 --> 00:03:18,090
And from five or from six, there
will be no path back to three,

49
00:03:18,090 --> 00:03:21,450
so three is also
a transient state.

50
00:03:21,450 --> 00:03:24,840
And it is clear that
for the same reason,

51
00:03:24,840 --> 00:03:29,750
one will be a transient
state, and two will also

52
00:03:29,750 --> 00:03:32,550
be a transient state.

53
00:03:32,550 --> 00:03:39,740
Now, if a state is transient,
like three, four, one, or two,

54
00:03:39,740 --> 00:03:45,600
it means that it will be visited
only a finite number of times.

55
00:03:45,600 --> 00:03:47,860
And in the long
run, the probability

56
00:03:47,860 --> 00:03:51,990
of being in a transient
state will converge to zero.

57
00:03:51,990 --> 00:03:54,660
The recurrent states
of a Markov chain

58
00:03:54,660 --> 00:03:57,620
can be grouped in
different classes.

59
00:03:57,620 --> 00:04:01,400
Here, in this example,
we have two such classes.

60
00:04:01,400 --> 00:04:10,760
Here, this is one class, and
here you have another class.

61
00:04:14,080 --> 00:04:16,970
What is so special
about these classes?

62
00:04:16,970 --> 00:04:21,800
Well, within one class,
for example, in this class,

63
00:04:21,800 --> 00:04:25,790
all recurrent states
have a way to communicate

64
00:04:25,790 --> 00:04:30,290
between each other, but
there is no communication

65
00:04:30,290 --> 00:04:34,262
between recurrent states
of different classes.

66
00:04:34,262 --> 00:04:38,080
From that recurrent
state here, you

67
00:04:38,080 --> 00:04:43,000
have no communication with
that recurrent state here.

68
00:04:43,000 --> 00:04:44,920
No communication in
the sense that there

69
00:04:44,920 --> 00:04:50,110
is no path joining that
state to that state

70
00:04:50,110 --> 00:04:54,300
or that state to that state.

71
00:04:54,300 --> 00:04:58,650
The existence of more than
one class of recurrent states,

72
00:04:58,650 --> 00:05:01,330
like in this example,
in a Markov chain,

73
00:05:01,330 --> 00:05:05,540
will be the telling
sign that initial states

74
00:05:05,540 --> 00:05:08,230
will matter in that chain.

75
00:05:08,230 --> 00:05:12,970
It's pretty clear here that
if your Markov chain starts

76
00:05:12,970 --> 00:05:18,290
in state five, you will never
end up in state six, seven,

77
00:05:18,290 --> 00:05:21,870
or eight, and if you
start in one of the states

78
00:05:21,870 --> 00:05:26,410
six, seven, eight, you will
never end up in state five.

79
00:05:26,410 --> 00:05:30,270
So depending on
where you started,

80
00:05:30,270 --> 00:05:34,159
the long range probability of
being eventually in state six

81
00:05:34,159 --> 00:05:36,390
will not be the same.