1
00:00:00,876 --> 00:00:01,376
All right.

2
00:00:01,376 --> 00:00:03,720
So, as we've seen in
the previous clip,

3
00:00:03,720 --> 00:00:07,460
another way that initial
conditions may matter

4
00:00:07,460 --> 00:00:12,030
is if a chain has a
periodic structure.

5
00:00:12,030 --> 00:00:14,450
There are many ways of
defining periodicity.

6
00:00:14,450 --> 00:00:17,130
Some of them are more
mathematical than others.

7
00:00:17,130 --> 00:00:20,880
Let us consider one of the most
intuitive ways of doing that.

8
00:00:20,880 --> 00:00:22,610
So here is the definition.

9
00:00:22,610 --> 00:00:24,460
The states in a
recurrent class are

10
00:00:24,460 --> 00:00:28,720
periodic if they can be
lumped together, or grouped,

11
00:00:28,720 --> 00:00:34,756
into several subgroups so that
all transitions from one group

12
00:00:34,756 --> 00:00:36,500
lead to the next group.

13
00:00:36,500 --> 00:00:38,780
So what does that really mean?

14
00:00:38,780 --> 00:00:44,320
Let us try to parse out this by
looking at the given example.

15
00:00:44,320 --> 00:00:47,130
So here we would
have a situation,

16
00:00:47,130 --> 00:00:53,790
a structure of a diagram,
in which d is equal to 2.

17
00:00:53,790 --> 00:00:59,540
Whenever you are at a given
time in a state in that group,

18
00:00:59,540 --> 00:01:03,570
in the next transition
you will go to that group.

19
00:01:03,570 --> 00:01:07,410
And if you are in that group,
at the next transition,

20
00:01:07,410 --> 00:01:09,789
you will go to that group.

21
00:01:09,789 --> 00:01:12,180
So in some sense,
there is periodicity,

22
00:01:12,180 --> 00:01:15,680
and there is somewhat of
a deterministic behavior,

23
00:01:15,680 --> 00:01:17,860
according to this transition.

24
00:01:17,860 --> 00:01:21,000
But there is also
some randomness left.

25
00:01:21,000 --> 00:01:24,490
We can be in any of these
states within that sub-group.

26
00:01:24,490 --> 00:01:26,640
But one thing that
is for sure is

27
00:01:26,640 --> 00:01:28,730
that whenever, at
a given time, you

28
00:01:28,730 --> 00:01:31,530
are into one of
these states, here,

29
00:01:31,530 --> 00:01:34,750
in that group, if
the Markov chain has

30
00:01:34,750 --> 00:01:37,710
this specific structure,
in the next transition

31
00:01:37,710 --> 00:01:41,180
you will transition to
one of the states, here.

32
00:01:41,180 --> 00:01:46,190
And the next transition
after, you will go back here.

33
00:01:46,190 --> 00:01:48,720
So essentially, if
your Markov chain

34
00:01:48,720 --> 00:01:54,420
started in one of these states
here, at time n equals zero,

35
00:01:54,420 --> 00:01:57,860
then at time n equals
1, it would be here.

36
00:01:57,860 --> 00:02:01,020
And then at time n equals
2, it would be here.

37
00:02:01,020 --> 00:02:05,050
Times them into 3, and
so on and so forth.

38
00:02:05,050 --> 00:02:09,759
So every time you would
have an even number,

39
00:02:09,759 --> 00:02:12,850
then you are guaranteed
that the Markov chain will

40
00:02:12,850 --> 00:02:14,700
be in one of these states, here.

41
00:02:14,700 --> 00:02:17,150
Clearly, with this
kind of structure,

42
00:02:17,150 --> 00:02:20,890
it is impossible to have
convergence of the steady state

43
00:02:20,890 --> 00:02:22,040
probabilities.

44
00:02:22,040 --> 00:02:24,750
This is another
example where you

45
00:02:24,750 --> 00:02:27,530
would have a structure
of a diagram in which you

46
00:02:27,530 --> 00:02:29,400
have a period of 3, here.

47
00:02:29,400 --> 00:02:32,800
So in that case
here, d equals 3.

48
00:02:32,800 --> 00:02:36,310
So here again, if you are in
one of these states at a given

49
00:02:36,310 --> 00:02:39,970
time, then at the next time
the following transition

50
00:02:39,970 --> 00:02:42,660
will guarantee to bring
you in that group.

51
00:02:42,660 --> 00:02:46,450
And during the next transition
you will go to that group.

52
00:02:46,450 --> 00:02:48,890
And then again,
here, and again you

53
00:02:48,890 --> 00:02:52,560
have this kind of behavior
in a very systematic way.

54
00:02:52,560 --> 00:02:55,640
So if you started here
at time n equals 0,

55
00:02:55,640 --> 00:02:59,590
you would be in that group,
here, at times n equals 1.

56
00:02:59,590 --> 00:03:02,200
And you would be
here at n equals 2.

57
00:03:02,200 --> 00:03:10,670
And again, n equals 3 here, n
equals 4, n equals 5, 6, 7, 8.

58
00:03:10,670 --> 00:03:12,910
And you see the pattern.

59
00:03:12,910 --> 00:03:15,980
If you look at any
time in the future,

60
00:03:15,980 --> 00:03:21,760
if the time is of the form
3 times k for any k greater

61
00:03:21,760 --> 00:03:23,800
than or equal to 0,
you're guaranteed

62
00:03:23,800 --> 00:03:27,030
that your chain will be
in one of these states.

63
00:03:27,030 --> 00:03:29,950
Otherwise, it will
be here, or here.

64
00:03:29,950 --> 00:03:32,850
So here again, you do
not have convergence

65
00:03:32,850 --> 00:03:34,500
of the steady states,
because if you

66
00:03:34,500 --> 00:03:38,690
are told that you started in
one of the states here, then

67
00:03:38,690 --> 00:03:41,950
you know that whenever
you have a time that

68
00:03:41,950 --> 00:03:46,360
is the form 3k plus
1 you will be here.

69
00:03:46,360 --> 00:03:48,940
And so the probability
of being here would be 0.

70
00:03:48,940 --> 00:03:53,010
All right, so we have been able
to explain a little bit what

71
00:03:53,010 --> 00:03:57,550
a periodic state is,
using this definition.

72
00:03:57,550 --> 00:04:00,540
Now given a Markov
chain, how can we

73
00:04:00,540 --> 00:04:04,390
tell whether a Markov
chain is periodic or not?

74
00:04:04,390 --> 00:04:06,380
There are, in fact,
systematic ways

75
00:04:06,380 --> 00:04:08,870
of doing it mathematically.

76
00:04:08,870 --> 00:04:12,940
But usually within the types of
examples we see in this class--

77
00:04:12,940 --> 00:04:16,100
most of them, these
will be easy to see-- we

78
00:04:16,100 --> 00:04:19,160
just eyeball the chain and we
tell whether it is periodic

79
00:04:19,160 --> 00:04:19,959
or not.

80
00:04:19,959 --> 00:04:24,110
So let us see if it is that easy
to do, and consider that chain,

81
00:04:24,110 --> 00:04:25,200
here.

82
00:04:25,200 --> 00:04:26,810
OK, so let's see.

83
00:04:26,810 --> 00:04:31,260
Is this chain periodic or not?

84
00:04:31,260 --> 00:04:35,140
I will give you a couple of
seconds to think about it.

85
00:04:35,140 --> 00:04:37,360
Now I'm going to
help a little bit.

86
00:04:37,360 --> 00:04:40,140
I'll just decide that
this one will be red,

87
00:04:40,140 --> 00:04:43,680
this one will be red,
this one will be red,

88
00:04:43,680 --> 00:04:45,726
and this one will be red.

89
00:04:45,726 --> 00:04:47,100
Now I'm asking
the same question.

90
00:04:47,100 --> 00:04:49,180
What do you think?

91
00:04:49,180 --> 00:04:52,040
Well, from the
structure it is clear

92
00:04:52,040 --> 00:04:55,159
that this Markov
chain is periodic

93
00:04:55,159 --> 00:04:57,850
and has a period of d equals 2.

94
00:04:57,850 --> 00:05:00,060
In some sense, the
red state, here,

95
00:05:00,060 --> 00:05:01,870
could be one of these groups.

96
00:05:01,870 --> 00:05:03,560
These are the red ones.

97
00:05:03,560 --> 00:05:07,780
And the white, here,
would be those, here.

98
00:05:07,780 --> 00:05:10,720
And it is clear, if you
look at this diagram,

99
00:05:10,720 --> 00:05:14,370
that whenever you are in a red
state, at the next transition

100
00:05:14,370 --> 00:05:17,270
you are guaranteed to
be in a white state.

101
00:05:17,270 --> 00:05:18,770
From this one, you
will be in white.

102
00:05:18,770 --> 00:05:20,620
From this one, you
will be in the white.

103
00:05:20,620 --> 00:05:22,010
From this one , as well.

104
00:05:22,010 --> 00:05:23,800
And from this one, as well.

105
00:05:23,800 --> 00:05:27,820
And whenever you are in a white
state, this one, this one,

106
00:05:27,820 --> 00:05:31,350
this one, or this one,
at the next period

107
00:05:31,350 --> 00:05:33,800
you are guaranteed
to be in a red state.

108
00:05:33,800 --> 00:05:36,080
So this tells you
that sometimes it

109
00:05:36,080 --> 00:05:39,159
is not that easy to
eyeball and decide

110
00:05:39,159 --> 00:05:41,860
if the Markov chain
is periodic or not.

111
00:05:41,860 --> 00:05:45,110
If you have, for example,
lots of, lots of states,

112
00:05:45,110 --> 00:05:47,870
you might have some trouble
doing this exercise.

113
00:05:47,870 --> 00:05:50,240
On the other hand, something
very useful to know.

114
00:05:50,240 --> 00:05:54,270
Sometimes, it's extremely easy
to tell that the chain is not

115
00:05:54,270 --> 00:05:58,330
periodic, even if you
have a lot of states.

116
00:05:58,330 --> 00:05:59,390
What is that case?

117
00:05:59,390 --> 00:06:03,810
Well, look at this case,
here, and suppose for a moment

118
00:06:03,810 --> 00:06:07,720
that you have a self
transition here.

119
00:06:12,360 --> 00:06:15,640
Well in that case, you
would have a transition

120
00:06:15,640 --> 00:06:18,920
from a white state
to a white state.

121
00:06:18,920 --> 00:06:22,650
And you're not guaranteed
anymore that from a white state

122
00:06:22,650 --> 00:06:25,260
you would go to a red state.

123
00:06:25,260 --> 00:06:28,940
In that sense, as soon as
you have a self transition,

124
00:06:28,940 --> 00:06:31,210
the Markov chain is aperiodic.

125
00:06:31,210 --> 00:06:35,220
It cannot be periodic.

126
00:06:35,220 --> 00:06:37,620
So whenever you have
a self transition,

127
00:06:37,620 --> 00:06:39,850
this implies that the
chain is not periodic.

128
00:06:39,850 --> 00:06:43,590
And usually that's the
simplest and easiest way

129
00:06:43,590 --> 00:06:45,340
that we can tell,
most of the time,

130
00:06:45,340 --> 00:06:47,961
that the chain is not periodic.