1
00:00:00,101 --> 00:00:00,600
All right.

2
00:00:00,600 --> 00:00:03,550
So let us revisit the
example from last lecture.

3
00:00:03,550 --> 00:00:06,880
So we have a Markov chain
with two states, one and two,

4
00:00:06,880 --> 00:00:11,502
and this Markov chain has
a single recurrent class.

5
00:00:11,502 --> 00:00:12,950
All right.

6
00:00:12,950 --> 00:00:15,970
And then also it's
not periodic right,

7
00:00:15,970 --> 00:00:19,060
because we have self
transition of this type.

8
00:00:19,060 --> 00:00:23,810
So as a result, this is well
defined and these steady state

9
00:00:23,810 --> 00:00:32,208
probabilities from 1 to m, in
that case for us, m = 2, right?

10
00:00:32,208 --> 00:00:34,970
So let us write the
system and solve

11
00:00:34,970 --> 00:00:38,940
the system of linear equation
for this example here.

12
00:00:38,940 --> 00:00:41,176
So what we have is pi
1 equals pi 1 times 0.5

13
00:00:41,176 --> 00:00:51,880
plus pi 2 times 0.2.

14
00:00:51,880 --> 00:00:55,500
So that's the first equation
that corresponds to j equals 1.

15
00:00:55,500 --> 00:00:58,024
Now, for j equals 2, pi
2 equals pi 1 times 0.5

16
00:00:58,024 --> 00:01:07,410
plus pi 2 times 0.8.

17
00:01:07,410 --> 00:01:09,580
So we have a system
of two equations

18
00:01:09,580 --> 00:01:13,390
with two unknowns,
pi 1 and pi 2.

19
00:01:13,390 --> 00:01:18,060
Let us rewrite them, I
pass this one on this side

20
00:01:18,060 --> 00:01:19,432
and this one on this side.

21
00:01:19,432 --> 00:01:21,182
So we get pi 1 times
1 minus 0.5 minus 0.5

22
00:01:21,182 --> 00:01:30,084
equals pi 2 times 0.2.

23
00:01:30,084 --> 00:01:31,834
And this one pi 2 times
1 minus 0.8 is 0.2

24
00:01:31,834 --> 00:01:41,340
equals pi 1 times 0.5.

25
00:01:41,340 --> 00:01:45,700
We realize that these two
happen to be the same,

26
00:01:45,700 --> 00:01:50,350
so they are not enough to
define a unique solution,

27
00:01:50,350 --> 00:01:52,240
so we have to add
another equation,

28
00:01:52,240 --> 00:01:54,580
and we know that these
are probabilities.

29
00:01:54,580 --> 00:01:59,100
So pi 1 plus pi 2 has to
be one, and so now we're

30
00:01:59,100 --> 00:02:01,930
going to keep one of these
two, let's say this one,

31
00:02:01,930 --> 00:02:03,141
I'm going to write it here.

32
00:02:03,141 --> 00:02:05,266
And we can rewrite it by
saying that pi 1 times 1/2

33
00:02:05,266 --> 00:02:06,182
equals pi 2 times 1/5.

34
00:02:14,730 --> 00:02:18,770
So now, we're going
to take that, replace

35
00:02:18,770 --> 00:02:26,010
pi 1 equals 2/5 of pi 2
is the result of that.

36
00:02:26,010 --> 00:02:30,690
And we're going to use that
pi 1 and replace it here.

37
00:02:30,690 --> 00:02:38,400
So we end it by 2 times
2/5 plus 1 equals 1,

38
00:02:38,400 --> 00:02:46,730
which means that from here, we
get that pi 2 equals 5 plus 2/7

39
00:02:46,730 --> 00:02:52,540
so 5/7, and then we use
that and place it here

40
00:02:52,540 --> 00:03:00,280
and we end up having pi 1
equals 2/5 times 5/7 equals 2/7,

41
00:03:00,280 --> 00:03:03,510
and we check 5
plus 2 equals 7, so

42
00:03:03,510 --> 00:03:05,350
these are real probabilities.

43
00:03:05,350 --> 00:03:07,370
So the probabilitiy
that you find yourself

44
00:03:07,370 --> 00:03:12,627
at state one at time 1 trillion
would be approximately 2/7.

45
00:03:12,627 --> 00:03:14,210
The probability that
you find yourself

46
00:03:14,210 --> 00:03:19,760
at state one at time 2 trillions
is again approximately 2/7.

47
00:03:19,760 --> 00:03:22,860
So essentially what we have
here is the probability

48
00:03:22,860 --> 00:03:27,370
of being in that state one
settles in a steady value.

49
00:03:27,370 --> 00:03:30,140
That's what the steady
state convergence means.

50
00:03:30,140 --> 00:03:32,960
It's convergence of
probabilities, not convergence

51
00:03:32,960 --> 00:03:34,130
of the process itself.

52
00:03:34,130 --> 00:03:37,350
Again, the process will
keep jumping back and forth,

53
00:03:37,350 --> 00:03:40,240
but the steady state probability
will settle for a given value

54
00:03:40,240 --> 00:03:44,300
here in one, that will be
2/7, and the steady state

55
00:03:44,300 --> 00:03:49,720
probability in being in
two will settle to 5/7.

56
00:03:49,720 --> 00:03:52,355
And finally in this
example, and more

57
00:03:52,355 --> 00:03:55,420
generally when we have a single
class and no periodicity,

58
00:03:55,420 --> 00:03:58,520
the initial state
does not matter.