1
00:00:00,500 --> 00:00:02,790
Let's do a little
warm-up now before moving

2
00:00:02,790 --> 00:00:05,230
to the central topic
of today's lecture

3
00:00:05,230 --> 00:00:08,450
on how to apply these transition
probabilities in order

4
00:00:08,450 --> 00:00:10,450
to calculate some
more interesting path

5
00:00:10,450 --> 00:00:13,470
behaviors of Markov chains.

6
00:00:13,470 --> 00:00:17,200
So suppose that you are given
the following Markov chain,

7
00:00:17,200 --> 00:00:21,210
and you are asked to calculate
the probability that starting

8
00:00:21,210 --> 00:00:29,570
in state 1, you go successively
to state 2, then state 6,

9
00:00:29,570 --> 00:00:31,950
and then 7.

10
00:00:31,950 --> 00:00:34,810
So this is really what
we are asked here.

11
00:00:34,810 --> 00:00:39,120
You start at 1, then
you go to 2, 6, and 7.

12
00:00:39,120 --> 00:00:41,460
So how to calculate
such a probability?

13
00:00:41,460 --> 00:00:44,675
Well, we can use a version
of the multiplicative rule

14
00:00:44,675 --> 00:00:46,390
we have introduced before.

15
00:00:49,380 --> 00:00:52,660
And so what is the specific
format of that rule

16
00:00:52,660 --> 00:00:54,580
that I'm going to use here?

17
00:00:54,580 --> 00:01:00,300
You have three events like that,
B, C, D. They are conditioned

18
00:01:00,300 --> 00:01:04,965
on A. And I will say that this
is the probability of B given

19
00:01:04,965 --> 00:01:11,130
A, times the probability of
C given A intersection B,

20
00:01:11,130 --> 00:01:18,190
times the probability of D given
A intersection B intersection

21
00:01:18,190 --> 00:01:23,130
C. So this is a version of
the multiplicative rule.

22
00:01:23,130 --> 00:01:28,080
And this is what we're going
to use here, so let's do it.

23
00:01:28,080 --> 00:01:31,370
This is going to be
equals to the probability

24
00:01:31,370 --> 00:01:38,479
that X1 equals 2 given X0
equals 1, times the probability

25
00:01:38,479 --> 00:01:44,090
that X2 equals 6
given X0 equals 1,

26
00:01:44,090 --> 00:01:47,810
and X1 equals 2
times the probability

27
00:01:47,810 --> 00:01:57,310
that X3 equals 7 times
X0 equals 1, X1 equals 2,

28
00:01:57,310 --> 00:02:00,080
and then X2 equals 6.

29
00:02:00,080 --> 00:02:03,070
So multiplication rule.

30
00:02:03,070 --> 00:02:05,020
Now here, this is p12.

31
00:02:08,139 --> 00:02:11,690
And here, we are using
the Markov property.

32
00:02:11,690 --> 00:02:14,250
What it says here is,
what is the probability

33
00:02:14,250 --> 00:02:21,350
that I will be in state 6, given
that I was in 1 and then 2.

34
00:02:21,350 --> 00:02:23,530
And we know that having
the entire trajectory

35
00:02:23,530 --> 00:02:25,710
doesn't matter, it's
just the last step.

36
00:02:25,710 --> 00:02:31,140
So it's essentially
times, this one is p26.

37
00:02:31,140 --> 00:02:36,930
And for the same reason, this
one is nothing else than p67.

38
00:02:36,930 --> 00:02:39,490
The important message
here is that to find

39
00:02:39,490 --> 00:02:43,210
the probability of a specific
trajectory like this one,

40
00:02:43,210 --> 00:02:46,050
you just need to multiply
the transition probabilities

41
00:02:46,050 --> 00:02:49,770
that you find along
the trajectory.

42
00:02:49,770 --> 00:02:52,610
So this is what we have here.

43
00:02:52,610 --> 00:02:54,730
Now suppose that you want
to find the probability

44
00:02:54,730 --> 00:02:57,490
that in four times
steps, you find yourself

45
00:02:57,490 --> 00:03:02,570
in a specific
state say, 7, given

46
00:03:02,570 --> 00:03:07,410
that you started in a
specific initial state say, 2.

47
00:03:07,410 --> 00:03:09,140
This is really
what we want here.

48
00:03:09,140 --> 00:03:14,240
You're here, and then you
end up here after four steps.

49
00:03:14,240 --> 00:03:16,910
So how do you calculate that?

50
00:03:16,910 --> 00:03:20,430
Well, one way is to use our
recurrence formula for rij

51
00:03:20,430 --> 00:03:22,810
that we have described before.

52
00:03:22,810 --> 00:03:26,010
Another, for small
examples like this one,

53
00:03:26,010 --> 00:03:28,070
when the number of times
steps in the future

54
00:03:28,070 --> 00:03:30,920
is small-- in that
case four-- one

55
00:03:30,920 --> 00:03:34,070
can perhaps use a brute
force calculation.

56
00:03:34,070 --> 00:03:36,020
So what is a brute
force calculation?

57
00:03:36,020 --> 00:03:40,630
Well, you try to enumerate
all possible trajectories,

58
00:03:40,630 --> 00:03:43,390
and then you sum all
their probabilities.

59
00:03:43,390 --> 00:03:47,820
So in that case, I think
we have three trajectories,

60
00:03:47,820 --> 00:03:52,810
but I'm not so sure, so
let's try to enumerate them.

61
00:03:52,810 --> 00:03:55,510
So you start here,
and one possibility

62
00:03:55,510 --> 00:03:59,240
in a one-step transition
is to go to 6.

63
00:03:59,240 --> 00:04:04,220
Then from 6 you go to 7,
then from 7 you go back to 6.

64
00:04:04,220 --> 00:04:08,630
And then from 6 you
again go to 7, all right?

65
00:04:08,630 --> 00:04:12,050
So that, if we look and use
the rule that we have developed

66
00:04:12,050 --> 00:04:14,940
before, it would
be the probability

67
00:04:14,940 --> 00:04:24,820
of going to 6 times p67
times p76 and times p67.

68
00:04:24,820 --> 00:04:27,820
So this is one way of doing it.

69
00:04:31,000 --> 00:04:33,350
What would be another path?

70
00:04:33,350 --> 00:04:36,780
Well, from 2,
instead of going to 6

71
00:04:36,780 --> 00:04:40,940
you could go to 1
in one transition.

72
00:04:40,940 --> 00:04:47,392
Then you go back to 2, then
we go to 6, and then right up

73
00:04:47,392 --> 00:04:48,159
to 7.

74
00:04:48,159 --> 00:04:53,700
So essentially here, what
we have is plus p21 times

75
00:04:53,700 --> 00:05:00,173
p12 times p26 times p67.

76
00:05:03,150 --> 00:05:05,710
And what is the
third way to do that?

77
00:05:05,710 --> 00:05:07,630
So there is a third path.

78
00:05:07,630 --> 00:05:10,060
You're starting from here, yes?

79
00:05:10,060 --> 00:05:12,810
You go here in one step.

80
00:05:12,810 --> 00:05:20,410
Here you do a jump on
itself, and another jump,

81
00:05:20,410 --> 00:05:22,980
and then you go to 7.

82
00:05:22,980 --> 00:05:32,570
So in that case, you would
have plus p26 times p66 times

83
00:05:32,570 --> 00:05:37,870
p66 times p67.

84
00:05:37,870 --> 00:05:42,270
So here we have the entire
solution, all right?

85
00:05:42,270 --> 00:05:44,490
So this is what is
called, by brute force.

86
00:05:44,490 --> 00:05:47,930
Now of course, if
instead of 4, here,

87
00:05:47,930 --> 00:05:54,390
you had something like 200,
the number of trajectories

88
00:05:54,390 --> 00:05:57,880
would grow exponentially
with this number of steps.

89
00:05:57,880 --> 00:06:00,230
And this is not
practical anymore.

90
00:06:00,230 --> 00:06:02,490
Using the recursion
formula would

91
00:06:02,490 --> 00:06:05,380
have been a much
better approach.

92
00:06:05,380 --> 00:06:08,700
And in some sense would have
a much better complexity.

93
00:06:08,700 --> 00:06:11,550
Essentially, a linear
growth as a function of time

94
00:06:11,550 --> 00:06:13,580
steps in the future.

95
00:06:13,580 --> 00:06:19,685
More precisely, for a chain
with a state space of m,

96
00:06:19,685 --> 00:06:24,140
at each time steps you need
to update all our rij's.

97
00:06:24,140 --> 00:06:26,970
So for each pair of
ij, so each of these

98
00:06:26,970 --> 00:06:29,740
would take about m squared.

99
00:06:29,740 --> 00:06:32,440
And so the total
computational complexity

100
00:06:32,440 --> 00:06:39,630
would grow about n times m
squared as a function of n.