1 00:00:00,640 --> 00:00:03,900 We have already introduced the concept of the conditional PMF 2 00:00:03,900 --> 00:00:08,720 of a random variable, X, given an event A. We will now 3 00:00:08,720 --> 00:00:12,150 consider the case where we condition on the value of 4 00:00:12,150 --> 00:00:18,690 another random variable Y. That is, we let A be the event 5 00:00:18,690 --> 00:00:22,500 that some other random variable, Y, takes on a 6 00:00:22,500 --> 00:00:26,310 specific value, little y. 7 00:00:26,310 --> 00:00:28,760 In this case, we're talking about a conditional 8 00:00:28,760 --> 00:00:33,240 probability of the form shown here. 9 00:00:33,240 --> 00:00:34,420 The conditional probability-- 10 00:00:34,420 --> 00:00:38,110 that X takes on a specific value, given that the random 11 00:00:38,110 --> 00:00:42,160 variable Y takes on another specific value. 12 00:00:42,160 --> 00:00:46,300 And we use this notation to indicate those conditional 13 00:00:46,300 --> 00:00:48,200 probabilities. 14 00:00:48,200 --> 00:00:52,750 As usual, the subscripts indicate the situation that 15 00:00:52,750 --> 00:00:54,070 we're dealing with. 16 00:00:54,070 --> 00:00:56,280 That is, we're dealing with the distribution of the random 17 00:00:56,280 --> 00:00:59,310 variable X and we're conditioning on values of the 18 00:00:59,310 --> 00:01:02,260 other random variable, Y. 19 00:01:02,260 --> 00:01:05,990 Using the definition now of conditional probabilities this 20 00:01:05,990 --> 00:01:10,020 can be written as the probability that both events 21 00:01:10,020 --> 00:01:15,450 happen divided by the probability of the 22 00:01:15,450 --> 00:01:16,700 conditioning event. 23 00:01:21,840 --> 00:01:25,490 We can turn this expression into PMF notation. 24 00:01:25,490 --> 00:01:28,330 And this leads us to this definition 25 00:01:28,330 --> 00:01:30,390 of conditional PMFs. 26 00:01:30,390 --> 00:01:34,289 The conditional PMF is defined to be the ratio 27 00:01:34,289 --> 00:01:35,900 of the joint PMF-- 28 00:01:35,900 --> 00:01:38,440 this is the probability that we have here-- 29 00:01:38,440 --> 00:01:41,330 by the corresponding marginal PMF. 30 00:01:41,330 --> 00:01:44,200 And this is the probability that we have here. 31 00:01:44,200 --> 00:01:47,190 Now, remember that conditional probabilities are only defined 32 00:01:47,190 --> 00:01:51,210 when the conditioning event has a positive probability, 33 00:01:51,210 --> 00:01:53,770 when this denominator is positive. 34 00:01:53,770 --> 00:01:58,200 Similarly, the conditional PMF will only be defined for those 35 00:01:58,200 --> 00:02:03,990 little y that have positive probability of occurring. 36 00:02:03,990 --> 00:02:08,680 Now, the conditional PMF is a function of two arguments, 37 00:02:08,680 --> 00:02:10,570 little x and little y. 38 00:02:10,570 --> 00:02:14,500 But the best way of thinking about the conditional PMF is 39 00:02:14,500 --> 00:02:21,270 that we fix the value, little y, and then view this 40 00:02:21,270 --> 00:02:25,100 expression here as a function of x. 41 00:02:25,100 --> 00:02:28,390 As a function of x, it gives us the probabilities of the 42 00:02:28,390 --> 00:02:32,690 different x's that may occur in the conditional universe. 43 00:02:32,690 --> 00:02:37,480 And these probabilities must, of course, sum to 1. 44 00:02:37,480 --> 00:02:39,579 Again, we're keeping y fixed. 45 00:02:39,579 --> 00:02:42,510 We live in a conditional universe where y takes on a 46 00:02:42,510 --> 00:02:43,870 specific value. 47 00:02:43,870 --> 00:02:46,110 And here we have the probabilities of the different 48 00:02:46,110 --> 00:02:47,840 x's in that universe. 49 00:02:47,840 --> 00:02:49,385 And these sum to 1. 50 00:02:52,130 --> 00:02:56,370 Note that if we change the value of little y, we will, of 51 00:02:56,370 --> 00:02:59,740 course, get a different conditional PMF for the random 52 00:02:59,740 --> 00:03:04,090 variable X. So what we're really dealing with in this 53 00:03:04,090 --> 00:03:08,910 instance is that we have a family of conditional PMFs, 54 00:03:08,910 --> 00:03:13,500 one conditional PMF for every possible value of little y. 55 00:03:13,500 --> 00:03:16,170 And for every possible value of little y, we have a 56 00:03:16,170 --> 00:03:20,380 legitimate PMF who's values add to 1. 57 00:03:20,380 --> 00:03:22,320 Let's look at an example. 58 00:03:22,320 --> 00:03:25,630 Consider the joint PMF given in this table. 59 00:03:25,630 --> 00:03:31,690 Let us condition on the event that Y is equal to 2, which 60 00:03:31,690 --> 00:03:34,480 corresponds to this row in the diagram. 61 00:03:39,520 --> 00:03:44,829 We need to know the value of the marginal at this point, so 62 00:03:44,829 --> 00:03:50,800 we start by calculating the probability of Y at value 2. 63 00:03:50,800 --> 00:03:53,560 And this is found by adding the entries in 64 00:03:53,560 --> 00:03:55,400 this row of the table. 65 00:03:55,400 --> 00:03:58,920 And we find that this is 5 over 20. 66 00:03:58,920 --> 00:04:01,700 Then we can start calculating entries of 67 00:04:01,700 --> 00:04:03,730 the conditional PMF. 68 00:04:03,730 --> 00:04:09,230 So for example, the probability that X takes on 69 00:04:09,230 --> 00:04:14,580 the value of 1 given that Y takes the value of 2, it is 70 00:04:14,580 --> 00:04:20,320 going to be this entry, which is 0, divided by 5/20, which 71 00:04:20,320 --> 00:04:22,360 gives us 0. 72 00:04:22,360 --> 00:04:27,710 We can find the next entry, the probability of X taking 73 00:04:27,710 --> 00:04:33,180 the value of 2, given that Y takes the value of 2 will be 74 00:04:33,180 --> 00:04:38,400 this entry, 1/20 divided by 5/20. 75 00:04:38,400 --> 00:04:39,730 So it's going to be 1/5. 76 00:04:42,990 --> 00:04:46,180 And we can continue with the other two entries. 77 00:04:46,180 --> 00:04:53,490 And we can actually even plot the result once we're done. 78 00:04:53,490 --> 00:04:59,560 And what we have is that at 1, we have a probability of 0. 79 00:04:59,560 --> 00:05:02,860 At 2, we have a probability of 1/5. 80 00:05:07,588 --> 00:05:13,200 At 3, we have a probability of 3/20 divided 81 00:05:13,200 --> 00:05:19,070 5/20, which is 3/5. 82 00:05:19,070 --> 00:05:23,665 And at 4, we have, again, a probability of 1/5. 83 00:05:26,520 --> 00:05:31,040 So what we have plotted here is the conditional PMF. 84 00:05:31,040 --> 00:05:36,060 It's a PMF in the variable x, where x ranges over the 85 00:05:36,060 --> 00:05:40,710 possible values, but where we have fixed the value of y to 86 00:05:40,710 --> 00:05:43,120 be equal to 2. 87 00:05:43,120 --> 00:05:46,560 Now, we could have found this conditional PMF even faster 88 00:05:46,560 --> 00:05:50,600 without doing any divisions by following the intuitive 89 00:05:50,600 --> 00:05:53,200 argument that we have used before. 90 00:05:53,200 --> 00:05:55,740 We live in this conditional universe. 91 00:05:55,740 --> 00:05:59,150 We have conditioned on Y being equal to 2. 92 00:05:59,150 --> 00:06:03,740 The conditional probabilities will have the same proportions 93 00:06:03,740 --> 00:06:06,660 as the original probabilities, except that they needed to be 94 00:06:06,660 --> 00:06:09,050 scaled so that they add to 1. 95 00:06:09,050 --> 00:06:13,160 So they should be in the proportions of 0, 1, 3, 1. 96 00:06:13,160 --> 00:06:17,670 And for these to add to 1, we need to put everywhere a 97 00:06:17,670 --> 00:06:19,640 denominator of 5. 98 00:06:19,640 --> 00:06:24,470 So the proportions are indeed 0, 1, 3, and 1. 99 00:06:24,470 --> 00:06:30,730 Pictorially, the conditional PMF has the same form as the 100 00:06:30,730 --> 00:06:36,640 corresponding slice of the joint PMF, except, again, that 101 00:06:36,640 --> 00:06:40,159 the entries of that slice are renormalized so that the 102 00:06:40,159 --> 00:06:42,810 entries add to 1. 103 00:06:42,810 --> 00:06:45,480 And finally, an observation-- 104 00:06:45,480 --> 00:06:49,280 we can take the definition of the conditional PMF and turn 105 00:06:49,280 --> 00:06:53,300 it around by moving the denominator to the other side 106 00:06:53,300 --> 00:06:57,040 and obtain a formula, which is a version of the 107 00:06:57,040 --> 00:06:59,450 multiplication rule. 108 00:06:59,450 --> 00:07:03,850 The probability that X takes a value little x and Y takes a 109 00:07:03,850 --> 00:07:07,880 value little y is the product or the probability that Y 110 00:07:07,880 --> 00:07:11,080 takes this particular value times the conditional 111 00:07:11,080 --> 00:07:15,420 probability that X takes on the particular value little x, 112 00:07:15,420 --> 00:07:19,970 given that Y takes on the particular value little y. 113 00:07:19,970 --> 00:07:23,960 We also have a symmetrical relationship if we interchange 114 00:07:23,960 --> 00:07:29,430 the roles of X and Y. As we discussed earlier in this 115 00:07:29,430 --> 00:07:32,520 course, the multiplication rule can be used to specify 116 00:07:32,520 --> 00:07:34,210 probability models. 117 00:07:34,210 --> 00:07:36,990 One way of modeling two random variables is by 118 00:07:36,990 --> 00:07:39,580 specifying the joint PMF. 119 00:07:39,580 --> 00:07:43,060 But we now have an alternative, indirect, way 120 00:07:43,060 --> 00:07:45,260 using the multiplication rule. 121 00:07:45,260 --> 00:07:49,580 We can first specify the distribution of Y and then 122 00:07:49,580 --> 00:07:54,200 specify the conditional PMF of X for any given 123 00:07:54,200 --> 00:07:56,510 value of little y. 124 00:07:56,510 --> 00:08:00,540 And this completely determines the joint PMF, and so we have 125 00:08:00,540 --> 00:08:04,560 a full probability model. 126 00:08:04,560 --> 00:08:07,540 We can also provide similar definitions of conditional 127 00:08:07,540 --> 00:08:10,460 PMFs for the case where we're dealing with more than two 128 00:08:10,460 --> 00:08:12,120 random variables. 129 00:08:12,120 --> 00:08:16,080 In this case, notation is pretty self-explanatory. 130 00:08:16,080 --> 00:08:21,930 By looking at this expression here, you can probably guess 131 00:08:21,930 --> 00:08:26,230 that this stands for the probability that random 132 00:08:26,230 --> 00:08:31,720 variable X takes on a specific value, conditional on the 133 00:08:31,720 --> 00:08:36,380 random variables Y and Z taking on some 134 00:08:36,380 --> 00:08:38,520 other specific values. 135 00:08:38,520 --> 00:08:41,429 Using the definition of conditional probabilities, 136 00:08:41,429 --> 00:08:48,270 this is the probability that all events happen divided by 137 00:08:48,270 --> 00:08:51,810 the probability of the conditioning event, which, in 138 00:08:51,810 --> 00:08:56,930 our case, is the event that Y takes on a specific value and 139 00:08:56,930 --> 00:09:01,280 simultaneously, Z takes another specific value. 140 00:09:01,280 --> 00:09:07,080 In PMF notation, this is the ratio of the joint PMF of the 141 00:09:07,080 --> 00:09:12,000 three random variables together, divided by the joint 142 00:09:12,000 --> 00:09:19,670 PMF of the two random variables Y and Z. As another 143 00:09:19,670 --> 00:09:24,280 example, we could have an expression like this, which, 144 00:09:24,280 --> 00:09:28,370 again, stands for the probability that these two 145 00:09:28,370 --> 00:09:35,120 random variables take on specific values, conditional 146 00:09:35,120 --> 00:09:41,530 on this random variable taking on another value. 147 00:09:41,530 --> 00:09:45,020 Finally, we can have versions of the multiplication rule for 148 00:09:45,020 --> 00:09:47,460 the case where we're dealing with more 149 00:09:47,460 --> 00:09:49,360 than two random variables. 150 00:09:49,360 --> 00:09:51,770 Recall the usual multiplication rule. 151 00:09:51,770 --> 00:09:55,580 For three events happening simultaneously, let's apply 152 00:09:55,580 --> 00:09:59,430 this multiplication rule for the case where the event, A, 153 00:09:59,430 --> 00:10:04,490 stands for the event that the random variable X takes on a 154 00:10:04,490 --> 00:10:06,050 specific value. 155 00:10:06,050 --> 00:10:11,160 Let B be the event that Y takes on a specific value, and 156 00:10:11,160 --> 00:10:15,600 C be the event that the random variable Z takes 157 00:10:15,600 --> 00:10:17,470 on a specific value. 158 00:10:17,470 --> 00:10:20,720 Then we can take this relation, the multiplication 159 00:10:20,720 --> 00:10:23,820 rule, and translate it into PMF notation. 160 00:10:23,820 --> 00:10:27,470 The probability that all three events happen is equal to the 161 00:10:27,470 --> 00:10:32,690 product of the probability that the first event happens. 162 00:10:32,690 --> 00:10:36,000 Then we have the conditional probability that the second 163 00:10:36,000 --> 00:10:39,550 event happens given that the first happened, times the 164 00:10:39,550 --> 00:10:43,270 conditional probability that the third event happens-- 165 00:10:43,270 --> 00:10:45,210 this one-- given that the first 166 00:10:45,210 --> 00:10:46,650 two events have happened.