6 - PROBABILITY GENERATING FUNCTIONS Certain derivations presented in this course have been somewhat heavy on algebra. For example, determining the expectation of the Binomial distribution (page 5.1) turned out to be fairly tiresome. Another example of hard work was determining the set of probabilities associated with a sum, P(X +Y = t). Many of these tasks are greatly simplified by using probability generating functions. . . Probability Generating Functions - Introduction A polynomial whose coefficients are the probabilities associated with the different outcomes of throwing a fair die was given on page 5.8: G(n) = On0 + an+ + En2 + 6?3 + 6n+3 + =nÂș 1 1 1 The coefficients of the square of this polynomial give rise to a higher-order polynomial whose coefficients are the probabilities associated with the different sums which can occur when two dice are thrown. There is nothing special about a fair die. The set of probabilities associated with almost any discrete distribution can be used as the coefficients of G(n) whose general form is: G(n)=P(X=0)nÂș+P(X=1)n1+P(X=2)n2+P(X=3)n3+P(X=4)n4+ ... (6.1) This is a power series which, for any particular distribution, is known as the associated probability generating function. Commonly one uses the term generating function, without the attribute probability, when the context is obviously probability. Generating functions have interesting properties and can often greatly reduce the amount of hard work which is involved in analysing a distribution. The crucial point to notice, in the power series expansion of G(n), is that the coefficient of n" is the probability P(X = r). Properties of Generating Functions It is to be noted that G(0) = P(X = 0) and, rather more importantly, that: G(1)=P(X=0)+P(X=1)+P(X=2)+ ...= >P(X=r)=1 r In the particular case of the generating function for the fair die: 111111 G(1) = 0 + ++ 6+ 6 + 6+ 6 + 6 = 1 - 6.1 -
Next, consider G(n) together with its first and second derivatives G'(n) and G" (n) (the differentiation is with respect to n of course): G(n)=P(X=0)nÂș+P(X=1)n1+P(X=2)n2+P(X=3)n3+P(X=4)n4 + ... G (n) = 1P(X=1)nÂș+2P(X=2)n1 +3P(X=3)n2 +4P(X =4)n3+ ... G" (n) = 2.1 P(X = 2)nÂș +3.2P(X=3)n1 +4.3P(X =4)n2+ ... Now, consider G(1), G'(1) and G" (1): G(1)=P(X=0)+P(X=1)+P(X=2)+P(X=3)+P(X =4) + ··· G'(1) = 1P(X=1)+2P(X=2) +3P(X =3) +4P(X = 4) + ··· G" (1) = 2.1 P(X =2) +3.2P(X =3) +4.3P(X =4) + ... At this stage, recall the general formula for the expectation of an arbitrary function of a random variable: E(f(X))=>f(r).P(X=r) r Then, express G(1), G'(1) and G" (1) in sigma notation and derive the following results: G(1)=>P(X=r)=1 r G'(1) =>r.P(X=r) = E(X) r G"(1)=>r(r-1).P(X=r) = E(X(X-1)) r (6.2) By differentiating the generating function one can directly obtain the expectation E(X). By differentiating again one can obtain E(X(X -1)). These two results lead to the rapid derivation of V(X). Note: G (1) + G'(1) - (G(1))2 = E(X(X-1))+E(X) - (E(X))2 = E(X2)-E(X)+E(X) -(E(X))2 = E(X2) - (E(X))2 = V(X) (6.3) A generating function is particularly helpful when the probabilities, as coefficients, lead to a