PROBABILITY These notes accompany the twelve lectures of the course on Probability given to Part IA Computer Scientists. An outline schedule of the lectures is presented overleaf. Prerequisites It is assumed throughout this document that the reader studied Mathematics to A-level or equivalent and followed the Michaelmas Term course in Mathematics given to Part IA Natural Scientists. No familiarity with Probability is assumed but a little account is taken of those with pre-University experience of Probability, perhaps having studied the topic at A-level. Even in the earlier lectures material is presented in a way which is likely to be different from the approach used at A-level and many of the exercises are quite demanding. Recommended Books There are numerous books on Probability. In the preparation of this course the book which was most extensively consulted was Probability: An Introduction by G. Grimmet and D. Welsh. An important standard text is An Introduction to Probability Theory, Volume I by W. Feller. Appendix This is not a course on statistics but, in response to requests from previous Computer Science students, there is an appendix at the end of this document which addresses a particular question: Why is it that the formula for population variance is E(x- x)2 /n but the formula for sample variance is 2(x - x)2 / (n - 1) ? Acknowledgements Some of the material in this document has been adapted from a previous version of the course which was given by Prof. A.W.F. Edwards. Many of the exercises are taken directly from those set on the earlier course. Many other exercises, including all of Exercises VIII, were supplied by Prof. P. Robinson. Exercises were also supplied by Mrs K. Edgcombe and Mr R. Stratford. Dr J.K.M. Moody and Mr D.M. Sheridan have made a number of comments which have been taken into account in preparing the current edition. F.H. King January 2008
SCHEDULE OF LECTURES 1. SINGLE RANDOM VARIABLE. Probability in Computer Science. Sample space. Event space. Probability space. Relationship to set theory. Random variables: discrete versus continuous. The P(X=r) notation. Probability axioms. The inclusion- exclusion theorem. Conditional probability. Mapping. 2. TWO OR MORE RANDOM VARIABLES. The multiplication theorem. Independence and distinguishability. Array diagrams. Bayes's theorem. Event trees. Combinatorial numbers. Pascal's triangle. The binomial theorem. 3. DISCRETE DISTRIBUTIONS. Uniform distribution. Triangular distribution. Binomial distribution. Trinomial distribution. Multinomial distribution. Expectation or mean. 4. MEANS AND VARIANCES. Use of derived random variables and generalised expectation. Variance and standard deviation. Geometric distribution. Poisson distribution. Revision of summation (double-sigma sign). Mean and variance when there are two or more random variables. Covariance. 5. CORRELATION. Mean and variance of the Binomial distribution. Correlation coefficient. Complete positive and complete negative correlation. P(X+Y=t). A polynomial with probabilities as coefficients. 6. PROBABILITY GENERATING FUNCTIONS. Generating functions. Means and variances of distributions. Application of generating functions to P(X+Y=t). 7. DIFFERENCE EQUATIONS. Introduction to linear, second-order difference equations with constant coefficients. How these equations are found in Probability. Solving homogeneous and inhomogeneous difference equations. 8. STOCHASTIC PROCESSES. Random