Chapter 8: Random Vectors and Random Functions

Learning Objectives 

  • Define a Mixed random Vector as a Measurable Function 
  • Use Mixed Random Vector Radon-Nikodym Probability Densities 
  • Compute Expectations of Random Functions

The objective of Chapter 8 is to provide mathematical tools for using probability distributions of mixed random vectors whose components may include both discrete and absolutely continuously random variables. In particular, the Radon-Nikodym density is used to specify the probability distribution of a mixed random vector. In addition, for the purpose of providing a better understanding of the Radon-Nikodym density, Chapter 8 introduces measure theory concepts including the concepts of a sigma-field, probability measure, sigma-finite measure, Borel-measurable function, almost everywhere, and Lebesgue integration. The concept of a random function is introduced, and then conditions for ensuring the existence of the expectations of random functions are provided. The chapter also includes discussions of important concentration inequalities for obtaining error bounds on estimators and predictions. In particular, the Markov inequality, Chebyshev inequality, and Hoeffding inequality are used to show how to estimate generalization error bounds and estimate the amount of training data required for learning.

The podcast LM101-086: Ch8: How  to Learn the Probability of Infinitely Many Outcomes provides an overview of the main ideas of this book chapter, some tips for students to help them read this chapter, as well as some guidance to instructors for teaching this chapter to students.