Finals

Scope of finals exam:

Format: 2 hours, 4 questions (30 mins/qns), no special integration needed
- Math manipulation expected, though no need calculators
- May follow examples from lecture notes
Topical hints:
1. Binomial and normal distributions: Need to know basics of probabilities (mean, variance, normalization), priors (conjugate, Jeffrey's, (im)proper), posterior predictive distributions.
2. Sampling with Metropolis-Hastings: Markov chains and proofs for stationarity, MH algorithm + formula + idea (i.e. why rule chosen)
3. Model comparison using Savage-Dickey ratio: Bayes' factor (less of model validation)
4. Binary classification and kernel methods: Validate using confusion matrix parameters, derive ROC, feature maps and kernel methods, validating kernels (less of regression and neural networks)

Topic 1: Binomial/normal distributions

$$$$ p(\theta|x) = \frac{p(x|\theta)p(\theta)}{p(x)} $$$$

Some terminology:

Basic definitions

$$$$ E[\tau] \equiv{} \langle{}\tau\rangle{} := \int dx\,\tau(x)p(x) $$$$

$$$$ \mu_n := E[(x-\mu)^n], \qquad{} Var[x] \equiv{} \sigma^2 := \mu_2 $$$$

Show that for two random variables x, y, E[x] = E[E[x|y]].

Show that for two random variables x, y, Var[x] = E[Var[x|y]] + Var[E[x|y]].

Binomial distribution

\begin{align} \text{Bin}(x|n,\theta{}) = \left(\begin{matrix}n\\x\end{matrix}\right) \theta{}^x (1-\theta)^{(n-x)} \end{align}

Binomial distribution: