2. Bayes Decision Theory¶

19th, Sept.

2.1. Assumptions¶

$\sum_{i=1}^cP(\omega=\omega_i)=1$

The a priori or prior probability reflects our knowledge of how likely we expect a certain state of nature before we can actually observe said state of nature.

$P(\omega=\omega_i)$

A feature is an observable variable, scalar $x$ or vector $\mathbf{x}$
Likelihood (Class-Conditional Density): the probability density function for feature, given that the state of nature is $\omega$

$p(\mathbf{x}|\omega)$

Note that lower-case p refers to probability density function and upper-case P refers to probability.

Posterior probability: the probability of a certain state of nature given our observables

$P(\omega, \mathbf{x}) = P(\omega|\mathbf{x})p(\mathbf{x}) = p(\mathbf{x}|\omega)P(\omega)$

$P(\omega|\mathbf{x})=\frac{P(\omega, \mathbf{x})}{p(\mathbf{x})}=\frac{p(\mathbf{x}|\omega)P(\omega)}{\sum_ip(\mathbf{x}|\omega_i)P(\omega_i)}$

$\omega^*=\arg\max_iP(\omega_i|\mathbf{x})$

2016_09_19_c585278e78e9a334af8b5126f41f413d

$\min P(error)=\int_{-\infty}^{+\infty}P(error|\mathbf{x})P(\mathbf{x})d\mathbf{x}$

Decision making relies on both the priors and the likelihoods and Bayes Decision Rule combines them to achieve the minimum probability of error