ML-Likelihood Function

Probability Perspective

The likelihood function is a function of the parameter w about the statistical model

Probability describes the distribution of the random variable x when the parameter w is fixed.

Likelihood describes the influence of different parameters w on the distribution of a known random variable x.

Maximum Likelihood Estimation (MLE）

Assume are samples from , the probability of the observed samples occurrences is .

Likelihood function

Maximize the likelihood function to get

where is maximum likelihood estimation and is maximum likelihood statistic.

Log Likelihood Equation：

Bayesian Perspective

When the training data is relatively small, overfitting will occur, resulting in inaccurate parameter estimation

Add prior (先验) knowledge to the parameters

Bayesian Learning:
Consider the parameter as a random variable.
Objective: Given a set of observation data , find the distribution of parameter .

is also called posterior distribution (后验分布)

Bayesian rule
The relationship between and :

Table Of Contents
ML-Likelihood Function
Probability Perspective
Maximum Likelihood Estimation (MLE）
Bayesian Perspective