Kullback-Leibler Divergence

A general number describing the divergence between two distributions. Often used as a Loss Function

Formula

$D_{KL}(P||Q)={\sum }_{x\in X}P(x)\log (\frac{P(x)}{Q(x)})$