Math — Shannon Entropy
Information content of a distribution.
Rendering…
Make it your own.
H = -\sum_i p_i \log_2 p_i
H_{max} = \log_2 n\ \text{(uniform)}
I = -\log_2 p\ \text{(self-information)}
\text{measured in bits}
Information content of a distribution.
H = -\sum_i p_i \log_2 p_i
H_{max} = \log_2 n\ \text{(uniform)}
I = -\log_2 p\ \text{(self-information)}
\text{measured in bits}