# HTW Berlin - Angewandte Informatik - Advanced Topics - Exercise - Entropy

## Introduction

In this notebook you will calculate the entropy of a probability mass function (pmf).

In order to detect errors in your own code, execute the notebook cells containing assert or assert_almost_equal. These statements raise exceptions, as long as the calculated result is not yet correct.

## Requirements

### Knowledge

To complete this exercise notebook you should possess knowledge about the following topics.

• Proability mass function (pmf)
• Proability density function (pdf)
• Entropy
• Expected value

### Python Modules

# External Modules
import numpy as np
import matplotlib.pyplot as plt
import scipy.stats as ss
from numpy.testing import assert_almost_equal

%matplotlib inline

## Exercise

At a party you are offered a free drink $x$ by playing a special roulette game. With probability 40% you receive 1 of 3 possible kinds of beer. 30% for 1 of 5 possible kinds of wine, 20% for 1 of 10 different kinds of schnapps and with a probability of 10% you receive vinegar. The different kinds in each of those categories (beer, wine, ...) are uniformly distributed.

Estimate the entropy $X$ of your drink in bits.

Reminder:

$H(X) = \sum_{x \in \mathcal A x} P(x) \log_2 \frac{1}{P(x)}$

or

$H(X) = - \sum_{x \in \mathcal A x} P(x) \log_2 P(x)$
# Complete this cell

# (...)

hX = 
# Executing this cell must not throw an Exception
# The solution is obfuscated so you can solve the exercise without unintendedly spoiling yourself

obfuscated_solution = 126361276584.6638/32894689023
assert_almost_equal(hX, obfuscated_solution)

## Literature

The following license applies to the complete notebook, including code cells. It does however not apply to any referenced external media (e.g., images).

HTW Berlin - Angewandte Informatik - Advanced Topics - Exercise - Entropy
by Christian Herta, Klaus Strohmenger