Question

A source transmits symbols from an alphabet of size 16. The value of maximum achievable entropy (in bits) is \(\_\_\_\_\).

Hint:

Maximum entropy occurs when all symbols in the alphabet are equally likely. Use \(\log_2 N\) to calculate maximum entropy for a source with \(N\) symbols.

Answer 1

Step 1: Use the formula for maximum entropy.
The maximum entropy of a source with \(N\) equally likely symbols is given by: \[ H_{{max}} = \log_2 N. \] Step 2: Calculate for \(N = 16\).
Substitute \(N = 16\) into the formula: \[ H_{{max}} = \log_2 16. \] Since \(16 = 2^4\), we have: \[ H_{{max}} = 4 \, {bits}. \] Final Answer: \[ \boxed{{4}} \]