We are given the summation:
\[
\sum_{k=1}^{n} \left( \alpha^k + \frac{1}{\alpha^k} \right)^2 = 20
\]
where \( \alpha \) is one of the roots of the quadratic equation:
\[
x^2 + x + 1 = 0
\]
The roots of the quadratic equation are the cube roots of unity, given by:
\[
\alpha = e^{i\frac{2\pi}{3}} \quad \text{or} \quad \alpha = e^{i\frac{4\pi}{3}}
\]
For both of these roots, we know that:
\[
\alpha^3 = 1 \quad \text{and} \quad \alpha + \frac{1}{\alpha} = -1
\]
Thus, we can simplify the term \( \left( \alpha^k + \frac{1}{\alpha^k} \right)^2 \):
\[
\left( \alpha^k + \frac{1}{\alpha^k} \right) = 2\cos\left( \frac{2k\pi}{3} \right)
\]
So,
\[
\left( \alpha^k + \frac{1}{\alpha^k} \right)^2 = 4\cos^2\left( \frac{2k\pi}{3} \right)
\]
Now, we sum this for \( k = 1, 2, \dots, n \). We know that for the cube roots of unity, the sum of the squares of \( \left( \alpha^k + \frac{1}{\alpha^k} \right) \) over three terms gives 3 (since each term contributes 1). Thus, to get a total sum of 20, we must have \( n = 5 \), as:
\[
5 \times 4 = 20
\]
Therefore, the correct value of \( n \) is \( \boxed{5} \).
Thus, the correct answer is (3) 5.