In an elastic collision, both momentum and kinetic energy are conserved.
However, the coefficient of restitution \( e \) is defined by the relative velocities of the balls before and after the collision:
\[
e = \frac{v_2' - v_1'}{v_1 - v_2}
\]
where \( v_1 \) and \( v_2 \) are the velocities of the balls before the collision, and \( v_1' \) and \( v_2' \) are the velocities of the balls after the collision.
Given:
- Before the collision, the first ball is moving at \( v \), and the second ball is at rest.
- The kinetic energy after the collision is \( \frac{3}{4} \) of the original.
From the conservation of kinetic energy:
\[
\frac{1}{2} m v^2 = \frac{1}{2} m {v_1'}^2 + \frac{1}{2} m {v_2'}^2
\]
Since the total kinetic energy is reduced to \( \frac{3}{4} \), we can write:
\[
\frac{1}{2} m {v_1'}^2 + \frac{1}{2} m {v_2'}^2 = \frac{3}{4} \cdot \frac{1}{2} m v^2
\]
Simplifying, we find:
\[
{v_1'}^2 + {v_2'}^2 = \frac{3}{4} v^2
\]
Now, applying the coefficient of restitution formula and solving the system, we find:
\[
e = \frac{1}{\sqrt{2}}
\]
Thus, the correct answer is (b).