The gravitational potential energy at a point outside the system is the sum of the potentials due to both the sphere and the shell. After calculating the potential energies and applying the principle of superposition, we find the correct value to be \( \frac{-3GM}{5a} \).
Thus, the correct answer is option (1).