Step 1: Understanding the problem.
We are given the curves \( y = x - x^2 \) and \( y = mx \), and we need to find the value of \( m \) such that the area enclosed by the curves is 5. This can be done by setting up the area integral and solving for \( m \).
Step 2: Conclusion.
The value of \( m \) is \( -2 \), corresponding to option (2).