Step 1: Define the domain. We are maximizing the function over the triangle: \[ x \geq 0, \quad y \geq 0, \quad x + y \leq 1 \]
Step 2: Compute partial derivatives. \[ f(x, y) = x^2 y^3 (1 - x - y) \] Compute: \[ f_x = x y^3 (2(1 - x - y) - x), \quad f_y = x^2 y^2 (3(1 - x - y) - y) \] Solve: \[ x = \frac{1}{6}, \quad y = \frac{1}{3}, \quad 1 - x - y = \frac{1}{2} \]
Step 3: Evaluate the function at critical point. \[ f\left( \frac{1}{6}, \frac{1}{3} \right) = \left( \frac{1}{6} \right)^2 \left( \frac{1}{3} \right)^3 \left( \frac{1}{2} \right) = \frac{1}{36} \cdot \frac{1}{27} \cdot \frac{1}{2} = \frac{1}{1944} = \frac{1}{432} \]
Step 4: Verify boundaries.
On boundaries (\( x = 0 \), \( y = 0 \), \( x + y = 1 \)), the function becomes 0. Hence, the max is interior.