Question

Given data \( \{(-1, 1), (2, -5), (3, 5)\} \) of the form \( (x, y) \), we fit a model \( y = wx \) using linear least-squares regression. The optimal value of \( w \) is:
(Round off to three decimal places)

Hint:

For linear regression using least squares, the optimal weight \( w \) can be found by minimizing the squared error function, which can be done by differentiating the error with respect to \( w \) and solving for it.

Answer 1

To find the optimal value of \( w \), we need to apply the least-squares regression method to minimize the error between the predicted and actual values of \( y \). The least-squares error function for linear regression is given by: \[ E(w) = \sum_{i=1}^{n} (y_i - wx_i)^2 \] Where:
\( x_i \) and \( y_i \) are the data points.
\( w \) is the parameter we need to optimize.
The optimal value of \( w \) is obtained by minimizing \( E(w) \). We take the derivative of \( E(w) \) with respect to \( w \), set it equal to zero, and solve for \( w \). First, express \( E(w) \): \[ E(w) = (1 - w(-1))^2 + (-5 - w(2))^2 + (5 - w(3))^2 \] \[ E(w) = (1 + w)^2 + (-5 - 2w)^2 + (5 - 3w)^2 \] Now, expand the squares: \[ E(w) = (1 + 2w + w^2) + (25 + 20w + 4w^2) + (25 - 30w + 9w^2) \] \[ E(w) = 1 + 25 + 25 + 2w + 20w - 30w + w^2 + 4w^2 + 9w^2 \] \[ E(w) = 51 - 8w + 14w^2 \] Next, differentiate \( E(w) \) with respect to \( w \): \[ \frac{dE(w)}{dw} = -8 + 28w \] Set the derivative equal to zero to find the critical point: \[ -8 + 28w = 0 \] \[ 28w = 8 \] \[ w = \frac{8}{28} = \frac{2}{7} \approx 0.286 \] Thus, the optimal value of \( w \) is approximately \( 0.286 \).