Question

Consider the neural network shown in the figure with \[ \text{inputs: } u = 2, \, v = 3 \] \[ \text{weights: } a = 1, b = 1, c = 1, d = -1, e = 4, f = -1 \] \[ \text{output: } y \] R denotes the ReLU function, \( R(x) = \max(0, x) \). 

Given \( u = 2, v = 3, a = 1, b = 1, c = 1, d = -1, e = 4, f = -1 \), which one of the following is correct?

• A. \( \frac{\partial y}{\partial a} = 8, \frac{\partial y}{\partial f} = 0 \)
• B. \( \frac{\partial y}{\partial a} = 1, \frac{\partial y}{\partial f} = 0 \)
• C. \( \frac{\partial y}{\partial a} = 1, \frac{\partial y}{\partial f} = -1 \)
• D. \( \frac{\partial y}{\partial a} = 2, \frac{\partial y}{\partial f} = -1 \)

Hint:

When calculating derivatives in neural networks, remember that the derivative of a ReLU function is:
\( 1 \) if the input is positive,
\( 0 \) if the input is non-positive.

Answer 1

The Correct Option is A

Given the inputs and weights, we can compute the output \( y \) as follows:

1. The first ReLU unit gives an output of \( R(5) = 5 \),
2. The second ReLU unit gives an output of \( R(-1) = 0 \),
3. The final output \( y \) is \( R(19) = 20 \).
Now, we compute the derivatives:

\( \frac{\partial y}{\partial a} = 8 \),
\( \frac{\partial y}{\partial f} = 0 \).
Thus, the correct answer is Option (A).