Step 1: Expression for \(\vec{c}\)
The vector \(\vec{c}\) is given as:
\(\vec{c} = \vec{a} - \vec{b}\).
Substitute \(\vec{a} = \alpha \hat{i} + 4 \hat{j} + 2 \hat{k}\) and \(\vec{b} = 5 \hat{i} + 3 \hat{j} + 4 \hat{k}\):
\(\vec{c} = (\alpha - 5)\hat{i} + (4 - 3)\hat{j} + (2 - 4)\hat{k}\).
Thus:
\(\vec{c} = (\alpha - 5)\hat{i} + \hat{j} - 2\hat{k}\).
Step 2: Area of the triangle
The area of the triangle is given as:
\(\text{Area} = \frac{1}{2} |\vec{a} \times \vec{c}|.\)
Substitute \(\text{Area} = 5\sqrt{6}\):
\(\frac{1}{2} |\vec{a} \times \vec{c}| = 5\sqrt{6}.\)
\(|\vec{a} \times \vec{c}| = 10\sqrt{6}.\)
Step 3: Cross product \(\vec{a} \times \vec{c}\)
The cross product \(\vec{a} \times \vec{c}\) is given by:
\[ \vec{a} \times \vec{c} = \begin{vmatrix} \hat{i} & \hat{j} & \hat{k} \\ \alpha & 4 & 2 \\ \alpha - 5 & 1 & -2 \end{vmatrix}. \]
Expanding the determinant:
\(\vec{a} \times \vec{c} = \hat{i} \begin{vmatrix} 4 & 2 \\ 1 & -2 \end{vmatrix} - \hat{j} \begin{vmatrix} \alpha & 2 \\ \alpha - 5 & -2 \end{vmatrix} + \hat{k} \begin{vmatrix} \alpha & 4 \\ \alpha - 5 & 1 \end{vmatrix}. \)
Calculate each minor:
1. For \(\hat{i}\): \(\begin{vmatrix} 4 & 2 \\ 1 & -2 \end{vmatrix} = (4)(-2) - (2)(1) = -8 - 2 = -10.\)
2. For \(\hat{j}\): \(\begin{vmatrix} \alpha & 2 \\ \alpha - 5 & -2 \end{vmatrix} = (\alpha)(-2) - (\alpha - 5)(2) = -2\alpha - 2\alpha + 10 = -4\alpha + 10.\)
3. For \(\hat{k}\): \(\begin{vmatrix} \alpha & 4 \\ \alpha - 5 & 1 \end{vmatrix} = (\alpha)(1) - (\alpha - 5)(4) = \alpha - 4\alpha + 20 = -3\alpha + 20.\)
Thus:
\(\vec{a} \times \vec{c} = -10\hat{i} - (-4\alpha + 10)\hat{j} + (-3\alpha + 20)\hat{k}.\)
\(\vec{a} \times \vec{c} = -10\hat{i} + (4\alpha - 10)\hat{j} + (-3\alpha + 20)\hat{k}.\)
Step 4: Magnitude of \(\vec{a} \times \vec{c}\)
The magnitude is:
\(|\vec{a} \times \vec{c}| = \sqrt{(-10)^2 + (4\alpha - 10)^2 + (-3\alpha + 20)^2}.\)
\(|\vec{a} \times \vec{c}| = \sqrt{100 + (16\alpha^2 - 80\alpha + 100) + (9\alpha^2 - 120\alpha + 400)}.\)
\(|\vec{a} \times \vec{c}| = \sqrt{25\alpha^2 - 200\alpha + 600}.\)
Set \(|\vec{a} \times \vec{c}| = 10\sqrt{6}\):
\(\sqrt{25\alpha^2 - 200\alpha + 600} = 10\sqrt{6}.\)
Square both sides:
\(25\alpha^2 - 200\alpha + 600 = 600.\)
\(25\alpha(\alpha - 8) = 0.\)
Since \(\alpha > 0\), \(\alpha = 8.\)
Step 5: Calculate \(|\vec{c}|^2\)
Substitute \(\alpha = 8\) into \(\vec{c}\):
\(\vec{c} = 3\hat{i} + \hat{j} - 2\hat{k}.\)
The magnitude squared is:
\(|\vec{c}|^2 = 3^2 + (1)^2 + (-2)^2.\)
\(|\vec{c}|^2 = 9 + 1 + 4 = 14.\)
Final Answer: Option (2).