1. Express $\mathbf{c}$ as a linear combination of $\mathbf{a}$ and $\mathbf{b}$
Since $\mathbf{c}$ lies in the plane of $\mathbf{a}$ and $\mathbf{b}$, we can write it as:
$\mathbf{c} = x\mathbf{a} + y\mathbf{b}$, where $x$ and $y$ are scalars.
$\mathbf{c} = x(i + 2j + k) + y(2i + j - k)$
$\mathbf{c} = (x + 2y)i + (2x + y)j + (x - y)k$
2. Use the perpendicularity condition
$\mathbf{c}$ is perpendicular to $\mathbf{a}$, so their dot product is zero:
$\mathbf{a} \cdot \mathbf{c} = 0$
$(i + 2j + k) \cdot ((x + 2y)i + (2x + y)j + (x - y)k) = 0$
$(x + 2y) + 2(2x + y) + (x - y) = 0$
$x + 2y + 4x + 2y + x - y = 0$
$6x + 3y = 0$
$2x + y = 0$
$y = -2x$
3. Substitute $y$ in the expression for $\mathbf{c}$
Substitute $y = -2x$ into the expression for $\mathbf{c}$:
$\mathbf{c} = (x + 2(-2x))i + (2x + (-2x))j + (x - (-2x))k$
$\mathbf{c} = (x - 4x)i + (2x - 2x)j + (x + 2x)k$
$\mathbf{c} = -3xi + 0j + 3xk$
$\mathbf{c} = x(-3i + 3k)$
4. Use the unit vector condition
$\mathbf{c}$ is a unit vector, so its magnitude is 1:
$||\mathbf{c}|| = 1$
$\sqrt{ (-3x)^2 + (3x)^2} = 1$
$\sqrt{9x^2 + 9x^2} = 1$
$\sqrt{18x^2} = 1$
$3\sqrt{2} |x| = 1$
$|x| = \frac{1}{3\sqrt{2}}$
Therefore, $x = \frac{1}{3\sqrt{2}}$ or $x = -\frac{1}{3\sqrt{2}}$
5. Find the possible vectors $\mathbf{c}$
6. Match with the given options
$\mathbf{c} = \frac{1}{\sqrt{2}}(-i + k)$
Answer: The correct answer is option 2.