Method 1: Using Coordinate Transformation
Step 1: Write down the transformation equations for rotation of axes.
When the coordinate axes are rotated through an angle \(\theta\) in the anti-clockwise direction without shifting the origin, the relationship between the old coordinates \((x, y)\) and the new coordinates \((x', y')\) is given by:
\[
x = x' \cos\theta - y' \sin\theta
\]
\[
y = x' \sin\theta + y' \cos\theta
\]
Given \(\theta = \frac{\pi}{4}\), we have \(\cos\left(\frac{\pi}{4}\right) = \frac{1}{\sqrt{2}}\) and \(\sin\left(\frac{\pi}{4}\right) = \frac{1}{\sqrt{2}}\).
Substituting these values:
\[
x = \frac{x'}{\sqrt{2}} - \frac{y'}{\sqrt{2}} = \frac{x' - y'}{\sqrt{2}}
\]
\[
y = \frac{x'}{\sqrt{2}} + \frac{y'}{\sqrt{2}} = \frac{x' + y'}{\sqrt{2}}
\]
Step 2: Substitute the transformation equations into the original equation.
The original equation is \(x^2 + y^2 - 2x - 4y - 20 = 0\).
Substitute the expressions for \(x\) and \(y\):
\[
\left(\frac{x' - y'}{\sqrt{2}}\right)^2 + \left(\frac{x' + y'}{\sqrt{2}}\right)^2 - 2\left(\frac{x' - y'}{\sqrt{2}}\right) - 4\left(\frac{x' + y'}{\sqrt{2}}\right) - 20 = 0
\]
Expand the squares:
\[
\frac{(x')^2 - 2x'y' + (y')^2}{2} + \frac{(x')^2 + 2x'y' + (y')^2}{2} - \frac{2(x' - y')}{\sqrt{2}} - \frac{4(x' + y')}{\sqrt{2}} - 20 = 0
\]
Combine the terms with common denominators:
\[
\frac{2(x')^2 + 2(y')^2}{2} - \sqrt{2}(x' - y') - 2\sqrt{2}(x' + y') - 20 = 0
\]
\[
(x')^2 + (y')^2 - \sqrt{2}x' + \sqrt{2}y' - 2\sqrt{2}x' - 2\sqrt{2}y' - 20 = 0
\]
Combine like terms:
\[
(x')^2 + (y')^2 + (-\sqrt{2} - 2\sqrt{2})x' + (\sqrt{2} - 2\sqrt{2})y' - 20 = 0
\]
\[
(x')^2 + (y')^2 - 3\sqrt{2}x' - \sqrt{2}y' - 20 = 0.
\]
Step 3: Identify the coefficients \(a, b, h, g, f, c\) in the transformed equation.
The transformed equation is of the form \(ax^2 + 2hxy + by^2 + 2gx + 2fy + c = 0\).
Comparing \((x')^2 + (y')^2 - 3\sqrt{2}x' - \sqrt{2}y' - 20 = 0\) with the general form (using \(x'\) and \(y'\)):
\(a = 1\) (coefficient of \((x')^2\))
\(b = 1\) (coefficient of \((y')^2\))
\(2h = 0 \implies h = 0\) (coefficient of \(x'y'\))
\(2g = -3\sqrt{2} \implies g = -\frac{3\sqrt{2}}{2}\) (coefficient of \(x'\))
\(2f = -\sqrt{2} \implies f = -\frac{\sqrt{2}}{2}\) (coefficient of \(y'\))
\(c = -20\) (constant term)
Step 4: Calculate the determinant \(\begin{vmatrix} a & h & g
h & b & f
g & f & c \end{vmatrix}\).
Substitute the identified coefficients into the determinant:
\[
\begin{vmatrix} 1 & 0 & -\frac{3\sqrt{2}}{2} \\ 0 & 1 & -\frac{\sqrt{2}}{2} \\ -\frac{3\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} & -20 \end{vmatrix}
\]
Expand the determinant along the first row:
\[
= 1 \left( (1)(-20) - \left(-\frac{\sqrt{2}}{2}\right)\left(-\frac{\sqrt{2}}{2}\right) \right) - 0 \left( \text{minor} \right) + \left(-\frac{3\sqrt{2}}{2}\right) \left( (0)\left(-\frac{\sqrt{2}}{2}\right) - (1)\left(-\frac{3\sqrt{2}}{2}\right) \right)
\]
\[
= 1 \left( -20 - \frac{2}{4} \right) + \left(-\frac{3\sqrt{2}}{2}\right) \left( 0 + \frac{3\sqrt{2}}{2} \right)
\]
\[
= \left( -20 - \frac{1}{2} \right) - \left(\frac{3\sqrt{2}}{2}\right) \left(\frac{3\sqrt{2}}{2}\right)
\]
\[
= -\frac{41}{2} - \frac{9 \times 2}{4}
\]
\[
= -\frac{41}{2} - \frac{18}{4}
\]
\[
= -\frac{41}{2} - \frac{9}{2}
\]
\[
= -\frac{50}{2} = -25.
\]
Method 2: Using Invariants (Shorter Method)
The value of the determinant \(\Delta = \begin{vmatrix} A & H & G \\ H & B & F \\ G & F & C \end{vmatrix}\) from the general second-degree equation \(Ax^2 + 2Hxy + By^2 + 2Gx + 2Fy + C = 0\) is an invariant under rotation and translation of axes. This means its value does not change after coordinate transformation.
The given original equation is \(x^2 + y^2 - 2x - 4y - 20 = 0\).
Comparing this to \(Ax^2 + 2Hxy + By^2 + 2Gx + 2Fy + C = 0\):
\(A = 1\) (coefficient of \(x^2\))
\(B = 1\) (coefficient of \(y^2\))
\(2H = 0 \implies H = 0\) (coefficient of \(xy\))
\(2G = -2 \implies G = -1\) (coefficient of \(x\))
\(2F = -4 \implies F = -2\) (coefficient of \(y\))
\(C = -20\) (constant term)
Now, calculate the determinant using these original coefficients:
\[
\Delta = \begin{vmatrix} A & H & G \\ H & B & F \\ G & F & C \end{vmatrix} = \begin{vmatrix} 1 & 0 & -1 \\ 0 & 1 & -2 \\ -1 & -2 & -20 \end{vmatrix}
\]
Expand the determinant along the first row:
\[
= 1 \left( (1)(-20) - (-2)(-2) \right) - 0 \left( \text{minor} \right) + (-1) \left( (0)(-2) - (1)(-1) \right)
\]
\[
= 1 \left( -20 - 4 \right) - 0 - 1 \left( 0 + 1 \right)
\]
\[
= -24 - 1
\]
\[
= -25.
\]
Since the determinant is an invariant, the value of \(\begin{vmatrix} a & h & g \\ h & b & f \\ g & f & c \end{vmatrix}\) in the new coordinate system is also -25.