If A and B are square matrices of the same order and B is a skew-symmetric matrix, then we need to determine the nature of \(A'BA\).
Since B is a skew-symmetric matrix, we have \(B' = -B\).
Let \(C = A'BA\). To determine if C is symmetric or skew-symmetric, we need to find \(C'\).
\(C' = (A'BA)'\)
Using the property \((ABC)' = C'B'A'\), we have:
\(C' = A'B'(A')' = A'B'A\)
Since B is skew-symmetric, \(B' = -B\), so:
\(C' = A'(-B)A = -A'BA = -C\)
Since \(C' = -C\), C is a skew-symmetric matrix.
Therefore, the correct option is (D) Skew symmetric matrix.