There is no solution when the matrix is inconsistent.
The solution of a system of equations is not defined when the system is inconsistent or the equations are dependent. Let's discuss each case in the context of matrices.
Inconsistent system: An inconsistent system of equations has no solution. In matrix form, this occurs when the system of equations can be represented as an augmented matrix [A|B], where A is the coefficient matrix and B is the constant matrix. If the row-reduced echelon form (RREF) of [A|B] contains a row of the form [0 0 ... 0 | b], where b is a nonzero constant, then the system is inconsistent and has no solution.
Dependent equations: Dependent equations occur when one equation can be obtained by adding or multiplying other equations in the system. In matrix form, this means that the rows of the augmented matrix [A|B] are linearly dependent. To determine linear dependence, we can row-reduce the augmented matrix to its RREF. If the RREF contains a row of the form [0 0 ... 0 | 0], where the corresponding row in the coefficient matrix A is not all zeros, then the system has dependent equations. In this case, there are infinitely many solutions, as the equations are not providing independent information.
To summarize, the solution of a system of equations is not defined when:
The system is inconsistent, and the augmented matrix [A|B] in RREF form contains a row of the form [0 0 ... 0 | b], where b is nonzero. The equations are dependent, and the augmented matrix [A|B] in RREF form contains a row of the form [0 0 ... 0 | 0], where the corresponding row in the coefficient matrix A is not all zeros.