If A*B is defined then matrix A must have the same number of columns as B has rows. In other words,
dimensions of matrix A = m x n
dimensions of matrix B = n x p
So for now, matrix AB is m x p. Note how the n terms match up. The 'n' terms are the inner terms
(m x n) x (n x p)
--------------------------------------
We're told that A*B is a square matrix, so that means m = p. We have the same number of rows and columns. This means
dimensions of matrix A = m x n
dimensions of matrix B = n x m
(m x n) x (n x m)
So matrix A*B is an m x m matrix.
--------------------------------------
If we swap things around to compute B*A, then we can see that this is possible. Why? Because the 'm's now match up
dimensions of matrix B = n x m
dimensions of matrix A = m x n
The 'm's are now the inner terms.
(n x m) x (m x n)
meaning that matrix B*A is an n x n matrix. This proves that B*A is defined.