Take
\begin{align*}
A&=
\begin{bmatrix}
1&0\\0&0
\end{bmatrix}
&
B&=
\begin{bmatrix}
0&0\\1&0
\end{bmatrix}
\end{align*}
Then $A$ is already in reduced row-echelon form, and by swapping rows, $B$ row-reduces to $A$. So the product of the row-echelon forms of $A$ is $AA=A\neq\zeromatrix$. However, the product $AB$ is the $2\times 2$ zero matrix, which is in reduced-echelon form, and not equal to $AA$. When you get there, Theorem PEEF or Theorem EMDRO might shed some light on why we would not expect this statement to be true in general.
If $A$ and $B$ are nonsingular, then $AB$ is nonsingular (Theorem NPNT), and all three matrices $A$, $B$ and $AB$ row-reduce to the identity matrix (Theorem NMRRI). By Theorem MMIM, the desired relationship is true.