Rather than jumping into the two halves of the equivalence, we first establish a few items. Let $B$ be the unique square matrix that is row-equivalent to $A$ and in reduced row-echelon form (Theorem REMEF, Theorem RREFU). For each of the row operations that converts $B$ into $A$, there is an elementary matrix $E_i$ which effects the row operation by matrix multiplication (Theorem EMDRO). Repeated applications of Theorem EMDRO allow us to write \begin{equation*} A=E_sE_{s-1}\dots E_2E_1B \end{equation*}
Then \begin{align*} \detname{A} &= \detname{E_sE_{s-1}\dots E_2E_1B}\\ &=\detname{E_s}\detname{E_{s-1}}\dots\detname{E_2}\detname{E_1}\detname{B} &&\knowl{./knowls/theorem.DEMMM.knowl}{\text{Theorem DEMMM}} \end{align*}
From Theorem DEM we can infer that the determinant of an elementary matrix is never zero (note the ban on $\alpha=0$ for $\elemmult{\alpha}{i}$ in Definition ELEM). So the product on the right is composed of nonzero scalars, with the possible exception of $\detname{B}$. More precisely, we can argue that $\detname{A}=0$ if and only if $\detname{B}=0$. With this established, we can take up the two halves of the equivalence.
(⇒) If $A$ is singular, then by Theorem NMRRI, $B$ cannot be the identity matrix. Because (1) the number of pivot columns is equal to the number of nonzero rows, (2) not every column is a pivot column, and (3) $B$ is square, we see that $B$ must have a zero row. By Theorem DZRC the determinant of $B$ is zero, and by the above, we conclude that the determinant of $A$ is zero.
(⇐) We will prove the contrapositive (Proof Technique CP). So assume $A$ is nonsingular, then by Theorem NMRRI, $B$ is the identity matrix and Theorem DIM tells us that $\detname{B}=1\neq 0$. With the argument above, we conclude that the determinant of $A$ is nonzero as well.