The spin-flavor wavefunctions of the ground-state baryons are symmetric, and color was invoked to recover the required antisymmetric character. You should notice, however, and some people did, that we can construct a totally antisymmetric proton wavefunction, for example,
$$
|\mathrm{p} \uparrow\rangle=\sqrt{\frac{1}{2}}\left[\mathrm{p}_A \chi\left(M_S\right)-\mathrm{p}_S \chi\left(M_A\right)\right]
$$
and forget about color! Write this function in an explicit form, comparable to (2.71). Obtain $|\mathrm{n} \uparrow\rangle$, and hence show that
$$
\frac{\mu_n}{\mu_p}=-2 .
$$
So this option is ruled out by experiment. In fact, glancing at your derivation, you will notice that $\mu_p$ is negative. It is measured to be positive. Long live color.