N20 motors are good but as you said, quite loud. I would try with a dedicated dc motor driver with your existing motor first.
Another thing to keep in mind is that of you spin the stirrer too fast, the centrifugal force coupled with the fluid viscosity literally flings the stirrer bar towards...