There's a reason quite a few people go without sand, they can become huge magnets for detritus and other crap (literally and figuratively), and unless you have critters going through each layer to keep it clean any benefit of them is quickly lost and then you're left with a big problem.
That said, if you do clean it, you should add that to part of your monthly cleaning regimen. As others have said a gravel vac works good, and can be useful. If anything to start over. Just be prepared to do a good sized water change afterwards, not only from the water you pull out, but also for any stuff in the water column that managed not to get sucked up by the vacuum.