Smaller crushed coral, maybe you could use a siphon, but the larger crushed coral, I have never successfully cleaned it without have to stir up the crushed coral.
Not cleaning your crushed coral is like telling someone to not clean your sand. So, if you’re in that camp, sure, don’t touch it.