I've seen cups used before, but generally just for species that stay on the sandbed. They do keep the Nem from travelling in many cases, but normally the cup is buried under the sand. Basically a buried "flower pot."
For species that tend to travel looking for the best spot (ie BTA's, etc) a cup won't keep them at all unless the lip is above the water line.
I've used coffee cups for anemones getting treatment in QT but I don't think I would ever try any kind of cup in a display, especially something glass that mounts with a suction cup.