I've used castors and lockable castors for years and to be honest, I would have absolutely everything on castors. It makes for a far more versatile workshop - especially if you're pushed for space.
If you buy really good quality lockable castors (that lock all movement), then you won't get movement. I have tools mounted on units with four lockable castors and they are solid and movement free (remember the good quality bit).
Really good quality castors are not cheap, so if you are not pushing and pulling the thing from all angles, then try combining them with two non-lockable swivel castors at the back. Or cheaper still, try fixed castors at the back.
Jonny