Sounds to me like your engine mounts are gone, if its moving around on bumpy roads like you said. Seeing how its directly connected to the motor, if the engine mounts have a lot of play so will the shifter