We have always reused screws, and had no problems, but you make me worry slightly

If the collar moves, I do not think it would be by much. I have to push HARD to move the collar with no screws installed.
The leak would be minor.

I guess a hose clamp would make a crude stopper, but with the risk of catching things

Some sailors have to worry about the shaft backing all the way out of the boat, but ours is positively held in the coupling.