It would need to be something substantial to cause any real damage. People have been wearing Tilley style hats with chin straps on boats for many decades, I've never heard of anyone loosing their head over it. I wear my Tilley when I'm cruising but I agree with roverhi that they can get in the way at times and I wear a ball cap when racing.