I agree, but it doesn't always turn out that way... There's good reason why the Newport-Bermuda Race is one of the classics, and that passage is considered one of the greats... The heaviest weather I've ever experienced aboard my own boat was sailing to and from Bermuda, and that was in JUNE... (grin)
After arriving at the Bitter End YC last December, I had an interesting chat with the captain of a Swan 60 owned by the owner of the resort... the boat makes the round trip back to New England every year, the skipper was hugely experienced, has probably made that trip more times than Don Street...
After a particularly brutal passage a few years ago when he feared they were gonna lose the rig, that particular Swan 60 (about as well-found an offshore boat as you'll ever find) now makes the trip - at the captain's insistence - aboard a Dockwise ship... (grin)
Actually, I'd prefer a late-season trip over an early spring return, anytime... I think coming back in, say, April, is way riskier than heading down in December - March and April is the time of the year that scares me the most off the east coast...