It's always ready to pull alongside and jump in, put on a pfd(always secured inside)and start rowing(and calling 911...). The little time we spend even out of cell phone range is minimal so I long ago settled on a basic set of coastal cruising safety equipment. It's easy to meet and keep up.
A VHF is still a good safety device along the east coast. I'm comfortable with how I define our limits but they don't work for everyone.
Fire is my number one sinking concern on and away from the coast. It's amazing how fast an electrical fire can take off(I've seen one onboard, yikes!). Sunk or not, a fire is soon out of control, and you've gotta go fast.
Next is sinking due to a plumbing failure. I keep close track of the potential problem areas. Collisions with other boats is a concern but whales are not something I worry about hitting, and we sail near them occasionally.
Those two things, fire and plumbing failure induced sinking away from the coast, plus the rocks closer to shore I figure are what will (have...) sink our boat if I'm not careful. Truth is I'm sure, most boats sink at docks or moorings than have problems on the water. I feel safer on the water than I do on land.