Who will get more joy and satisfaction out of the boat: a one arm man a thousand miles offshore in a hurricane or you and your wife in a quite, snug anchorage? If you believe you will use the boat and enjoy it then you're good to go.

Having said that all boats are a compromise and a boat completely suited for heavy weather offshore sailing may not have the same accommodations suited to coastal cruising as another boat. I'm thinking of things like the size of the cockpit, dinghy davits, size of the berths, open space below etc.