If you can sail a nice beam reach or run up Blue Hill Bay on a bright day, you should do it. It's such a beautiful bay to see and lobster buoys are pretty thin, especially as you head up.

You could come back down the next day, leisurely to Swans for the night. Blue Hill to Frenchboro is an easy day sail as well. Easy to the Merchant island area in a day sail as well with a morning start.