We never went into Cape Poge Bay, but I had the same thoughts as you. The charts show a rock somewhere in the narrow part of the entrance channel.

Was in Katama 2 years ago and reversing currents ran upward of 5 knots. We didn't go all the way south into the Katama Bay because I read anchoring was prohibited "to protect shellfish beds" but we took a town mooring at the north end in Edgartown for a reasonable fee.