I guess most anchorages I like are flawed in one way or another. That's why they're usually open. I've anchored in there several dozen nights I guess now over many seasons. It's a little narrow through the inner harbor but we've shared it with as many as 3 other anchored cruising boats. This was taken in 2009
What turns most people away is getting into it through the winding narrow entrance.
But the biggest complaint I hear from people that have anchored inside, is dinghy access. Because most have inflatables, the tide doesn't allow much time to get in and out of the landing in the middle of Blue Hill village.
If you have a rowing dinghy, there's only a few hours it will be dry at the landing when you can't get in or out.
That plus the former problems(and Blue Hill Bay being off the beaten path for most cruisers) means I can always rely on finding space up here. When we go up, we like to spend at least a couple days and nights, maybe more. I know of no more snug anchorage.
David Buckman just did a nice article in Points East on Blue Hill and the inner anchorage(it's online now at PE site). He's the type of sailor and boat you'll likely find in there for a few days. He sails in!