Yeah, they keep vigilance in the anchorage but I've never known it to be as bad as some rumors. I never found it to be bad at all unless you wanted to anchor and leave the boat for an extended time.
When we anchored and wanted to leave the boat for a few hours I'd stick a note in the cockpit where it could be seen, usually leaving our phone number and a return time. We were never bothered by the harbor patrol. I considered them to be working on my behalf. It's a pretty small anchorage.
The policy in the anchorage is that you can leave the boat for general duties,,,, grocery shopping, taking a walk, having dinner ashore, visiting friends, etc. but not to take the ferry to Avalon or spending the night at a local friends house.
Your singlehanded friend's experience is out of the ordinary for the anchorage. If I pulled in as a single hander and couldn't leave the boat it would suck big time. I have no problems with the anchorage rules. It's an anchorage of refuge for a couple of days, not an anchorage you hang out in for weeks.
Then again, Steve on Brendon has a different take on harbor enforcement. Still loves the anchorage but can tell you stories