Most of the surrounding water front is public, you can anchor off beaches and go ashore in many places. There's a very good bus system we took advantage of. It's cheap and comfortable with good routes to get to see the island most anytime.
We took a trip to Siasconset for a few hours and had a nice walk through the old village and on the public beaches. Many parks to enjoy on the island.
I was very impressed with how much of Nantucket is still quite wild as the bus took us through the middle of the island.