I was heading down that way a couple years ago and got some good replies here, including specific comments on OC and Chincoteague. See the link below as well as other posts in the thread.

I've come into Chesapeake Bay at night, lots of lights and traffic as SG says but it is doable. Upon entering the channel you will get called on VHF and asked to identify yourself, and the harbor patrol folks will give you a heads up for any ship traffic that you'll need to pay attention to. We always stop at Little Creek; twice I've arrived there at night and instead of going right in, have anchored just outside the entrance to Little Creek to wait to go in in daylight. You can't anchor outside in a blow, but in reasonably settled conditions it was fine and easier than going in and trying to find a place to tie up at night.

http://cruising.sailboatowners.com/csbb/index.cgi/md/read/id/173892/sbj/i-m-with-david-cape-may-would-be-my-first-choice/