No worries---Bubba won't be playing at North Neck. He'll be either at the deluxe course on Beach Road near the little bridge in OB, or maybe at Mink Meadows. The course at North Neck is too low rent, they don't even have carts.
Edgartown has pretty good public access--all of the town beaches are open to the public. You could probably chain your bike to something at the end of Water Street, and from there, access plenty of shoreline between the Lighthouse and Eel Pond. I'm kind of hazy on the place, I haven't been there in about 11 years. If I was there, I could find it, but I can't picture it in my head.
I'm not sure whether the beach alongside the Chappy side of the ferry dock is public or is owned by a beach club down the road a piece. But you shouldn't have any problems fishing there at night.
I think the last ferry from Chappy is at midnight, but they'll probably run it until all the cars/people in line at midnight are off. If you miss it, it's a loooooooooong walk off the beach. Despite what your Senator would have you believe, the current is too fast to make a swim for it
Speaking of which, if you head farther down the Chappy road from North Neck, and go straight down a dirt road at the point where the main paved road hangs a sharp 90 degree turn to the right, in another 3/4 mile you'll come to Dike Bridge. They have bike racks there, and East Beach on the other side of the dunes might have some action with blues, bones and albies---maybe even some Spanish macks.