Grew up in Bay Shore and know the area well. (Although it is ALOT more developed nowadays) You have a couple options. The closest is around Fire Island inlet area. As the others have mentined, take Robert moses bridge to the last parking lot and walk to the inlet. You can also fish cedar, gilgo or the sore thumb area ...and if it doesn't pay off head over to Oak Beach Inn for a couple drinks
The bridge is very close to babylon. If you want to drive a bit, you can always fish shinnicock or Montauk.