Going down isn't bad, it's walking back up in waders!
OTOH, I can think of worse things than dragging a 50# up the cliff.
Last year the beach at the bottom was all medium side round stones. Not the easiest to walk on, especially at night!
But lots of people still keep talking about that spot so by default I have to presume there's something to it. Just wish I could get some more info.