I've been injured more often and more severely while rockhopping than while skishing, that's fact. My opinion is that I'm much safer after I gear up and get in the water than I was while driving through Providence, RI traffic to get there.
Everyone brings up two dangers, boats & sharks. Both dangers are real, and can be deadly.
I'll talk about boats first.
Most of the places I skish have very little, if any, boat traffic. Many of the places I skish are inaccessible by boat or shore, so swimming there or kayaking there is really the only option. When swimming at night I am equipped with a VERY bright light for emergency use, and can make myself quite visible if I need to. I've never had a close call. I am quiet, and can hear even the quietest boat coming long before it presents a danger. In the event that I do find myself in a situation with a boat bearing down on me, I can easily dive to 30+ feet in a matter of seconds and have a 4 minute + breath hold. I can get out of the way when I need to.
Sharks... They get a bad rap in my opinion, through sensationalized TV & other media. I do avoid swimming with seals as much as possible once the water warms enough for sharks to be present. For the most part, I don't worry about them. I swam with the shark (I think it was a blue shark) that got Horseneck Beach shut down a few weeks ago. It wasn't interested in me, and would even let me get close enough for a quality photo (underwater visibility was only about 6 feet and hazy). Sharks are keen hunters and even in low viz, murky water, they can tell what it is that they are hunting. I really don't think there are many sharks out there who will choose to tangle with a swimmer who can present an 20' profile in the water and shows no fear. I do carry a large toothbrush with me while skishing, just in case I do get close enough to lend a helping hand to passing sharks,,, they don't get to brush very often. A sign of good will
. They can be quite playful really...