I look at divers the same way I look at boaters who drift rather than anchor up---a momentary minor annoyance. Most divers aren't going to stay in one place once they enter the water. Gives me a chance to check my leader for abrasion and re-tie if need be. By the time I'm done, they're usually gone. I also don't have a problem holding up a cast for 30 seconds or so if some guy drifting an inlet in a boat happens to be in range temporarily.
Jet skis, and boaters who drop the hook 100 feet in front of you, well, that's a different story
When you fish a rocky area, you often choose your perch based on ease of access to the water to land a big fish. Divers also look for the same types of places. A little common courtesy would be nice ("excuse me, do you mind if we take a couple of minutes to go in here?") but common courtesy is an uncommon virtue these days---I've learned not to expect it.
Of course, night fishing usually eliminates 99% of these hassles
Threaded Mode