IMO it's usually because a single fish is holding on that one piece of structure you made your cast to. I've had situations where I had that hit on the first cast, missed it because my brain wasn't fully engaged yet, made the next cast and had the same fish hit in the exact same spot--and usually, that second time, I'm ready for it
Then, it's a question of whether it was a single fish lurking there, and whether another fish will eventually take up the spot.