I entered in a soapbox derby once when I was a kid

It can only help the fish by releasing the larger fish. If you keep removing all the large fish, then the fish will onl;y grow to just enough of a size to keep reproducing and the 100 pound capability is severely diminished since the genes have not been passed on from other large fish capable to grow over 100 pounds.
I have let many large go free, I have also kept my share of large, but only about 3-4 fish over 30 pounds. I try to revive them now. I hope it helps.
Some people will keep on killing for food or tournaments and I have no problem with that. I worry about myself, and whoever is fishing with me makes their own decision too.