In salt water, I don't see an issue. These fish WILL die in a few hours (Hopefully sooner
)
In fresh water I might be able to see how they MIGHT cause a problem. That's MIGHT, MAYBE, etc...
But for the salt, IF they take pressure off of native bait I say
