I always laugh when guys talk about wearing jeans under their waders. I can't imagine doing that. For me it's a bathing suit or mesh shorts in the summer and breathable long johns in the colder months.
Sweats or shorts - keep in mind with Shorts you have the problem of the seam where the waders join the boot (or stocking foot) rubbing and irritating your skin. So where high calf socks.
No matter what, don't go Commando when going commando