move on. Life it too damn short to spend working a job all the time. Scale back your excesses and work less, fish more. It can be done. Look at it this way. You just missed the best inshore tuna bite in 2 decades...because you were working. We all have to work but we don't have to be slaves to the job. (also, don't wait to retire to do these things...chances are you will be dead (from working too hard and worrying) or be in poor health and unable to do the things you want to do)
You need to ask yourself (and spouse) one question
.... Is this what we want? If so, keep at it. If not, change your lifestyle. Do it. You will enjoy life a hell of a lot more.