Yup, this is just the way it is with the legacy airlines. You pays your money and you takes your chance! I pride myself on getting the best fare I can and we usually fly with United to MCO but that doesn't always happen.
For our fall trip I watched fares like a hawk and decided what my "trigger" price would be. The fares stayed high for weeks and I feared that was what I was going to have to pay. We are very constrained on our travel dates so we couldn't fly on another day. So one day the fare dipped down a hundred dollars per ticket and I bought them. Sure as anything, the fares came down another hundred a few days after that but it was too late. Then one day, I was checking our reservation for changes and there at the bottom was an offer to upgrade to first class for only a couple hundred more. We often upgrade to Economy + seats because we have a long day's flying (8 hours travel time) to get to Disney, so I did the math and sure enough, we could get the first class upgrade for about the same cost as the E+ seats would cost and get priority boarding, meals, and free checked luggage in the bargain. So at the end of the day we got burned a bit with the higher fares but got first class tickets for about 500 dollars less than a first class regular fare.