One of my co-workers said to me recently, "You travel more than anybody I know." (DH and I usually go away for one long weekend per month--3 nights, then a one-week vacation once or twice a year).
This is how we do it: for one thing, we live in Florida, and we hardly ever leave (I always say, we live in Paradise, why would we leave on vacation??

)
When we go to WDW (our cool weather destination), we stay offsite at a relatively inexpensive hotel (although we do plan to try the Swan/Dolphin this fall or winter, when we can get the $149 nurses' rate!). We generally only do 1 or 2 Disney sitdown meals per visit--the rest of the time we eat offsite or counter service. We will probably try the Value resorts agian sometime, too.
When we go to St Augustine (our warm weather destination), we stay at an inexpensive but cute waterfront motel, not a pricey bed-and-breakfast. We eat out for every meal but often that means a great diner breakfast or the best pizza-by-the-slice in St Aug. We do love to eat, though, and will splurge on dinners at restaurants with great waterviews.
We save a lot of money by driving everywhere and not having to spend $$$ on a flight or rental car.
Also, as somebody mentioned, it's a matter of priorities. My co-worker's top priority is being debt-free--making double payments on his mortgage. While I think that is a worthwhile goal, I don't mind stretching out my mortgage to the full 15 years or our student loan payments to 15 years. All our debts will be fully paid off by the time we retire when we're 63 (16 years from now). We're also saving an adequate amount of money every month for retirement.
Meanwhile we are having the times of our lives instead of waiting to have fun "someday, when our debts are paid off." Again, I'm not criticizing people who would rather make double mortgage payments than go on vacation--I'm just saying that DH and I have different priorities.