It helps, of course, if you have an income that is higher than your living expenses and savings goals like retirement, college fund, car fund, and you already have emergency reserves. If that's the case it's easily justifiable to set aside some money for fun / frivolous things every paycheck like vacation. I put $300 a pay into my vacation fund, and I have a Southwest Visa Card that gives me points for quite a few freebie trips on Southwest each year.
That gives me $7200 and two freebie round tickets to Orlando and two freebie round trip tickets elsewhere each year.
We usually do a one week vacation somewhere in the US or Mexico spending about $5000 to $5500. That's for our family of three. That trip is never Disney. Favorite places are Rocky Mountain National park -- less expensive than the budget by quite a bit, Puerto Vallarta -- right on that budget. We can also do Victoria, British Columbia, Yellowstone, or South Dakota on a budget like that, for example. And that's flying not driving.
After that there is still $2000 - $2500 left over. With that more modest budget DS and I (DH stays home) usually do a little mini trip to Orlando (just a couple of days each year). Our air is always free with Southwest points. If other relatives want to come too, we split a condo and rental car cost and can sometimes do a week offsite with that.