How involved is your DF in the hands-on childcare? That's where my DH sometimes doesn't get why I want to keep going back to Disney... I'm a SAHM and the primary caregiver, and that pattern doesn't change when we travel. When we go other places, "divide and conquer" generally ends up being him enjoying the destination with our older kids while I keep the baby from getting bored. I know we'll outgrow the annual trips to Disney at some point but with an almost-3yo there just aren't a lot of other destinations that are as much fun for all of us.
For example, we took a few days up north last summer. DH & the older kids did a dune buggy ride, went hiking in the woods, went golfing, and had a blast. I mostly sat on the beach watching our 2yo splash in about 3" of water because that's as far as she'd get into a very cold Lake Michigan. Yes, she enjoyed it - a kid that age is happy anywhere there's a beach or a playground or a patch of grass to run around - but it wasn't much of a vacation for me. That's fine for a cheap weekend away and the older kids love the one-on-one time with DH who sometimes works crazy hours, but for our big vacations I want places where we can ALL participate and have fun. That simply isn't an option at a lot of the destinations that DH suggests. Even "family friendly" resorts like the ones a previous poster quoted the snippets of aren't really "toddler friendly" - a ski resort, dude ranch, the Bahamas, Hawaii... All pricey trips where a 3yo can't enjoy the vast majority of what is offered, so unless you're willing to find a sitter at your destination one parent ends up sitting out.
Fortunately my DH does enjoy Disney, so our compromise is to stick to those trips for another couple years (Universal, Sea World, and
Disney Cruise Line will work their way in there too), then move on to seeing more of the world when it doesn't involve strollers, diapers, and a child too young to partake in so many activities/attractions. We still take smaller non-Disney trips around our area, but to places that we've been before and/or will go back to again, where if I spend the day walking the baby around the Mall while the rest of the family enjoys the Smithsonian or sit on the beach while they snorkel I'm not missing out on something new, exciting, or once-in-a-lifetime.
Have you tried talking to your DF about it from that perspective? If he's anything like my DH, he's hearing "annual pass" and you planning two trips back-to-back and thinking "OMG, we're going to be going to Disney and only Disney forever!". Once DH & I sat down and had a heart-to-heart about the reasons I'm all about Disney for our big trips at this stage of the game he understood a lot better, and knowing that I do intend for us to go elsewhere when the timing is better made a big difference in how he sees our frequent Disney trips right now.