I would never take a family vacation with only part of the family-so if DH and I go both kids go, not leave one home and take the other, mine are so close in age I can't imagine why that would even be an issue..
I will probably do a mommy and me trip with each of my kids, a long weekend trip, but that's different.
DH goes away alone, I go away alone, a long weekend with friends each, each year.
But we don't go away together without the kids.
The biggest reason being I have no one to watch the kids for us.
Both their grandmas passed away.
My FIL is 80 and not in great health, he could never watch them longer than a few hours, I don't talk to my Dad really, and even if I did he's not a "kid" person, so he wouldn't take them.
My brothers and their wives, and my BIL and his wife all work so they can't take the kids.
It's an impossibility so it's never been something that we've even thought about.
My SIL and brother have taken the kids for a weekend Fri-Sun, but it's for things like we have a wedding, so we are without the kids, but we are home.
Now my SIL works weekends as does my brother(police) so they can no longer do that for us.
My other SIL helps in a pinch, she slept here so we could go to DH's HS reunion(local) and sleep at the hotel.
It's not that I feel we have to be attached to our kids all the time, we go on dates, we ditch the kids whenever we can for the evening,

, I'm thrilled to have the few weekends we've had alone with just DH, I'd go away for a few days with him in a heartbeat if it were possible, but a full out week plus vacation to us, at this point in time, means all 4 of us.
We were together 10 years before we had children, we traveled a lot, and before we know it the kids will be grown and not traveling with us anyway, so we are enjoying all the family vacations while we can.
I know who we are as a couple, we've been married 15 years, happily.
We certainly don't lose that just because we aren't alone together somewhere other than home a few days a year, and to be honest I sort of resent when it's implied that because a couple doesn't go on vacation without their kids that they're losing their "coupleness", especially because for some of us, it's just not an option because we' donlt have anyone to take the children.