I read your original post and just couldn't respond because I felt I didn't have enough information. There are just so many things to consider. Do you and your dh have custody of the children? How is their relationship with the 2 of you? How is your (and your dh's) relationship with their mother? I think all of these things are important.
I really am surprised that you received so many responses without being asked for more information. I have 2 step-kids, and I find being a step-mom infinitely harder than being a mom. I know many of you do not feel I should even refer to them as step-kids, and I respectfully disagree. I am not their mother, and she (their mother) would be furious if I claimed to be. She would also be absolutely livid if she ever heard me refer to either of them as my children. I will also add that I do love my step-kids, but not as I love my children. However, it is not a biological thing. I simply don't spend enough time with them. I don't get to care for them and do the things with them that build that love bond. Like I said though, I do love them, and I want to and try to treat them as I do my bio children, but they don't (and never really have) allowed me to hug them and get close to them. If that makes me a horrible step-mom, then so be it.
All this just to say that I know where you are coming from OP, but I don't know your particular situation. The only thing I do know is that you stated your dh wants them to go with you, and I do think that needs to be honored. Remember though a lot can happen in 2 years, and neither of you really know how this new baby will change things -- for all of you. Regardless of that though, you need to sit down with your dh and talk about your expectations -- and his. Including all the children on big family trips is wonderful. What about the small week-end getaways though? Will they also be included in all of those? In other words, does your dh expect you, and him, and any children you 2 have together to sit home twiddling your thumbs because his oldest children are with their mother? I know that wording is a bit harsh, but I use it because it gets the point across. Will you also not go to the zoo or the circus if the older 2 can't make it? Again, so much of this depends on where the children live and the kind of relationship all of you have, and really without knowing more, I couldn't begin to comment on so many of the other dynamics that play such an important part of this very complicated family model.
Good luck to you, OP. Sorry, I wrote a small novel for you, but really, in the 12 years that I have been a step-mom, I have learned that nothing is ever really so cut and dry.