The latest I've gone was 22 or so weeks, but I'm thinking of this very situation with our 3rd baby. I'm not yet pregnant (and won't start trying til March), but we have a trip planned for early November and I'm wondering if I should delay TTC another month to be safe.
Like a poster above, my kids were early-1st was just under 38 weeks and 2nd was 36 weeks. So my fear is that going at 32-34 weeks would be really risky. It's a lot of walking, commando or not, so it might be more taxing on the body than you (or I) would think.
FWIW, my SIL is pregnant with her 3rd and both of her kids were term deliveries (41 weeks 1st time, induction at 38 weeks with the 2nd) and her OB told her not to leave the area past 28 weeks. She didn't have any preterm issues with the other 2, but he just doesn't think it's wise to be far from home at that point.
And I will also add that with my 1st I had terrible swelling that started around 18 weeks. We went to WDW in October around 19 or 20 weeks and it was exhausting and uncomfortable. The heat seemed to make me swell more and I spent a lot of time with my arms and legs elevated, a lot of time resting or sleeping. Now, that wasn't a problem for us in terms of touring as we go often as well, but it was somewhat a problem for me as it was sort of annoying to have to take a lot of breaks and not really be comfortable resting on a hard bench.
One last thing, neither of my kids were planned pregnancies, so don't count on it not working out the 1st time around!! Of course, I'm right there with you, worrying about how long it will take to conceive cause this 3rd one we are actually planning, but based on not just my experience but that of friends as well, there's a good chance the first month will be it!