Not that I would say definitely don't go but.....I think you'd all have a lot more fun if you postponed the trip even just 3-4 months! Maybe go in May? I took my DD to DL at a little over 5 months and it was a great trip! She was able to easily ride all the rides that didn't have height restrictions. She loved looking around on rides like small world and she was so easy to tote around in a sling. She slept in the sling and we just kept riding rides. She could sit in a highchair while we ate our meals and pick at some cheerios! Also, nursing was much more stabilized at that point and quicker. I just remember the 45 minute long nursing sessions at 6 weeks followed by another feeding less than 2 hours after that one ended. If you are nursing, you will spend a lot more time in baby care centers or sitting and nursing while your older child enjoys the rides with dad. For an older baby, you can nurse on the rides, but when they're that young they wouldn't be done during the length of a ride most likely. And, like someone said, if you have middle of the night crying fits, that means none of you are getting any sleep.
If you are close and you go often, I would say all of this is no big deal. But, if you are flying and this is a once every 4 or 5 years kind of trip, I'd wait. I would also be concerned with germs on the plane but not so much at WDW because the baby isn't touching things and you can keep them pretty close to you and covered when in the stroller. You just have to be careful washing your hands.
Whatever you decide, I hope you have a great time!
Alison