OK - will admit I could bring myself to read all the posts!!!
I say go for it. OP seems like a pretty laid back mom. she knows her older children and knows they will be happy swimming and being at the hotel for the day. She is not planning on handling all 3 kids at the parks by herself!
My only deal on this is feeding the baby. I never bottle fed, but that seems like it might be a little difficult to keep enough bottles with you, to be able to warm things up in the room, etc. If you are breastfeeding - no problemo! When I was there with infants (4 and 6mo), the CM's told me I could breastfeed anywhere! In shows, during parades, anywhere I wanted. I used the baby stations a couple of times, but did not feel confined to those places - which were VERY nice with wonderful people, nice rockers, special rooms for breastfeeding moms. I think especially if you use a sling or snugli.....the baby will be fine.
Ok - have to make a comment regarding the husband who gets sick everytime he goes to WDW because he touches the rails at the rides and then his face!!!!! How in the world would he know that is how he got sick? Didn't other family members touch the guard rails? Some people tend to get sick on vacation - the relaxing, the different schedule, not eating right, being tired, stress, letdown of stress....lots of reasons- all different people!
BTW - babies, esp if breastfed, have better immunities when young than they do with older!!!