I agree it is totally up to you because it is your wedding. That being said, as a mom, if I was spending a good bit of money to go to WDW (no matter what the reason) I would definitely bring my child along with me. I guess I would look at how many guests it would affect. For example, if it's half of your guests that may not be a feasible thing to ask.
If you provide those with children good options, then I think it could completely work out. I think I read something where you could have a separate kid's reception where they are entertained and fed. Don't know if something like this is in your budget, but it might be worth looking into. Also, providing parents with baby-sitting options and (depending on budget) offering to help cover some of the costs would be a reasonable idea. As a mon, I would be OK with either of those choices.
It's your day and ultimately whatever you are comfortable with is what you should do.
Lisa