I think its important to have a few days to yourself with your husband or wife..
we went away when my son was two for an 8 day tour of New England.. and I have to say I hated being away without my son!!
However, in the eight days, my son didnt mention me or my husband once!! I was so upset!!!
its also important for grandparents to have that special time with their grand kids.
Sadly my mother died the night before my sons 4th Birthday. He has very special memories of his grandmother and they were very close.
Make the most of the grandparents.. sometimes they get taken from us suddenly, with little warning. My son no longer has a grandmother as DH's mother died before our son was born- and that makes me sad as my grandmothers lived until their late 80's - Neither of our mothers made it past 62...
Anyway.. sorry, I digress... the point is that we now have no one we can ask to have our son overnight...
What i would do for a romantic weekend away - just the two of us!
So no, you of course are not bad parents! You MUST go away alone together!
Although. (and this is really a separate issue!). I do feel that your kids will get something amazing out of WdW no matter what the age (they need to be sitting up in the push chair at least!)
My son went aged just 1 to WdW for the first time. He looks almost daily at the photographs from that trip.. and treasures those memories so much! He is now 5 and has spent over 24 weeks at WdW - and has literally hundreds of amazing memories, video and photographs from that time. NEVER has he missed out on doing a ride -
I think taking younger children gives you a chance to smell the roses a bit more.. and I certainly would feel cheated not to have taken my son at an early age.. we all have so many magical memories.