I personally couldn't go without my dd.. but that is MY personal decision.. my dh feels the same, so we at least agree on it! LOL However, I had her young, so when she is "all grown up" I will have lots of years left to have fun solo in WDW just me and dh, until then though, it will always be a party of three!!
However, i would never tell anyone that it is WRONG to go without kids.. it is just a personal decision.
Good luck and have fun with whatever you decide!