I do not believe that your husband's co-workers have ever been to WDW with the pre-school set. If they had, they would realize there is nothing like it in the world. I think my favorite trip was DD's first trip when she was 2 years 10 months old. Everything was absolutely real and magical to her, and she spent an entire week with a HUGE grin on her face. The interactions with the characters when they are 5 and under is adorable- they believe in the characters and the characters make a huge fuss over them in some instances. They can go on most of the rides as most don't have height requirements. Even the ones with height requirements, by age 4 they are usually 40 inches and can ride most of them-- if they are little thrill seekers. Some kids you can't get on the thrill rides even if they are 10 years old, so waiting is just silly in my opinion.
They start realizing that characters and things are not totally "real" at a young age these days. In kindergarten other kids were already telling DD that the characters weren't real. An 8 to 10 year old, certainly doesn't totally buy into the magic, although WDW is still magical even to older kids. DD is 7 now, and I wouldn't give up those 5 and under trips for anything. It is a little bit of a different trip now, she still loves the characters, but there is something just totally special about the way the characters interact with the pre-school set.
If you were only going on 1 trip in their lifetime, MAYBE I would say 8-10 (but that is a toss-up because I still wouldn't want to give up the younger trip magic), but if you are going more than once, I would totally say you MUST have a 5 or younger trip!
For those who say the kids won't remember, YOU will remember. WDW is fun for the grown ups too. And YOU will remember how happy they are. That is like saying don't bother having Santa come because a 2, 3 or 4 year old won't remember Christmas anyway. DD (7) can still give us specific details about her 2 year old trip, even though she is 7 now, but even if she forgets that trip, I won't forget. Seriously, who says, "Don't bother doing anything that they enjoy or makes them happy at age 2 or 3 because they won't remember it anyway?"