I have taken infants to both parks and as long as you are aware that the baby will not ride many rides, I would still take the baby and just swap with your spouse so you can have some fun. We took infants with us and always had a great time as a family. At IOA the baby can ride One fish two fish, and probably Cat in the Hat, and the carousel (if you are in a nonmoving seat). At US I think the baby can ride Shrek (nonmoving seat), and Jimmy Neutron (nonmoving seat). Definately not like the Magic Kingdom where infants can ride 90% of the rides, but Universal is still a good time anyway (at least to us it was!).