Go!
Universal has Shrek, Jimmy Neutron, Twister, Beetlejuice(sp), Earth Quake, MiB and ET that should be accessible to all of your children. For the younger ones there is a great play area, a Barney show and Animal Planet - for the more adventurous there is Jaws (if it's running), Back to the Future, The Mummy and my personal favourite, Terminator.
As for Islands of Adventure, personally I think this is a park for older children. Sure there are rides for smaller kids in the Dr Suess area but this park has bigger, faster rides that your 5 & 3 year olds wouldn't be able to go on. I'm not saying 'don't visit IOA' just check out what they have to offer before going.
As Alison says - take a change of clothes as there are opportunites in both parks for your kids to get very wet!