Oddly enough, I'd say Islands of Adventure. I certainly saw more characters out and about there, and Seuss Landing will be especially good for the 4 year old. And even if they don't want to go on Spiderman or Jurassic Park, the theming in those areas is so cool they'll probably appreciate it.
There's also a water park area which I personally skipped, so I can't comment on how appropriate it is for their ages.
I think even the mild adult rides (Jaws, Back to the Future, Shrek) at US might be scary for kids who don't like being scared. Jimmy Neutron was fun, but might be a lot, depending on how you define "scary."
The Seussville rides in IOA are totally mellow.
Oops, just remembered there is a kiddie area in US that we skipped. Nevermind! (We did see a good animal act show there -- but then the Sinbad show was fun at IOA, too)
I think either way you can find a way to entertain everybody.