We've been many times with young kids and had a blast! At 3, it was two of my kids second favorite park of the 7 main Orlando parks. (4 Disney parks, IOA, US, and SW). My youngest has some sensory issues and refused to do rides at our last US/IOA trip two years ago when he was 4. But we still managed to have a lot of fun - and so did he. Just his idea of fun wasn't what we would have planned for him. We had to not worry about making his experience ours. This meant lots of rider swap, but it worked well. (He also refused to do Disney rides on our next Orlando trip, a year later at 5 for the first half of the trip). We've been with a timid and tall 3 year old who would do a lot - though we found by trial that we shouldn't have started with Spiderman. We've been with a bold petite 3 year old as well who had a blast.
Rider swap is very easy and makes it so the other parent doesn't have to miss out. For rides with less interesting lines, like Hulk (which is closed now anyway), we did single rider line and that meant being split up for a short time. But for something like Forbidden Journey, it was worth us all going through the line.
Seeing how I haven't been in two years, I realize there have been some changes. There was plenty to do last time but now I expect it will take even more time for my family. My middle child is upset that Disaster is closed at US, but I'm certain the interactive wands are going to make it take much more time! We managed to do 6 days at US/IOA on our last trip - and that's with the 4 year who refused to do rides. So, yep, most families would do just fine.