We took our first trip with kids when the oldest was barely 3 years old and 40" and the second child was 2 months old. We always find more than enough at IOA. We have run into US not being a full day, but then we walk back over to IOA. Our trip before the last one had kids ages 11 months, 3.5, and 6.5 - and we had 2 day hoppers and did 1.5 IOA and 0.5 US. WWoHP was not yet open then. We just returned a few days ago and had a two park one day ticket, so we stuck to IOA. My oldest, who is 8, is 54" but didn't want to go on any of the higher height limit rides except for FJ, which is 48". My middle child is 6 and 44" and found plenty to do. My youngest is 38" and 2.5 and he also found enough to do.
Initially, our last trip was supposed to be a Disney-only trip for the first time. We've always done both parks in our trips and figured it was time to focus on just one. The plan was to wait until DD was 48" and YDS was 42" (Since Men In Black was a favorite of our oldest at 3.5 years old when he hit 42") But we got free one day tickets so we couldn't handle staying away.
If I knew I wouldn't be back to IOA/US for several years and if I had a HP fan, then I'd wait it out to 48" with shoes because FJ is an amazing ride. (If my hazy memory is right, then Mummy is also 48" and ODS did do that one at 6). However, if you're going back sometime, then it's a huge issue. Also, keep in mind that just because a child is tall enough, doesn't mean they'll want to do those rides. My oldest is tall for his age, but we usually can only get him on thrill rides if his little sister does them! (FJ was the exception due to it being HP inspired. But there was still no way we were going to get him on Dragon Challenge despite him making the height).
With both Disney Universal, I figure we'll always be back in three years or less. So, it's okay if my kids aren't quite tall enough for everything yet - there's still enough to do. Plus I've learned that being tall enough doesn't mean they'll want to do everything anyway with my first.