As noted above, you can get most of the popular, crowded rides done by noon. We are kids were smaller, we always left about noon for an afternoon nap then came back.
However, we they go older, we made PS at one of teh three hotels on the monorail route (about 1pm) and used that as a break to relax.
When we get back, about 3ish, it's still crowded. But that's when you see COP, TK, Pirates, Hall of Presidents. You can also time to get back right before the afternoon parade and hit some rides while that is going on.
I just don't buy the claim that EE days are much more crowded in the afternoon than non-EE. They're all crowded in the afternoon. You may just notice it more because you got up so early and are already feeling the drain.