We were there on Wed, the 11th, and I was surprised by the large crowds.

Very unusual for a weekday. I heard from several visitors that AZ schools were on fall break that week so that may explain it. Also, Disney is heavy promoting the Halloweentime at DL. It seems like all the AP are going to check it out.
If your only choice is to go Sunday, or not at all, I would go Sunday. If you can go another day, that would be my pick. If you go Sunday, get there at opening (and NO, they don't let you in early-you'll have to wait in the line to get in) Still, you'll be able to ride alot of rides first thing, before the 11 a.m crowds arrive. Added-Ooops, just re-read your OP and saw your getting there at noon.
Personally, when I go on a weekend, I arrive at opening and stay til it gets crowded. But then that's the benefit of having an AP. I plan on going one day this week as my DD7 is on min. day all week long. That's another thing to consider for anyone thinking of going next week. The Corona-Norco SD elementary schools are all on min. day for the entire week due to Parent/Teacher conferences. That may affect the crowd levels as many of us here in Corona have AP.
Have a great time!