We went during a Star Wars weekend in 2003 (first week in June, if I recall) and again this year (the last weekend in May, right before Memorial Day). Probably due to the fact of the holiday weekend, plus it being Ep. 3 release year, but it was INSANELY crowded this time. But for some reason we were able to walk on Star Tours...
All that being said, we try to plan our summer trips around Star Wars weekends (early summer 2004 I was engaged in little stuff like starting my own business).