I went to the very first Celebration of Harry Potter event and arrived in Orlando ahead of time because I came from a fair ways away. I went to the park on the Thursday so I could tour the park as normal before being tied up with all the Potter events over the weekend, and as I recall the park was not super busy. Busy is subjective to everyone I suppose, but there weren't exaggerated wait times for all the rides or anything.
Now that said, I haven't been back for the Celebration since and if its gained in popularity since the first one then I guess it could affect Thursday crowds more than it did when I went.