I haven't been on the 4th, but I've been there over New Year's. Dec 31 is the single busiest day on the WDW calendar. Expect things to get better day-by-day with a big drop off on Jan 2nd as people travel back home that day so the kids can start school on Monday.
The decorations will start to come down shortly after New Year's.