The Polynesian was the first on-site resort that I stayed at so perhaps that first stay has permanently altered my judgement, but the only thing that I don't like about the Polynesian is that it costs as much as it does
There are some folks that say there is a 'Poly smell', and there was one time that the first room they took us too had a smell that annoyed my wife, but the bellman called back to the front desk and we were taken to a different room/floor without any delay.
If you were in one of the longhouses near the Luau site you might not like the noise from the Luaus - but who knows maybe you would like the music/etc...
As someone has mentioned the Poly is spread out compared to the WL so you need to be 'careful' about your selection of longhouse. I personally like the Tokelau Building because of it's location, but we've stayed in Tahiti and Rarotonga and they were fine really. I didn't like the parking lot view we had from the Rapa Nui building

- but I have to admit it was handy to be able to drive the car to the door when we were packing up (folks we were traveling 'with' arrived a few hours before we did and didn't know any better when they were assigned the room).
The WL was perfect at Christmas, I thought the AKL was like an exotic safari, and I would really like to try the Beach Club so that I could just walk into EPCOT (also Storm-a-Along Bay

, but the Polynesian is always the first place that comes to mind when planning the 'next' trip!