Have stayed at Poly twice and AKL twice. We've had nothing but good experiences at either resort. I know there are people though, who have had bad experiences at both. The above could be said for every resort on property.
Plusses at the Poly: Monorail, monorail, monorail. Longer pool slide (though the AKL pool slide is probably more a 2 yr old's cup of tea). Ability to view Wishes from the pool and beach area. Monorail again: Easy access to dining at GF, Contemporary, MK, Epcot. Larger rooms.
Poly negatives: Cost. You have to go outside to get to your room (can be a challenge during those periodic Florida monsoons).
AKL Plusses: Theming. Newer resort. Food court is better than Poly's IMO. No need to go outside to get from your room to other areas of the resort.
AKL Minuses (compared to Poly): Smaller rooms. No monorail.
We love them both, but opt for AKL and WL these days because the cost is much better and we're still dazzled by our resort. With a 2 yr old, though, I would go with Poly if it were me for the convenience of the monorail.