So it's just between the Poly and GF now, right? I'd go with Poly. But I do have to say that GF is the only deluxe we haven't stayed at ~ but we have eaten there several times so I sort of have a feel for it. The themes are totally different, so I'd base your decision on that. Poly is laid-back and tropical. GF is prestige and upscale. Both great resorts, just not sure which theme suits your family best. I think Poly has the better transportation with the TTC right there so you get direct monorail service to both MK and Epcot. You've already stayed at Contemporary, so no need to sell you on how great the monorail is....
And you've also stayed at the Swan, so no need to sell you on how great the Epcot resort area is....if any Epcot resort creeps back into your choices, let us know and we'll give lots of advice on those! That is my favorite area to stay. We love walking to the Epcot countries for most of our meals.