@Regan117 You're bringing a gun to a knife fight bringing Tower Club into this discussion!
OP - You've been given some good opinions and solid pros/cons above.
@Babsy's rundown is excellent.
I've stayed at Poly quite a bit and CR several times as well - I love both, but in different ways and for different reasons. My kids have been 2-6 in my travels so in the same general zone as your crew.
Do you mind if I ask the length of your trip? For longer stays I prefer Poly, for shorter stays I prefer CR.
One main reason at my point in life is the pool. My kids get a TON of enjoyment out of the Poly splash pad area and the zero entry pools. We are heavy resort people, so this is a big factor in the way we tour and our resort decission making. We spend hours and hours at the Pool/Splash Pad area.
CR's pool has a nice, large pool deck, is rarely very crowded, and has a beautiful setting on the shores of Bay Lake. But a very dinky kids splash area and no zero entry pool. We enjoy it, but just not as much as Poly.
Admittedly you're going during a time where weather could be hit/miss for full pool enjoyment, but we've been 2-3 times that week and tend to have more pool-weather days than not.
My other favorite feature of Poly is walkability to TTC to go to Epcot. We tend to hit Epcot quite a bit when we stay at Poly because it's so easy - even impromptu outings. It's still easy from CR, but just an added step/time coming/going having to transfer monorails.
Lots of pros/cons here, in my opinion!