We do the same thing -- as you can see by my signature....right when we get home, we start planning for the next trip. We also have 2 young children and I will say that we spring for the more expensive resorts (monorail/Epcot resorts) mainly because of convenience with the kids/stroller. It changes the whole outlook of the trip when we can walk/monorail/boat to a park and know that the kids can fall asleep in the stroller and we don't have to wake them up, unload the stroller, and fold it up. It's nice to just easily come and go without all the hassles. Anyway, as far as a resort choice, we've stayed at your 2 options of Poly and CR and love them both, but for different reasons. Here are a few tidbits about each:
Poly--relaxed hawaiin theme, separate longhouses with the rooms & one main building, Ohana fun family-style dinner, Ohana character breakfast, Kona great food, even fast-food restaurant is very good, fun pool, great "standard" view rooms, rooms large and newly renovated, resort monorail and TTC monorail huge plus.
CR--more contemporary theme, can be conventions there, magical feeling being right next to MK, if you stay in tower you can elevator right down to main levels (so one building), tower room MK view offers fantastic view of Poly, GF, MK and wonderful viewing of Wishes and water parade from from room balcony, walking path to MK, Chef Mickeys character breakfast and dinner great fun.
On the flip side of that, as you are thinking this through, do consider WL as one of your options. Especially making the jump from a value resort to a deluxe resort. WL (and AKL) are the least expensive deluxes, but yet are very gorgeous resorts. WL is considered an MK resort and is not far from the CR. You can take a boat to MK and also watch the water parade from WL.
Happy planning!