We've stayed CL at Poly, BC, BWI, WL, AKL, GF and CR. So the only one we're missing is YC (although we've stayed in a regular room there.) Our kids are now school-aged through older teens, but we've been taking them to Disney since they were tiny so we have lots of little-kids-at-Disney experience!
While not my favorite CL overall, with your travel party I would recommend Poly. Lots of reasons: the vibe in the lounge is super laid back, the kids' TV area is wonderful for little ones, the proximity to both pools from the CL rooms and lounge cannot be beat, the monorail is super convenient with strollers, the proximity to MK is wonderful with little ones who may want to spend more time in MK than other parks and may need an afternoon break, the views from the room balconies on either side of the CL building (Hawaii) are peaceful (necessary for a parent wanting a few moments out on the balcony!) Also, I loved the ability to watch the fireworks from the lounge and then get the kids right into bed afterwards. I previously ranked Poly CL food among the lowest on property in terms of quality and variety, but I've read recent reports that suggest they have significantly improved!