No doubt you'll get much larger accomodations for a much less price at WH. I've never stayed there, but they do look nice. No debating that.
But that said, considering the ages of your children, and guessing they'll want to spend most of their time in the MK, I don't think you can beat the Poly for convenience and themeing. This is your first trip and it's only 4 nights....you'll want to maximize your park and fun time. The kids can have fun on the beach at the Poly (can't swim in the lake though)....while you sit out in a hammock. The pool there is themed like a volcano---you climb up and slide down through the volcano. At night, you don't even have to be in the MK to enjoy the fireworks. You'll see & hear them right there on the beach. Also, every night around 9:00--9:15 pm, the Electric Water Pagent water parade goes around the lake, right past the Contemporary, Poly, WL, FT WIlderness and GF :
http://allearsnet.com/tp/ewp.htm (it's a small parade, but your kids will love it).
Taking the monorail from the Poly directly into the MK beats having to drive there, park in the Ticket and Transportation Center parking lot, wait for a tram, then take the monorail or Ferry over to the park any day. This is coming from someone who stays both on and offsite. It can take 30-45 min from the second you pass through the MK parking lot entrance until you actually get into the MK park when you stay offsite. If you stay until the park closes, it can take 45 min to an HOUR to get out (been there done that). Guest staying at the MK resorts, jump right on the short line for the monorail back to their resort, while the offsite guests are herded like cattle sometimes, back to the MK parking lot. IF you do decide to go with WH (and nothing wrong with that), just make sure you leave the MK at least one hour before the firework show or it will take you a good amount of time to get out through the crowds.
The Poly rooms are larger than other "deluxe" category rooms such as the Beach Club, Yacht CLub etc. They are about 409 sq feet (I know...no comparison to WH size), but you have to factor in the onsite perks/ambience etc. You factor in being able to enter the parks 1 hour before offsite guests, and that's a couple extra spins on Dumbo, Peter Pan and Pooh before the crowds start coming in. The evening EMH's tend to be more crowded, but the morning ones can be life savers to saving time in line with little ones.
MY tip : IF you stay at the Poly, request a room with a balcony. Not every room at the Poly has one, though most do. But I would make this my #1 request.
Good luck with your decision....it's a tough one....these are two very different options and vacation experiences.