We have stayed at both and we're from Long Island too!! We have a family of 5, but as you can see from my tag it would probably be the Polynesian hands down unless you are looking to save a ton of money and you haven't said what the price savings would be. Also to consider: ages of those in your family, what parks you will be spending most of your time at etc. We have 3 teen ds and spent april in the poly and now in nov/dec. we are at the swan, my dh has a conference and I am only spending the rest of the time at the swan because we got a great mousefest rate of $129, a night. So cost was a factor to me this time.
Here are the positivies:
Polynesian: On monorail line, can go to other monorail restraunts etc.
Wishes and the ELP from the beach
Water craft rental etc. right there
Volcano pool
Neverland Club onsite if you need that
Captain Cooks for quick snacks
Swan:
Heavenly Beds, just love them
You get starwood points
First bus stop, close to epcot,mgm, Boardwalk dining/entertainment options
Grotto pool, beach volleyball, basketball etc.
Things minus about the Swan:
No charging back to the room
Not disney themed
Extra charge for roll away for 5th person, $25 per night, we are bringing an aerobed this time.
Hard to do the mug refills, We solved this by stopping at the hess mini mart and buying some soda packs
Snacks are at tubbis at the dolphin as well as the laundry facilities
Good luck deciding.
Read the board for a year, but this is my first post. Got call from CM to last week, who upgraded my 5 days at CR to the Poly for same price. Don't know whether to keep all 5 days at Poly or split stay with Swan for better price. Never been to either resort. Opinions please?
But it would not be my 1st choice. We had a bad experince here in 04' and the front desk was less than helpful. 
Room overall was ok, not what I expected, many suits around when we where there. pool is farther away. Of the 2, I would pick the dolphin, has more resturants, closer to pool, has better shops, and laundry. (Bigger)
