On this one, I know, but I have stayed at both, and I would go with POR with kids. The food court is better (bigger, more choices, more convenient), the transportation to MK is better, and it has a more "Disney" feel.
The Swan is a beautiful hotel, but it is like many other beautiful hotels that you might stay at for a conference. At POR, we felt like we were in Disney.
Maybe we were there at a bad time, but getting to the Magic Kingdom and back was a monumental pain in the butt...felt like we stopped at every bus stop in greater Orlando and then drove right by the one we had been picked up at!!! (I am exagerrating, but you get the idea).
At POR, most of the folks were on vacation like us, at the Swan, most of the folks were at a conference. It just had a different feel.
One big plus for the Swan is the close proximity to Epcot and MGM, but we had no trouble getting to either of these via the POR bus either. We walked back from Fantasmic (to the Swan) one night, and it was really nice. Also, the boardwalk area is fun to explore.
Either way - have a great time, and think about what is important to you on your vacation.....to each his own.