I've stayed at both (Swan & POFQ). I was really expecting to love POFQ after reading so much about it on these boards, but I could honestly take it or leave it. It was fine, for the most part clean, etc., but as far as moderates go, we prefer CBR. I think even POR looked nicer - we enjoyed Boatwright's, and they had a much better gift shop than POFQ!
It's true that the Swan is more "convention-y" and doesn't have that 24/7 Disney feeling, but the beds are nice and the rooms are large. It was definitely a better quality room and overall service than POFQ.
I have stayed (and will be staying again) on SPG points when at the Swan - it's defintely not my first "deluxe" choice on property were I to be paying for the room. But if I were purely deciding between the two, I'd probably take the Swan if the price wasn't much of a difference (or try another moderate).