I originally responded agreeing with your choice of moderates, since we like many of the things the POR has in common with WL, but for $500 extra our choice would be (and has been for our three trips) to stick with WL.
In particular the things that keep us going there and make it worth the extra to us are the fact it is all under one roof and not as spread out as many other resorts, which is great in the rain. Also as others have mentioned the lobby is so impressive that it is an awesome place to come back to each day. The theming is nice but not overdone, and we really enjoy the location on Bay lake. Not only is the view great, but the boats to MK are nice, and it is a nice 15 min walk or a short boat ride to get to Fort Wilderness, which we enjoy visiting.
Either way you will likely be relying on busses to get to the parks besides MK, although we often do take the boat to CR and get on the monorail there to go to Epcot, but the bus is quicker. We love the quick service (Roaring Fork) at WL, but it is a lot smaller than the food court at POR and can be crowded during busy times.
I'm sure you have seen it, but a great source of WL info is the big WL FAQ thread
here. There is a nice map and a lot of pictures to push you over the edge.
Either way, enjoy your trip and be sure to visit both resorts.