Depends on what you want from the resort and if that extra money is going to hurt. I agree with above poster about Swan/Dolphin. I stay there just about every trip since I can usually get a room for about or close to the price of a moderate. Back to that in a sec.
If you are wanting the location BC is a fine option but would not be my first pick of the 4 resorts in that area (I'm counting Swan/Dolphin). Like you, I never use the main pool but do enjoy a dip in the quiet pool every now and again so taking that out of the equation, BC is fine but has a few drawbacks for me. #1 there are very few rooms (if any other than on the DVC side) that have full balconies. There are some rooms that have the practically useless standing balcony. If I'm paying what you pay for that resort, I want to sit on my balcony in the morning with my coffee and enjoy the atmosphere. #2 (this would be a plus for some) it tends to be a tad more kid friendly than the other Epcot resorts. I prefer the more adult sedate feel of YC, BWI or S/D. But, you can't beat a quick walk to Epcot and either a boat, sky liner if you must, or walk to HS.
Given the three choices you mentioned, I'd probably do POR. I would request a room on the mansion side though, the rustic side is just a tad too rustic for me. I've stayed at all three you have mentioned and all have their up sides. I'm not a fan of Pop because I seem to be one of the few people who actually doesn't like the sky liner. Now days, if I stay at a moderate it's usually CSR.
Now S/D. Cons - it's not Disney owned so you aren't going to get hit in the face with Disney if that is what you want. You will not be able to use your MB to get into your room, you will have an old fashioned key card. You will not be able to charge back to the room from the parks or anywhere else on site other than S/D. You will have to pay for parking if you have a car. The buses do not drop you off at the same place WDW buses do. At AK, I actually think they drop you off a tad closer but at MK they drop you off at TTC, which is fine for me but some people don't like it. Lots just walk over to YC/BC or BWI and catch a bus there.
Pros - walking distance and boat distance to both Epcot and HS. Lots of food options at both resorts. They share amenities so you can use either the Dolphin's big pool or the smaller lap pool at Swan. Deluxe perks like extra magic hours at the parks. If you want to pay a tad more, you can get a room with a smallish balcony that does at least hold a chair and a little table. The price is great if you can find a discount, I use the senior discount more than the AP discount.