Personally, I will never again book off site, unless I plan on doing other things besides Disney. Since you asked for opinions, I would suggest that if you can switch without a penalty, to do that, if you are there primarily for Disney. I Drive is a good 20 minutes away, (if this is your first time there you might add in another 20 minutes lost time to get used to it).
You sound like I was on our trip in 2000. I was so undecided, trying to save money. I got 3 nights through Priceline at a really nice hotel - the Marriott Orlando World Conference Center. The MOWCC is rated by the Unofficial Guide as 92 quality points (or is it 94). And, they strongly suggest that you not stay at the All Stars (size of room, noise, etc). They give the All Stars I believe a 76. Lots of difference.
Reality turned out to be that there was lots of difference. We hated the MOWCC. They had laundry facilities on site. That broke down, with our clothes in them. We called 3 times for maintenance so we could to Disney. They finally showed at 6 (our first call was at 3), and they did take care of our clothes for us, but the Candlelight Processional dinner reservation we had was at 5:45. The place was completely secluded. And, they didn't tell us about the extra fees (almost doubled what we were paying). After 2 days, I switched to the All Star Sports. Should have went there from the start. There was lots of difference - for us the All Stars were so much better.
If it's going to cost $450 to switch (we only lost $60), I'd say stay, but if there is not charge, I'd say switch. Just my 2 cents.