For me, the deluxe resort experience is worth it mainly for the location of the resorts, and the Poly, IMO, has the best location of any of them.
That being said, even at the Poly, you have to take busses to HS and AK.
For that reason, we love doing split stays with at least one portion at a MK resort and at least another portion at an EP resort (we don't mind changing resorts as most of our vactions involve staying at multiple places).
A lot of this has to do with our touring style:
1. We use magical express and don't rent a car (for us, not having to deal with the car rental hassle, driving, and parking is a big perk of going to Disney)
2. We have young kids who wake up early, need a nap in the middle of the day, and go to bed early. So our touring style is to hit the parks from RD to around 1 pm, then go back to the resort for naps/swimming. Given their convenient locations, deluxe resorts are just soooo nice for this touring style.
3. The best restaurants on property are the table service restaurants in the deluxe resorts and we love having dinners at those. Again, staying at a deluxe resort makes this soooo convenient and by doing a MK resort/EP resort split, we get to experience lots of different places.
4. In the event we decide to stay late for Wishes or Illuminations, it's soooo much faster and easier to get back to our room afterwards, and with young kids, this is a big deal (another reason for the MK resort/EP resort split).
We stayed moderate once (at CSR) and while it was a lovely resort (and, as far as the resort itself goes, more bang for the buck than any deluxe IMO), we just spent too much time on busses (and the resort had good bus service - our touring style just requires so much back and forth). We'd probably consider staying there again (or POR) once our kids are older and our touring style changes but, for now, we're a deluxe resort split stay family.