It really depends on what your money is worth to you and how much you enjoy certain aspects of resorts. Personally, I love staying onsite and don't plan on ever staying offsite, but I can see how it may not be worth it to everyone.
The rooms are definitely not bigger when comparing similar-priced resorts. If you take just the rooms and what's in them, I'd compare All-Stars to lower priced chains (I guess, I've never really stayed in one). The moderates would be like Hampton Inns or something similar. The deluxes are more like Hilton or Hyatt rooms. As you can see, the prices are far higher than what they would be at any "regular" resort. I consider $120/night for a moderate a pretty good deal, but I wouldn't pay $120 for a Hampton Inn.
There are other benefits to staying onsite. For us, transportation is a huge factor. We drive there, but we park our car when we get there and rely on Disney for the rest of the trip. I'd much rather wait for an airconditioned bus under an awning with fans instead of crossing a busy parking lot to get in a hot car that's been in the sun all day. The ammenities at the resorts are also better. Many hotels have very few options for food, and what they have is often over priced (like room service). Disney resorts have a bigger variety of places to eat. There are also more recreational options.
Another good thing about Disney resorts is the theming. At other hotels, the theme is that you need a place to relax and sleep. Disney resorts are fanciful and make you feel like you're in another world.