The one time we stayed at a Downtown Disney hotel, I hated it. The shared bus for DTD hotels took soooo long and got really crowded (way worse than any Disney property resort). And it wasn't that much of a monetary savings.
That said, I would be willing to offsite at one of the nice places that's close to WDW. But, I would require that we have a car and drive ourselves to the park. Also, I would want my kids to be older so that we last longer at the parks each visit (my young kids can't last longer than 4 or 5 hours at a shot... they get overstimulated).
I agree that the folks on the resorts board might be able to point you in the best direction for a reasonably-priced off-site resort.