We've only been once and stayed off site because we had a large group and rented a house. It would have been my choice to stay onsite, and we will in the future. BUT I will say that I didn't mind being offsite as much as I anticipated. Getting to and from the parks didn't take that long (potentially the same as onsite depending on the location). The food was cheaper, and in our case since we had a house, it was nice with small children not to be in a hotel room. So, it probably is a matter of personal preference.
There are so many "it depends," but you'll enjoy your trip regardless.