If DH thinks that it isn't worth it and is happy paying as you go, why would that be a problem?
DVC isn't for everyone. I think that for DVC to work to your benefit, both of the parties making the financial decision need to be on-board and understand the benefits. Going once a year or once every other year, it may turn out that DH is right. You do understand that even in the years you don't go, you are going to be paying your $4/point in annual dues? Are you/DH prepared to fork out $500 or $1000 in the years you don't go? I'd say that's a pretty tough one to swallow - it would be for me.
In our case, based on the amount of money we were spending on just the accommodations, and the frequency we go, our membership paid for itself very, very quickly.
BCVOwner2002 expressed my thoughts exactly. You cannot just "think" you will come out ahead, you need to sit down and put the costs in a spreadsheet, look at where your break-even point is and work from there. Use the nightly room rates you paid, look at the
point charts for the days and time of year you went, and figure how many points you'd need to equal the accommodations. You may find that DH is right that paying as you go is better, provides more flexibility, and from a financial perspective is better. You should also consider how it fits into your personal financial profile - that is, will you be financing, or paying the for the whole thing immediately? Personally, I don't think anyone should finance a timsehare purchase like DVC. There is good debt and bad debt - I see DVC no different than financing a car. It is important to realize what the financial benefit (if there is a benefit) is before going in, because in 39 years, it will be worth nothing.