I think I know where your husband is coming from (not agree with it - just get it). DH and I both get a certain amount of spending money every 2 weeks. I always use my spending money to buy new clothes when I need them. The clothes are neccessary (I've had to replace my entire wardrobe), but I do really enjoy getting them and wearing them, so I consider them fun. DH would never say anything to me about using our joint money on them, this is just something I choose to do so we can put the money elsewhere.
Maybe your husband thinks that because you get some enjoyment out of getting new clothes/pajamas, they should come out of your gift money. Again, not agreeing - just thinking of what he may be thinking.