This was my biggest complaint when we switched to cash-only/envelope system. Going in every time I get gas, especially with a little one, is a total pain. I solved the problem by purchasing gift cards at the gas station for dh and myself. I bought a gift card for myself and one for dh and then each week I take the cash we have budgeted for gasoline to the gas station along with both gift cards and have the cash amount put onto the cards sometime when I'm able to get out without my little one. We can use the gift cards to pay at the pump so I don't feel I'm being "punished" by not using a credit card.
This has worked great for us. Many people in our Dave Ramsey class had the same complaint and loved this suggestion.