I think you're missing out, but you do you. It would make me terribly sad to be steps from Epcot and not go in and wander the countries and booths at the very least.
I will say that as a frequent (biweekly, if not more often) visitor, the posted wait times are worthless so I hope you're not making all your decisions based on that. I felt bad for people paying for
Genie+ when I was walking on things that they had paid for!