The difference between EP and non-EP varies quite a bit, even over the course of a single day. For us, we've been to Orlando enough times that we often choose to skip any attraction that has a long wait if we don't have EP (and maybe check back later in the day). As others have said EP rarely exceeds 20minues, often is far less, but also fluctuates a bit unpredictably between 5 and 20 min per attraction.
With EP, we ride just about everything whenever we want to ride it. Even without EP, standby waits fluctuate over the course of the day. Flight of Hippogriff is a good example. Sometimes the standby is 30+, other times it is short. Personally, I just wouldn't really ever wait in a long line for it. Your touring style might be different. Our attitude is also relative to things like weather and how hungry we are.
So for us, EP = very relaxed touring, where non-EP = more selective touring.
IMO, if we don't see an onsite hotel price we like, then we don't tend to buy EP.