I had the same same issue w/the otterbox case on my phone. It was one of the oldest ones they make for the iPhone 4, the newer ones are designed differently, and seem to fit a lot tighter. The otterbox served its purpose well that's for sure, it has saved the phone from pretty good impacts on parking lots, tile floors etc.. The rubber part is kind of loose and starting to tear around the charging port. I know the rubber part can be replaced, I didn't know that the company might replace it for you though. Definitely worth looking into. I ended up giving the phone to my mom otterbox and all. I told her that she might consider getting a another case because the one on it, although functional, is pretty junky looking now.
I got a Lifeproof for my newer 4s, was just in the mood for a change. I didn't much like it at first, but it is really starting to grow on me. I like how it doesn't seem to be too bulky.