We stayed at POR in April, right after Easter, and it was still pretty crowded. I rented a car and NEVER used it. We actually had trouble finding it when it was time to go back to the airport. The only delays we encountered were leaving MK after Wishes. There were only 2 showings that week and it was really crowded. We had our traditional snack at the bakery and waited for "stroller derby" to finish, to try to avoid the long wait for a bus. It didn't work! There was an accident somewhere near the TTC. We waited about 20 minutes in a huge line and then they rerouted the packed bus all the way around the back of MK to avoid the traffic. I wish it hadn't been dark out, I'm sure there were interesting sights I've never seen back there. The wait seemed longer than it was because there were so many people ahead of us and the other bus lines were pretty empty, plus we were tired after a long day, but it really wasn't that bad.
I would suggest though that if you go during a busy week, use the south bus stop near front lobby entrance if it's not too much of a hike. There were a few mornings when the bus stopped for us there and filled up, so we went straight to our destination, without stopping at the other bus stops at the resort. I wondered how much longer the people at the other stops had to wait, but then they may have just sent a full bus right over there.
I had a great time at POR and would not let a few bus delays change my plans. I had three impatient pre-teen boys with me and they would have made life miserable if the lines had been really bad! We also loved the boat ride to DTD and back. The line to return looked long, but it moved quickly once a boat arrived. Relax, you'll have a great time!