Having stayed at POFQ with a 5 and a 3 year old- my advice is to rent the car!
The bus system is fine- but it does take a while to get to the parks, particularly when you have to go all around the Port Orleans Riverside stops first. (sometimes the buses do go direct to the parks however) When you need to make the extra stops- you can count on 20 to 25 minutes to any of the parks. If not- then at least 15 to 20 minutes.
In fact, it felt like we were always on the buses. Either waiting for one to go- just missing one, waiting for ones from the parks- and on the bus itself. This is particularly rough if you do take an afternoon break.
This is how it goes- wait for bus (usually pretty good 5 to 10 minute wait or if lucky bus pulling up). Load kids and stroller in bus. When you are unlucky- you head over to Port Orleans Riverside and need to stop at 4 additional stops to pick up people. Finally head out to parks, struggle to get kids and stroller off bus. Then when taking afternoon break- do all this in reverse. I found our wait midday to go back to the hotel from the parks was about 15 to 20 minutes as the buses don't seem to show up as frequently. Load kids, stroller, backpack, etc. Lucky to get a seat sometimes on the way back. Unload everything at hotel. Go back to the parks at 5:00- do it all over again. If you close the parks, wait for bus, stand in long line, wait for what seems like a million
ECVs to load, then stand on bus. After a while, it becomes a drag. If you want to eat at other resorts or go to the waterparks, it's also a bit of a bust. We were forever running to catch a bus to wherever- then get that resort bus to the restaurant.
I'm sorry I sound so negative on the buses- but I found with two little kids and two strollers it was all very overwhelming. This year we are renting a car. Last time I wanted to try the Magical Express and the bus service- just to see what they were about. After having ridden many buses- I can honestly say- I prefer to be at my own command and drive a car.
Good luck in your decision!