I hate to stay it but you are going to do a LOT of walking whenever you stay at the BoardWalk. Epcot and DHS are only serviced by walking paths and the boats. Boats are nice because you don't have to fold up the stroller, but I just despise waiting for them to arrive.
The walks are about 15 minutes to Epcot and 20 to DHS. And if you're headed to Future World at EP, figure on another 5-7 minutes of walking once inside the park just to reach the attractions.
Pushing the stroller those distances is no treat. When going to Epcot there are two noteworthy hills on the path--one just beyond the ESPN club and the other once you are inside of the park.
Unless you're on the ground floor or don't mind hauling the stroller up and down the stairwells, you will have to walk to the lobby each time you arrive or depart as there is only one elevator in the building.
My kids were about the same ages (5, 3) when we once stayed at the BW with a stroller and it was not a lot of fun. Honestly we did better at SSR with the car parked right outside the building. We drove to every park and the stroller stayed folded-up in the trunk until needed. Now that the kids are older and they can walk to the parks, BWV is much more manageable.
Hope that doesn't ruin your day but I'm just trying to give you realistic expectations. I'm a pretty healthy dude and a Disney vet, but I was icing my knees by Day 3 after pushing those kids on the double-stroller from BWV!
Technically you could drive to both EP and DHS. DHS is practically a straight-shot from BWV while you have to take a more circuitous route to get to Epcot. It will take you more time to reach those parks by car but it should be less hassle (leave stuff in the car) and will require less walking--particularly if you valet park at the resort.