Nope, not at all. The streets are very well laid out, lots of signs. You park and grab the tram going by if your too far to walk, we get there early so can walk it easily enough. We grab our car to park hop as well. MK takes abit longer as you have to take the Ferry or monorail but again no biggie. On the ride home, we often stop for supper or the grocery store if we need anything, its handy. We have also got to know Orlando this way as sometimes we just drive around and check things out, lol.
Having the space a condo provides (and the quiet) is worth it to us and honestly, driving is not a concern although I should mention that we tend to go in offseason. I have no idea what the traffic is like during peak times.