Yes it is very close. Once I was riding one of those bikes you can rent and I must have taken a wrong turn and ended up riding to the Fantasia golf parking lot.
It can be a little complicated to get there though. Whenever we go, we end up walking through some little pavilion that the Swan and Dolphin has that looks like it hasnt been used for a year or so. We always end up getting there though. Not sure how sometimes.