It's a beautiful ten minute walk along the path from the Poly to the GF. We stayed at the GF last year and walked over to the Poly for breakfast and dinner a few times. We also enjoyed walking over to the TTC to catch the monorail to Epoct. The grounds are so beautiful, I'm sure you'll enjoy seeing it all first hand.
Hope you enjoy your trip!
Susan