Before construction on the Poly Tower began, I'd have recommended walking from VGF to Poly to the TTC.
I'm just plain chapped that that lovely walkway from VGF to Poly is so nastily detoured currently and will be as long as major construction is underway on the new tower.
We'd hoped for over 10 years that walking from MK to the TTC would eventually be possible. Upon hearing that the new GF to MK walkway was open prior to our trip last fall, we decided we simply had to experience walking (well, rolling for DH) from MK to the TTC.
It was great fun, a lovely path the whole way. Took around 45 minutes at a brisk but not exhausting pace.
I suggest you plan to do that walk once the Poly Tower is done. We rarely saw more than maybe 8-10 people, mostly runners, on the segment from MK to VGF which we used several times. Didn't try it at MK closing, though, but I'd pick it over anything else for getting back to VGF sooner just to see if I'm right that it'd be fastest.