It's actually very easy to get from MK to the Poly - you can just hop right on the monorail and get off at Poly...if you end up at the Ticket & Transportation Center (via monorail or boat/ferry), you can easily walk (probably a 5-10 minute walk) to the Poly.
It actually may be in your best interest to leave MK before Wishes because the crowds trying to leave MK after Wishes are absolutely horrible - if we have time, we usually grab a snack and sit for an hour or so in the park before leaving and even then, the crowds are usually pretty heavy (on the buses, the monorail, and the boats leaving MK). If you're driving, drive to MK that day, drive to Poly and it's even better - you can just hop right in your car and not have to worry about the crowds after a long day/week. Parking is free is you self park - I think they have valet parking too for a fee - we usually just self-park!
If you're not driving, just walk back to the TTC from Poly to catch your bus or take the monorail if you don't want to walk. TTC will be crowded by then, but most of the crowds will still be in MK.