A few minutes at the most. Straight shot and we always do that on our MK day. We park at the TTC, walk over to the Poly, eat breakfast at Kona, then hop on the monorail to the MK.
Note: When you take the path to the Poly, just stick to the "outside " side walk that runs along the parking lot. It's the most direct route to the Great Ceremonial House, the monorail station,etc.