1. If pressed for time, taxi service (allow 30 minutes for bell services taxi call plus check in time).
2. If reliant on Disney transport, bus to Magic Kingdom then monorail or boat to Polynesian (allow 75 minutes at the very minimum - 90 is better).
Love the place! We are going back in November and that will one of our top breakfast stops!! Stay at the Poly so you only have to decide between the elevator or stairs.
I think bus to magic kingdom and monorail/boat/bus to Poly is your best bet but you could also walk or boat to then entrance of HS and the bus to Poly.
ETA If you're using disney transportation I would allow at least 90 minutes no matter which route you choose.