Forever, trust me.
The quickest way is to take a cab.
Otherwise Montana's advice is sound. It would work as well or better to take a bus to Epcot, then the monorail to TTC, and walk to the Polynesian from there (5 min).
Don't do anything that involves Downtown Disney or changing buses.