Generally speaking with Disney transportation: the fewer number of transfers and stops you have on your way, the better.
Since time is crucial here, and assuming you don't have a car to drive yourself, I'd take the bus to MK, and then boat from MK to the Poly.
The reason I'd take the boat is that Poly is three stops away from MK via the monorail and the monorail can often get slowed down at CR and the TTC. The boat will probably have to stop at GF before going to the Poly, but it is a quick stop, and then on to the Poly. This is a little quicker going TO the Poly from MK than the monorail.
If you're going to MK after the Poly, I'd take the monorail, as it makes a quick stop at GF to pick up a small number of people (normally), and then zips on to MK.
You certainly could go to EPCOT and then take the monorail from there to TTC and then walk to Poly, but that will most likely be quite a bit slower than what I'm suggesting.