We just got back Monday and caught the bus at the resort. We had no problems and never waited more than a few minutes.
I'd say whether you go to the Poly bus stop or the TTC stop would depend more on the location of your longhouse. A few of the longhouses might be closer to the TTC, in which case the TTC might be easier, but if you're closer to the Poly stop which is right by the Neverland Club, I'd certainly go there.