I would say if you are at the front of MK near the train station you would be first out and have no problem getting to the Poly within 20-25 minutes.
Since you will be in the thick of it at the dessert party I would say an hour back to the Poly is just about right.
Personally, if you have the energy left in you at the end of the day I would wait it out and spare myself the mass exodus.