If you allow yourself no more than two hours to eat.....and tell the castmember when you sit down that you have to be somewhere by whatever time it is....allow an hour for travel if you are using Disney transportation, it will be close but you can do it..
If you have a car and not using Disney transportation, it may make it easier for you to get back and not cut into dining time. It is an all you can eat buffet, so hopefully they are on that night and you get the food fast.