IMO, it is worth it. The food there is so good. The easiest thing to do would be to plan on going to Kona on whatever morning you are going to be at the Magic Kingdom. Try to get a reservation before the park opens. That way you can go eat and then head over to the park to begin your day.
If you are driving - park your car in the Magic Kingdom parking lot, get to the TTC, and take the resort monorail to the Poly. Then once you are done with breakfast take the monorail over to the Magic Kingdom.
If you are using the Disney buses - when you get to the Magic Kindgom just get on the resort monorail and go over to the Poly. After your meal, get back on the monorail and go back to the park.
I believe Kona opens at 7:30. I would try to get a reservation as close to that time as possible so that you can be done with your meal and at the Magic Kingdom close to opening.
Allow about a half hour for travel in the morning if you are driving, and about forty five minutes if you are using Disney transportation. Getting back to the Magic Kingdom from Poly will probably only take 10 minutes or so via monorail.