I have not had it so I can't comment on how good it is, but if it is anything like everything else they serve in Japan I would say you will not be disappointed.
So to your question, yes, Mitsukoshi Tempura Kiku is a full service restaurant with seating around the Tempura Bar. They do not take reservations, they are strictly walk up, but they do serve lunch and dinner. The sample menu shows: Batter-fried items including beef, chicken, scallops, shrimp, and vegetables. Also serves sushi and sashimi with prices ranging between $10-$25. It is actually located off the same lobby as the main restaurant Mitsukoshi Teppanyaki.