I would definitely recommend making a reservation. When my mom and I went during a slow time, they had a 25 minute wait for one pair of walk ins that came up behind us, which doesn't sound too bad except that the tea room isn't open but for a few hours - 2:00 to 5:00, I think.
It's yummy - if you click on the link in my sig line you can read my review from last March, and as mentioned above, Oybolshoi has one in the "Hungry are the damned" review thread from October.