Thanks for reminding me! Just booked a lunch for one of our Epcot days! And yes you need reservations. We tried to walk up 2 years ago. Long walk to be turned away.
It is considered a table service restaurant, and as others have said, reservations are recommended because it is a very small, popular dining spot. However, if you are just looking for ice cream, most of the ice cream/shake options (not the kitchen sink, though), are available for takeout. Snack credits can be used for some of the selections, I believe.
If you see a reservation available, grab it. My kids want to eat there in June and I waited too long and the only thing available was a 3:15pm lunch during the entire week that we are there, so I booked that. I've heard of people being able to walk up and get seating if they are lucky and go at an off-peak time, but it's definitely not guaranteed and I wouldn't count on it.
Last May we walked up at opening without a reservation (we were staying at BCV and wandered over from the pool).
They were able to seat us at the counter which was fine.
B&C isn't bad, but it's not worth going out of the way for, IMO. I'd rather go to "The Fountain" at the Swan & Dolphin, which doesn't require being tied to yet another ADR.