Perhaps things have improved at Yacht Club Galley. Last time we ate there 3-4 years ago, the service was slow and incompetent and the food was blah. We have not been back.
If you are willing to go elsewhere, we had a very good lunch at the Turf Club at Saratoga Springs recently. It was quiet and food was delicious. Service was slow and not very polished but perhaps if they're aware in advance that you're coming, they'll be a little prepared with extra help. We liked that it was quiet and out of the way.
Grand Floridian Cafe serves lunch and is a pretty setting for a wedding luncheon.
At Down Town Disney, Wolfgang Puck Cafe might be nice. House of Blues is pretty quiet at lunch, likely could accommodate a large party and has good food. Portabello Yacht Club has tasty Italian food and has not been crowded at lunch when we've been there (unlike dinner which can be very crowded).
Another thought -- plan tea at the Grand Floridian Garden View Lounge at 2 p.m. Follow this with dinner in the wine room at California Grill! Or dinner at Narcoosee's. Or even dinner in the separate side room at V&A! Seatings start at 6 p.m. at V&A, and probably earlier at the others, like 5 p.m. perhaps.
Whatever you decide, enjoy!