I agree that Yacht Club has more adults, and was more sedate and quiet, but not boring nor stuffy. The Yacht Club lobby was calmer and less crowded than the Beach Club lobby, which was usually busy with families and kids. Cape May Cafe, with the character breakfast is right off of the Beach Club lobby. Both resorts share access to the same amenities.
I would go with the Yacht Club. We asked for a quiet room, and got one right off of a courtyard; it was lovely and quiet.