My kids have both taken swim lessons since they were very young (1.5 or so). None of their swim instructors has ever approved of floaties or vests for young swimmers. Floaties and vests cause parents to relax their guard a bit more, but young children can EASILY slip out of most floaties and vests.
What we've always done is each of us parents takes one child and we are responsible for THAT child. If we didn't have two parents, then the kids could just play in the kiddie wading pool where one parent could keep an eye on them. It was sometimes a pain when the kids wanted to go in the pool but one of us adults wanted to do something else, but that's life.
My kids are 7 and 5 now and are EXCELLENT swimmers (they take swim lessons year 'round and DS7 has been on Swim Team for the last year) and we still assign one parent to each child for busy pools and waterparks. There's no telling what could happen in a busy pool. Better to be safe than sorry.
BTW, many of the larger resorts have a "quiet" pool that doesn't have as much theming, but IS quieter and less crowded. My kids actually preferred the quiet pool last year.