Remember, these people are councilors, NOT lifeguards (even though some may be). During the day, the pool is usually crowded, so in order to maintain a "secure" environment, they would have to ask everyone else to leave the area during their time there. That would make for a lot of unhappy cruisers. You notice that when the kids clubs go somewhere else on the ship, they usually have the area to themselves so the councilors can maintain control of the group. Wouldn't want any child to become lost or worse, taken!
Bottom line: if your kids want to swim, You have to take them.