The rules prohibit pool-hopping by guests. The only exceptions to this are guests who are staying at "sister" resorts that share one pool, and in some cases DVC guests/owners.
To make an exception for grand gatherings would be unfair to guests in groups smaller than grand gatherings who paid just as much, per person, to enjoy the resort pool.
I would appeal to everyone's sense of fair play in this regard. You may be able to get away with it, but please consider others who paid just as much as you for the pool, who could very possibly have pool and chair usage limited by this type of thing. Sure, there are worse things that can happen, but that doesn't make it fair.
The best solution for those staying at separate resorts, and who wish to swim together as a group, would be to go to a waterpark. If you can get by without using other resorts' pools, you can do lots of things at other resorts that won't interfere with other resort guests, such as eating at the restaurants, browsing the gift shops, checking out lobbies, etc.