I think the answer always depends on how many. Our last trip my daughter, son and I often went to the gates early - like up against the gate early. My husband would often join us about 15 minutes before we would be let through but that is amending a party of 3 to a party of 4. I have seen others do this as well. This seems to be acceptable. However, if you are adding more to the party than is already there, then I think it's somewhat frowned upon. Last time we saw in the line next to us 1 person having 5 people join them at the front of the line and there was some grumbling of others in that line.
Gate is not as big of a deal though as ride lines, as going through the gate is pretty fast! Holding a place in a ride line is much more frowned upon. People do it, but I think it's rude. Leaving with a kid to go potty and coming back is entirely a different thing.