. If you take a child for a potty break and return, you are rejoining your original place, and not cutting. What op is talking about is blatant line cutting and in mho is wrong.
I agree that meeting your group after you pull a fast pass is wrong- but I kind of don't understand the potty break exception either. It seems blatantly unfair to single parents. So if you came as a group, you can "double dip" by line waiting and potty breaking at the same time. But if you are a single parent with your child, you can't do these things at the same time. You'd be stuck getting at the back of the line, whether you had to delay getting into the line or if you had to leave the line for the potty break.
I certainly wouldn't say anything- but it still doesn't seem 'fair'.