Baby swap and FastPass are two seperate things. To ride with a FastPass, everyone riding needs one.
With Baby Swap , one person asks the CM at the entrance for a coupon and waits in the designated area. The rest of your party rides the ride using either standby or FastPass and when they exit the ride, the other person can enter the FastPass line to ride.
Some rides, you can get right back on as the rider swap area is at the front of the ride.
The Baby Swap pass is good for up to 3 riders.