I'm sure a big part of the reason that rider swap, in practice, is not offered for everyone who "doesn't want to ride" is because of the potential for abuse. People could claim that their tall enough child doesn't want to ride to get the rider swap pass (while part of the group uses their fast passes) and then take that same child on the ride later on using the rider swap pass and essentially double up on fast passes for rides since they wouldn't have to have a fast pass to use the rider swap pass. It's situations like this that rider swap is not meant for, but it's impossible to really police it except by enforcing the rule that if you're tall enough, you don't qualify for a rider swap pass, which is what they mostly seem to do.
From what I've seen, they do sometimes seem to be a bit more lenient if the child seems frightened (but is tall enough) or if they are really young (actually I'm a little surprised pp didn't get it for their 4 yo on TOT).
A good thing to keep in mind though, if there's a ride that you're sure your tall enough kid will not ride, get fast passes for everyone else and the adults can just take turns staying back with the kids and everyone still gets to ride that wants to. If you're not sure if your kid will ride, get a fast pass for them too and you can always take them out the chicken exit if they get scared.
In regards to getting a rider swap for an adult who doesn't want to ride, why would that even be a thing? I understand that the website indicates it could happen, but as several people have pointed out, it's not likely to, and there really wouldn't be a need for rider swap as the adult could wait by themself while everyone else rides, or go through the line with everyone and use the chicken exit. The point of rider swap is to enable families to ride when they have someone too short to ride (and therefore not allowed in the queue) and too young to wait outside the ride alone.