We did once, but not by choice.
We were getting on TOT on our first trip down, my DH couldn't ride due to an injury so he waited outside. My oldest DD was 9 at the time and was psyched up to ride, until she got to the front of the line, and then she freaked. We couldn't go back, and with it being our first time, I didn't know about rider swap or chicken exits, so I was in a panic too. I was trying anything to bribe her to just go ahead and get on, and close her eyes until it was over, because I honestly didn't know what else to do, or how to get out of there.
Thankfully a nice CM seen what was going on, and offered to take her to the exit to meet with her Dad, while I road the ride with my younger DD. I was relieved, however, when we got off the ride, DD was wandering the gift shop alone. She said that she didn't see her Dad, so the lady just told her to wait there. She was fine thankfully, but leaving her alone would have never been my first choice.
If it's the only choice you have though, I agree with the person that suggested the baby swap area, where she would most likely be the safest.