Remember that there is always a chicken exit where the rides board. The first time my husband and I asked for the castle tour. The second time, he was definitely going to ride, I wasn't sure. We saw different parts of Hogwarts going through the ride line and more of it with more time to look around. When we got to the boarding area, he went ahead and I chickened out and they let me leave through an exit door into Filch's Emporium. So, my advice is if there is no castle tour that you can just carry on as if you're riding and then don't.