They aren't asked to leave. But since they don't can't get the wristbands that those staying at WDW hotels have, they just can't (or at least aren't supposed to) get on any rides.
As the night goes on, you usually see fewer and fewer guests without wristbands.
HBC