Unless you happen to get on an elevator where someone is riding who has already inserted their card and punched the concierge floor. That's happened quite a few times to me at CR (although I was good and didn't go with them to look around - I stopped on my own floor). It would happen to me more when I was getting on the elevator on the 4th floor and there were others already in it.