Technically only people that have a key that allows them to the fifth floor are allowed up there. Will they say anything about people visiting in your room? Probably not but the problem we've been having is people will try to bring friends up to their room, go into the lounge, load up on stuff and take it all back. When you are talking appetizers, wine, beer, etc., that throws off our inventory big time. We know how many guests are CL guests and order accordingly. I'm sure you wouldn't even think about doing that but others have done this and have been asked to leave. Makes it very uncomfortable.
If it were me, and this is only my personal opinion, I'm not talking as a CM, I would not take anybody up to the fifth floor unless they had fifth floor access themselves. I would feel very uncomfortable. I'd visit them in their room or in the lobby.