As I recall there is always a sign indicating the way for guests without bags. DH goes that way while I get my bag checked.
I always make sure that all the compartments in my bag are open before I get to the guard. It speeds up the process. I get annoyed when I'm behind someone who has a huge bag with lots of compartments and does not open anything until the guard tells them to. I have also observed that often those people get a much more thorough search.
In any case I don't think I've ever been delayed more than 2 or 3 minutes.