I agree with previous posters to keep the kitty confined to one very small room (such as a bathroom) for a few days. It's often better for a new kitty for other reasons than just the litterbox. It can be overwhelming and scary to go from living in a cage at the shelter to having access to the whole house in one go. Start them off in one small room and gradually give them access to more of the house but be sure to spend LOTS of time with him so he can get to know you!
Also, he might not like the location of the litterbox. If it's too out in the open that could be a problem. And as others have said, he might not like the litter.
Last summer we trapped four feral kittens. We kept them in a large cage for a week or so with a bed, a litterbox and food and water bowls. I was amazed when they took to the litterbox immediately! So I think most kitties do figure it out pretty quickly, provided there aren't too many other options! When we moved them from the cage into one room, I wondered if we would have problems but they continued to use the litterbox!
Good luck !