We stayed at Bay Lake Tower last August and used the Contemporary pool. It wasn't crowded at all. It's a pretty big pool and every time we swam in it there were less than 10 other people. Lots of empty chairs, too!
We were out there during the morning and afternoon on several occasions. We never made it out for an evening swim, but looking at the pool from our balcony, it never seemed crowded at night.