Like any other land, the best time to go is when it first opens. There will be characters out as soon as they open the gates, and the area is obviously less crowded. If Toon Town is a priority for your family, I highly recommend this. I very much don't like going there in the late afternoons. It's just maddening.
last couple of times we have done Toontown. we get there for gate opening.. walk straight to Mickeys and get photo.. then make way round and get the other photos with Minnie , pluto and goofy and can be getting on a ride or two all comfortably within an hour.
We always try to go after dark. It looks great at night with all the neon and lights. In daylight it doesn't give the full effect of being a cartoon land like at night.
W went last Friday right when it opened and Mini Mouse came out when they opened the gate and grabbed my daughters hand and walked her and her BFF up to her house for a pic! Right after that the girls rode gadgets coaster 6x in a row! Definitely go right when it opens!