We just went with my 23 month old. It's kind of funny, he doesn't like walking up to characters to meet them (he just watches from a safe distance), but he didn't mind the character meals, where he stayed in his high chair and the characters came to him. Kind of odd.
My other son (who is now 4), has always been great with the characters. He's been to character meals at 18 months, 2 3/4 years, 3 years and 4 years, and runs right up to them for lots of hugs.
Every child is different. Like one of the other posters said, it's always worth a try. If it doesn't work out, just eat somewhere else.