I've done both, several times each, and I prefer breakfast. It's a much calmer atmosphere, less hectic and loud. Maybe I prefer it partially because at breakfast we've always gotten a window table, and coincidently gotten the same server every time, who's been there since it opened.