Probably not - but try one out at home first. If you have a typical American Japanese steakhouse in your town, or can go to one nearby, it's the same experience. Someone will cook at your table, etc. The only times my small children had issues were with the fire (usually the chef will pour some sort of liquid onto the grill and make a huge flame) but it doesn't last very long and I could always hold them and/or turn them around so they didn't feel the flame too much.
Foodwise, I don't know what your baby would be eating, but you can get a child's meal. It will have plain white rice (Teppanyaki doesn't do the fried rice, sadly - that's my favorite part!!) and some sort of grilled meat, depending on what you order. You can also get a broth-like soup and a salad at the beginning of the meal. I think my girls have had some sort of ginger ice cream there before - yum!
Don't worry about noise, etc. Each table seats 8, so you'll be eating with strangers anyway, and it's very noisy and colorful - NOT a quiet, sedate dining experience at all!!!
We love it - I think you'd find it fun, too!! If you're going this August, make sure it will be open (I guess they'd tell you when you tried to make the reservation!) as I understand it's undergoing a renovation right now. It wasn't available during our Spring Break trip last month.