I heard that it was a super adult type atmosphere as well, but it did not seem that way to me. We actually went there twice. The first time we went to see the view and sat at one of the little round lounge tables near the window and ordered drinks and just relaxed, with three kids.
The second time we made PS for dinner and decided that we did not want to sit in the resturant part, because we found a table in the lounge area with a great view of the fireworks. We ate our dinner at the low tables in the lounge area with four kids from 2 to 16. It just did not seem stuffy to me, and the server was great too. He went out of his way to make all of us feel comfortable, and was very accommodating to the kids as well.
There were people dressed up and some dressed sort of casual, not park casual, but not formal either. You would probrably have a great time. Go for it!
Sam