This is where I am from. Have you visited gulfcoast dot org? It's our chamber of commerce website with a list of attractions from Bay St. Louis to Pascagoula. All of the communities in our area sort of run together and are a relatively short drive to each other. While I agree with our beaches-go out to Ship Island to see the pretty water-the Mississippi Sound and the brown water is because of the islands holding in the silt. Not pretty to look at, but the nesting ground for great seafood. Us locals prefer to go to the islands or our fresh water creeks and rivers. Along the beach/Gulf there are many "comfort stations" with quick shower and restroom facilities. If you do go to the beach, the beach in Ocean Springs at Fort Maurepas Park is quiet. There is a nice park there for young kids, a pier to walk out over the Gulf and watch the seagulls or people fish and another activity we like is to "walk the bridge" located near the park. There is a 3 mile bridge between Biloxi and Ocean Springs with a very nice walking lane built into the bridge. Quite safe for walkers and joggers and strollers. Walk out onto the bridge as far as you'd like and you'll be over the Biloxi Bay and can see miles around. Nice view day or night.
How old are the children you are traveling with? There are splash pads all over the area that are free and a nice place to play for younger children. We have Gulf Islands Waterpark in Gulfport, a nice day or half day place to cool off. The Infinity Space Museum is in Bay St. Louis right off I 10 and is comparable to the Huntsville Space Center. In Biloxi, there is a nice visitor's center in an antebellum home replica that is free and also has historical interesting things about the area. Go early and climb the Lighthouse across the street-tours are only done in the morning. Charter a fishing boat and go deep sea fishing or take a shrimp boat excursion-boat ride out in the Gulf and learn about catching shrimp.
There is a Mardi Gras Museum in Biloxi and Bay St. Louis. Island Strikes Bowling alley is in Biloxi, a nice bowling alley where kid's parties are held. There are a couple of places to play minigolf in Biloxi--nice little courses. Ocean Springs Gulf Islands Seashore is a very nice large park with trails and places to walk and explore and have a picnic. Biloxi has Beauvoir, the last home of Jefferson Davis of the Confederacy. No matter what you believe about the Civil War, it's an interesting place to visit and see an antebellum home on the water. There is a petting zoo there.
Last year, we had MGM park open across the street from the Beau Rivage. It's the home to the Shuckers Minor League Baseball team. It's a fun, safe, family environment. Last year was the team's first year in existence and they were good. We enjoyed a nice family evening at a ball game. There is a new Biloxi Maritime and Seafood Museum which details the history of the seafood industry in the area. And near the museum is a splash pad and playground.
Pascagoula has a new nature center and area that I've not been to, yet so I don't know a lot about it. Look up Pascagoulariver.audubon.org. Take a swamp or river tour on a pontoon boat. Links at the above website.
Bay St. Louis and Ocean Springs, on opposite ends of the Coast, are nice artsy towns to walk through and stop and have a bite to eat. All across the coast there are restaurants with delicious seafood, comparable to New Orleans.
The information about smoking in restaurants is incorrect. I've not been to a restaurant with smoking in quite a while. Perhaps there are a few towns with smoking, still, but I cannot think of a restaurant I've been in lately with smoking allowed.
Ask if you've got specific questions.
