Crab Trap in Destin. Actually they have 2 locations, one down between Destin and Ft Walton, and the other is past Destin proper, in the area of the outlet malls. Both are on the beach and very casual. The second one is really close to the beach. They put the windows up so you can feel the sea breezes and after you're done, you just walk down the steps and go play in the water. We go there every time we're in Destin.
Mobile has a few good spots--Ed's on the causeway(hwy 90 linking Spanish Fort/Daphne with Mobile) and Catalina's, I think on Azalea Rd. Both are very casual. Catalina's caters to locals and their seafood is very fresh. Eds has a more extensive and varied menu. don't bother with Wintzells in Mobile. Yeah, it's famous for being famous but I don't think the food is all that great. Stick with those dives on the causeway and I think you will be happy.