Check out Turner Field and the Braves. They play the Mets on Friday, April 6 and Saturday, April 7. They have fireworks every Friday night. If you have kids, there is a great thing called "Tooner Field."
Stone Mountain is fun. At least go out and hike up (or take the cable car) to the top of the mountain, and stay for the Laser Show (shown on the face of the carving on the mountain).
The aquarium is very nice. The "World of Coke" is a decent way to spend a few hours as well.
The High Museum of Art has, what is supposed to be, a great traveling exhibit from the Louvre that will be on display while you are here. It is the cheapest way any of us will ever get to see some of this art.
The Fernbank Science Center is a great place where you can easily spend an afternoon, or whole day. They have an IMAX theater, but also a great museum of natural history.
The Atlanta Zoo is quite nice now with some beautiful habitats (not the caged animals of the past). You may also like to see the Cyclorama (a Civil War museum located near the zoo.
The "Georgian Terrace" is a beautiful hotel directly across the street from the Fox Theater. They have very nice suites, and the hotel is reasonably price (mid $200's). The Westin downtown is also nice, with the best views in the city, and a nice rotating restaurant on the top floor.