I lived there for a year, back in my young, sassy days.It's a lot of fun just about any time, but even more so during halloween.
I always thought there was plenty to do even with kids, as long as they were old enough to understand the difference between the "Hocus Pocus" witches, and the poor women who were burned long ago.
I recommend taking a tour, seeing the Peabody Essex Museum, and of course, some of the tourist traps. The town was actually a site of great international commerce long ago, and there are many old sea captain's homes, and maritime attractions.