Lets see-
There are many quaint 'old' towns, well the center of towns. I have a house a stones throw from me that was built in 1750 but there are older ones (I live about 5 miles out of 'town'). I love to walk around my town because I love to look at the old houses there. Plus the surrounding towns by me (I live in a touristy area, somewhat).
There is the Essex Steam Train & Riverboat. I finally know what it is like to be bored on it since we go on it yearly when we get the 'free' local ride as well as the North Pole Express. But it is still nice if you are not a local to see the surrounding towns. It was also in a few movies, most recently the newest by a few years Indiana Jones Movie.
Gilettes Castle is pretty neat as well especially when you listen to them talk about the house. You can get to that via the train since they make a stop there.
Beaches in CT are not free & if you don't live in that town you pay more & if you are out of state expect more. We go to Rocky Neck (about 20 min from me) vs Hammonasette state park because it is a bit smaller & it is in a bit more open water, aka cleaner.
There are 2 covered bridges that I know of in CT. One is in East Hampton (I think, it is just north of me by about 20 ish minutes). The other one is in Litchfield County.
There are a few river cruises (CT river) that take you on to LI Sound (haven't done them yet) or on the CT river.
You might just want to get some scenic maps & take some drives. It is really pretty. Some days I get off my exit in my town & look to the right (that is the direction of the center of my town) & I have to stop & think that I am not in VT but in CT.
One of my friends moved here from Minnesota & she said it was hard to get used to the curvy roads that we have here in New England. So that goes with above & take a map & go for a drive.
There is Mystic (already discussed that above), the Maritime Center in Norwalk, the Discovery Museum in Bridgeport (we just went there last summer), Beardsley Zoo in Bridgeport which is a small zoo & let me tell you it has grown since I was a kid (I used to live in the town next to it before moving east to the boonies).
Dinosaur State Park which we finally took our kids to last year after telling them for the past 3 or 4 years we would go. It actually has dinosaur prints. My dh remembers when he was a kid actually walking on them but you can't do that anymore.
This place looks neat but we haven't gone (son is dairy allergic so I am not sure how he would tolerate bison milk)
http://www.creamerybrookbison.net/index.html
FYI in August the Jellyfish start to come out so the water might not be the best swimming water at that time.
Another site to look at since we get the magazines since we live in this part
http://www.mysticcountry.com/
http://www.lighthousemaps.com/index.html
Someone mentioned Lenny & Joe's Fishtale. We go there. We are about 10 min from the Westbrook location & order take out. Yes it gets crowded in the summer but the one is Madison more so.
There is the Chester/East Haddam ferry which I think is the oldest running ferry in either CT or the US (I forget what they say). It is just basically a ferry ride in your car to the other side of the CT river.
If you were to go to Mystic Aquarium, go early. We were members there for about 2-3 years & we'd get there for about 9-9:30 & head right over to the bird cage which opened at 10 to feed the hungry birds. The few times we got there for about 11, the birds were full & had no interest in going on our little sticks.
Oh & it is very easy to take a drive to RI & Mass for a short day trip. I am hoping to get my dh to take a day off during Feb vacation so we can go to Yankee Candle & the butterfly museum up in South Deerfield, Ma.
One other thing, fairs start in CT in August. Orange has the Firemans Fair as does North Haven. Chester (cough & you'll miss it) has their fair in late August. I think Moodus has one around then too. And maybe Goshen too.