Best advice, get away from the tourist areas (anywhere near the beach or rt1) Portsmouth (just down 95 or rt 1 depending on where you are) has a little bit of everything to eat. The Rusty Hammer is a good local bar with great food, Flatbread Company has amazing pizza. Gilley's is a tiny burger place that has been there since the 1940's, they have burgers, hotdogs, fries, ect during the winter the seating consists of 4 or 5 stools and everyone eating standing up. It is DH's and I's must visit any trip home, their chili cheese fries are amazing. Or you can go to Newington for chain restaurants. Since you'll be in Kittery the best Chinese food in the local area is East Mountain on rt 236 in Eliot (the town over) it is in a strip mall type thing on the left, there is also a bar called The Shipyard Brewpub (great place to get a local beer with locals), and Village Pizza which isn't very good. Of course you have to have to try Bob's Clam Hut on Rt 1 in the outlets. It USED to be really really good and worth it, however since it has become "famous" the prices have gone up, portions are smaller, and it's trying to be more then "bob's". However the food is still really good and Mike the owner treats his employees amazingly. However they SHOULD be closing for a couple of weeks soon come new year.
Yes DH and I have family up there but given the choice both of us would rather go up there in the winter and visit. I used to go to the beach more in the winter then the summer (parking is free normally this time of year). You will also be treated better.
That post ended up being alot longer then I thought! Have fun!