We stayed a couple of nights at Old Key West last October and LOVED it. Olivia's is a great restaurant, and you can take a boat to DTD. It has a very casual Florida vibe, which may or not be what you're looking for, depending on where you are coming from.
Wilderness Lodge is another favorite. Yummy restaurants, great pool, nice ambience in the lobby.