You don't say if you are retired or working, but keep in mind that the commuting on and off the island for people who have to leave to work is horrendous. Also, unless you have big bucks, the property is pretty expensive.
Bluffton is a lovely small town dripping with Southern charm and atmosphere. But it is a very small town. Have you looked at Beaufort, about 45 minutes from HH and on the coast. It's a really nice small size city that's developing rapidly, but still small enough to retain it's charm. An even bigger city along the same lines is Savannah, more more touristy than Beaufort.