Princess, my MIL was actually born and raised on Smith Island. She moved off when she got married, which was quite a long time ago. She still has family there, and my husband and I have some pretty good friends still living there.
There is a book you might want to check out if that kind of environment is your thing. It's called "An Island Out of Time" but I forget the author. It has some background and some cute stories in it, it's all about Smith Island.
You're right, it's a completely different world down there. Everyone knows everyone -- and it's pretty darn hard to keep a secret, because the only thing people have to do down there is talk.
They are pretty hard workers, mostly all of them are watermen, which is a bit of a different class of people (mostly enjoyable). But they're also a tight knit community, so be forewarned that they look a little closely at "foreignors" for a while before making any decisions about them.
They are so hospitable, and so down-homey. I swear, if somehow you missed the ferry back and got stuck on the island, I'm sure someone would take you back by their own boat, or possibly even put you up in their home for the night.
Truly a different world.
There are three islands that make up Smith Island. Each island has it's own church, and the pastor goes back and forth between the churches. Most of the cars on the island are old beat-up cars with no license plates (or plates from the 60's or 70's).
They have some of the best cooking. It's different, but sooooo good. You'll definitely have to try a piece of authentic Smith Island cake (the state dessert) while you're there.
If you want, you can PM me, and I'll try to answer your questions.
But I agree, it's a great place to raise a child. It's its own world. And it's not a bad world. But there isn't much to do other than relax and talk to your neighbors.