I am a stay-at-home father of 4. I have been on 2 solo trips so far. I went last August and May of 2011 to get away for a weekend as I was getting pretty burnt out. I chose Disney because I had gift cards to offset some of the cost, and I love it there (which is why I was getting the gift cards in the first place.

) I offered for my wife and kids to come but she said no, it was too much work to take all the kids. But, I went. I felt a little guilty going without the kids but I just didn't tell them about my last trip. I told them I went to FL for a car show and just took pics of the older cars I saw (like at Pop Century and Hollywood Studios).
I am going on my 3rd trip in October to run the Tower of Terror runDisney race. I feel less guilty about this one as I am more going to run the race as my reward for getting in shape and less to ride rides and stuff. I'm getting divorced now too so this will be a little different for that reason as most of my future trips will be solo as well.
As far as safety, I am a man so it is a bit different but as long as you stay on Disney property I can't see why you wouldn't be safe. You walking from your room to the lobby to eat would be no different than if you went with your husband and he stayed in the room with the kids while you went to get something to eat. Nobody, unless you tell them, knows your alone while you're walking around. Nobody will know if your husband isn't 2 minutes behind you at any second.
As long as you have the money and your husband supports it I say GO FOR IT! It is so much fun not worrying about bathroom breaks, picking someplace to eat everyone will like, kids being too tired and cranky to watch the parade and fireworks, etc...
If you book it for the weekend I am down there I will happily meet up with you for a meal or some rides, if you like.