My daughter's definitely a daredevil; when she made it to 40" tall she went on Tower of Terror with me and she was only three!
However, you can hold onto your kid on ToT, you can't hold onto them on Space Mountain! They have to sit in front of you.
So, we waited until she was 6 and I felt she was strong enough to hold on by herself before we went on Space Mountain. That being said, I was still totally freaked out the entire first ride, waiting for her to go flying out of the seat in front of me.
But, she made it, no problem (it was Mom who had the white knuckles), and is now one of her favorite rides. But she was about 46" and about 50lbs of muscle before I felt comfortable letting her on it.