My DD, now 6, first rode ToT at age 4. She loved it! We thought she would be afraid of the story line (she's still nervous on HM!! lol!!), but that she would love the ride. Turned out that she thought the story was funny! Like the OP, we bought a ToT souvenir after her first ride--a t-shirt just like Mom and Dad bought when they rode ToT for the first time on their honeymoon!
By the way, DD's first trip to WDW, she was too short for ToT. They gave her the "Future Rider Certificate" that functions like a fastpass for her next trip. Needless to say, she made a beeline to ToT as soon as we entered the gates of MGM.
She now loves all the thrill rides--Splash, Space, BTMRR, Test Track. I can hardly wait for her to try MS and Soarin' on this next trip. We also will be there for the DVC/AP preview for EE and are trying to work her out a ride on that. Following in her Mommy's footsteps...