And as your little one gets older, you might do what we do. I typically budget $25 or so per kid and buy clearance and sale items from Disney.com. They sometimes have free shipping. Cups, vinylmation, pens, autograph books. I even found the Sorcerer's hat my DD had been begging for at WDW on sale. That one was a birthday present.
I bring everything on the ship and the Castaway Cay Fairy leaves surprise gifts for the kids every day or so (depending on the length of the cruise). We got away with just one purchase of 6.95 on board last time (DS needed a pen for his autograph book!). The gifts I bring are smaller than what you'd find on board, so instead of one $25 gift, they get three or four smaller gifts that "magically" appear!
It wouldn't work for everyone ( hello, airline baggage fees!), but we drive. And we're always on a budget, so letting them loose in the shops is not an option