For on a budget, pressed pennies are great.
On our last trip we did something different. Some of the restaurants (TS and CS) have drinks that come in souvenir cups. We made a point of trying to find the places and then buying the mug with the drink in it. You can't use
DDP points for these but, for example, at DHS Pizza Planet and Backlot Express had souvenir mugs that came with the juice/soda in for $4.25. It would have cost nearly $3 for a soda anyhow so I didn't think it was so bad. Epcot has a build your own mug at Club Cool and you can fill it there too - a little more expensive but a more elaborate mug.
Animal Kingdom has various locations with the Animal sports topper cup around $7, but then for the rest of the day it is less than a dollar to refill it.
We didn't even do this for budget, it just became a fun distraction if DD was getting a bit cranky with the stimulation overload so we changed the tune by starting a search for a mug (I did try and find out so that I had locations in mind so as not to draw it out TOO much if she was really thirsty).
I read somewhere about someone who gave her kids disposable cameras and they had contests to find hidden mickeys/drinking fountains. Whoever found one would get their picture taken in front of it with their camera. At the end of the trip if I did this, I would get the kids each a photo album and have the cover autographed by their favorite character with a sharpie marker and they can then put together their souvenir that the spent the trip putting together. Disney is expensive without adding to the cost with extras, but some of the best things - like memories and the experiences we all share - are free.