My Target ponchos cost about 69 cents each, and tuck easily into a pocket/fanny pack/purse.
They're so inexpensive that I treat them as "disposable" (throw them away after one use), so that I don't have to try to wrestle them back into whatever I'm carrying them in. The Disney ponchos are much nicer (heavier weight), but if you're only going to use them once, the disposable ones work better. The Disney ones do fold back down, but you'd need something pretty big (like a backpack) to carry them around in. I travel much more lightly than that, and would rather tuck a disposable poncho into my pocket than wrestle with a bigger one.
If it was probably going to rain several days in a row, I would bring my nice, waterproof rain jacket with a hood. It gets very windy in Florida and an umbrella wouldn't work very well.