Every water shed in this country will have it's own blend minerals and micro-organisms in it that (in most cases) are perfectly fine for consumption. However, we can still smell and even taste the difference from what we're accustomed to back home. In my home town the water is from wells and is laden heavily with iron to the point that our rivers and ponds are all stained orange. We're used to it of course, and so the water and anything mixed with it doesn't seem funny to us, but a few visitors will comment.
When we first went back to Disney World after a 15 year break we thought the water was horrendous and I blamed it for making us sick (it is very possible to get ill off of other regions water if your body isn't ready for it) Now, after a number of visits we're accustomed to it, in fact I like it better than at home, perhaps just because if I'm drinking it it means we're on vacation.
Soda from the large manufacturers such as Pepsi and Coke, be it bottled or from the fountain, will taste like the water in the region because it is mixed locally with local tap water. Fountains mix the syrup with the carbonation and tap water on site while bottled comes from a regional bottling plant. The plants could, if they wished to, filter the water before bottling but I doubt they do. I have never heard of a fountain machine filtering the water.
I recommend parents of bottle fed infants either buy bottled water of a brand they're used to in order to mix the powder with. Canned formulas should be purified and safe regardless of where they're bought.
Worse than the tap water making you sick: the pool water.

But that's a discussion for another thread.