We faced the same issue when we took our infant son last year.
I know the water available on-board is obviously potable. But on one of our previous cruises, we filled the bathtub with water and you could see a yellow tint to it. It looked crystal clear when you had only a glass of water (only a few inches thick of water), but when there was a foot of water to look through, there was a definite color to it. (It's sort of like air polution... you can't see anything in the air between you and the person you are talking to 5 feet away, but you try to look at a distant mountain and you can see plenty of air polution.
Since I already owned a backwoods water filter, I just took that and "re-filtered" water from the sink to use for formula. Specifically, we took a couple of (small) bottles of bottled water and kept re-filling them with the re-filltered water.