I wasn't aware that FL water tasted so terrible or that these portable water filtration devices existed until the DIS. So, I read all about the Brita Fill 'n Go. It sounded great... BUT who wants to buy something that is discontinued and would require that you HUNT to find the containers and the refills. Uh, not me.

So, I found Culligan on ebay. I bought two portable water bottles for $11.50 (including shipping). The Culligan brand is well known in our area so I trust the quality. Culligan also has a website:
http://www.culligan-store.com/
They are VERY similar to the Brita Fill 'n Go. The filters lasts through 15 gallons and removes 97% lead and reduces chlorine, bad taste, and odor. I haven't had any problems with the bottle and it works great. I can't wait to try them in FL. You should definitely consider purchasing.
My bottles each came in a box with directions. The (summarized) directions say to wash the bottle with mild dish soap (they are not dishwasher safe!). Rinse the filter with water only. Push the filter onto the cap. Fill the bottle with tap water to within one inch of the top. Screw the cap on and squeeze one bottle of water through the filter before using. Then refill the bottle and begin using it.
Did you slowly squeeze a bottle of water through the filter before drinking it? That may be what the problem was. I hope this helps.
P.S. The directions also say that you can keep your filter in the fridge to increase the storage life and quality.
That bottle looks great and I am sure that it works great too, but it is a little pricey because it filters water that is untreated and unfiltered. Like when you go camping or hiking so that you can drink from a spring, etc. I am sure that it would work fine with the water in FL too but it is really unnecessary if that would be its only use. Just trying to save you some $$.