Speaking from my vast experience of one cruise on the Magic where I was also in search of brewed iced tea...
It is available in the 3 main dining rooms, and also at Parrot Cay when it's serving a buffet. Topsiders does not have brewed iced tea, and it was somewhat of a pain to brew it there myself, so that's just one of many reasons I don't like Topsiders and won't go back there unless it's the only dining locatin open.
The pre-made iced tea at the drink stations is the yuck stuff from the fountain. But they have tea bags and hot water so you can quickly brew your own from the hot drink station. I found that two tea bags in a cup of hot water for about 2 minutes would be perfectly brewed and watered down just right if poured over a cup of ice, and then perhaps topped off with more ice.
On the next to last day of my cruise, I discovered that the hot water from the bathroom tap was hot enough to brew the tea in my room, though it may take a few more minutes to fully brew. Once I figured this out I was able to keep some extra "iced-tea" supplies in my room, and along with the bucket of ice that our room hostess kept full for us in our fridge I was able to prepare my iced tea whenever I wanted from my room.
Yes, I'm an iced-tea addict and must have it to drink throughout the day! I brought my own 16 oz disposable insulated coffee cups and would use one each day. Would start the day with it full of iced tea and carry it around with me and refill, and then just throw it away whenever I got tired of carrying it. This worked better for me than something like a Tervis cup or other refillable mug because I could discard it whenever it became inconvenient, and I started each day with a clean cup.