In my years around here I tend to see two divided camps to this answer - the Garden Wing does have a loyal fan base who tend to appreciate it for the more quiet vibe, quick walk to the pools, no need to deal with elevator congestion in the Tower, and the lower price point. Non-fans tend to not like the inferior views (although some are quite lovely), grungy stairways and elevators, the lack of balconies on the second and third floors, and more isolated location requiring a 'longer' walk to building amenities. The Garden Wing rooms are ever so slightly smaller in depth than the Tower rooms which is hard to really notice, but is technically the case.
Personally, I'm a Tower guy - if I'm going to spend the time/money to stay at CR, I sort of want the "full deal" experience of the monorail, a balcony, and views (as the views on both sides are pretty awesome). The Tower isn't without flaws though - elevator congestion can happen, especially the evenings, some find the parking lot views on the TPV side less desirable, the TPV side will bake in the afternoon sun on a cloudless day/afternoon, Chef Mickey's and central atrium noise is nearly ever-present.
Like anywhere you have a mix of pros/cons, and where you fall on either side is very subjective. And ultimately I don't think you can make a wrong choice here, so that's good!