Love that idea. We're members at a new Y and it has a men's changing area, a women's area, and a smaller family area. The family area is similar to what you are describing, but there are only 4 private rooms and there is usually a line to use them. We usually wear our suits to the pool anyway, but I like to change out of my wet suit before driving home.
One night recently, the pool was really busy and there were so many people waiting to use the family rooms that I just went over to the locker area, took our stuff out of the locker, and started pulling my clothes on over my wet bathing suit. The problem with this was that there was a man standing there waiting for his family and he was watching me like I was doing something dirty. It was so uncomfortable. I can't really explain it, just the intensity with which he was staring at me was weird. I mean, I was putting clothes on, not taking them off! But the way the family area is laid out, the locker area is sort of isolated from the changing rooms, down the hall and around the corner, and when this man was staring at me and we were the only adults in that area the isolation of the area suddenly became very apparent to me, I've never really noticed it before because I never had any reason to feel unsafe in the locker area before. I fumbled with the clothes a bit since I was still so wet--pulling jeans up wet legs is hard--but I rushed as much as possible and hurried my kids out of there. I've avoided the common area of the family area every time since then, the incident kind of put me off it. I wish I could just change in the women's room but my son is uncomfortable in the men's area by himself.
I am well endowed in the chest area and over the years I've learned that many men seem to have the attitude that women's bodies are on display just for them, especially when the women have a feature that is pleasing to these men. I shared in another post about an issue I had with a stalker who got physical with me, and I'm sorry to say that is not the first time something like that has happened to me. That's what worries me about unisex bathrooms. Let me be clear though, I have nothing against people using the bathroom that is appropriate for the gender they associate with. Like you said, OP, I prefer to be modest and I never change in the open, I always go into one of the bathrooms or shower stalls when I am in the women's locker room. But, for someone like me with a history of child sexual abuse, or a woman who has been a victim of rape or domestic violence, a situation like that has the potential to get scary quick. If I am in a room alone with more than one man, or just one man who resembles my abuser, I become a shaking, cringing, petrified mess. 20-some years after my abuse ended and I still feel like a victim. And having a man staring at me like that, in a place where I felt vulnerable because I was alone with 2 kids, was a horrible experience, but it makes me wonder if stuff like that would happen more often if unisex bathrooms became the norm.