Due to the obesity crisis in America (and the effect of unopposed hormones produced by excess fat), children *are* starting puberty at younger and younger ages. It's no longer one of those "middle school things" and it absolutely should be a consideration for parents for a myriad of reasons beyond "should I take my child into the restroom with me".
I have no issue with parents bringing their *young* children into the restrooms with them. But once a child is old enough to understand social safety and attend to their hygiene needs, the bathroom corresponding to their gender is the one that is most appropriate for them.