I really just can't abide by that logic. You don't know what someone has been through on any given day, and sometimes certain conditions are not easily visible. Clearly people with that kind of mindset don't stop to think about those things.
I would never feel like I'm more deserving of sitting down just because I'm a woman, versus a man who may have had a much more trying day than me, that maybe has back or knee issues. Same thing with the younger versus older situation. Not in the sense that I don't abide to leaving your seat to the elderly, but younger people shouldn't be automatically judged for not giving up their seats. Physical disabilities don't discriminate on age. Honestly, it aggravates me a lot more to see four seats taken up by two parents with their young child seated in the seats next to them (I'm talking 2, 3 years old) rather than in their laps when the bus is full.
As was mentioned above - elderly people, pregnant women and injured people should have priority in sitting down. Same rules as in any other public transportation. Sure, there can be exceptions, like stated above if there's a parent holding up a sleeping child because that can definitely affect their capacity of staying upright if something happens and there's a sharp turn or something.
Personally, if I want to sit down on a bus and I know the first one to come by is gonna be full, then I'll just wait for the next one.