See I didn't mean in that way. But I believe Pegasus928 explained it better than I would. It helps to be tolerant and understanding if you know the circumstances.
Anyway, your last comment can also be seen as extremely ignorant and offensive.
Let me tell you a story of what happened to a friend of mine who gets that comment at times and every single time it put her close to depression.
She and her DH had been trying for years to get a child. After several treatments of IVF and going through some really rough times, they had given up. About 2 years after giving up, she got pregnant. They were obviously extremely delighted. However, she went into early labour and the baby was born at 27 weeks (or thereabouts), so for weeks her baby boy was in intensive care and for a long time it really was a matter of touch and go. After 3 months in hospital, they finally got to take their son home, but the struggle wasn't over yet. He got ill very easily for a long time, was obviously behind in growth, had trouble learning to walk, and they still didn't know if he would have any permanent brain damage as he was starved of oxigen at birth. At age 2, it looks like he made it through the worst and is doing well. Then 5 months ago, they're in a car accident and their son died. Can you begin to imagine what they have gone through??? Can you begin to imagine how much it hurts for her to get the comment "you'd understand if you'd have kids" given to her by perfect strangers???
People with kids often don't realise how painful it can be to say such a thing to other people. For all you know they might be desperate to have kids and such comments only rub it in that they don't have any. Or worse, they might have lost a child.
Maybe a bit of tolerance is required towards people without kids too?
then I remembered my screen name!
Do you honestly think we've never spent time with infants? 
