if he enjoys seeing his family, and you've not been one to join in (are you this way with your side of the family?) perhaps in the back of his mind there is the concern that once the kids are adult, move on to their own lives and CAN make their own decisions they will choose not to visit both of you. he may harbour worries that ds will marry someone who is unwilling to facilitate visits that are'nt easily convenient, and if dd is begining to show your tendancies-that she will make no effort.
it will feel very different when you are in the place of the relatives that long to see their children/spouses/grandkids.
my bil has always had a very close relationship with his boys, but you could tell at the very few extended family get-togethers (on his side of the family) that he was not thrilled to be there, basicly when the kids were little the only time gma/gpa got to see them was if they traveled to one of their school events or they were needed as babysitters (they spent more time with-but not much, his dw's family). bil is now having a very hard time adjusting to the fact that his boys, one about to be married, one very involved with a young lady are spending ALL their time with those young women and their families. he is confused why the "kids" are not taking time from their lives to come for dinners, visits, to spend every holiday. he fails to see that this is the way his kids were raised-that "immediate family" is where you spend your time, that once you reach adulthood-your parents and sibs are not much more than those you send a holiday card to (if that), but certainly not people you would consider "wasting" time to go see.
i don't feel that spending time with a family member should be done out of a sense of obligation, but there are times when we put others needs and desires first-if the grandparents, uncles/aunts want to see us or our kids we make an effort, and our kids know it won't nesc. be the most fun (they are the ONLY grandCHILDREN-all the rest are adults, so no chance of cousins to play with)-but they know these people love them and the little opportunity we get to spend time together is special to them.