That you have to deal with all of this.. Not to sound "grim" but is your Dad's diagnois/fall "life-threatening" or is it more a matter of stay in the hosptial XX days, rehab XX days.. If his diagnosis is not life-threatening than I would go, as he is in the best place possible to be cared for, and I don't think you going is being selfish or should make you feel guilt-ridden, listen to those who love you most/know you the best, your sisters/dh.
Good luck with whatever you decide, I know it won't be easy.
Chip