Thanks for all the replies.
All 3 of my dad's children were abused, and it all happened by different people (not our dad). So we're sensitive to it, and paranoid about it happening in yet another generation. I wasn't sure if we were just being too sensitive and reading too much into things, or if we were right to think this behavior was not normal. We don't know why our younger brother is acting this way, but at least we're not being completely unfair to him. I live in a different state than they do, so I can't go, but next Wednesday my older brother will be joining my younger brother in his therapy session and he will bring up his concerns then. Hopefully we'll have more of an understanding then.
Until we sort through this, my niece will remain very well protected by her father and he'll continue being extremely careful for the remainder of her visit.
Thanks again, everyone.