I don't think it's weird at all, especially knowing your different vacation styles. We travel often with family members (though not to WDW), and have stayed at different hotels before. It's usually a matter of $$$ (we got a great deal on a certain hotel, but my dad didn't, so he stayed somewhere cheaper nearby). I would say it's not as convenient, but at WDW that isn't really true since you wouldn't be doing all the same things at the same time anyway. As long as make a few plans together (ADR's, shows, same parks/same days), I think you'll be fine, and everyone will enjoy their vacation more, knowing they can just go on their own schedules. Really, with cell phones these days it's so easy to hook up with people and change plans I don't think it matters where you stay.
In fact, you might have fun taking breaks at each other's resorts. You could get a piece of tie-dyed cheesecake, and your DN's might enjoy the pool slide at POR (I don't think that would be considering pool-hopping, would it?).
Go for it, and have fun!