We have stayed at both (although Alligator Bayou section at POR), and we love both resorts.
Here are some differences:
AoA: The suites have two bathrooms and three "beds." There's a queen in the bedroom, a murphy bed that pulls down over the table and the couch pulls out. There are two separate areas if the kids need to go to bed earlier - and also two TVs. AoA also has one bus stop, as opposed to many at POR.
POR: The Royal Rooms are cool, but you are only going to have one bathroom and two queen beds. The Alligator Bayou section has an additional bench/murphy bed, but a room there isn't guaranteed (you can call and request one though) - and those are a more rustic theme than the Royal Rooms. POR also has multiple bus stops, but you also have the boat transportation to Disney Springs.
Pools/splash pads: AoA has the Big Blue Pool, which is zero entry, as well as a Nemo-themed splash pad (which would be great for your youngest). POR has a pool and a slide (which AoA doesn't have), but there is no splash pad. However, you can also use the pool (w/a dragon slide) and the splash pad at Port Orleans French Quarter.
We love them both, and we tend to go back and forth between them...