My family loves POR. We love the grounds, the food court, the pool and the boat to DTD.
That being said, we are starting to run into issues as the kids get older with 4 people sharing one bathroom. For that reason alone we considered AoA for our last trip. We ended up doing 2 connecting rooms at Pop because it was cheaper (and Grandma decided to come). Having 2 bathrooms was great. My husband and I walked around AoA one evening and the theme was fantastic. With 5 people I would lean towards AoA.
Also, I am not sure how familiar you are with the boat to DTD from POR. l think it is about a 25-30 minute ride to DTD with a stop at POFQ. I just wanted to mention that since you wrote about letting your sons have some freedom to DTD. The ride is not a short one. Also you have to disembark at DTD if others are waiting for a ride back, so you are not guaranteed a round trip sightseeing trip on the same boat. Also, if there is inclement weather, the boats are sometimes shut down. Your kids would have to be prepared to take a bus in this case. I am not saying that you shouldn't let you kids do this, I just wanted to give you some info about the boats so they would be prepared. I remember we were surprised how long the trip took the first time we did it.