We have stayed at Port Orleans-Riverside, and Caribbean Beach. Personally I found CB much more kid friendly- they loved the beach to play on. We had a long walk to the PO pool too, so it wasnt very convenient. But honestly, we just didnt spend much time at the hotel on either visit, so Im not sure it mattered too much to us.
Eva