We stayed in a pirate-themed room this past summer, actually. We really enjoyed the rooms, but they are the farthest from EVERYTHING at the resort. We had a room in the second-farthest building in our section and decided to time how long it took us to walk from our room to Old Port Royale to get a pizza and pick up our packages. It took us about 10 minutes from walking out our door to get to the food court, order the pizza, and get our packages. OF course, then we had to wait for the pizza, but that gives you an idea of how long of a walk you will be looking at.
I love both POR and CBR. I think that the pool is definitely better at CBR. It's absolutely huge and has a more detailed theme, in my opinion. There are multiple water slides, and just a lot of fun areas. We also were able to see the Epcot fireworks from outside our room one evening, and that was really great. POR, though, is just beautiful. We have a girl and ended up staying in one of the Royal Guest Rooms which was very nice and in one of the closer buildings to the main one, but with a boy, I'm really not sure that would interest you. If you are planning to just stay around your resort on your rest day, I'd probably recommend CBR. You can spend a ton of time at the pool and Caribbean Cay. Also, your son might be interested in doing the Islands of the Caribbean Pirate Cruise (in the mornings) that they have there or pirate camp (in the evenings). With Pirate Camp, you and your husband could go have a nice dinner for the two of you while your son is learning how to be a pirate. They also have fun activities in the Old Port Royale Courtyard during the day such as tie-dying or arts and crafts. You also have the option of renting bikes or going fishing.
Now, Port Orleans Riverside does have some of these, as well. While they don't have pirate camp, they do have a pirate cruise that docks at POR and then heads to POFQ, Saratoga Springs, Old Key West, and maybe Disney springs (can't remember for sure). You can also rent bikes or go fishing here. For me, the pool just isn't as much fun. They also don't have the arts and crafts, but they do things like bingo and hula hoop games. It's nice to be able to take the boat to Disney Springs, as well, and while there, you could play at Splitsville, take a ride on Characters in Flight, or enjoy one of the fun dining experiences such as Raglan Road.
In all honesty, I think you will have a good time with either choice. I go back and forth on which one is my favorite moderate.