We've stayed at both POR and CBR, and while we liked both resorts we still prefer POR. We like the theming better, but CBR was great, nothing wrong with it at all, just a personal preference.
I would check out all of the Moderates and see which atmosphere you like best. We have stayed at all of them. Caribbean Beach, by our stays has been the favorite for us. I just like the happy island atmosphere there.
Port Orleans does not have elevators either (at least in the bayou section). I like the beach setting a bit more than the rustic bayou, so I liked CB slightly more than POR. Both are wonderful, it really depends on what theme you'd like the best.