What type of stroller do you have? Some strollers are just not that comfortable for kids. My first child I had a big Graco stroller & she always slept in it. For my second I switched to a Maclaren & could never figure out why he wouldn't fall asleep in it. For my third, I again broke out the Graco & he slept all the time in it. We made no changes to our schedule or his schedule when we were in Disney. We did all day in the parks and he would sleep soundly (like for 2 hours at a time in the stroller). We would put a blanket over it, lay him back & walk around. I would try to start training him now. Take him to the mall around his naptime & see if he will sleep. You may have to do this numerous times but eventually I bet he will.
When we only had 2 kids we tried to go back to the rooms for naps & it was a disaster. Our DS would sometimes fall asleep on the bus. We'd get back to the room and we would try to get my DD ready for the pool and with all that commotion he was just wide awake. Then he would just cry for an hour & eventually we'd just all end up at the pool. We'd then have have a really early night. It really wasn't a great trip. We felt like we wasted so much time either trapped in the room or trying to get back to the resort.
Honestly, I think if your child has to be in a crib for 2 naps a day, maybe it's just not the right time for Disney. What will your other kids do while he is napping? If you all go back, I would think he would never fall asleep. And if you or your DH just takes him back, that doesn't leave much family time together.