I really didn't enjoy travel with my kids when they were very small. But I learned to let go of my 'old' vacation mind set (see everything in the most efficient manner and keep going until the place closes).
Give yourselves breaks. Fort Wilderness is a lovely place, there is a petting zoo there that small children would really enjoy. An go with the flow. If the kids get tired, go back to the room and rest.
Enjoy!
DJ