Travel time at Fort Wilderness can be time consuming and a lot of walking to get to the buses from your cabin. There are bus loops within the campground to get around and buses that take you to the parks and other areas. With your large party and a young one in tow that may be something important to consider. Many people feel that when they stay in the cabins it is "necessary" to rent a golf cart for the day to save time and energy. Keep in mind that Disney charges a hefty price to rent them per day ($65, with no weekly discounts). Outside vendors rent carts for less, but on the latest threads that I have read they mention that Disney frowns upon this and it isn't as simple as it used to be.
The plus side to the cabins is the remote privacy and outdoor feel. It's especially nice if you want to get away from the hustle and bustle.
We haven't stayed at AOA but I've been researching it a lot lately for ourselves. If you're into the Disney feel it looks like this place is ideal - especially for young children! There seems to be a lot of room in the suites. I've read good reviews about the food court as well. There are several videos out there to show you size and space like this one -
http://www.youtube.com/watch?v=4hbxKWPesBw
Another thing to consider is meals.
If you are planning to prepare some meals yourself you have a full kitchen in the cabins (just a mini kitchen at AOA).
If you are on the dining plan Fort Wilderness has both table service and counter service, not to mention the wonderful Hoop-Dee-Doo Review!
AOA just has counter service.
Lastly, there are a lot of recreational things to do at Fort Wilderness.
... and the pools at AOA seemed to be better themed for a Disney resort!