You won't know what your child will do until you get there. We took our son to the zoo when he was a year old, and he was more interested in the people than in the animals. Even at the petting zoo, he wanted to play with the other kids, instead of petting the animals. At first I was disappointed, but then I realized that he had a good time, and everyone had a good time watching him, so that made it a successful day.
As long as you decide that your trips to DL may end up being a little different than when you went without a child, and that it will be fun to have a new adventure, you should have a good time. You do need to look into rider switch passes, if you want to go on rides where your child doesn't reach the height minimum, and you probably should find places that people like to use for nap time. And if you try it one time, and decide it isn't the best place to take your baby, at least you tried! You have a AP's so you won't be out any money, so I would go for it!