I only planned 2 park days at MK the first time with a 2 and 6 yo. I left our off day open - we slept in, swam, had lunch, napped, and then went to MK, which our 6 yo requested (more MK!). I think that as long as you leave that day open and "read" your kids (do they want another day at MK or would they rather just hang out all day?), you should be good.
There is stuff for small kids to see at the other parks, but you could consider skipping one if it doesn't appeal to you/there isn't much to see for small kids. I personally wouldn't skip Epcot, because I love it, or AK, because everyone in our family loves animals, but I could have skipped DHS even with the PHD characters.