If you are planning on visiting all of the parks and waterparks or going to Universal or Sea World, then I don't think you'll have enough time. It's about 75 minutes to Kennedy Space Center and then you need 4-5 hours touring the center, looking at the exhibits, etc.
But if someone in your family is really into the space program, then take the time to go - it is worth it.