When I taught fourth/fifth grade, I had a family that wanted to take their kid on a
Disney Cruise for a week, and they were very concerned I would be upset. I responded with, "Wow, I am SO jealous! Go have fun!" I do think that once you get past elementary school the work is harder to make up, but for the younger ones...as long as it's not a testing week, and they are doing reasonably well in school, I say go for it.

They are only young once, and school is not the only place where children learn. Remember that parents are VERY important teachers themselves. Just discuss it with the classroom teacher ahead of time. We're taking our daughter out of Kindergarten for four days in December to go to WDW, and speaking as a teacher, parent, and current doctoral student in education, I have no qualms about it.