Honestly, I think it would depend on how much is academic side/how much physical if that makes any sense to you.
My older son physically doesn't have any issues but his are academic (I felt guilty enough when we went last summer - he ended up missing 2 weeks of summer school - which they fully are fine with missing, they just ask what dates you are going to be gone. Had a progress report at the beginning of the school year and his resource teacher said he was just now getting back to where he was at the end of the school year last year!! I felt that was partly my fault for him only getting 1/2 his normal amount of summer school!). Hence my hesitation to take him out where it's regular school (he's in 3rd this year so I know things will just get tougher) and he has all his subjects to deal with not just his trouble area.
ON THE OTHER hand....my Kindergartner has an IEP for physical stuff, he gets speech therapy & occupational therapy and needs support in that way. He didn't qualify for summer school because academically he's fine (so far, it's hard to tell in Kindergarten, 1st grade is where academics stand out more at least here). He gets summer speech though but us missing that for 2 weeks didn't even affect anything.
So, the answer to that question is, it depends on what the IEP is for and how intensive. I can't imagine one in Kindergarten is going to be that intense that missing 5 days is going to have much of an impact either way.