Lol - that's how it goes. When is the party? I send out invitations less than 2 weeks before the party, and put the RSVP date a few days before the party (we have them out, so the venue wants to know).
Ds10's party was this weekend. 10 were invited. About 7 RSVP'd. Two who RSVP'd yes didn't show up. Two who didn't RSVP did show up. So, it all worked out. What I've learned to do was do a goody item that can be flexible (either stuff I can return, or something like a candy buffet, which I did this weekend, and it worked great).
This is VERY common with kids parties. Luckily, everyone dd10 invited was a GOOD GOOD friend, and I'm really close with the parents. Twelve were invited, 12 RSVP'd, 12 showed up.