I know you're annoyed and I would be too, but honestly, it sounds like your BIL just made an "oops" offer, probably thinking such a last-minute invitation would be rejected. I've been known to do that too.
Since you do vacation with them and generally enjoy their company, I'd be annoyed for a bit then rearrange your thinking to remind yourself you are still getting a pretty sweet deal to stay for free, even if you have to put up with folks you don't like for a day. You have five days to adjust to the idea, and I'm sure you will still have a great time!
