From the time my guys were very little, we've gone away for weekends without them. They've stayed with grandparents or friends, but we've worked hard to spend time alone together. Partly for me, as I just NEED to get away sometimes, and partly because I've thought it will do them good to know that they can rely on others and themselves (ie., each other) and be comfortable even if mommy and daddy aren't there. Other than one time with Miss Zoe, they are typically mad that we're back so early.
I'm sure your guy will be just fine, and will enjoy the time alone with whomever he's staying with. It will probably be much harder on you than your DS.