You could, I suppose. I'm not sure I'd do that, though. While it doesn't amount to much, it does translate into a little less of a tip for the server. (Of course, if you're a generous tipper, then you can feel less concerned about that, but of course the server doesn't know you're a generous tipper until afterwards...)