My rule of thumb is this - I personally think parental choice should be removed from this whole decision (unless there is a serious delay with whom the family is working with a doctor), but with that aside - my opinion is that I have no problem with people holding their child back AS LONG AS they are ok with them being taught to a year LOWER curriculum. As in, don't hold him back - let him learn to do a ton of things (start reading, start math, etc.) and then expect the teacher to 'enrich' him in the Kindergarten setting simply because he already knows all the stuff they are teaching. The teacher is supposed to be focusing on those whose ages are expected in the classroom based on cutoff dates. And honestly - if the parent wanted the child challenged - they should not have held him back to begin with - then he would be doing 1st grade work. You can't have it both ways. So, would you be OK if your son is way bored by the work next year if you decided to hold him back? Obviously he should be bored by it - he will have 12 months more learning than others in the class.
I think it can potentially take too much away from the kids who are aged appropriate for the class if the teacher is trying to teach to kids who really should be in 1st grade and at the same time trying to teach 4 year olds. And, I think it is unfair to the teachers too.
Like I said - a hot topic for me.