I think the best spot to catch the headless horseman is just over the wooden bridge along the parade route. I believe it was in Liberty Square near the Sleepy Hollow Refreshments that we were. On the opposite side of the street there is a wooden railing, great to have to lean on while waiting for the parade.
Hearing the horse coming was awesome, but only got better once it was coming over the wooden bridge. Very nice touch to add to the mood. I wouldn't want to see him in any other spot, wouldn't be the same.
I believe he does come down the whole parade route though, but I would highly recommend you be on the other side of the wooden bridge to get the most out of hearing him come.