I think the best way to refer to anyone is by their name...unless they ask you to call them something else.
But if you don't know their name, and you want to point them out to someone else, you do have to have
something to fall back on.
My son, 14 at the time, was stopped by a police officer. The officer began asking him his name, where he had come from, where was he going, etc... Then, a moment later, another police officer pulled up in a squad car and leaned out the window, saying, "Not that kid, THAT kid!" and pointed down the road at another boy.
Both boys were young teens. Both were tall. Both had curly dark hair and neither was wearing a jacket on that cold winter's night. However, there was one critical difference between them - my son is white and the other kid was black. Whomever had called the police had very politely refrained from mentioning the ethnicity of the young man, even though, in our town, "black" is something that sets you apart from 98 percent of the rest of the population (I looked up the demographics!). They probably felt it'd be rude to mention race.
With regards to the OP, "Little person" or "dwarf" works just fine. Though, I only use "dwarf" to describe people who are actually dwarves - ie, shortened limbs, average size torso and head (ie, the spectacular Peter Dinklage

). If I'm not sure, "little person" is the fall back descriptor. "Midget" has far too many negative connotations to use these days, although I've always rather liked the phrase "mental midget" as applied to certain politicians who will not be named here.
