Everybody remembers him as a quiz show contestant but, in the 1960s, I saw him on Broadway in the musical Skyscraper, with Julie Harris and Peter Marshall, and he was quite wonderful. He was also memorable as the "villain" on the TV version of The Ghost and Mrs. Muir. In fact, he was probably the only memorable thing about the show.
And yes, he was quite and accomplished Broadway director, which also gets lost in the shuffle.