For years, my family used to go for the whole week after Thanksgiving, mainly because the crowds just disappeared! I'm not entirely sure about Sunday, cause we always arrived on Sun and weren't always in the parks, but your theory is correct -- most people head to the airport to fly home on Sunday.
Are you just considering spending Sunday in the parks or are you looking at a longer time that week?
actually we are checking into saratoga springs resort on saturday(probably will not be in the parks) and will be checking out on thurs dec 2. (I'm so excited!) We are going to magic kingdom on sunday and have tickets to stay that evening for MVMCP!
Last year we arrived on the Friday after Thanksgiving, but didn't attempt any parks until Sunday. Just from being around for those other two days, the crowds really dropped off by Sunday afternoon. There were probably still a fair amount who at least went into the parks for a few hours in the morning, but not bad at all.