I'm going to say at the outset of this comment that I am not going to speculate that it was Charles, or anyone else, because I feel based on what I saw that Harry never meant for this to become a witch hunt; I truly don't think it occured to him that Meghan would share that, or that if she shared it, that the press would insist on finding out who said it. Does that mean he (again) wasn't thinking? Yeah, probably. He seems to be a mostly kind and caring person, but also perhaps not the sharpest arrow in the quiver.
I look at the words Harry used: he said "awkward" and "a bit shocked", but not "appalling" or "absolutely horrified". That sounds much more the way you describe an encounter with kindly-meant ignorance, not overt hostility. Ignorance in a kind person is easily remedied if one has patience.
Yes, there is an equally strong possibility that someone might have said it to be catty, or because they really are the kind of racist who indulges in colorism, but as what we have is purely hearsay, what I have seen so far does not support any need to put a head on a pike.