I really don't think this is fair. You did ask for honest opinions, right? People are allowed to give their honest opinions unless you say you don't want any negative replies at all. I actually agree with him--I have had that same feeling when staying on Harbor. This is Anaheim, after all.
The difference is that you will be staying at the
Disneyland Hotel. So I can say (as someone who has stayed on property and off many times) you will be FINE. There is a huge difference walking home at night on Harbor and walking home at night through downtown Disney to the Disneyland Hotel.
So being that you are stay at the Disneyland Hotel, I think we can all agree that you will be fine and have a great time. I hope you post your positive review when you get home.
Take care,
Lisa