If your child is freaking out and screaming and crying, no, you might want to consider keeping them in that year. I had to do that once. Dragging my daughter around while she was shaking in fear just didn't seem like a good idea. Call me crazy.
The problem is that you're seeing the reaction to the outside of the building and then to people screaming. You're not seeing the reaction to the RIDE.
My son once had a very negative reaction to the LINE at Peter Pan. We'd been on the ride during that trip, he loved it. But the line felt bad to him that day, it was too early, we'd gotten too little sleep, and I think we were hungry, too. (we're much better at getting sleep and food now but this was early days) He loves the RIDE. That morning he really disliked the line. At the time DH and I were on the bigger side where getting out of a narrow line like that was incredibly difficult (downright impossible for DH), so we would have to go through the whole line anyway and then exit.
We did indeed cajole him into getting on the ride, and he did indeed LOVE the ride, just like he always did. But if you were far away from us where you couldn't hear us talking about it, you wouldn't have known that it was the line he was worried about, and would have thought we were awful.
The ride for HM is different than the line. Or the screamers. I have never once felt the urge to randomly scream at something that wasn't actively scaring me at the time, so I just don't comprehend it.
(DS went on WDW's HM once with his cousin, and did just fine. I was in the doombuggy with them and they were laughing. He likes the reality of the ride itself (though he doesn't think he does). He hates the idea of it, though, and the screamers. If only there were a way to just beam him into doombuggy, darnit)
...the last time we were there he let out the most horrific, blood-curdling scream right before people usually scream and it jumped the group standing in front of us who had been getting ready to scream too. We aren't screamers and I almost dropped him on the floor, it shocked me so badly. He was incredibly proud of himself.

That's pretty funny.
I read it to my son, who is 9 and still hates HM, and thought "hey, that's a good idea", but then I realized then WE would be part of the problem. (
you weren't, you were holding a small child who had a righteous scream moment...you weren't coaching him or practicing it) If a big part of my son's problem with HM is the screaming of fellow guests, and we become screamers, then we are absolutely no better than the people we want to be quiet. Darn, thought I had a solution there for a minute.
Heck,
I have to wear earplugs in the stretching room b/c people physically hurt my ears with their noise.
They essentially do force their children on and aren't willing to skip the ride or take the "chicken" exit even if the child is crying and obviously terrified.
It's good then to know that the chicken exit is pretty scary, too. First time the ride was open during our first trips we thought DS (at 2) could handle it. And then the other adults started screaming.
We tried and tried to help him along as we walked, but the adults kept screaming.
When we finally found a CM before seating, I asked for the chicken exit. It's a long, long LONG hallway. At that time (maybe now, I do not know) the walls were painted BLACK. There were no signs. It was a LONG, turning hallway.
Then when you're finally at the end of it, it lets you out where the FP machines are (were?) at Halloweentime, which is a very SPOOKY area if you're a child who was spooked by the screaming of adults and the dark line.
Worse? It didn't take that much longer for DH to ride the ride than it did for us to get out of there. It likely would have ended up being better to just ride it than walk all that way without any clue how much longer it would be.
But I think i've already established that I have a questioner who doesn't blindly follow what he's being told, which is great in some ways but very exhausting in others, like when you're trying to find your way out of a ride (and I should mention that I have problems in low light places, and that black-walled hallway wasn't what I would call well lit) and you have no idea where you're going or where it will end.
So in the vein of being prepared, be prepared for the fear to not end if you take that chicken exit!