What a bummer! I've done the HEA dessert party twice and had wonderful experiences both times! The first was last summer. I was by myself and got there plenty early because I was by myself and didn't have to wait for anyone

I got a seat right by the railing facing the Plaza garden area. CMs made me feel at home, and told me to take my time. I headed down to the garden early to snag a good spot. I was in the front right by the railing. I didn't realize until it started that I was right behind a small obstruction, but I sucked it up and stayed there because I didn't want to get in anyone else's way.
The next time was in October with DD15. It was a Saturday night between MNSSHPs so I decided we should do the dessert party, knowing it was be crazy. Boy oh boy was it! We ended up being one of the last groups there. The usual seating area was actually full, so they seated us in the back section of the terrace viewing party area and we went to that buffet. Never once were we told to rush and get down to the garden. In fact, they've always told us to take our time and head down whenever we're ready. The second party (October), it was crazy getting down to the garden. The CM leading us down did his best, but half of us got separated. Luckily I knew where we were going so I guided everyone where they needed to go. The garden area still wasn't crazy, but this time DD and I decided to go towards the back. I liked it back there because I could lean against the fence. I will not sit down because I don't know if I'd ever be able to get back up lol. Funny thing at this party though, there was a family of 3 that decided to hop the back fence to get into the garden viewing area. I was kind of in awe that someone would do that, but as they were handing the baby across a CM came over and told them (very nicely and calmly) that they could not sit in the garden. They yelled, they cursed, but they left. They knew very well that they weren't supposed to go in there, and I was happy a CM came and asked them to leave.
I'm sorry you had a bad experience. I'm surprised, too. Hopefully Disney will get in touch with you and make it better. I do highly recommend doing a dessert party during October between Halloween Parties, though. It was a madhouse that night!