Hello and Welcome!
I'm a bit of a newbie myself, but on my first (and only) trip to the World, which was October 2005, I went to Mickey's Not So Scary Halloween Party with my 5-year-old daughter. We had a blast! We both dressed up in costume (Minnie and Cruella), and it was so much fun to see lots of other guests costumed too. It's a great conversaton starter. DD was not into the parade, so we used that time to go on Splash Mountain several times in a row without a wait. We did a little trick-or-treating, but again, DD was much more interested in the rides, so I didn't push it. Also, she is a very brave kid, so we went to the Haunted Mansion on "Halloween night" and it was so special to be there at that time! So much fun! We both loved the fireworks at the end.
I would say the magical atmosphere is definitely worth the price of admission and I would recommend going.
Two tips: It was rainy when we were there, so it might be worth investing a transparent rain poncho so that people can still see your costume if it starts to pour.
Also, no matter what your costume, wear comfy shoes!
Happy Halloween!