My son tries to, but before we even get to Wizarding World he is usually taking off his robes. It is SO hot there most of the time. And those robes are hot; either the ones sold there (lovely fabulous $100+ robes) or the cheaper nylon ones sold elsewhere.
And carrying wands means you can't go on the big rides, because the wands don't really fit into the lockers.
And since DS has red hair and fancies himself as Ron, he's also usually carrying a plush Scabbers (though his Scabbers has been renamed Max because this is the "good" rat that Ron gets that JK Rowling didn't tell us about and probably doesn't even know about LOL), and that gets stuffed into the backpack as well.
Basically, go for it, but make sure the clothes you're wearing under the robes are good for a day in the parks, and have a backpack to put everything in once you start melting.
That said, if you go on a cold cold cold day, you will be warmer. Buy a $35 house-color scarf and you'll be good to go.