As a WDW fan since birth and a Potter nerd since book one I can absolutely say that I thank the heaven's that Disney did not get the rights to Potter for the very reason you stated and I was not alone in that fact. When JKR tossed out the idea of going to a theme park to the fans, everyone was excited but the largest cry was "just please not Disney!"
The reason for the outcry on that was simple..."JEDI MICKEY". While the majority of the fandom was positive that Jo would never ever allow such a thing, there was very real fear that if Disney got the rights that our world, our stories would be decorated up with mouse ears and fairytale princesses.

It gave me shivers just even thinking about it and not happy shivers.
Thankfully that didn't happen but for a good while that was a very real fear and the Potter fandom was very much against Disney getting their hands on the rights from the get go. Not that Disney can not create a quality product, of course they can, but it's on their terms and with their "magic" and Potter has it's own magic, it doesn't need Disney's.
But I am sure you are right. If Disney had gotten Potter, all the "but Potter is fad" stuff and all the "it's nothing special" stuff often spouted here would not be happening.