If they are true HP fans, then probably not. Much depends on the season. You have at least two major rides with no Express Pass, so at least 90 minutes of queue time total (again, you could get lucky and be less). Whether or not you want to ride the Hippogriff and dueling dragons coasters, that will add 45 mins to an hour. The train is probably a good 30 minutes. There are some stage shows to be seen. If you are going to get a wand, especially an interactive wand, that adds time. Two really cool restaurants, you will want to at least tour both and eat at one. Ice Cream treats, and enough time for at least 3 Butterbeers in between, oh yeah, and dozens of shops and at least 100 photos.
Again, it depends on season, wait times, and how immersive the desire is. Too walk around a bit, eat, have a Butterbeer, ride SOME of the rides, and hit up a shop, then yes - half a day is good. But at $175 for park to park one day tickets, take your time and enjoy.
For example, we are there for 8 days in June, we will spend easily 20 hours in HP world over the week - that does count riding the major attractions multiple times. And we are casual HP fans at best - my wife has not read one page or seen one minute of any of the movies - it is just that awesome!