The kids are definitely getting wands. I think I'm getting one too although don't tell the wife

Unfortunately that means at least one early morning to use the wands before its busy. Probably have to do a breakfast too at the park as none of us are eating the food they serve at lunch and dinner at the harry potter places.
Agreed on the pools as well. I have a pool and the community has a pool. That doesn't mean my kids don't want to go swimming.
So like assuming you have the express passes and just hitting each ride once, dragon challenge will close so its just the hulk my son can't go on, how long does it take to do all the rides in one of the parks? I'm just trying to get a gauge on whether we will be express passed through everything in 2 hours or is it quite a bit longer.
While I'm sure you will want to use the wands a bit as soon as you get them...Shhh... we find that generally the best time to use wands is at night. For one thing, the crowds aren't as bad. Second, well, the whole thing is just way cooler after dark.
Also, well, you might not get picked the first time you go through Olivanders. At that point, you can buy any wand you choose, or try again. Lines for Olivanders tend to be long in the morning, and almost nothing later in the day. Once you have a wand though, there's no (real) reason to go through Olivanders again. So it isn't always bad to make a second try, maybe trying both shops in the process
No, you won't be able to do all the rides in 2 hours. For one thing, it takes time to walk to each attraction. Express pass is a tiny bit like WDW's Fastpass, in that you still go through a portion of the ride queue, and in some cases the pre-show or part of the pre-show. Also, you may well want to do some rides twice, like MIB. Who doesn't want to try for a higher score?
Beyond just the rides, there's a great many details to see everywhere. Some are funny, some are photo worthy, a great many are interesting, but some stuff might not interest you at all.
The amount of time it takes to cover the parks is going to be widely variable. It depends on a great many variables. Before HP, we could cover most of what we considered to be the main attractions in one very full day with express pass, but that was far from seeing everything. With 4-5 days, we still don't see EVERYTHING, but we DO ride a number of rides multiple times.