Agree with the others - do park to park and try to do two days if you can. You could probably swing one day if it's the only option, but you'd be rushing and inevitably would end up waiting in one long line - be it Gringotts or Forbidden Journey. Neither have express pass, so we always do one at rope drop on day one and the other at rope drop on day two. And you need park to park passes for Hogwarts Express which is a must do.
As for the rest of the parks, as two adults, we always spend more time at Universal than IOA. The rest of IOA doesn't take us long at all.