We've stayed at both Riviera (we're owners at Riviera) and Portofino Bay. I feel like the theming of Portofino Bay is more "old world" Italian, especially the exterior areas. We enjoyed wandering around and exploring the various nooks and crannies of the resort. Lots of nice details in the architecture. The room we stayed in was nice and getting Express Pass at Universal is a great benefit.
Riviera has a more modern feel. We enjoy all the artwork on the walls with subtle Disney content in a variety of artistic styles. I would describe the rooms as elegant.
If planning to visit Universal, I would probably opt for Portofino Bay which is walkable to the original parks (plus getting Express Pass). We also really enjoyed staying at Royal Pacific on a previous trip to Universal.
If going to WDW, I'd go with Riviera. It's a long drive from Portofino Bay to WDW and you'd have to pay for parking since you're not staying onsite at WDW. For WDW, staying at Riviera offers the skyliner to two parks (or free parking if driving to the parks).
