Don't worry about it too much. On our Magic cruise last March, on the formal night, some men wore a tuxedo, but more wore a dark business suit -- either of which are completely appropriate. From there, men's attire ranged from sportcoat-with-shirt-and-tie all the way to those who made no effort at all.
"Semi-formal" is one of those undefined cruise terms that means different things to different people. In the strictest sense, semi-formal is a different kind of tuxedo than formal -- but nobody goes by this definition. Some people really dress up, while others think semi-formal means casual. My advice? For men, just wear whatever you wore on the formal night. If you wore a dark suit with a white shirt and elegant tie on formal night, consider a blue shirt and sportier tie on semi-formal night. If you rented a tux, get your money's worth out of it.
Remember, you're requested to wear a jacket at Lumiere's and Palo, so you'll probably want to bring a sportcoat for that. I can't imagine bringing a tuxedo and a suit and a sportcoat, although undoubtedly some men do.