These point swap transactions are interesting. In my view, these are not really pure bartering transactions because there is cash involved in the transaction and you never really relinquish your points. Essentially, you're renting your points and agreeing for the cash proceeds to be paid directly to the broker who, in turn, uses that to purchase your cruise. So, all you did was rent your points (and have rental income) and then purchased something with that cash. What the brokers have apparently decided is that, because the cash is never paid to you, they have no 1099 reporting obligation. And, if the IRS never receives a 1099, they have very limited ability to ever discover the transaction. No bank account statement, no credit card statement - probably some e-mail confirmations. But, unless you find yourself the subject of some intense audit (which is unlikely to happen absent some sort of very complicated tax situation), the IRS will not be going through your e-mails.
TBH, I think the IRS would be more likely to inquire about this if you attempted to report the income than if you simply never reported it all.