I really think that may be more of a parking thing. They don't want non-resort guests parking at the POLY and then taking the monorail to MK. It's the same thing at the CR, GF (monorail resorts). When you pull up they ask for your ADR number or the name the ADR is under because people dining at the resort are the only others allowed to park there. Since you couldn't give that information, they may have thought you just wanted to park there for the day. I have heard the same thing about the BWI/BWV (Epcot resort). You need an ADR # or the name.
Now just VISITING a resort is no problem if you are using Disney transportation and not trying to park there. In fact at Christmas time it's encouraged by CM's just to see the decorations.
Plus lots and lots of people take the monorail to the POLY to eat dinner even though they may be staying at say, the YC. Lots of people do the same thing at all the other monorail resort restaurants-Chef Mickey's, Calif grill, etc. PLus lots and lots of people walk or take the boat from Epcot to eat at an Epcot resort restaurant (Flying Fish, Yachtsman Steak house...) or have a drink at one of the bars, look around, walk on the boardwalk, etc...
Are you saying this IS NOT allowed or SHOULD NOT be allowed?

It is most definately allowed, encouraged in guide books (for example, some books suggest taking the monorail to a monorail resort restaurant for lunch while at the MK for the day), suggested by CM's. For example, in Dec a CM asked me if we had taken the monorail over to the GF to see the gingerbread house...I said yes, two days ago. Are you saying we had "no business" going to the GF lobby and buying a gingerbread house-that I had to carry on the plane by the way???
I think you're mistaken and that the POLY's refusal had everything to do with parking and not actually "visiting" the POLY. Next time you're in the MK, take the monorail over to the POLY for dinner at OHana. I promise they will not kick you out!