If you're going at a slow time and ask the lifeguard on duty, you may be allowed to. I wouldn't try it at some of the more popular deluxe resort pools (Stormalong Bay at Beach Club/Yacht Club would be a definite NO, as probably the Poly would be). But if one of you is staying at the All-Stars or Pop or even a Moderate like CSR, I think the answer would likely be yes. Resort IDs are checked more often at the deluxe resorts than the others.
But in general, as stated above, pool-hopping is not allowed. If you're the guest of a resort guest, you might get permission. The policy is in effect so that all pools are not wide open to all Disney resort guests at will.