The far building in Trinidad South is the farthest you can possibly be from the main section of any resort on Disney property. It puts you about a mile and a half from the front desk, and about a mile from the restaurants if you're in the far building (or at least it feels like a mile )
May I recommend the Jamaica section (Building 45 specifically)? It's where Scott and I were both staying when we met in 2003. It's just across the bridge from Old Port Royale, so it's close, but since it's across the lake, it's not as crazy or loud. The best of both worlds.