Most countries require nothing but a passport. Since both parents must agree to the passport, the assumption is that any travel is approved.
This is not universally true. I recently completed a (Canadian) passport application for my son, and participation of both parents was "requested" but not necessarily required. Specifically, the form says this:
"
Both parents are requested to participate in obtaining passport services for their child and to sign the application form. The other parent may be contacted."
Note "requested" and "may".
The application also says this, under the heading of "Documentation to Facilitate Travel Abroad":
The following documents may be required to support the entry of a child to another country:
- Birth certificate showing the names of both parents.
- Documents pertaining to custody.
- If the child is travelling with one parent, a letter signed and dated from the other parent authorizing travel (letter may have to be notarized).
Not trying to dispute your knowledge of the US rules, but it's interesting that customs agencies in general would base policy on assumptions that may or may not be correct, depending on the issuing country.
Personally, I'd probably carry a signed letter. I don't imagine I'd bother getting it notarized, since the likelihood of even being asked for it is low. But a signed letter costs nothing, and if I *was* asked, it would probably at least be better than nothing. And like you said, it's the "usually" that you have to worry about.