The Customs and Border Patrol (CBP) site says this:
Adults traveling in or out of the U.S. with children under the age of 18 should be aware of the following: because of increasing incidents of child abductions in disputed custody cases and as possible victims of child pornography, Customs and Border Protection (CBP) strongly recommends that unless the child is accompanied by both parents, the adult have a note from the child's other parent (or, in the case of a child traveling with grandparents, uncles or aunts, sisters or brothers, or friends, a note signed by both parents) stating "I acknowledge that my wife/husband/etc. is traveling out of the country with my son/daughter. He/She/They has my permission to do so."
(emphasis mine)
While it's likely that no one will ask for the note, I think it's better to have it than not simply because the CBP recommends it.
-Dorothy (LadyZolt)