Obi's right that it can't hurt to have a letter.
It just dawns on me another place you might possibly encounter questioning is at the airline check in, as you leave on your trip.
when you check in, the airline ground agent will ask for both of your passports (since there is no outgoing government run passport control when you leave the US). It's possible that at this point the airline might ask you something. So perhaps the airline itself might be able to answer this question.
But it's easier to just come with a letter in hand already just in case. However, how would they know that it's your ex's signature? If a letter is required, it would seem a notarized letter would be required (otherwise you could just forge his signature).
Why not call your airline and the British embassy and find out.