To start off, we always get questioned at the border - not a blended family, but our daughter is adopted from China. She has always been directly questioned at the border going to the states, but never on the way back (I think because we all have passports which we use even when crossing in a car). I also get routinely searched when we travel by air - I guess they like my looks

. I also used to do some immigration law, and I wanted to chime in on two posts - just for information.
Actually, you could be kidnapping the child from your American Ex (not you, of course, just telling you what they are thinking). It does happen, unfortunately.
And to dancing Disney Style, I would suggest
always taking your letter, even just for the day trips. I say this because if they don't let you cross, they will likely note in their system that you were refused entry - and that will make for a HUGE problem on your next trip.
If it is noted that you were refused entry, that is grounds for a border officer to decide your are ineligible for entry (i.e. never able to cross the border or even make a connecting flight in the U.S.). They will likely ask on your next run if you were refused entry, and like most people you will say no (because you didn't realize that when you turned around that meant you were refused entry), and that will be considered lying to the border officer (also grounds that make you ineligible for entry). Depending on the officer you get, they can even confiscate your vehicle (yes it does happen).
This is a worst case, but not uncommon, scenario. As long as you carry the paper, you should never have a problem.