No matter what, I still have you beat with what happened at DD's last birthday party.

One kid's mom never bothered to RSVP, so I assumed she wasn't coming. The girl and her mom walked in one hour after the party started....and it was a party with an activity at a venue, so walking in that late threw a major monkey wrench in things. (Thank goodness, I had an extra goody bag.) The mom asked when she needed to pick her up and I said, "In one hour, at 5:00." Mom left.
Mom never picked her up. Never. Ever. The poor party venue lady had to stay open an hour and a half late while we waited and waited and waited. We couldn't call because they'd recently changed all their numbers. The little girl claimed she had no idea what the new numbers were. We suspected (and DD confirmed later) that she was lying because she preferred to stay with us rather than go home.
Finally, after waiting an hour and a half, we decided to find their house and hope someone was home. Mind you, we had no legal right to take the kid in our car, but what choice did we have? (DH wanted to call CPS.

) Family had driven in for DD's birthday (90 minute trip each way) and our dinner out was delayed, so now they had an even later drive back home. Major PITA. We drove up to her house and there were FOUR vehicles there.
They had never even noticed she was missing. The front door was unlocked, so DH just walked in with the kid and no one even noticed that either. He was so irritated that he just left. So from all we observed, they would have taken HOURS to have figured out they had neglected to pick up their kid. No wonder she wanted to stay with us.

At least we notice when a kid is AWOL.
In your shoes, I wouldn't feel guilty. You made an honest mistake, corrected it, and if someone failed to phone you, they walked into what happened. People need to learn to RSVP.