You are NOT being too dramatic! I would absolutely be upset. Is your DH afraid of his own child???
Maybe my DH is an exception, but that would not have happened at our house. Now, I'm not saying our kids *never* ended up in day care while I was working and he wasn't. If he had a doctor's appointment, or a full list of errands, or a home improvement project going on, then yes, the kid(s) may have spent time in day care on that particular day. However, it was almost never the whole time. Maybe the baby got dropped off in the morning, errands run, baby got picked up. However, if my DH had had "free" afternoons all summer, at least half the time (likely more) the kid(s) would have been picked up.
DH typically has had more vacation that I do. Our kids are older now (8 and 11), but DH took three days off the week of Thanksgiving last year to hang out with the kids.
I think you should have a heart-to-heart talk. Maybe he just doesn't know how to play with the baby? You don't say how old your little one is. Some men just can't connect until the baby can smile, crawl, or something. He's missing out on precious bonding time that he may wish he had back someday.
Maybe your DH needs some time out. Yeah, I know, we Moms hardly ever get it, but maybe he sees that as his "guy time." He may need to find other "guy time" one evening a week -- go play softball, bowling, whatever.
You are NOT overreacting. Your DH doesn't necessarily need to spend every minute off of work with the baby (though again, we moms are expected to), but he should be spending a lot more of them with his child than he is. He's a FATHER now, and that entails extra responsibility.
Good luck.