As long as his name is on the birth certificate and there is no court order to the contrary, he has just as much right to spend time with the child as your daughter. She is not breast feeding and there's no reason that that child shouldn't be allowed time to bond with the father--who you should be glad WANTS to spend time with the child.
My brother and his ex split up just after my niece was born, and my brother has had her for overnights since she was about six months. In fact the court very much sided with him on the entire custody issue, as he was more stable in a lot of ways. He gets her two evenings a week and at minimum every other weekend. He also gets her for Thanksgiving, Christmas, and Fathers Day, plus the weekend of our annual family reunion, which is usually the first or second weekend in August--even if it's not his "weekend." Here's the best part, she isn't allowed to move with the baby more than 15 miles from his house. So she's stuck. Period. He did allow her to move 16.3 miles away, but it's in writing that this is a one time deal, and not to be construed as a permanant allowance or revocation of that clause in the court order.
Also, they have joint physical and legal custody, and HE is allowed tot ravel outside of the State or even country with her by simply giving notice to her mother. her mother must give notice and get his approval before doing to.
All this because the ex was being a jerk at the beginning about letting him see her, so he got a very good lawyer and the mothers actions were brought up in court. It looked like she was being an unreasonable witch, and the judge was very sympathetic to my brother because he was being reasonable. The judge gave him more than he was asking for.
So I'd be very, very careful, you are walking a slippery slope. Unless there is a VALID reason for the father to not spend time with his child (he lives in a crack house, sells drugs, is an alcoholic, as a record as a child molester, etc.) then there is absolutely no reason for him not to. The best interest of the child trumps those of either parents, and unless there is a compelling argument otherwise, it is in the childs best interests to have a relationship with both parents. Your daughters "feelings" are simply not important as far as the law is concerned.
And for those posters who think six months is too young for overnight visits, or it's too far to travel... What if the OP wanted to take her daughter and grandchild to WDW? Would it still be too far to travel? My guess is you wouldn't think so. And if you had a six month old and needed to go away for a weekend, would you leave your child with your husband? I can't imagine most of you saying no. So how is this ANY different?
BTW OP--if God forbid something were to happen to your daughter, you do know that full legal and physical custody of your grandchild would AUTOMATICALLY go to the surviving parent, and unless you can prove him unfit, you wouldn't have a chance at custody. And he could even deny you visitation and potentially win. It is in everyone's best interests to allow him the occasional over night visitation he wants, and to keep at least a civil relationship with him.
Anne