Let me offer the same advice as others plus a slightly different insight.
I have a good male friend. We've been friends for over 20 years, long before I met DH. We've never been anything more than friends, almost like siblings, especially since I have only one sister and he's an only child. There were no sexual undertones to our relationship or anything like that as far as I could ever tell.
I'm married now (15 years), and the male friend has made a good effort to b/c a friend to my DH, too. The male friend has been in a serious relationship for about 5 years, and i've made an effort to be friends with his DGF. Things appear absolutely fine.
Even with that positive scenario: My good friend and I, because we value the fact that we are "with" other people, have been careful since my marriage and (perhaps even more) since his relationship began, to back off on the close and private side of our relationship (if that makes sense). We do talk on the phone once in a blue moon (he lives 7 hours away now) and email/facebook a bit more but everything else includes DH and his DGF. In fact, We know that somehow this is best. Men and women can be friends, even after one gets married, but the friendship must take, and must be seen to take, a secondary place to the marriage/serious relationship or it will bring friction, IMHO.
Anyway, your DH's relationship with this young woman has crossed boundaries just by making you uncomfortable. I think the "hanging out alone" stuff needs to end, also. That's a recipe for disaster, especially for a vulnerable newlywed.
Trust your instincts on this--they're guiding you correctly.
Any, just my take on things.