I think so. I'm going to a wedding in September that's at 3pm and I'm wearing black. So is the wedding party for that matter. (Also, so did my wedding party and I had a 3:30pm ceremony.)
I did to the last two I went to, and so did a lot of other women. I think it's okay. As long as you don't wear white, or a really super bright color that distracts from the bride.
I think it depends on how dressy the dress is and the type of wedding reception you're attending.
I am going to a wedding this Sat. and only had a very nice black dress, very elegant. The wedding is at 3p and the reception is at 4p at a local hall, not casual, but not elegant either. I opted to buy a new dress last night as I thought I'd be overdressed in the black dress. The dress I bought was just right~ not too casual, not too dressy!
I think you'll see a wide range at a daytime wedding anyway and if you like your dress, it doesn't matter what the color is!