It was just my husband and I with our 2 year old last summer (and this one), so a very similar trip! 8am-11am-ish is the beautiful golden time - cool weather (often cloudy from the marine layer), low crowds. Fantasyland especially is wonderful at that time of day! We did then take a midday break around 11:30-12 after an early lunch (avoiding lunch crowds), and then headed back around 2:30-3:30ish. It was definitely more crowded then in general, and noticeably the most crowded on the grad night day. But since we got most of the rides done in the morning, we stuck to low/no wait time things (train, carousel, tea cups) in the afternoon and we were just fine. We plan to follow a similar schedule this year.
The actual grad night event takes place at DCA after hours, so it’s likely that DCA will have larger afternoon and evening crowds than DL. Schools have to pay for more expensive packages to do both parks.
My uneducated guess is that May grad nights might also be less crowded than June. CA schools start getting out in late May, but most of them get out in June. I see most schools do grad nights closer to when school gets out. But I don’t know for sure, just my hunch.
If you’re mostly worried about AM crowds, you’ll be just fine.
Disneyland is a magical unicorn in that sense - we were shocked by how crowded rope drop time is at WDW!! It’s very different.