It also depends a little on which you prioritize more. For the last several years, Memorial Day weekend has been the kick-off to the summer season. With that comes longer operating hours and increased entertainment. Prior to that, the park hours are generally shorter, there is less entertainment (e.g., fireworks are only offered on the weekend), and a higher chance of ride closures due to refurbs.
Grad Nites typically start in early-to-mid-May and run through mid-June. They don't impact the operating hours of the parks because Grad Nite is an after-hours event. However, it can impact the crowds in the parks during the day because the attendees have park hopper tickets. Some folks said they'll go on a Grad Nite day again, while others didn't think it was much different than any other summer day. I personally wasn't that bothered by the Grad Nite groups; there are good groups and bad groups on any day in the parks.
In terms of crowds...I've been there Memorial Day weekend and I've found that the weekday (Wed and Thu) leading into Memorial Day were pretty manageable. The weekends were busier, but still manageable. I'm curious if that will be the case this year, or if the pattern of less busy weekends and busier weekdays will continue to hold. The changes in pricing structure (peak versus regular), AP blockouts, special ticket offers (e.g., Military or Southern California), and special events can all have a major impact on the crowds. It seems like (lately) the weekend have been pretty manageable, whereas the weekdays are busier than expected (at least during Spring Break). So it's hard to say whether the trend will be similar going into the future or if it'll change by the time you go.
Also it's worth noting (regarding APs) that there are two levels of Southern California APs. The SoCal Select AP summer blockout starts in early June and the SoCal AP summer blockout starts in late June/early-July. This year the blockouts start on Jun 9 and June 30, so I would expect similar dates next year. They've already lost most Fri & Sat by the time the full blockout goes into effect, but I would expect a bump in attendance the last few days (to week) before the blockouts go into effect.