Yes - the late night and the early morning extra magic hours are free to resort guests as long as you have a park ticket for that day. You can't use a water park plus and then do the extra magic hours or anything like that.
Honestly though - you really want to avoid the park on extra magic hours. Or, use the early morning extra magic hour and then stay in the park until it starts to get crowded - then hop over to another park. The EMH's funnel park guests into those parks - meaning greater crowds. We did the MK extra magic hours in Late June and it was an absolute zoo. We were literally pressed into the exiting guests as we tried to enter. We only rode 5 rides and we stayed until the 2AM closing. It was pretty cool to be in MK in the middle of the night. But go for the atmosphere - not for the rides.
Best time to go to the MK? The morning after a late night EMH. Check the EMH schedule and go to the park the day after the EMH.