I do think there is some truth to how you as a couple work out this situation in terms of compromise and relationship for the future. No reason to be mean to them but no reason either to let them take over the entire trip if this truly upsets you.
For how your bf might feel wanting to spend time with his family it might have made more of a difference if this was discussed 13months ago where you might still feel like they invited themselves but at least planning-wise things would have been different. You could have set it up where you had your time with your family and time with his family all together too. Alas that ship has sailed so I get not wanting to just scrap everything or to try and stress yourself out accommodating them. Regardless of the location and having been there before it is still your vacation.
Are you staying for any nighttime entertainment? Obviously with a baby that may not be feasible. That might be a way to spend time with them (and have an excuse of being tired for the night afterwards lol) without interferring too much with other plans (unless you had dessert parties booked or ADRs booked right around fireworks). Or maybe a parade if you're into that though that impacts more of your plans. {ETA: I missed that the baby's bedtime was before 7:30pm so yeah nighttime entertainment would probably be a no go on that}.
I totally get wanting to just do the trip the way you've got it planned. I wouldn't really change any of the plans at this moment personally but I would at least leave it open to spending some time with them if the plans work out--basically don't try to avoid them entirely if you really can spend time with them lol. That might appease the masses.
Also if you end up finding out they are available for a time period depending on the park and your plans maybe you could arrange for them to go on some rides with you if the wait is short enough (you could discretely look at the
MDE app for the wait time listed to at least get an idea before voicing it out loud). Like it's a small world for instance where one of his family members could be sitting up with you (or your bf) with your baby. IDK thinking about having them included in a meaningful way if it worked out.