From everything I've read, it seems like others drop their party size pretty regularly and don't have a problem with it. From personal experience I too had a party size drop and just called and spoke with a CM. They had a customer service rep make the change to the reservation and all is well now. I think either way, you should still plan on going. I seriously doubt they will turn you away when you have a reservation.