I've asked at the front desk at check in or as early in the trip as I think of it. The one time they told me no asked later in the trip. They can probably only say yes to so many guests leaving the same day. We do the same thing, we like to go to a later breakfast on check out day and be able to return to the room afterward, not be rushed etc..