Although I have never eaten at Olivia's, I have stayed at OKW several times, twice in November. I really don't think you will need reservations, but it wouldn't be a bad idea to make them anyway, that way you would have a guarantee. However, if you chose not to make reservations, and if by chance you found it crowded, there is counter service available, and dining at the picnic tables by the marina is certainly not a bad thing!