I actually had breakfast in the resturant by myself when I was there last March, it was early on a Monday morning, and virtually no one was there. This was shortly after sunrise (around 7:30a I think) which I got up to watch fromt he "dock."
they also sell some breakfast items in the general store, for you to take back to your room.
Have a wonderful trip!