Used to live in SoCal (including San Diego - is that where Cabrillo beach is? - I have never heard it called that, but Cabrillo Monument is out on Point Loma and I have been there). Personally, I would never go to the beach in SoCal in Oct or Nov. The water and air temps are too cold. If you are a surfer and have a wetsuit, that is different.
If you want to walk on the beach or visit tide pools, then you can do that year round. But laying out on the sand and building sand castles - I wouldn't do it. I would save those activities for May-September (better yet, June-August).