We have planned 3 nights & 2 days in San Fransisco and a beach day in San Luis Obispo.... we love silly roadside oddities as well as more traditional stuff.
My family and I love to take road trips, and have driven cross country twice stopping at fun, touristy places along the way. I use this website to get information on quirky places to see and things to do:
http://www.roadsideamerica.com/location/ca
These aren't the sort of places you'd consider as a destination trip, but if you are passing by already, they are neat to see. You can search by state and city to plan stops along your route. Have fun!
Oh, another place I've always wanted to go is The Tonga Room in the Fairmont Hotel in San Francisco. I've been to SF and done the typical touristy things, but never made it there. It looks totally cheesy, and wonderfully tacky:
http://youtu.be/ac_9B6nWJHk
It's a bar, but I read online you can take kids there because it's also a restaurant. And, they have fun, fruit non-alcoholic drinks for kids.
http://www.tongaroom.com/
I second seeing the elephant seals, although in May I'm not sure how much there is to see in May. All the info can be found here:
http://www.elephantseal.org/
And, I agree, if time and money allow, Hearst Castle is a neat attraction to stop at.
You mentioned staying in San Luis Obispo. While there, check out the Madonna Inn. Even if you don't spend the night there, you can go there for breakfast or hit up their bakery for their famous pink champagne cake:
http://www.madonnainn.com/cakes.php
If you don't have to stay in San Luis Obispo, our favorite place to stay is about 16 minutes away in Pismo Beach. It's the Pismo Lighthouse Suites. Great for families, and with a view of the ocean. If you want, you can see a little review I posted about it here:
http://www.disboards.com/showthread.php?t=1836870&page=13
For a fun dinner nearby, check out Klondike Pizza.
Another neat place as you're making your way down the coast is Solvang. It's a cute little Danish town. Fun to walk around, and make sure to get some Danish pastries!
http://www.solvangusa.com/
Anyway, hope some of this is useful to you. Happy planning!
