Can't wait for your trip report! Sounds like you had fun and skipped the usual tourist stuff! If you ever go back or for others planning a trip heres my take on the tourist areas.
Cable cars can be a fun ride but the lines can be horrible on the tourist routes. They are part of the SF bus system so a bus pass will also allow you to ride the cable cars.
Fishermans Wharf is lots of fun despite it being very touristy. We enjoy Scomas which is a great restaurant on the water. It is a bit back behind all the tourist action and is why its one of our favorites.
The seals on pier 39 are always entertaining but the rest of the pier is packed with tourists.
Kids love the exploratoriam. A large place for kids to learn about science while having a blast playing with all the gizmo and gadgets.
If driving south to
Disneyland etc. and have the time, take 101. Much more scenic than I-5. The boardwalk at Santa Cruz reminds me of decades gone by with fun rides and a fun beach.
Monterrey Wharf is what SF wharf used to be many decades ago. For lunch or dinner try Dominico's on the pier. Great food, sights and atmosphere. Monterrey Aquarium is one of the best in the country.
Much further south stop for breakfast or lunch at Fat Cats at Port San Luis at Avila Beach. It's a locals favorate with fantastic food and a great ocean view.