Oh, no! I am so sorry you had such a miserable experience your first day in San Francisco! It's my very favorite city (I live in the Bay Area), and it makes me sad when tourists don't enjoy their trips here.
Unfortunately, you picked summer, which is not very summery at all. It's very cold and foggy and damp. I find it utterly charming, but I also live here, come prepared with our clothing choices, and am usually escaping the East Bay heat.
It's too late now, but in case anyone else is reading or you visit again, there is a second Ghiradhelli store in the Square. It's all the way in the back corner, and many tourists have no idea it is there. The store itself is much more open, and the lines for ice cream are much shorter, even during the height of tourist season. A trip down to that area (which we do often with our young girls), is never complete without ice cream from there, so we go often and have never had more than a 10 minute wait in line, at the longest. There is plenty of seating outside.
That In-n-Out is perfectly awful. Please don't judge other I-n-O's by that one, and do try them again. Fisherman's Wharf is just not the best, foodwise. It's such a mess of people and garbage. We never went there until we had kids, but they love the carousel (so glad you got on that), the little playground by the aquarium, and the Musee Mechanique. So we often end a day in the city with a visit down there. If you return, stick to Boudin's (nothing fancy at all, but it's edible) or for a nicer experience and a lovely view, try Fog Harbour Inn.
I hope things improved, especially with your poor sick little one! It looks like you had fun at the Museum!
