We have also taken the Amtrak from LA to San Diego and we loved it! It is nice not to have to deal with the traffic and the train was practically empty. We went on a midweek day and stayed one night.
The one thing that we did not realize was that the San Diego Zoo was pretty far from the station and the trolley doesn't really go close to it. We just stayed in the "tourist" area and visited the aircraft carrier(we loved that!), the Russian sub, and the pirate ship. That worked out good for us.
Have fun.