St. Augustine is a GREAT stopping place! I would have guessed it more like 2 1/2 hours, but I guess it depends upon your driving habits.
Leave time to stroll the city, the market area, the historic sites. Certain times of year there are art shows and music concerts there. We often stay in the Casa Monica Hotel...right in the area. It isn't cheap, but is in the heart of everything.
Enjoy your trip, whatever you decide!