Drive time depends when you're going and if there's an event at Staples and/or Nokia Theatre. If you go at rush hour in the afternoon, allow at least 1hr 30mins, more if you can (definitely more if there's an event). If you go mid-day or on a weekend, allow 1 hr.
The area is icky. Okay to walk around a bit during the day, if you feel some crazy need to do so, but not at night. It's downtown LA and it's not a great place. Best to stick to the Staples/Nokia complex. Lots of dining and such there.