Although it's not that far from Miami to Key West, it takes longer than you might expect - at a guess I'd say around 3 hours. We flew into Miami a couple of years ago and drove to Marathon, which is about half way down the Keys, and I'm sure that took at least an hour and a half to two hours.
There's only one main road along the Keys, so the going can be slow if you're stuck behind something, but it's a laid-back place so it doesn't seem to matter! There are a lot of bridges (the only way to drive from one island to the next!), and we had a great time dolphin-spotting. The 7-mile bridge is quite something and it does seem to go on for ever!
We finished our trip in Orlando, and then drove back to Miami for the flight home, and I think that took us about 4 hours on a good road (don't remember which route we took, sorry).