Fastest way is by automobile (yours, rented, or chauffeured).
And hailing a taxi is very easy.
By bus you can take one from Hollywood studios to any resort along the way (Caribbean Beach, Saratoga Springs, Port Orleans, Old Key West, even Pop Century or Art of Animation), then transfer to a bus for Downtown Disney. Allow 75 minutes, considering you still have to walk from the bus stop to the restaurant.
If the boat is there (to the left as you walk out from the DHS entrance plaza) loading for Boardwalk, Beach Club, etc. you could take that to the first stop and then take the Downtown Disney bus. You would have to walk from the dock through the resort to the bus stop.