From Magic Kingdom you can either go to the Transportation and Ticket Center and get a bus directly to Downtown Disney or hop on the monorail to any one of the Resorts, or bus, monorail would be quicker and take a bus from the Resort directly to Downtown Disney.
EPCOT you must go to a resort and catch a Downtown Disney bus.
Animal Kingdom is same as EPCOT.
Above all, I'd say Hollywood Studios is the easiest. They have a bus that goes directly to Downtown Disney right at the end of the bus lot.