From Downtown Disney, there is a direct bus to Fort Wilderness. It will be stop #4. That bus travels to WL, and then to FW.
I'm assuming you're going to the Hoop Dee Doo. Once you get to the Fort Wilderness Outpost Depot (that's where the DTD bus drops off), you're going to want to catch a bus that says SETTLEMENT.
To get back to the All-Stars, you have a few options. If the MK is open, take the boat, and catch the bus there.
2nd choice for transfer point would be the Disney-MGM Studios. That bus picks up right at the Settlement.
3rd and 4th choices would be Epcot and DTD. Those buses are at the Outpost, so you'll need to hop on an Outpost bus to get there.
If you ever have any questions, make sure to ask someone wearing purple pants. If they don't have purple pants, they might not be giving you the most accurate information.