Is it possible?...Absolutely
Is it annoying...well, that depends. If the bus system at any other Disney Resort annoys you, then FW will also. If you're the Hakuna Matata type, the busses really won't be that bad. (I actually enjoyed the boat ride from FW to MK!)
As mentioned above you can rent a golf cart ($45/day approx) that you can leave at the bus stop/Settlement Depot, or near the Marina.
The big advantage we found to having a car is that you can park it right at your cabin, then drive directly to the parks.