the little boats are called Sea Raycers. Here's some info for you.
There's more info HERE on Allearsnet.com, but I don't know when it was last updated.
I don't know if you need to reserve them in advance. It might not hurt. I know I've walked by the Marina at the Yacht club numerous times and seen a few of them moored there so I don't think you should have a problem walking up and renting one.
Our Authorized Disney Vacation Planners help you compare resorts, tickets, packages, discounts, dining, and cruise options. There is no cost to use our planning services.