I think probably the ONLY way to get to Celebration without renting a car (or walking ) would be a cab. Not sure how much it would cost you.
You should plan ahead what types of things you'd like to do in Celebration, before you call the cab. This way, you know where to tell them to bring you and you know about when you'd be needed a ride back to WDW.
There are some really nice restaurants in Celebration. The downtown are is really nice to stroll around, sit in a rockingchair at the water front, do a bit of shopping. Without a car, the extent of what you see will either be on foot or from the cab as they drive you to and from WDW.
I suppose if you are not afraid to pay the fare, you could have them drive down a few of the residential streets to take a look at the quaint neighborhoods or see what houses are for sale.
Unfortunatley there is not "Disney transportation" provided to Celebration. If you wish to visit you would need to take a cab. Depending on where on WDW property you are Celebration is 5-10 miles away.
Just tell the driver you wish to be dropped of at Market Street in Celebration (that is the main road). As was stated there are some nice resturants there, an AMC theater, some shops, the lake and the park. From there if you wish to see some of the houses, there are a few streets worth that you can just walk up and down.
This would bring an overwelming number of people into the town. Many people falsely believe that Celebration is a Disney attraction. In fact, I have heard stories of people being surprised when random people walked into their Celebration home!