I educated my DD at home until last year. At that point, I sent her to a private high school, primarily due to the fact that she responded better to deadlines, etc. with an outside authority. I'm a single mom, and couldn't deal with teenage rebellion in all spheres as well as in education--consequently, the education was suffering.
The best thing I can tell you is that children educated at home tend to be socialized vertically while children in schools tend to be socialized horizontally. By that, I mean that in school you spend time with other 6 year olds...or 10 year olds. There is very little you can learn from children your own age except "bad language and bad behavior" (my daughter's input.) Children taught at home learn to get along with people of all ages, from infants to the elderly. They visit nursing homes, care for toddlers, etc.
There are many homeschool groups. We tended to decrease these activities as DD got older and the educational stuff was more intense. Then, there are church activities, music lessons, etc. We certainly never lacked for activities with other people.
DD is now 15. She was the freshman representative to choir council at church as well as one of several representatives to youth council. She is always the kid selected to help a new person in choir. She is also the kid who reported (turned in) a drug dealer at her high school, taught Bible school, was a "helper" at middle school retreat, etc. One of her proudest moments was when a Sunday School teacher asked the 6th grade class what they wanted to be when they grew up and one young lady replied that she wanted to be just like (DD's name.)
In April, we went to Europe. It gave us a lot of time alone together--no TV, no phone, no computer, etc. DD said that her biggest mistake in life was that she hadn't read more--she thought that she'd be "smarter" if she had read more. I replied that it wasn't an irreversible issue at her age. I told her that I wished I'd sent her to school sooner when she started the rebellion stuff. She felt that she might have done more academic work that way, but that she wouldn't have been "as good a person." I really think that says it all.
The best thing I can tell you is that children educated at home tend to be socialized vertically while children in schools tend to be socialized horizontally. By that, I mean that in school you spend time with other 6 year olds...or 10 year olds. There is very little you can learn from children your own age except "bad language and bad behavior" (my daughter's input.) Children taught at home learn to get along with people of all ages, from infants to the elderly. They visit nursing homes, care for toddlers, etc.
There are many homeschool groups. We tended to decrease these activities as DD got older and the educational stuff was more intense. Then, there are church activities, music lessons, etc. We certainly never lacked for activities with other people.
DD is now 15. She was the freshman representative to choir council at church as well as one of several representatives to youth council. She is always the kid selected to help a new person in choir. She is also the kid who reported (turned in) a drug dealer at her high school, taught Bible school, was a "helper" at middle school retreat, etc. One of her proudest moments was when a Sunday School teacher asked the 6th grade class what they wanted to be when they grew up and one young lady replied that she wanted to be just like (DD's name.)
In April, we went to Europe. It gave us a lot of time alone together--no TV, no phone, no computer, etc. DD said that her biggest mistake in life was that she hadn't read more--she thought that she'd be "smarter" if she had read more. I replied that it wasn't an irreversible issue at her age. I told her that I wished I'd sent her to school sooner when she started the rebellion stuff. She felt that she might have done more academic work that way, but that she wouldn't have been "as good a person." I really think that says it all.
having the freedom to choose is great- it doesn't mean that those of us who choose this are ignorant of the way life really works. trust me. 
We will raise him in a very active home where he will study martial arts (at a dojo with us, not with us as instructors, FYI), camp, fish, hike, swim, play any sport that tickles his fancy (we greatly encourage organized sports), and continue to go for the daily 2 mile walks we take as a family. We appreciate the great outdoors and will not be raising any couch potatoes! If that isn't going to suffice for "physical education"...I surely don't know what will. You do realize that many schools have completely done away with PE programs and even recess, don't you? 

(There were some girls on the team who weren't fitting in very well, however, and they were kids that go to the middle school.)
I DO know what schooling is all about. The fact is that I (personally) am opposed to the entire structure of public school, no matter how "great" the school itself is. I don't want my son being taught what others have decided is important for him to learn, and when. Period. We could live in the best school district in the country, and he still wouldn't go to school. I'm not saying every homeschooling family has the same objections. They don't. But I do understand very well what schooling is all about. Some schools work "okay", and some don't. Most of them have some major problems, from lack of funding to outdated curriculum to horrifically unhealthy cafeteria food. Many homes have the same problems, as well, I'm sure. I'm just saying that homeschoolers tend not to be as ignorant (and I use that in its neutral, literal sense) of public schooling as public school proponents are of homeschooling. Most homeschooling parents I know attended public school themselves, and most of them have had at least one child attend school for some length of time before pulling them out.
)