Growing up, we went on vacations often as a family. I remember being 4 or 5 and going to Lake Erie & Niagara Falls with my parents, and that was probably my first vacation. We were fortunate enough to go to Disney twice, once around 1981 and again in 1985, as well as trips to Cocoa Beach FL, Pensacola FL, and multiple 'vacations' to see my mom's family back in Pittsburgh after we moved to Indiana. Even though those were primarily to see my grandparents and extended family, we always included a day at an amusement park, zoo, tourist type stuff for us kids to enjoy. In 1996, my parents took my sister & I to Grand Cayman for 4 days, and that was fabulous!! Now that my dad's retired AF, he wants to take my mom all over the world but she doesn't like to travel. He did get her to Hawaii for spring break a few years ago, and they did a 3 week driving tour of the southeast two summers ago, plus yearly trips to Pittsburgh when he can convince her. She doesn't like to fly AT ALL and can't sit for more than an hour or two, which really limits her travel.
DH's family never took vacations. Ever. They couldn't afford it at all, and if they did go somewhere it was to visit family...he said they took one trip to the UP in Michigan when he was around 10, and that was to visit a friend of his mom's. They just aren't vacation/travel people. Amazingly enough, a few years ago they took a driving tour of the country...drove out to CA and back, I think they were gone 3 weeks or so? They were coming back through Las Vegas, and instead of stopping until the city was all lit up, his dad drove straight through at noon!

His attitude? "What's so special about a bunch of lights?" Um, a LOT, especially when you live in the middle of nowhere! They don't understand why we like to take the girls on trips.
We took our first vacation with kids when Emily, DD1, was only 6 months old. It was just a long weekend at Lake Michigan, but it was a blast. When they were 13 months and 2 1/2, we went to St Augustine/Daytona Beach FL, and then the next year we went to Ft Myers Beach. After that money got a LOT tighter, and we didn't "vacation" from 2006 until 2011. That being said, we did go to Pittsburgh in 2009 for a cousin's wedding, 2010 we went back for a week & stayed with my aunt & uncle, and then last summer we spent a week in Ft Myers Beach. This summer we are blessed in that my parents are taking all of us (including my sister, brother & his wife) to WDW. Otherwise, there wouldn't be a vacation this year.
I think a lot of PP's were correct...people have different priorities on where to spend their money. Some people don't like to travel. I'm amazed that my mom is willing to drive to Florida, but I also know they will take as long as they possibly can to get there while we drive straight through.

We are blessed with children who love to travel and do well in the car for long periods of time too!