We've done trips in the past with friends and family.
Biggest problem I've run into is the other folks don't seem to keep the same schedule as us. We like to hit the park first thing in the morning and some folks don't. When we're ready to go I do not like waiting for hours for folks to get going and then they want to take an hour for breakfast. We don't wait any more.
My girls are pretty patient when it comes to doing things in the park that might not really interest them because they know later something they like will be the priority.
Some of our friends kids throw a huge fit and it has to be about their needs all the time.
Just like other kids diapers, whining from other children not your own seems more annoying.
Also splitting up may be a good idea along age lines. Say take the older children to the bigger rides and the little ones to their age specific rides.
Now on to the adults. Are you compatible to spend a lot of time together?
Does your friends husband monopolizing the television, sleeping on the couch, with accompanied back side symphony, drink beer all day, not cleaning up after himself or watching his children really your idea of a holiday?
The larger the group the more decisions by committee that will be made, what parks, where to eat, when to rest etc....
Best advice I can give is do some activities as a group and some as a family.