It really depends on your likes and wants. We were a party of 10 last year, 7 adults and 1 baby, plus 2 4 year olds. The oldest of the group was in her 70's.
The group did stick together a lot, and we planned lots of sit down meals at places like CG, Narcoosee's, etc because that is how the group usually vacations.
We did also do some times where the group split up. Frankly, it's tough to travel with 10 people. I crave my alone time, so ducked away several times. My friend's brother, who was paying for the whole trip, seemed to get a little bored of the kiddie rides (he couldn't ride Pooh 100+ times) so 'uncle' and I went on some adult rides so that he could enjoy his vacation too.
Think about what you like to do as a group, and make plans. We enjoyed our nightly dinner together, and most nights watched Wishes as a group. That's the part of the trip that the adults remembered and enjoyed most. (Of course, the 4 year olds like Pooh best, and can still remember every detail of that ride!)