You won't have a problem holding your baby on your lap, especially if baby is being worn. My toddlers didn't have to sit on the seat until they were 2 and even then my daughter was able to ride on my lap because she's exceptionally small.
As for how many will fit, it depends on the size of the adults and older kids more than anything. We've fit my husband, my 6 year old, and my 2-year-old boy that's almost the size of a 4-year-old together. My husband is overweight, but not severely. So, conceivably, if your children were on the smaller side, you might all fit. That being said, I'd just go in assuming the 10-year-old has to ride alone.