I agree, it entirely depends on the child.
All our kids have (or will) go to our most local primary school- eldest daughter we assumed would fit in better there as it's small school, she was a timid child, extremely shy and we thought smaller environment= less chance to be over looked, for middle school we realised the smaller middle school was failing badly so she went from small school (260 kids) to big school (900 kids) and i was petrified- however it's been fabulous for her, the best move we could have made. She has flourished- the larger student body means she isn't such an outsider as there are other unusual children there (she really stood out at her old school as we don't at all match the "norm" ) being a larger school it has a fantastic gifted student program which she was put on immediately (at the other school-although they acknowledged it, it made her a social outcast as she was the only gifted student in her class) she's had the oportunity to enter national writing competitions, national book reviews, she's been in 3 drama productions, she's a distance runner now (where as before she was the non-sporty kid) she's learning to play flute- all things she wouldn't have had the confidence to do in a smaller school as she felt the need to blend & hide, in a bigger school she doesn't find the need to do that.
Eldest son however is a special needs student- he's on the gifted list too but also the special needs register, he has dyspraxia, dysgraphia, is colour vision deficient & is being assessed for the autistic spectrum & a couple of other coordination based needs. He has physio, occupational therapy, will need a scribe in later school... a big middle school isn't going to suit him because I am aware SN students can get lost into the system because in a larger school their underachievement doesn't lead to a big spike in class grades like in a smaller school.
Youngest daughter is a social butterfly- I know she's most likely to chose whichever school the majority of her little group goes to and we'll be happy to support that, I don't like her having to live in her siblings shadow anyway (2 elder siblings on the gifted register is alot of pressure)... youngest son is just learning to walk and talk so time will tell there.