I tried to do it without a life vest, jumped in, and jumped right back out. I'm not a strong swimmer.
DH did it with DD7 and DS10--the first time he used no life vest, and found it hard to keep with the kids (wearing life vests) and keeping himself afloat. He missed a lot, so he did it again, this time with a vest.
So--lesson learned--even if you're an adult, strong swimmer, you might find it a more relaxing swim if you grab a lifevest anyway.
And I agree with the other poster--go FIRST THING, so you can go through twice if you want with minimal lines.