We do this essentially every trip (we like to do a waterpark on our last day most of the time). I'm not small, and I wear a Miraclesuit, which is lined in 3 layers, so drying it can take a lot of time. I'm a great fan of the "stomper" method of swimsuit drying. It does require a dry towel, though.
What you do is rinse out the suit in cold water ASAP (or if you have a really pricey suit, a product like Suit Saver) to get the worst of the chlorine out, then lay out a towel flat and place the suit on it inside out. Roll the suit up in the towel, then stand on the rolled towel, walking along the length of the roll for at least about 5 full minutes. (I like to dry my hair whilst doing the stomp.) Your body weight will force most of the water out of the suit and into the towel. The level of water remaining should be less than if it has gone through a spin cycle in the washer. Put it in your suitcase in a double ziploc for the flight, then be sure to wash properly it as soon as you get home.
Putting the suit in checked baggage *is* a risk, especially if you haven't fully removed the chlorine. If the bag gets delayed it may end up mildewed by the time you receive it, or if it has gotten hot sitting in a warehouse, the elasticity of the Lycra may be shot.
DS and DH tend to just keep their suits on until we get to the airport, then change into long pants if it is colder at home. Of course, they tend to wear suits that look like regular shorts, so it isn't noticeable.