Now I know we're all the best judges of our own kids, but I really would welcome views from people who have done this. Are mine too young?
I know it's a cliche, but it really is impossible for someone to say who doesn't have a good knowledge of the children involved. What you'll have been reading is people who have kids of about that age (or who are judging on their own children at that age) who are deciding (maybe justifying not spending the money on DC yet) that it's not something their children are ready for yet. IMHO no one can judge the ability of someone, they don't know, to participate in this program.
We were at DC just a couple of weeks ago, my younger DD (who likes to make a drama out of a crisis) is now 15, I wouldn't have taken her any younger because of the fuss she makes about anything to do with deep ( more than 6 feet deep) water. This is despite the fact she is a good swimmer. Even at her age and ability she still wasn't comfortable in the coral reef and although she spent maybe an hour in the coral reef pool over the day, she would only go in while clinging on to my arm ( I still have the nail marks to show for it when a school of fish came too close to her LOL), my other daughter however was a real water baby and spent hours in the pool every day when we lived in Bahrain from before she could walk, she could swim many lengths by the time she was 4/5 and absolutely loved the water. I'd have had no problem taking her for the dolphin swim LONG before she was 6 years old. IMHO there are no hard and fast rules as far as kids, their age or even their ability to swim ( with the vests swimming ability required is not very much) by far the most important factors are the kids confidence in unusual situations and their liking of animals. If you have a kid that would run a mile when faced with a horse at a petting zoo then it may not be for them, if they are likely to give the horse a big hug and stroke it then there's a good chance they'll deal with things fine. IMHO the swim aspect is about 20% of the requirement to enjoy the program, which is why it's impossible for ANY stranger to say if it's going to be OK for your kids.
I would agree with SEVANS that if you haven't been to SeaWorld before and tried to feed the dolphins there an investment of a few $$s on a few trays of fish might be a very good one. See how the kids react to feeding and (hopefully) touching the dolphins that the feed, even if they don't manage to actually touch one of those dolphins, how they react will give you some clues as to the viability of the dolphin swim and it will also ( probably) raise the desire to actually get in with the dolphins.
The kids can opt out of the deep water aspect of the swim, although this is IMHO the best part of the experience, and just stroke, kiss and pet the dolphin from the shallows. DC's trainers are used to dealing with kids that have a change of heart about going through with the deepwater swim aspect. They will work to try and make sure the kids get as good an experience as possible, working within whatever limitations the kid's desire to participate allow.
In your situation, I would probably try to split the swim into two groups, so one parent and one child does a morning swim and one parent one child do a later one. This way it gives each parent the chance to concentrate on watching their child experience the event ( this is, IMHO a more rewarding experience than doing the swim yourself) and it gives the child you are most concerned about the chance to see what goes on, see their sibling get through the experience and decide how much they want to partake. I'm sure if they decide to back out the other child would LOVE to have another go LOL. This would also give you the chance to take your own photos of the event and not rely on the (rather pricey) momentos that DC will happily provide for you.