SAB for sure! Our boys were 4 and 5 when we stayed there last spring. They LOVED it. And, so did DH and I.

It is such a fun pool! We decided that next time, we'll stay longer JUST so we can spend more time in the pool. Our youngest wore a life vest the entire time --shortly thereafter, he began swimming independently, so our next trip will be vest-free

Our oldest was a strong, independent swimmer, so he was allowed to go on the big slide, which he must have done about 100 times! DH and I took turns going down the slide when he wanted to do so. The baby pool in the back of the ship slide has a mini slide that my 4 year old loved. The pool itself is about 3 ft deep, though. BUT, the best part of the pool was the HUGE sandbar wading area. Wow! This area rocks for kids of ALL ages. Water is from 0" (at the sandbar) to a maximum of about 18" at the edges. There were toddlers and kids of all ages playing here, making sand castles, etc. My younger son actually LOVED the lazy river area too! He would just float on his back (with the vest on) and go around and around. DH and I had fun diving down to the bottom of the 8ft section! Can't do THAT at most other WDW resorts. Also, we loved how the hot tubs were integrated into the pool, so DH and I could sit in there while the boys played in the sandbar. So cool!
I love all the "fear monger" posts you see about SAB and it being overwhelming for young kids. Let me ask you: do you put your kids in the pool and then sit on a chair and take a nap? If not, I don't see how SAB is any more dangerous or overwhelming than any other resort pool. At 4 and 6, you'll probably be in the water with them, no? Plus, there were more lifeguards at this pool than I've ever seen at any other WDW resort pool, and they were VERY vigilant about enforcing rules and squashing reckless behavior when they saw it.
I'd recommend SAB to families with kids of ALL ages.