Have also stayed at both. I really like the Silver Points location - the key is to stay off Irlo Bronson - and come in the back way (Sherbeth Rd). No traffic that way, and you come into the side of the resort. Never really hit any traffic coming to/from Vistana and the parks either, and we go in the summer.
The other comments are spot on - the Vistana unit was more comfortable and the grounds are more resort-like. Both are nice off-site properties, and I wouldn't hesitate to stay at either again.
We'll likely be at Silver Points this summer, as Vistana has a 1 in 4 rule, so it's off the table.