Bring them.
If you need them you have them. If you don't, no harm is done.
We have had one or two times (definitely once, not sure twice) where we NEEDED them, even DH. The time I totally remember was Feb 2013 and it wasn't that long after a big storm had hit Castaway. That changed the consistency of the sand (dry and under the water) completely and it HURT to walk. I had water shoes. DH and DS didn't. Until we spent ~25 each in the island giftshop to get them.
The other times we've been there (with the double dip counted, that's either 3 or 4 other times) we haven't needed them. But we had them, just in case!