The security pin is four digits. That means a thief has a one in ten thousand chance in getting it correct. Maybe two in ten thousand, if a given retail outlet allows them to try the pin twice.
In other words, losing your credit card is way more scary.
If you lose a band, go and have it deactivated and reissued with a new band. But don't panic.