I totally get that in an aircraft emergency, you really have no idea what you will and won't grab. And instincts will take over. But PLEASE, if at all possible, override your instinct to grab the backpack and just evacuate the airplane!
The reason they say to leave all belongings is not just because it takes a few seconds to grab the bag. It's because that bag will be in the way and will slow down the evacuation. Think about how skinny airplane aisles are and how easy it is for a bag to get hooked on an armrest as you're walking down the aisle (that happens to me when I'm just slowly boarding a plane, let alone when I'm moving and in a panic). You'll need two hands to safely leave the plane and get down the shoot. The people around you will need the space that your backpack takes up. And once you're out of the plane, especially in a water landing, you sure as heck don't want the weight of a bag to deal with.
If your wallet's in your pocket - awesome. If you can easily grab your phone from the seat pocket, great. But please do *not* bring a bag with you when evacuating the plane.
Aircraft evacuation procedures are incredibly well designed and thought out. Everyone's life will be safer if you actually follow instructions.