I'm an electronic file extremist. I started scanning and shredding my paper documents back in the '90s. Now I get everything possible electronically because it saves me the scanning and shredding step. Even for accounts that keep electronic statements on file, I download and save those.
My physical filing cabinet is tiny now. It has tax documents, passports, social security cards, and insurance policies. That's about it.
I love having everything electronic for several reasons. It's easier to find things. I can make backup copies of everything and store a copy off site. It takes up virtually no space. Because it is well encrypted, it is more secure than paper files sitting in my house. It saves on paper and delivery costs.
Instead of relying on mail to let me know when to pay bills, I have them all handled automatically. My paycheck is direct deposited. Money is automatically sent to my various savings and investing accounts. My credit cards, utilities, memberships, and insurance policies all auto-draft my checking account. The only bills that I have to pay manually are taxes, HOA fees, and flood insurance. Those only happen once a year, so they aren't too bad. I wish they'd move into the modern era and set up direct drafts as well. I actually changed electric companies once because they couldn't handle auto-drafting well.
To keep track of everything, I use Quicken. I reconcile all my accounts frequently (usually weekly). It's very easy to do and it helps me catch accounting mistakes quickly. I also link images of my bills, checks, etc into my Quicken account. That way, if I'm looking at an entry for a credit card bill, paycheck, hand written check, or whatever, I can click on a button and pull up the electronic image.
As electronic focused as I am, I can see why some people prefer to live in a paper world. For me, I am most comfortable with everything electronic. With paper, I never feel in control. I feel like I truly "have" something when I have it in an electronic form that I can control. I know many people that feel the opposite and "having" something means having a physical copy. Whatever works for you is the right solution.