I was a SAHM for a very long time so I'm gonna be biased and say stay home to enjoy your family if that is what your heart is tugging you to do.
Only thing I would do differently is find something part time mid day, maybe temp work with your employer or similar just so you can jump back in later on without drawing venom from other people.
The one thing I was not prepared for is how horrible people, in particular other women, treated me when I was trying to rebuild a career after being home. The viterol was quite a shock and it wasn't from men, like at all. When I would interview women my age who chose to work and Millenial age young women acted like gatekeepers and denied entry, it was so obvious. As soon as I head the question, "Soooooo I see you weren't working from years X through Y, what were you doing?" I knew they had an axe to grind. They were also awful that I was a secretary at a school district as a re-entry job, "Why were you doing this?" So much for women having choices

. I could afford to stay home and wanted to do so and I did, that didn't justify the dehumanization I experienced. At any rate, I don't regret a moment but if I knew I would have done part time at maybe a CPA firm or something as a placeholder.