I grew up with a pool, it was in my back yard before I was born. There was never an issue. Watch your kids, keep the doors locked we have a dead bolt and the key is up high where the kids can't reach it. Well they can now, but now, but they are much older and know better, and really have no desire.
When the kids were little, we also had a baby fence around the pool, it was the law. Now we don't have it up.
You can also get an alarm on your door, it would wake the dead. Anyway, enjoy, just watch the kids and you will be fine. The kids will love the pool.