While I would NEVER open my door to a stranger, I have never once felt like a prisoner in my house.
Person knocks on my door.....I speak through the door and ask what they want........they tell me....... I offer to call someone who could help them, but I won't open my door to them...........they leave........I go about my day........no feelings of being a prisoner.
I live in the country. I can only see one neighbors house. I rarely get strangers at my door, but it does happen a few times a year.