We have Nexus for myself, wife and two kids. $100 for all 4 of us for 5 years is definitely worth it, but we live about 10 minutes from the border and I can drive across to fill up on gas, buy milk etc. The card has actually paid for itself in savings on those things. We'll do day trips down to Belingham, WA and the occasional weekend in Seattle.
If you're only crossing a couple of times a year it may not be worth it, but any more than that and I definitely feel it's worthwhile. It's great to see a 30 minute border wait, drive right by it, and be waved through (usually).
We all have passports too.