My kids used their passports as proof of age when they got their drivers' licenses, and for proof of citizenship when they got summer jobs. Also used them as ID before their had driver's licenses for signing up for school, and Little League and soccer. Frankly, most of the times my kids have used their Passports have been right here at home as opposed to traveling.
So you lose out on that by not renewing, and of course if the fee goes up, it will cost you more, but otherwise, there are no issues waiting.