Yeah, i found some lots during my simp!e search, for well under $100,k. But (again

) not having even the slightest bit of knowledge of Milwaukee real estae

no idea what zoning would allow.
Do we know that? What she paid 15-30 years ago?
If another poster is correct about this being the same person with different issues in the past, itbs possible the selling condition may not be what she perceives.
Based on the OPs subsequent clarifications, it sounds like it IS just one bad neigbor. And only periodically.
Visit the neighborhood at 4 AM?
Yup, all this..(especially agree with #3)
And remember, the OP has a very low threshold of what makes a bad neighbor. Just seeing two 24 packs of beer sitting outside the garage made her absolutely believe an upcoming "drunkfest." So, one has to question, what is the OP's threshold of cars coming in and out (she said they have teens,) that makes her think this is a drug house? What would a potential buyer really see? Looking at police reports, it would only be less than 2 calls per month. If they are by the same person, a potential buyer might write it off as a "Gladys" type neighbor.
OP, vent away. But if this is really as bad as you say and you cannot have your grandchildren there because of the house next door, you have plenty of options to make this better.
The best option is to move, even if you lose money on your house. There is no price on safety. And if the neighborhood is deteriorating, the longer you wait the more money you will lose.
Right now, assuming average credit and financial history, just by what you have posted, it seems very doable. Of course, you would need to talk to your accountant. You say you would have a $80,000 mortgage, but you did not factor in using the $15,000 in travel funds you have set aside for this year's travel if you suspend travel for one year. That brings it down to $65,000. A $65,000 mortgage at a current rate of 3.95% for 30 years is a payment of $300 a month. Over a 12 month period, that is $3,600. You could easily afford that by changing your travel habits to moving to RCI/Carnival instead of Disney for your cruises
or just dropping one trip to WDW. Out of the $15,000 of travel money a year, you would still have $11,400, a very sizable travel budget. Doesn't seem like too much of a sacrifice to give up just one out of 3 trips a year to be able to have my grandchildren in a safe house.
Also, if you used that $15,000 travel money to fix up the house, there is the possibility of netting more than the $15,000 investment, making your mortgage even lower.
Good luck!