It does depend on the state, but most states have a legal clause about squatting and letting others (even without knowledge!) use property. Here in Maine it is 7 years. If someone uses a piece of property I own for 7 years and thinks it is theirs, it is then theirs. That is very simplistic, and there are laws around it all. I know this because we inherited some property that my father in law owned - he allowed a neighbor to use some for a garden - neighbor is elderly and has alzheimer's and is not living there now - current owners/renters think the garden is in THEIR property. We have only 7 years to make them aware it is ours or else it is legally theirs. Was told this by both the town and a realtor.
For our case we are having a survey done in the spring and will be clearly marking ALL property lines and neighbors will NOT be using any of our property for their own use.
Good luck. But, honestly, if it were me, unless very valuable, I'd just let it go.
We got a call from our attorney two weeks ago regarding signing off on a piece of property that still showed a lien from my father in law. It was SO much easier to let it go than to pay to have it looked into and fight it...