LOL...We have those too. Thing is, they never do anything mean or threatening, so we're not afraid of them.
Ours turn the Christmas lights on in the middle of the night. We've woken up several mornings to find the tree and the mantle lights on, when we DEFINITELY turned them off before bed. (they sometimes come on in the evernings too, while we're making dinner.)
Let's see...they've rearranged the word magnets on the fridge a couple of times to make little sentences. But they were never of the "Get out now" nature...LOL
Oh, and ours smoke too. Neither Jen nor I smoke, but we distinctly smell smoke sometimes in the evenings. (We live in a 2-family flat, but the guy above us doesn't smoke either, and we smell it even when he's not home, so we know it's not guests smoking in his apartment.)
We've just sort of learned to live with him/her/them. They're like members of our little family now. Like I said, we've never felt threatened by them, so we feel no need to leave. I've never said anything here about them because I figured people would think I was nuts...LOL
But if you do feel threatened by them, there are ways you can "cleanse" your apartment. We looked it up when ours began, just in case we felt like we needed to get rid of them. But we never felt the need to do anything.
As far as them coming with you when you go, I think what I read at one time is that it sometimes can be attributed to "negative" energy in the home, and that if you take that energy with you, then you take the problems with you. So if you "cleanse" before you go to cast out the negative energy, you can rid yourself of the problem before you move.
I know you all probably think I'm loony...I'm just going by what I've experienced and what I've read.
LOL