It's not that they don't go places while they're down there, it's just that stumbling upon one of them would be like coming across someone you know while you're at Disney World. Sure, it could happen, but I'm not sure I would get my hopes way up. Anything's possible though (like when I saw Kurt Busch eating lunch at the Olive Garden in Charlotte recently), and if you happen upon one of them, just remember that they are on vacation just like you, and are deserving of the peace and relaxation that you too are in search of.
p.s. I work for the #38 M&M's Ford on the weekends (one of the weekend warriors), so I know a little about their schedules. While they tend to stay in the Daytona area during the month preceeding the Speedweeks, you can catch them out and about if there is some free time, and a lot of them have their families with them during that time, so it makes sense that Disney might be one of the better bets to find a driver and his family. Good luck driver hunting.