Yes, I had thought that too before our first trip. I was told that they tow you unless it is obvious that your car has mechanical troubles. Otherwise all of us Preferred holders would leave our car there.
But, it does make staying offsite more attractive if you have the preferred pass, since it is easy to find a nice place without parking fees.
Not that there aren't benefits to on-site. Our upcoming trip is onsite. But, I tell myself that the passholder hotel discounts more than pay for the parking.