You'll have to call your provider (AT&T?) and give them the IMEI of another phone, otherwise they will not cancel the data plan. As of Sept. 2009 it is a requirement for smart phones to have a data plan.
I have a Blackberry. I tried to cancel the data and wasn't permitted, so I switched out the BB for another phone we had that had been upgraded. I called and AT&T updated my account with the other phone which now does not have a data plan.