When you say driving yourself, will you be parking at the port or dropping off a rental car?
If parking at the port, give yourself about an hour. So, if your PAT is noon, you would want to leave around 11am.
If dropping a rental, there are two options.
First option - Drop your luggage and everyone except the driver at the port. The driver returns the car and then shuttles back to the port. Give yourself an hour before PAT from WDW
Second option - Everyone goes to car rental drop off and takes the shuttle. Give yourself and hour and a half before PAT from WDW
If you arrange a service, they will suggest a pick-up time based on whether you have a grocery store stop included. But, again, about an hour before your PAT.