We did 5 days, and that was good.
Day 1 was DLand
Day 2 was California Adventure
Day 3 was DLand
Day 4 was California Adventure
Day 5 we went back and re-rode the favorite rides, and went to a few more shows.
The park hopper is really good to have, so you can go catch the parade at one park, go to the fireworks at the other, etc..
I think you could do it in 3, but you would be rushing yourself, and you would miss things. I had it all planned out, so we didn't miss one thing. We still had time to stop for an ice cream break every afternoon and just sit and relax and people watch. On our second day, it was a busy saturday, so we left the park around 2, went to the hotel, slept and swam for 3 hours, ate supper, then went back to watch the electrical parade (CA) and Fantasmic (DL). We went in May, so it was fairly quiet, but weekends are always busy, any time of year. We went to Downtown Disney on day 6 (no pass needed) and spent a few hours shopping and eating at the Rainforest Cafe
We didn't do any character meals, I think those would take a little longer, we mostly ate counter service.
I would definately recommend 5 days. It is worth the few extra dollars.