Out of the PSP or the DS (any of the DSi's) I'd vote for the DSi XL. The PSP is a great handheld gaming system but, I feel, lacks in games geared towards younger kids. The DS, DSi, & DSi XL have some really great educational games, allows kids (in the older models) to connect (wi-fi if you will) to other friends that also have the game system. The biggest difference between the DSi and the DSi XL is that entire system is a bit bigger and the screen is a good bit bigger which is easier on the eyes. I think they might have also My two boys (5 & 9) had the DSi's and used birthday money this year to upgrade to the XLs since I saw no real reason to upgrade. The games from the Gameboys, DS, DSi's are all interchangeable and you can often find cheap used games at places like Game Stop. I don't believe that the cost difference between the DSi and the DSi XL is that much but that bigger screen is a huge plus in my opinion.
HTH