No, its not difficult at all.
There is a free internal tram service between all the terminals at CDG. You just look for the signs for the CDGVal. Hop on, and there are announcements as you arrive at each stop. The tram runs every 10 minutes or so, and it only takes a few minutes to get from terminal to terminal.