For DS's room we got a sheet of 1/2" thick insulating foam and I cut it to size. Spray painted one side black (for the darkening properties) using a paint that works on plastic and then spray painted it a second time with a color that worked for our room theme. Put white adhesive paper on the outside so the window had a clean look and made a couple of pull tabs on the bottom (used the same white adhesive shelf liner - just folded it together so it wasn't sticky and then used another strip of shelf liner to hold it in place on the back side) so we could remove it easily.
Not only does it help darken his room, but it also keeps his room warmer in the winter and cooler in the summer.
We don't take it out of the window very often - his room isn't intended for play, but for sleep, so we don't need to mess with it. (that's what the play room - and the rest of the house...LOL - is for)