I bought a couple of knit hoodies at KMart a few years back, and love them! The fabric is a fairly heavyweight knit, so it doesn't wrinkle easily (don't get anything too lightweight, or it will wrinkle).
They come in solid neutral colors (gray, navy, black, white) as well as other fun colors like pink, red, etc. They're less than $15 and work really well. They even have matching sweatpant-type pants, if you want to make an "outfit" of it!
My other option that I've used in the past is a long-sleeve denim shirt. Wrinkles a little more, but it's a nice, lightweight layer.
Final tip: I've found that if I roll up the item before putting it in my backpack, it doesn't wrinkle as much. Similar to what you'd do when packing a suitcase!
Have fun on your trip!