You will be fine. When you think about it, how much of the "stuff" did you really use? Probably not much. And if you need it you can either 1) keep what you want in a locker, 2) return to the hotel and get it or 3) buy it in one of the may gift shops.
My bags have gotten smaller and smaller over the years. Last year I carried a small cross-body Vera Bradley bag - didn't hold more than a change wallet, camera, medications (Epi Pen, Bendadryl, Motrin), a pen, and a small hand sanitizer. I love not having to worry about a big backpack of stuff.