Yes, I would stick with it. We were there last year from Sept 14th-22nd. There were several 1's and 2's that week, and we stuck to the calendar. We were just stunned by how few people were in the parks! I am a Florida native and have been to WDW many times, but I had never, ever seen crowds that small. We rode a ton of rides and there was just a more relaxed vibe around the park. We went to Epcot on our arrival afternoon, and despite it being a low number (3 or 4), it was still pretty busy because it wasn't recommended. You won't hate the crowds on a non-recommended day at that time, but you will want to take advantage of the recommended parks.