Touring Plans: How accurate are Crowd Predictions??

[GBBTomorrow]

Not a chance of accuracy beyond making very broad generalizations that anyone could make. So many variables and subjective things that at best it's as accurate as

I really disagree with this. These folks are statisticians and know what they are doing. They collect data constantly. Maybe an English major like me doesn't understand the algorithms and the magic of their math, but I definitely believe that they are capable of a whole heck of a lot with the data they have, definitely far more than flipping a coin. It's not just hocus pocus. It's math.

 

[Dan Murphy]

I really disagree with this. These folks are statisticians and know what they are doing. They collect data constantly. Maybe an English major like me doesn't understand the algorithms and the magic of their math, but I definitely believe that they are capable of a whole heck of a lot with the data they have, definitely far more than flipping a coin. It's not just hocus pocus. It's math.

Very true.

 

[stinkpickle]

They actually post the "How We Did Yesterday" section to show how close they came.

 

[Cheyflan]

Can someone who has a subscription to Touring Plans tell me what they expect MK to be on September 24th on a scale from 1-10?

 

[jessiegirl1982]

Can someone who has a subscription to Touring Plans tell me what they expect MK to be on September 24th on a scale from 1-10?

3 out of 10!

 

[pitpat]

I really disagree with this. These folks are statisticians and know what they are doing. They collect data constantly. Maybe an English major like me doesn't understand the algorithms and the magic of their math, but I definitely believe that they are capable of a whole heck of a lot with the data they have, definitely far more than flipping a coin. It's not just hocus pocus. It's math.

Do those wait times look remotely plausible to you? Forget about FOP - 5 minutes for Everest in mid-afternoon?

 

[mesaboy2]

If math and statistics alone could predict the future, I would not be the middle-income schmo that I am.

 

[lentesta]

I have a spreadsheet that tracks the crowd calendar predictions for all 6 Orlando parks, It's what we use on our weekly call with the stats team to review their performance. Here's a screencap with the latest relevant stats.

For 2017, the Magic Kingdom predictions are +/- 1 about 70% of the time - so 5 days out of 7 in the week are exactly right or off by 1.
Epcot's predictions are +/-1 just under 76% of the time.
DHS is +/-1 around 72% of the time.
AK is +/-1 about 65% of the time.

Universal is 81% and IOA is 69%. The average for all 6 parks is just +/-1 72% YTD for 2017.

We think weather, ride breakdowns, and Disney changes to ride staffing are the biggest variables for the predictions right now. Our same-day forecasts adjust for all of those, but that's a separate process from these way-in-the-future predictions.

Ride breakdowns at Epcot, DHS, and AK are particularly challenging because they have so few big rides with posted wait times. A half-day outage at Test Track, for example, can add 2 points to a crowd level because of increased waits at the other headliners.

 

[Klayfish]

I really disagree with this. These folks are statisticians and know what they are doing. They collect data constantly. Maybe an English major like me doesn't understand the algorithms and the magic of their math, but I definitely believe that they are capable of a whole heck of a lot with the data they have, definitely far more than flipping a coin. It's not just hocus pocus. It's math.

So it can account for weather, current events, economy shifts, perception of what someone calls a "1" and what another person calls a "5", what park what people will randomly decide to go to on a given day? Yes, of course they can collect historical data and project it for the future. Doesn't really take a lot of math skills to do that. So you can then say "Based on what happened last September, an average crowd is x", but to say that 5 Tuesdays from now the wait at FoP is going to be xx minutes at 10am is pure hocus pocus...IMO.

 

[stinkpickle]

... but to say that 5 Tuesdays from now the wait at FoP is going to be xx minutes at 10am is pure hocus pocus...IMO.

True. TP is assigning a value (crowd level) based the expected average wait time between 11am and 5pm for a group of 24 attractions.

 

[jaysmom4285]

So it can account for weather, current events, economy shifts, perception of what someone calls a "1" and what another person calls a "5", what park what people will randomly decide to go to on a given day? Yes, of course they can collect historical data and project it for the future. Doesn't really take a lot of math skills to do that. So you can then say "Based on what happened last September, an average crowd is x", but to say that 5 Tuesdays from now the wait at FoP is going to be xx minutes at 10am is pure hocus pocus...IMO.

This is a very flippant rejection of sites like Touring Plans, which use many solid research methods to come up with their crowd predictions. Len Testa just noted that of course they can't account in advance for things like weather and ride breakdowns. But overall their predictions are solid. And this has nothing to do with what someone perceives as a "1" vs. what someone perceives as a "5." It has nothing to do with perception or how crowded the walkways feel or how long you had to wait in line for a counter service lunch or for a bus. They make it very clear that they are basing their predictions on wait times for the bigger attractions during the major part of the day, and this is very finite and measurable. Every day they publish an accounting of how close they were (or weren't) to their predictions compared to actual wait times, and they are very accurate, as Len noted above - much more accurate than a coin flip. No one can predict this sort of thing with perfection, and to dismiss them for not being perfect is just petty. But they provide a solid basis on which to help people tour the parks more efficiently, and, IMHO, I think anyone with any sense would take their information as a guide and not as a guarantee and would operate accordingly.

 

[shaqfor3]

I love touringplans, I subscribed prior to my trip in May (first trip since 1994) and now again prior to my upcoming trip in October.

That being said, you can verify the historical data of what they predicted and what they saw that day in the park. I did that and usually they were off by 1 or 2 points (example, predicted 7 but it was a 6, or predicted 6 and it was a 8)

But during my trip, it predicted my DHS day as 3 and my second MK day as 6. I was there and the parks were packed that no way i believe it was anywhere close to those numbers. When i checked the historical data, the DHS day (May 23, 2016 ) was a 9 and the MK day (May 26, 2016) was a 10, which makes me think that they are honest even when they are very wrong and that they truly only missed by a few points must of the times. (and i got real unlucky those 2 days).

But it didnt screw up the trip as I learned so much with their plans that I knew where and when to do stuff.

 

[TulipMom]

This conversation (my previous response included) is kind of confounding two issues, though, no? The crowd calendar forecasts are one factor and the projected wait times for specific rides are another. In this case, the focus is FOP. The crowd calendar forecast of AK being at 1 out of 10 on 9/6 is likely correct. But are the forecasted wait times reasonable? Peak wait times for FOP were at 130 minutes today on a 5/10 crowd level day and one week from now it will be at 33 min? I totally get that there is a lack of data for these new rides. Wouldn't it be a better strategy to lean toward inflating possible wait times for new rides and then let people be pleasantly surprised if the waits are less?

I have much respect for the data and forecasting. It's a huge asset for my trip planning. But, these Pandora wait times do seem to have an issue. Maybe TP will prove us wrong. @jessiegirl1982 - You'll have to report back to us next week!

 

[Dan Murphy]

Can someone who has a subscription to Touring Plans tell me what they expect MK to be on September 24th on a scale from 1-10?

No.

Information that is available only through subscription of TP, or other similar online sites that charge a subscription fee, are not available to be publicly posted. Subscriptions for TP, and other similar subscription sites, are laughingly inexpensive, even more so with readily available online discounts.

 

[ttintagel]

Their predictions have never let me down. Easywdw has also been a good resource.

 

[jessiegirl1982]

This conversation (my previous response included) is kind of confounding two issues, though, no? The crowd calendar forecasts are one factor and the projected wait times for specific rides are another. In this case, the focus is FOP. The crowd calendar forecast of AK being at 1 out of 10 on 9/6 is likely correct. But are the forecasted wait times reasonable? Peak wait times for FOP were at 130 minutes today on a 5/10 crowd level day and one week from now it will be at 33 min? I totally get that there is a lack of data for these new rides. Wouldn't it be a better strategy to lean toward inflating possible wait times for new rides and then let people be pleasantly surprised if the waits are less?

I have much respect for the data and forecasting. It's a huge asset for my trip planning. But, these Pandora wait times do seem to have an issue. Maybe TP will prove us wrong. @jessiegirl1982 - You'll have to report back to us next week!

Already planning on reporting back!!!

 

[hllb]

Yes, I think Touring Plans is very accurate, but the Pandora stuff won't be for quite some time since it's new. We've used them for both of our previous trips (around 15 days total) in May and September. They were spot on every day except one. I don't remember the details of that day, but remember thinking it was more crowded with longer wait times than they predicted. I signed up again for our January trip. I used it to plan my park days, even though for one or two days, they're predictions conflict with other crowd calendars. I've found them to be the better tool, so I'm going with it.

 

[BoatDriver]

I have a spreadsheet that tracks the crowd calendar predictions for all 6 Orlando parks, It's what we use on our weekly call with the stats team to review their performance. Here's a screencap with the latest relevant stats...

Along with the tools, this is why I subscribe to TP. Len and his team may not always be right, but to call what they do a wild-a guess is unfair to the time taken and the process itself. Heck, they grade themselves every day not only to see how they did, but to maintain integrity and, I would assume, make every attempt to get better.

The guy just responded to the question himself, gave an honest glimpse into the system, and even posted a snapshot of one of the flat files they use in their predictions. Can't really ask for more than that.

 

[jessiegirl1982]

Ok.. so late reporting back... 9/6 at AK was relatively dead. My dad was able to get a same day FP for FOP by looking 1 time, and he now thinks they are super easy to get (has no idea how rare that is). The lines for FOP were about 90 minutes that day in standby. I barely got a look at Pandora as there was a torrential downpour during our Navi FP time, and my second AK day was cancelled due to Irma. Did book a bounceback next Aug/Sept though (and hopefully that is a hurricane free trip!).

 

[Ouida]

More people need to post back. Let us know how it worked out, please. On my plan FoP showed 59 minutes at opening, and a crowd level of I think 4.