FP+... who hates it

[NavyDad]

I look at Tripadvisor a lot when I'm going to be staying at a hotel I've never been to before or eat at a place I've never eaten at before. I also look at the reviews on Amazon and Walmart and a bunch of other sites when I'm researching a purchase.

I've never thought about whether or not those reviews were representative of the entire human race or just some subset thereof because that wouldn't really matter to me. What does matter to me are the reported opinions and experiences of people who have gone to, stayed at, eaten at, or bought what I'm considering. I read both the positive and the negative and evaluate both before making my decision.

I suspect a lot of other people use that information in much the same way.

I agree. I do the same thing. They are great resources.

Sometimes a small sub-group's opionion can be very important because they are very interested or concerned with the subject they are reviewing. They are going to pick up on changes positive or negative and can provide great feedback.

Let's say you had a group of people that really like Olive Garden. They have been going to Olive Garden for a long time and they knew every knook, every booth and every server at that Olive Garden. Over time they might be upset that their favorite table was moved or thier favorite server was let go or even their favorite dish was taken off the menue. Beause they are so invested in Olive Garden they would probably take the time to fill out review cards or manager suggestion box cards. That feedback would be very important but would not be representative of all of the guests that visit Olive Garden including the random travellers and 1 time visitors.

 

[NavyDad]

May I suggest the possibility that you do actually care about the composition of the sample? You say that you read and evaluate both the positive and negative ... let us say that you have kids (no idea IRL), would it not affect your weighting of the negative reviews if a large percentage of them started of "Perhaps I would have felt differently if I had kids, but ... "?

For the individual, these sites are useful even if they do not completely represent the population because we are able to digest the individual reviews to identify those whose circumstances are similar to ours, or who have focused more on features that we particularly care about. Even if most reviewers don't have kids that would be OK, because you'll be able to focus on the few reviews that you identify as having high value to you.

A business though is trying to look from the standpoint of everyone in their market, instead of picking out the examples that are applicable to them. Here, not being able to find representative opinions from every pertinent demographic is a big problem.

That is a great point!

If company A knows that 35% of their customers are married, 25% have kids and 60% are first time buyers they would want a poll that represents their market.

A Poll that included 55% married, 45% have kids and 15% first time buyers would not be representative at all and would not give them any credible results.... even if it had a large sample size...

 

[Cormoran]

It's interesting, but Themepark Insiders vote of the week that started today is on this subject. It is still a biased poll in that it's theme park lovers and the it may be too early for definitive results. But the initial results are interesting.

View attachment 84653 (Screen shot as of 12:45 p.m. 3/27)

Based on early results, if you ignore the people who didn't visit 62% liked FP+, 33% didn't like FP+ and 5% didn't use FP+. Based on those results, like it or not, FP+ has a 95% adoption rate. I don't think legacy FP was ever that high. And by a substantial percentage, most people liked FP+. It will be interesting to see how the poll progresses. Of course after mentioning it here, everyone will run over skew the results.

FP+ has a 95% adoption rate because they have added it to everything including stupid stuff like EO and Figment. If you want to avoid long stand by lines for 3 attractions You have to use FP+. 26% of guests in this survey don't like FP+. Most companies who conduct these kinds of surveys, get really uncomfortable when 20 % of their customers are not satisfied. This doesn't look good.

 

[Shaden]

Its actually only 95% of the people who responded to that poll ... but you know. Representative surveys and stuff ... research methods ... who really cares about that stuff.

 

[Alfalfa]

FP+ has a 95% adoption rate because they have added it to everything including stupid stuff like EO and Figment. If you want to avoid long stand by lines for 3 attractions You have to use FP+. 26% of guests in this survey don't like FP+. Most companies who conduct these kinds of surveys, get really uncomfortable when 20 % of their customers are not satisfied. This doesn't look good.

Its actually only 95% of the people who responded to that poll ... but you know. Representative surveys and stuff ... research methods ... who really cares about that stuff.

Reminds me of the old quote:

“Statistics are like a bikini. What they reveal is suggestive, but what they conceal is vital.”

 

[Mkrop]

I have to think they are "monitoring" this and planning accordingly. Seems odd they would purposely increase the kiosk lines just to frustrate and have folks give up, but the theory is out there.

.

It wasnt a "theory" Christmas week. It was an absolute disgrace how long the line was at DHS with CMs with Ipads in front of the hat. And it was like that every time we walked by. They should have more CMs out there. We tried the other kiosks and their lines were long as well, we wound up at the hat and I got in line while the boys ran to the bathroom. They went and came back and I had moved up one space. We finally got up there and there was nothing left that we were interested in. Actually I think they were pushing Muppets and Voyage of Little Mermaid. We declined and left at 3pm.

 

[Shaden]

It wasnt a "theory" Christmas week. It was an absolute disgrace how long the line was at DHS with CMs with Ipads in front of the hat. And it was like that every time we walked by. They should have more CMs out there. We tried the other kiosks and their lines were long as well, we wound up at the hat and I got in line while the boys ran to the bathroom. They went and came back and I had moved up one space. We finally got up there and there was nothing left that we were interested in. Actually I think they were pushing Muppets and Voyage of Little Mermaid. We declined and left at 3pm.

Its not a matter of pushing the kiosk lines, its a matter of the kiosk lines being there on purpose. As we did the math earlier in this thread I think .... but a single 10 minute wait per day at a kiosk means essentially saving the demand for an entire attraction in a park ... how much does an attraction cost ? I mean, NFL gave us 3 for 400 million .... so ... 120 million ? pretty easy ... make em stand in line.

 

[Planogirl]

I once majored in a discipline that involved sampling principles so this interests me. How would some of you take accurate surveys? I see many comments that surveys can be skewed which is of course true but then how do you sample in a meaningful way?

 

[Cormoran]

I once majored in a discipline that involved sampling principles so this interests me. How would some of you take accurate surveys? I see many comments that surveys can be skewed which is of course true but then how do you sample in a meaningful way?

I have no idea , but it's not the accuracy of the survey that I question. It's the conclusions that people make when they look at the results.

 

[Planogirl]

I have no idea , but it's not the accuracy of the survey that I question. It's the conclusions that people make when they look at the results.

You make a great point. Interpretation of the results can be skewed just as the surveys themselves. And truly 25% disliking something isn't great. Will that number drop or increase?

Plus how does the casual visitor feel? That person doesn't go to themepark insider.

 

[Jennytoon]

I once majored in a discipline that involved sampling principles so this interests me. How would some of you take accurate surveys? I see many comments that surveys can be skewed which is of course true but then how do you sample in a meaningful way?

Well, there are two problems here ... one is construction of the survey itself, the other is collection from the sample.

When I say construction of the survey itself I mean the difference between the question "How do you feel about FP+ in comparison to FP- [I liked it better / I didn't like it as much / I like it about the same / I don't really have an opinion]" vs. "Do you feel that FP+ is 1000 times worse than FP- [True / False]". That is mostly not what has been under discussion here though.

As far as selection of the sample goes -- I think posting CMs at the exits is a tried and true method, and it guarantees one thing that many other approaches will have a hard time with, which is the ability to select from the entire population of people who go to the parks. It is true, as JimmyV says, that some number will still self-select by opting out, but there are ways to mitigate that ... "Sir / Ma'am, we know this will take a little time and we'd like to make that time up to you, in exchange for your participation we're happy to offer you a set of FP for any park for your visit tomorrow". It's also true that there's a potential for selection bias here: "they only go up to happy people", but honestly Disney is smart enough to know that that strategy would defeat the whole point. Biased data is not as useful to them as accurate data, because decisions made on bad information are inherently more likely to be flawed. Those CMs are not collecting statistics for Disney to splash across their marketing (although they may sometimes use them that way), they are collecting information to do exactly what the word implies -- inform their choices moving forward.

 

[Cormoran]

You make a great point. Interpretation of the results can be skewed just as the surveys themselves. And truly 25% disliking something isn't great. Will that number drop or increase?

Plus how does the casual visitor feel? That person doesn't go to themepark insider.

I don't think the 25% will drop unless Disney does something to improve the current system. For example remove tiering or allow the 4th FP to be booked on via the app. But there aren't enough attractions for either of these FP+ improvements to occur.

 

[Planogirl]

Well, there are two problems here ... one is construction of the survey itself, the other is collection from the sample.

When I say construction of the survey itself I mean the difference between the question "How do you feel about FP+ in comparison to FP- [I liked it better / I didn't like it as much / I like it about the same / I don't really have an opinion]" vs. "Do you feel that FP+ is 1000 times worse than FP- [True / False]". That is mostly not what has been under discussion here though.

As far as selection of the sample goes -- I think posting CMs at the exits is a tried and true method, and it guarantees one thing that many other approaches will have a hard time with, which is the ability to select from the entire population of people who go to the parks. It is true, as JimmyV says, that some number will still self-select by opting out, but there are ways to mitigate that ... "Sir / Ma'am, we know this will take a little time and we'd like to make that time up to you, in exchange for your participation we're happy to offer you a set of FP for any park for your visit tomorrow". It's also true that there's a potential for selection bias here: "they only go up to happy people", but honestly Disney is smart enough to know that that strategy would defeat the whole point. Biased data is not as useful to them as accurate data, because decisions made on bad information are inherently more likely to be flawed. Those CMs are not collecting statistics for Disney to splash across their marketing (although they may sometimes use them that way), they are collecting information to do exactly what the word implies -- inform their choices moving forward.

You could end up with several potential problems when posting CMs at exits. First of all, CMs are human and I would think that generally people would be more drawn to happy looking people than someone who looks angry or sad. Plus would offering more FP+ to those who are unhappy with it really make them more inclined to answer the survey? If they do take it could they feel more willing to forgive any shortcomings if offered more FP+?

I remember being taken into a room just backstage at both Disney and Universal and being asked a series of questions in years past. They offered A/C and something to drink which was really pleasing at the time.

 

[Alfalfa]

I was once taken into a back room at the Las Vegas Hilton - but it wasn't to ask my opinion.

 

[Angel Ariel]

It is true, as JimmyV says, that some number will still self-select by opting out, but there are ways to mitigate that ... "Sir / Ma'am, we know this will take a little time and we'd like to make that time up to you, in exchange for your participation we're happy to offer you a set of FP for any park for your visit tomorrow".

FWIW,I was approached for a survey in November. All they asked for at the time was name,e-mail addresses,and maybe city/state. The survey was e-mailed to me a few days after we got home from our visit. Nothing was offered for our time or anything since the in park time was 5 mins or less.

 

[wisblue]

Even with a proper sample, a survey has to ask enough questions to get useful information, depending on what you are trying to find out. A survey like the one above that only asks if someone liked or disliked FP+ leaves open a lot of other questions about why the responder did or didn't like it.

The first thing I would be asking is how many times a responder had visited WDW in the past and whether or not he had used paper FP. For guests who had used paper FP, I would be asking which system he preferred and some follow up questions about what features of each system he liked and disliked. For guests who hadn't used paper FP, I would ask what aspects of FP+ he liked or disliked. These follow up questions could include subcategories to bring out responses about things like advanced planning, number of FPs offered, selection of attractions, availability of same day FPs etc.

I would also be asking respondents when he made his reservations to see if satisfaction is greater for onsite guests who made reservations at or near the 60 day mark as opposed to guests who made the reservations later or only received their original FPs at the park.

Without a lot of this background information, I find the results of a simplistic survey like this to be somewhat interesting but not very conclusive of anything. The self selection bias makes them even less meaningful.

For example, if someone simply has a choice of like/dislike, someone who answers "like" might be saying "I like it, but it could be better", "I like it, but I liked paper FP better" or "I like it and I think it is superior to the paper FP system", or lots of other things. Similarly, someone who says they don't like it might be saying "I don't like it because I don't like planning in advance" or "I don't like it because I liked paper FP better" or "I don't like it because I think I should be able to get a FP for everything I want", or countless other possibilities. With enough information, someone interpreting the results can differentiate between things that could be addressed (like "I would like to be able to get additional FPs on my phone"), and things that are unrealistic within the constraints of attraction capacity and serving all guests (like "I want more FPs and no tiering").

I would also be including some questions about the guest's overall experience on his trip. A response of "I didn't like FP+, but I had a great time overall and will definitely be back" is a lot different than "I liked FP+ but I had a lousy time overall and will never come back". Of course there all kinds of points in between.

This is why a "dislike" percentage in isolation should not be at all surprising or troubling. There is no way that a company is going to satisfy every single person about not just the overall experience but about every aspect of the experience. I also think, and I have said this before, that for guests who have experienced paper FP (and especially those who mastered and made extensive use of paper FP), FP+ is starting with a distinct disadvantage. I suspect that the reaction to FP+ would be significantly different for a lot of people if it had been the first FP system instead of following paper FP. If FP+ had been first, people might think "I don't particularly like having to pick things in advance, but it is nice to be able to skip 3 of the standby lines at times that I can select to fit my touring style". Then if FP+ had been replaced with paper FP, people would be saying "What, now I have to take whatever time the system gives me and I have to wait to pick a second one. That stinks". If nobody had ever been exposed to a system that allowed certain savvy guests to get multiple FPs for some attractions while other guests were shut out, things like tiering and limits on the number of FPs would not be as controversial.

The bottom line is that, while simple online surveys like this can provide some interesting chat board fodder, any attempt to draw meaningful conclusions from them about what a company should or shouldn't do is pure folly. I have no doubt that the Disney company has much more meaningful data, and will make business decisions based on that data, not by counting up comments on websites.

 

[Jennytoon]

You could end up with several potential problems when posting CMs at exits. First of all, CMs are human and I would think that generally people would be more drawn to happy looking people than someone who looks angry or sad. Plus would offering more FP+ to those who are unhappy with it really make them more inclined to answer the survey? If they do take it could they feel more willing to forgive any shortcomings if offered more FP+?

I remember being taken into a room just backstage at both Disney and Universal and being asked a series of questions in years past. They offered A/C and something to drink which was really pleasing at the time.

I agree with you that CMs are human, but that's assuming that the CMs are given free choice, as opposed to some instruction like "one from every 20th group that passes you". Small remunerations (such a set of one any-where / any-time FP for each person in the group) are shown to generally not affect survey outcomes, and I think even people who are unhappy with FP+ don't have a problem with the use anywhere type. That said, they could just as easily offer a snack credit coupon or some other token, it was just a suggestion to illustrate the idea. I think taking you into A/C and offering you something to drink, which may affect your mood compared to being hot and thirsty, is more likely to produce a difference in the survey results than a small token of any type offered to take the survey on the spot.

FWIW,I was approached for a survey in November. All they asked for at the time was name,e-mail addresses,and maybe city/state. The survey was e-mailed to me a few days after we got home from our visit. Nothing was offered for our time or anything since the in park time was 5 mins or less.

That's really interesting. It doesn't match my (one) experience with this in the past which was several years ago. While I wouldn't expect them to offer you something if the time was as minimal as described, the fact that they are emailing it later tells me they likely are receiving what they consider to be a satisfactory response rate and representation using this method, or that they are using alternate methods as well and are interested in comparing the results that they receive from the two different populations. Emailing later does allow them to get around one big problem that they potentially have at the exits, which is language.

FWIW, when I was surveyed it was on the spot and was long enough to be noticeable but not terribly time consuming, and they did not offer me anything either. My description was not meant to be an instruction of how they should do it, just an answer / thought experiment to Planogirl's question about methods that would get around some of the issues raised in the discussion.

 

[wisblue]

In all of my visits, I have been asked several times to answer a couple of very quick questions that seemed to be designed just to get basic demographic information like zip code, number of times we've visited, etc. One time years ago I was asked if I would complete a survey and I was led into a room in guest relations to answer a number of questions on a computer screen. It was so long ago I don't remember the subject. I've also received a lot of surveys by email after I got home, but those usually seemed to focus on the experience at the hotel/resort.

 

[MrInfinity]

Just a friendly reminder of how easy it was to pull Legacy FPs. And these are just the ones we brought home and didn't use. This is from DL last year.

I came home with pockets full of them too -- we put them in our scrapbooks. That's bad tho right? It is a sign of waste. It means we spent time pulling a FP to something we didn't need, or didn't want, or ended up choosing not to use. That is one of the savings of FP+. Instead of going home with unused ones, you change them on the fly to something you will use.

 

[Cormoran]

I came home with pockets full of them too -- we put them in our scrapbooks. That's bad tho right? It is a sign of waste. It means we spent time pulling a FP to something we didn't need, or didn't want, or ended up choosing not to use. That is one of the savings of FP+. Instead of going home with unused ones, you change them on the fly to something you will use.

No, you can still waste FP's in the current system. If you decide not to use it and you don't cancel or change it. With Legacy, if I didn't use a FP, I usually gave them away. With FP+, I just let it expire, because "Giving it away" involves logging into the app and wasting battery life or standing in line at a Kiosk.