Crunching the numbers on WDWguruDH's resort satisfaction poll

[Granny]

Okay, first off, these are not scientific polls.

Secondly, different people interpret words differently, such as "disappointed, expectations, etc".

Thirdly, small samples such as SSR can really throw the numbers off....a few more votes could radically change the percentages.

Lastly, we cannot assume that these numbers represent the average DVC visitor experience. So we shouldn't project the numbers to all DVC visitors. And we shouldn't assume they mean anything more than fodder for discussion.


So....



As of 3:05 p.m. EDT on August 9, here are the results of the polls thus far:


The first summary is a combination of the first two responses....Excellent & Good. I included Good because it was defined as meeting all expectations. So the resorts should not be "punished" by high expectations. To me, if a resort is rated Good or Excellent, that's pretty impressive.

Votes to date by resort (Votes for Excellent & Good/Total Votes)

VB = 53/57 = 93%
BCV = 50/74 = 68%
VWL = 95/104 = 91%
BWV = 96/120 = 80%
OKW = 124/135 = 92%
HH = 47/48 = 98%
SSR = 20/28 = 71%



Or, to look at it the other way, we can rate dissatisfaction with a resort by combining the "Disappointed & Really Disappointed" votes.

Votes to date by resort (Votes for Disappointed & Really Disappointed /Total Votes)

VB = 0/57 = 0% (WOW!)
BCV = 8/74 = 11%
VWL = 5/104 = 5%
BWV = 8/120 = 7%
OKW = 2/135 = 1.5%
HH = 1/48 = 2%
SSR = 4/28 = 14%

 

[Doctor P]

Well, I notice that since your post, VB's numbers have gone in the tank--1.67% now report being disappointed.

 

[Doctor P]

It is also interesting, noting that some of this is a small numbers problem, that the two newest resorts have the lowest ratings and the three oldest resorts have the highest ratings.

 

[kathleena]

I agree Granny. It's hard to know what some define as "expectations".

Case in point (and not trying to flame anyone here, just making a point). Someone said they were disappointed in the small rooms at BWV and it seems like their rating played on that. I, on the other hand, considered the options purely on the room condition and service from the staff. I totally discounted everything such as pools, slides, restaurants, room size etc - because those to me are all part of the resort description - they are givens before you even get there. Room condition and service are not.

So unless you can control what everyone defines as "expectations" then the results are skewed.

I will also note, that BCV and SSR are the newest resorts. Perhaps after being open longer, and the learning curve starts to flatten out, their numbers will be more on par with others.

I would like to have seen this same poll with the others a few months or a couple of years after they opened...............

 

[Granny]

Doctor P....LOL on the VB rating going "downhill".

As for the "meaning" behind the numbers, my guess is that there are probably several explanations. Most people could only vote on a couple of the polls, so we don't have the same people rating each resort.

As mentioned, these polls are just for fun. I hope that nobody takes them as "resort bashing" or such. I just thought that it would be interesting to recap the polls and see how the numbers fell out.


And kathleena...I agree wholeheartedly. One person's disappointment might be someone else's "no big deal".

 

[spiceycat]

boy if makes you feel better - you should have been here for the polls at BWV when it was new - horrible - everyone here hated it - I kept my opinion to myself - but I brought points there - so it was pretty great in my opinion.

give your resorts time - they will get more popular - although they are both pretty darn popular right now - for them to get more popular only their owners could get in.....

 

[crisi]

Re: small sample sizes and radically changed percentages.....

Set the power to .95 (we want to be 95% sure we have the true number in her) and run these through my stat software

VB - 93% satisfied - could be anywhere from 99-77%
BCV - 68% satisfied - could be anywhere form 47-85%
VWL - 91% - 98-78%
BWV - 80% - 91-65%
OKW - 92% - 98-81%
HH - 98% 100-85%
SSR 71% 94-39%

None of these are statistically different at this point. Given an adquete sample BCV - which has the lowest rating of 68% could really have satisfaction numbers of 85%. And HH, with 98% satisfaction - could really be 85%.

To get real confidence in these numbers - were I designing the survey - I'd want at least 530 responses.

 

[crisi]

Oh, and I'd want a more randomized population to survey. People who have never stayed at BCV but need to put in their two cents about something they heard happened is not a good sample. Disboarders are too informed about other peoples problems - and sometimes what is not a problem without knowing about it because a problem because someone else has noticed it (the recent "lack of staff at the BW guest services desk" would be an example of mine).

 

[CarolMN]

Originally posted by crisi
...(snip)....To get real confidence in these numbers - were I designing the survey - I'd want at least 530 responses.

And of course you would also want a sample that was scientifically selected - so that we could say with confidence that the results of the sample reflect those of the entire population!



Best wishes -

edited to add:

I see we were typing and thinking the same thing!

 

[DrTomorrow]

I say let's non-normalize the whole thing! Skewness and Kurtosis Rule!

 

[Jimbo]

The other thing about this is that we have stayed at OKW 10 times. The resort's been open for 12 years, and I know that others have stayed many more times than we have. But we all get one vote, not one vote per visit.

Not being stats-oriented, I have no idea if this skews the polls. And in which direction, if it does. But I know there's a geek or two that will be happy to answer this.

 

[mom2alix]

Either way and on the "glass half full" side of things, it looks like the vast majority of resorts get an A for meeting or exceeding expectations. I owuld think these would be pretty impressive rating compared to the industry at large. (Not that I'm saying the negative experiences don't exist or shouldn't be corrected, but this certainly shows that the trend is to the positive!)

 

[PamOKW]

I think there's always a "mischief" factor in these polls as well -- someone who thinks it's fun to give a resort a low rating. Maybe we throw out the high and the low? I agree with mom2alix that the good news is that the majority of DVC'ers seem to be happy campers at all the resorts.

 

[Granny]

Originally posted by spiceycat
boy if makes you feel better - you should have been here for the polls at BWV when it was new - horrible - everyone here hated it - I kept my opinion to myself - but I brought points there - so it was pretty great in my opinion.

According to the poll, still only 80% of the BWV visitors have been satisfied with BWV. That's all I need to see to determine how these polls go.

And crisi...thanks for the detailed statistical analysis.

Obviously, those who venture off-site are very happy indeed!


And Dr. Tomorrow.....you continue to scare me!

 

[kathleena]

How about another poll to see what the dissatisfaction is? hahahaha I'm just half joking here.

One person may want dedicated self-parking, but to me a full parking lot justifies the "expense" of the valet and makes me less guilty for taking it.

One person may want bigger rooms and be disatisfied there, but for me - the smaller room for such a short period of time is a fine trade-off for the location.

It's all relative - that 80% - I would say if the poll were just for BWV owners, the number would be higher.

 

[spiceycat]

the survey is not asking just the owners - but everyone who stated there...

BWV is not favorite - but it is a great resort - especially if you like to party.....of course with SSR so close to PI - it might give BW a run for the money - but BW is close to 2 parks plus the BW - so I think it still wins....

 

[crisi]

Originally posted by Granny
According to the poll, still only 80% of the BWV visitors have been satisfied with BWV. That's all I need to see to determine how these polls go.

And crisi...thanks for the detailed statistical analysis.

Obviously, those who venture off-site are very happy indeed!


And Dr. Tomorrow.....you continue to scare me!

If you want to repost larger numbers (I don't want to dig in each thread today), I'll rerun the intervals. The margin should get smaller.

(And we could always transform all that non-normal data. Algebraic log functions are my favorite!)

Jimbo, I wouldn't expect the results to skew too much from people by including people who have had one trip and people who have had several - since the question is overall satisfaction. You have more opportunities to find problems with multiple trips, but most people who regularly stay at a resort are going to know when the problem is not the norm and factor it out of their response. Unless the quality of the resort has changed drastically over the past few years. We aren't really interested in what OKW was twelve years ago, just satisfaction as of today, and people are likely to weight their recent stays higher in a question like this.

Now, if I were designing a survey, I wouldn't ask anyone who'd stayed at the resort longer than a year ago, and I would phrase the question to ask about the satisfaction of their "last trip" I'd also put a correlating question in their about how many previous stays they've had at the resort. I wouldn't be surprised to find correlation between satisfaction and a fairly high number of previous stays - loyalty breeds satisfaction and satisfaction breeds loyalty.

The bigger problem with Disboard surveys (and all internet surveys) is that the sample is self selecting. We aren't typical DVC members (i.e. we have a greater rate of AP ownership than typical, I'd guess we have more points than the mean, etc.). We are better informed, not only about how DVC works, but about our fellow Disboarders stays, problems and gripes. I think we have both greater expectations for our stays (we know how the room request system works and when it doesn't work to our advantage can get upset), and lesser expectations for our stays (we've heard dirty room stories, so it disappoints us when the dishwasher wasn't run, but it isn't a complete shock.) There is nothing keeping me from answering very dissatisfied on the OKW poll - despite having never stayed there - just to skew the results, nothing keep me from answering very satisfied on the VWL poll (I've stayed at the WL and love it, never stayed at the Villas though) - because I have a soft spot in my heart for that resort and want it to do well.

 

[DrTomorrow]

Originally posted by Granny
And Dr. Tomorrow.....you continue to scare me!

I spent too many years writing statistical simulation software for CAD/CAM systems. Sometimes I scare myself; I wake up screaming "Beware the Pearson Beta" and "No! No! Not the Outliers". ::smile::

 

[poohnpiglet3]

You guys all remind me why I got a D in stats in college... my only one.



Sherri

 

[crisi]

Hey, at least you took stats in college. I haven't taken a math class since high school algebra and fully believed I wasn't good at math until a few years ago. Now I do stats for a living.....isn't life funny.