andrewducker (![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
andrewducker) wrote2010-02-12 04:10 pm
andrewducker) wrote2010-02-12 04:10 pm
It's how you ask the question that matters
The New York Times took a poll. They asked half the people whether they thought that gay men and lesbians should be allowed to serve openly in the US army. 60% said yes. They asked the other half whether they thought that homosexuals should be allowed to serve openly in the US army. 44% said yes.
One can only assume that people are made of crazy. And stupid.
From
One can only assume that people are made of crazy. And stupid.
From
no subject
What we're interested in is the chances of them producing such a differential through sheer chance - which will be staggeringly low for a sample size that high.
no subject
Anyway this sample is too small to be more than curious enough to merit further sampling, and it certainly doesn't merit wild accusations of stupidity or insanity on its own.
no subject
Assuming proper randomization then I reckon the power of this sample if probably enough to make generalisations about the wider population.
Dependent on the geographical area sampled of course. Although studies of relatively low sample sizes can be used to make all sorts of valid wider generalisations depending on the statistical tests used.
I'm not entirely clear what you mean by willfully incorrect answers?
Although I agree that the result doesn't merit wild accussations of stupidity or insanity. All it shows is that "homosexual" is a far more loaded term then gay men or lesbians. Or that there was some systematic bias in the way the questions were asigned to participants.
no subject
no subject
So how am I arguing against myself when I say that it I don't see it as definitive but it is a curiosity to look deeper at.
Wilful wrong answers refers to something I read about how any survey contains a small proportion of respondents who deliberately give answers opposite to what they think is expected. All I wondered was is this where the 4% +/- error comes from?
no subject
Fisherian statistics and significance is essentially what you are talking about - or classical statistics if you prefer. In that the higher your sample number the better chance you have of detecting a signal from noise in terms of inference.
Also where does the 100 million number come from? The population of America is just under 400 million and the population of NY state is 19 million.
The raw data doesn't tell us much and extrapolating that makes little sense. However using inferential stats. we could see how likely it is these distribitions are seen within the wider population.
500 to 100 million is not nessecarily that unconvincing.
no subject
http://research-advisors.com/tools/SampleSize.htm
no subject
I'm just naturally sceptical about broad statements based on minimal data, and its easy to say its not unconvincing but why isn't it? Why is a sample of less than 0.001% really comparable to the whole?
Maybe I just don't get it, but that extrapolating 500 to 100 million just seems like the claims that are made for homeopathy.
no subject
http://en.wikipedia.org/wiki/Law_of_large_numbers
http://en.wikipedia.org/wiki/Central_limit_theorem
If there are holes in the mathematical reasoning contained in these articles, then kindly point them out. If not, then accept that one of the things that mathematics is for is to override your instincts when your gut feeling about (e.g.) the necessary sample size is dead wrong.
no subject
Does that make sense?
Of course there is a possibility that this is what happened but it has not been reported that way.
no subject
1) Whether this scales up to much larger numbers, assuming that the population is similar - and from my understanding it does this pretty well.
2) Whether this would scale across the entire US - and on that we lack sufficient data, because people in Texas would not answer the same was as people in New York, and the differences are large enough for that to be a major factor, so scaling it in that way would be inappropriate.
And in any case, for this kind of research you want to do it at least twice, to prove that the findings are replicable and to make sure someone didn't do something stupid the first time.
Arh, the clasic six sigma question
My experience is from corporate sampling.. where the confidence is closer to 98 - 99 percent.. I believe the recognised confidence in public survey is 95%
The error rate goes down as the sample goes up.. The fact this is, and it IS a low sample rate is taken account by the error rate of 4% - which for a sample of 5-600 is about right..
so, rather than the percentages, let’s have a quick look at the actual figures..
1054 people were called.. that’s the ones that weren't busy and took the call.. and as many folks have pointed out that better be a representation of all states, and all backgrounds.. I assume not talking into account most serving armed forces, their phones just aren’t useful at the moment in Afghanistan, nor criminals.. nor anyone that works high up in most organisations, as their PA's would have taken that call..
Total yes % no % Yes # No #
gay & lesbians 542 60 40 325.2 216.8
homosexuals 542 44 56 238.48 303.52
so if that distorts when i post, 325 v's 238
when you take the 4% into account, you get a bunch of thresholds...
-4% +4%
gay & lesbians 303.52 346.88
homosexuals 216.8 260.16
what to look for here is the distance from the upper of one to the lower of the other.. if they overlap, the error is making the statistics show the same thing..
in this case the gap is around 43 people. which is 0.039667897
or 4% ( nothing to do with the error, if your not following )
remember that 95%..
"Wilful wrong" as it has been put here, isn't covering the 4% error it's the confidence I touched upon at the beginning.. ( although not something I had heard it called before, but from pigeonhed's description of whet it was when he read about it..
In my world, Its 99%, and the remaining 1% I call the "people that want to get fired" in the public world, i guess a better descriptive are tossers.. with a few that just didn't understand I'll let them off and more to the point, that’s a deficiency in the survey.
therefore the confidence percentage wipes out 54.2 peoples answers...
meaning our difference of 43 people can be looked at with a bit of statistical scepticism...
I know I am at the upper ends of all the threasholds, but if asking a few thousand more people would make it so I can't complain, then I'll belive the statistics. More to the point, if you belive the statistics as they stand, then follows that my statistics are just as valid..
For me.. I have to agree with pretty much all of what pigeonhed has said..
Is it significant, that’s what Andrew originally asked.. I think yes.. and are they stupid, I have agree, yes.. but are the statistics sound.. No.. not convinced.. up the sample rate to closer to 2k, and the error would drop, and it would blow my sums out of the water..