…intended to boost the patriotic feelings and confidence of statisticians in their work, according to the bureau’s Web site. Submissions should reflect on the development of the nation over the past six decades, discuss the role of statistics in national development and express the feelings of statistics workers towards this period in history…

WSJ kindly provides a translated sample stanza of “Love the Homeland, Love Statistics“

Life
Some mock me for doing statistics
Some loathe me and statistics
Some don’t understand what statistics are
Why is it that statistics
Put a calm smile on my face?
Because of statistics
I can solve the deepest mysteries
Because of statistics
I will not be lonely again, playing in the data
Because of statistics
I can rearrange the stars in the skies above
Because of statistics
My life is different, more meaningful
I love my life, my statistics

Gotta love statistical geekery.

What interests me (and what should interest you, if you’re trying to interpret this meaningfully or re-report the findings) is the method. The media report states:

“The Nielsen Global Online Consumer Survey, conducted by Nielsen Consumer Research, was conducted from 19th March – 2nd April 2009 among 25,420 Internet consumers in 50 markets across Europe, Asia Pacific, North & Latin America and the Middle East. The largest half-yearly survey of its kind, the Nielsen Global Online Consumer Survey provides insight into the opinions and preferences of Internet consumers across the world.”

So, we’ve got a sizable and diverse panel, but how many of those actually responded? No information. How were they recruited? No information. Maybe they were just the family, friends and contacts of Nielsen employees!? Do these people represent the internet population at large? Highly unlikely!

Unfortunately there is no indication of the statistical accuracy in Nielsen’s report. I know this report is aimed at the (generally) statistically naive media/bloggers, but it’s still important to give some indication of the statistical accuracy. Why?

You need to report the accuracy, otherwise people will make conflicting claims about how to interpret your data

If we don’t know the accuracy, we don’t know whether the numbers are really different or whether there’s so much noise in the data that the numbers are actually similar. Nielsen’s report has given us a real world example of this: people are simultaneously interpreting the numbers differently:

• Some people are saying 69% is close enough to 70% to be the same: e.g. GaugeInteractive claims “people view online virtual strangers who they’ve never heard of before, they trust their recommendations more than a newspaper editorial”.
• Others are claiming 69% is close enough to 70% to not be different: e.g. Marketing Exposed claims that “Brand websites carry just as much weight as posted opinions from other people online.”

Both interpretations are being made all over the web – but which is right? We could make arguments why either of them are right or wrong (and one of them surely is) but the real issue here is that Nielsen have a responsibility to ensure that right thing gets reported. A responsibility to their readers, to make sure we know the ‘real’ findings, and a responsibility to their own reputation: I’d hate to see my findings re-reported as contradictions!

More detail about errors and accuracy in your press reports please.

• use polls as a way to encourage audience participation, rather than to ask a genuine question
• ask bad questions, that lead –if not cause– their audience to respond a particular way or give a particular answer
• collect biased responses from their audience, and claim that the responses represent their target population at large
• analyse the data badly.
• misinterpret or overinterpret their analysis.
• market/report their findings misleadingly.
• don’t correct others’ misinterpretations of their “findings”, perpetuating further problems.

There are so many things that can be done wrong – so many oppotunities to stff up – that we need to be very very careful of the “research” people publish online.

Advice: evaluate research by its methodology, not its findings.

Great. We need a diagram to underscore that:

WebScience.com's image trying to explain what 'web science is.

(You can see a bigger, more detailed version at the webscience website)

The lack of overlap in this venn diagram clearly indicates they only look at a few particular approaches at once. Web Science isn’t a combination of all of these things - it picks and chooses what it feels like referring to!

…Somehow I don’t think that’s what they’re trying to say in this venn-inspired pseudo-graph.

I worry about such pseudo graphs. The public is increasingly data-aware, and are being exposed to real graphs more and more: data visualisation is becoming more and more prevalent. As a result, this sort of diagram holds special power that seems to be based on a psychological information-processing heuristic: graphs = based on data = professional = accurate = correct. That’s not necessarily the case (obviously).