www.publicinvolvement.org.uk

Notes from the field of public involvement

Wednesday, 09 May 2007

A Challenge to the Wisdom of Crowds and the Value of Collecting your own People Power Data

It is a salutary reminder that web entries have a long life when something written and posted a while ago (The Wisdom of Crowds and the National Health Service) can still catch someone's eye and prompt a response. It is rather encouraging. We have written before about the nature of a web readership in our article 'Cor, luv a lurker' and how little we ever hear back. So let us applaud Mr Paul Barker a consultant histopathologist who wrote to us recently saying

"You cite Surowiecki's criticism of medical pathologists. Since this is my profession, I know something about the issue of the accuracy of our work and I know that what Surowiecki writes is highly misleading. The evidence that he cites says nothing about the diagnostic accuracy of pathologists. See my critique at http://e-immunohistochemistry.info/Surowiecki.htm. If he is so wrong on something that I know about, I wonder how reliable is he on anything else?"

If nothing else, it does help us be cautious about what has been for some journalists like Surowiecki and 'Tipping Point' Gladwell a very good source of rich pickings - the ability to research scientific and other learned publications and and recognise the nuggets of value and translate these hitherto obscure trouvailles into books that lots of lay people can read, enjoy and write about in blogs.

But as Mr Barker points out

"Surowiecki has taken this information from a paper by Shanteau1 and, to be fair to Surowiecki, he is echoing the opinion of Shanteau that 'This [the correlation coefficient of 0.5] suggests a low level of competence.' Shanteau in turn was citing a paper by Einhorn2. Einhorn's paper, now 33 years old, reported a study of just three histopathologists, of who one was a resident (trainee)." (All the numbers of course refer to citations of the original works)

So I am afraid we have just re-learnt the lessons that nothing beats going back to the original sources and that the process of translation however interesting and readable the results can sometimes be based on shaky ground.

It is an argument for doing your own research amongst your own staff and users. We find in the health service that  many assumptions are made about user preferences and choices based on data that is old and out of date where the findings were based on specific times, places and circumstances and do not stand the test of transference and what we have now learnt to call 'contestability'. If the research is done in the light of your own circumstances then you have not only collected better data but there is a good chance that you will understand what the data are telling you.

The Moore Adamson Craig Partnership helps organisations understand their users better by organising surveys for them that address the concerns and experiences of their users. Their broad experience in such specialised areas as complaint handling allows them to interpret the findings with confidence and creativity while respecting professional standards of accuracy and organisation.

Colin Adamson | (1) comments | Trackback

 Replies to A Challenge to the Wisdom of Crowds and the Value of Collecting your own People Power Data

Hi Andrew

Just a quick hello and please keep up your good work on PPI. I regularly refer people to your MAC site. Thanks again for highlighting Ten Great Myths Paper in your resources section.

You may like to see Simplicity Blog today about my workshop ‘Trust Me I’m a Patient’

Warm regards

Trevor


Post a comment
Comments are disabled