More from the Horse’s Mouth – Part 2

happyhorse21Stepping back from a particular activity – inspection – this second paper takes a broader view of engaging the public or service user. Here we  match the views or the holders of views that are wanted with a methodology that offers the best chance of getting the output required and gives some indication of the above the line costs. The table below takes the exercise to be embarked upon as the “Course” and the matching technique/approach as the “Horse” who will run the course and give you the result or outcome that you need. 

Exploring General Issues

Before looking at the table, we explore some general issues that arise when we want to find out what the patient, carer or user thinks. The core assumption behind the various choices of methods is that the consulting body is going out to get answers to particular questions. The exercise is managed in a way that maximises the chances of finding those answers within is usually a tight schedule. However this is not the only context where we need to gather and understand patient and user data. Engagement with these stakeholders is an ongoing exercise.  In our previous blog, we talked about how patient and user data can gradually accumulate over time. Converting what is often fragmented information, scattered across a variety of channels, needs skill and patience. This is much more difficult than mining the tables in a quantitative survey where the numbers spell out the story.

“Weak Signals”

The art of interpreting these so-called “weak signals” calls for new skills in the interpretation of data. A photo here, a set of comments there, a complaint somewhere else take a lot of pulling together (and validation through some of the other methods discussed) before they make sense. It will take a while before others in the organisation trust the information gleaned in this way.

So when we say that the cost of social data is someone’s time, this is not a flippant aside. Staying tuned in to social data and building a profile to other users and becoming a skilled weaver of the stories that the new online channels provide is a new role and one that others may find difficult to accept. “He/she just sits at the computer all day twittering away” others may mutter.  NHS funds would not run to this form of specialised function  which is generally seen in organisations who can make some money by getting it right e.g. a nappy maker plugging into Mumsnet.

Elements of Success

More usually in a public sector context, it is a ‘gift’ of yet another task amongst the many you do already (thank you very much).  It is therefore important to identify the factors that will make that extra, often unwelcome job more successful. You need as ever the support and understanding of senior management – ideally someone who is a social network user who appreciates the nature of the medium. Also avoid relying on one channel of information – you are a synthesiser of lots of bits and bobs of information including visuals as well as the written word. Finally you must be an intelligent messenger  – get the message to where it will do most good and where the chances are high of information being turned into action. (You will probably have to have invested time and energy in these audiences by pre-packaging issues or bundling up case studies of how this goes).


One other technique is to involve allies – people who often have lots of information at their finger tips and who are already trusted sources of information within a hospital or healthcare provider.  These are the charities that specialise in certain conditions – Macmillans for cancer, the Motor Neurone Disease Association or the Parkinson’s UK – the list is long. They have access to people – that most precious resource – and can be considered as a standing source of information not just on the clinical aspects but as service users. Commissioners as well as providers don’t function effectively without this intelligence.

This is a combination of Caroline’s suggestion that you go where people gather  under the umbrella of their ‘own’ association , where that unifying interest – the reason why all the people are in the room – is a health matter. We have run research both with people with a particular condition and equally rewardingly in terms of the quality of the data, with carers. The insights are often disturbing and such groups merit the most sensitive handling, but the quality of the data is well worth the encounter for all concerned. Always bearing in mind the point we have already made about the data being heard and acted upon. If it is not, then that waste is a betrayal – wasting the time of vulnerable people.


The rule of thumb in cost is that the less you have to do, then the higher the cost above the line to employ consultants or agencies like Ipsos MORI or smaller organisations specialising in small group facilitation and analysis. Shell out some dosh and they are at your command. The smaller the above the line spend, then we are in the realm of DIY and Surveymonkey. We repeat the  point we have already made about making good use of your findings. That is not in essence a question of £ and pence – it is about being heard. Why do it if no one is prepared to listen to you? Our sympathies for those of you in situations where the answer to this question is ‘tick the box’.

Compulsory Reading to Lessen Risk

Another great source of waste and also – let us remember – risk (covering both money and other sorts of cost) is actually getting it wrong by a wrong interpretation of the findings. This risk is why reading Thinking Fast and Slow is compulsory. Daniel Kahneman’s analysis of the mistakes we make in interpreting data is both profound and humbling.  Chapter 20 ‘The Illusion of Validity” makes the point particularly directly. This is the biggest cost – misusing or misunderstanding the information you have.

So use the information in the following table as a starter for 10. It is not definitive but may save you some time and stop you wasting time and money – and wasting that most precious of resources: the  time, views and contributions of users.



Users – wide cross-section of views on their experiences e.g with Hospital/ GP surgery Survey of users within last 6/12 months: closed questions: structured questionnaire with limited room for free text comments Quantitative survey – reliable data statistically valid How to identify and contact sample; need emails/postal addresses. Postal more expensive depending on sample size, external agency involved, cost large but cost per respondent low if response rate reasonable – 20%+
Users of a particular specialist service If few of them, small group discussion for ½ hours Sense of main issues and the feelings of service users; vocabulary used; Valuable for getting sense of the issues and feelings around those issues; qualitative data is not definitive or complete picture Dependant on cost of venues; decision to pay people to come
The public / citizens/ users  – open house Meeting in town hall or similar accessible public space; marketed in advance – posters etc supplemented by targeted mailings to community orgs etc laying out reasons for meetings; who should attend and how the views expressed will be used, Collection of views and comments made on the night; wide spread of views or not; meeting can be hijacked by particular cause eg keep our surgery open. Scattergun approach to catch all in wide trawl of people means uncertainty about who has been heard from; what weight and significance should be assigned to which views. NHS consultations often attended by those for whom change in the NHS is sacrilege and a betrayal of 1948 ideals. Need strong chair Cost of  venue and promotional costs and having technology/ people to collect views (camera; recorders; note takers; microphones) attendance is highly variable – weather; competing events; issue featured on TV – can pack the house or empty it
Citizens/ public/ user views on current hot issues and what might be done about them More discussion needed and more structure to that discussion – invitations to active vol. orgs; other representatives including local government elected members; chairs of specialist orgs eg Healthwatch. Collection of views and opinions that have been considered, discussed and even acted upon by leaders of local community Homework essential – advance preparation/ local intelligence to find movers and shakers/ key influencers; what work has been done eg reports; who is not speaking to whom Costs as for small group. Worth considering how best to involve elected members –often touchy about being lumped in with all the rest. How political do you want this to get?
Staff of service providers – their take on the most important user issues All staff in provider with an e-survey; particular grades/ functions can be selected V interesting data – good indicator of staff support for initiative if response rate high;  compare with user data to identify shared areas of concern where high priority action needed. Staff surveys are often overlooked as a means of gathering information on user issues. Do not confuse with surveys on pay/ conditions etc but can be useful dipstick on organisational issues – eg would you recommend member of your family to use this place? Low if survey done by email and low cost data analysis available. Beware the ‘free’ survey – very limited for data analysis / xtabulation etc. Sample creation and respondent selection can be tricky – need to understand functional context of responses even if anonymity granted to respondents
Complainant satisfaction Structured questionnaire to establish what they are complaining about; who they complained to’ what happened; how long did it all take? Satisfied now? Recommend? Sample can be small but the data is structured; anchored to an identifiable event and v good basis for quality improvement action (Dis)Satisfaction is not indivisible – always a reason for this state. Worth asking ‘why’. Avoid surveying families of deceased or similar Low – sample to hand.
What are the things on peoples’ minds? What are they saying now about us amongst themselves? Start monitoring social data channels e.g. get a Twitter account and have a look for entries about you. Join Patient Opinion. Gives impression of having finger on pulse – peoples’ problems in real time not just the tip of the iceberg who come to you. Once started, very difficult to stop. Issues of staff discipline/ whistleblowing. See discussion on ‘weak signals’ Somebody’s time and development of new skills need to correlate data and create narratives



Leave a Reply

Your email address will not be published. Required fields are marked *