We thought we would pick up some of the points that Caroline made in her blog ‘The Inspectors Call (but are they listening)’ which reflected on a recent CQC Listening Exercise, to see what lessons might be drawn from this exercise and how it might have been made more productive. We are not claiming any inside knowledge of the CQC or its processes and intentions, but given the news that CQC are looking at how it can engage more effectively with the public, our thoughts might come in handy.
The context of the CQC exercise that Caroline participated in, can be sketched out as:
- National regulator
- Formal purpose i.e Inspection
- Access to local databases etc for invitations – unknown by us but let us say limited
- Local issues and engagement structure in locality: no CQC first-hand knowledge?
- Budget – not known but enough to pay and expense 20 ‘listeners’ in London
- Time – limited but includes both public engagement session and hospital inspection
The purpose of the evening was to gather healthcare experiences as they related to the local hospital. It would appear that the CQC team were under instructions to talk very little (about themselves and the process) and to listen attentively to whatever was said – unconstrained by disclosure of any particular agenda.. We do not know if they had done their homework (a Hackney report had apparently been sent to them but was not referenced at all in the proceedings).
There are many challenges for the CQC inspection regime. We focus on patient engagement and participation. The big hurdle for the inspectors is that gathering insights on the patient experience is a continuing process offering data that accumulates over time. These findings are interpreted best by those involved in the processes of local collection and discussion. That local aggregation of intelligence cannot be replicated or simulated by the one-off event in a crowded inspection diary. So the CQC needs to do its homework before embarking on inspection.
Know Your User
Hackney is one of the parts of London where for better or for worse, the citizens are ready to get involved either individually or on the basis of political or faith organisations. The CQC got a commendable turnout on a wet weekday evening – 30 people or so. However as Caroline wrote “What is needed here …is a much clearer distinction in the minds and processes of the inspection teams between different sorts of user intelligence and how they should be garnered. If the intention of this event was, as gradually became clear, to hear the voices of grassroots service users rather than to hear from organisations and their representative who have already absorbed, digested and processed the views and experiences of their members and others who approach them, then this should have been made explicit in the publicity materials for the event.”
Lesson 1: Match your promotion and publicity to your purpose
The discussion format
The round table approach was adopted. It is the favoured and most effective form of room layout and works well for an exercise of this scale and purpose. However the CQC team – no doubt with the best of intentions about not skewing the agenda to what the CQC wanted/needed – were very reluctant to create a platform or context for the discussion and were unnecessarily coy about their own identities. This reticence to impose an agenda and avoid muddying to taint the pure waters of patient recollection (we want to hear your story) translated into a reluctance to order matters. The consequence on Caroline’s table was that a contribution that was both too long and irrelevant took up a third of the allotted time. The next (very useful and to the point) contributor to the discussion was then held to five minutes – not enough to summarise the learning from a well-organised local survey. This is one of the most common dilemmas of patient engagement which all of us face who run public meetings. We remain enormously impressed and relieved that people turn up at all. So then to squash or curtail contributions risks being perceived as rude,ungrateful and disrespectful of that experience. How to deal with that situation in an empathic way – honouring both the person and the experience while suggesting that it is better heard (and will be heard) at another time or in another place, is hard. By the way, if you offer the chance to be heard another time, make sure before you leave the room that night that you have organised that opportunity.
So the better approach is to get it clear in the publicity, target the mailings and repeat who you want to hear from about what before the meeting and then tactfully enforce your own rules during the meeting. Respect means time for all to be heard not indulgence for the obsessive. “If I cut you off when you are talking, please forgive me. I am very keen that all should have their say. If you want to make sure your story is fully heard, we will contact you outside this meeting. Please leave your contact details with …(indicate member of team tasked with follow-up)”
Lesson 2: Explain what you want to hear and from whom, what the rules are and enforce them.
Inspection and engagement – can these two activities be reconciled?
Inspections, as practised by the CQC, are quick hits. Large teams descend for relatively short periods of time on a complex organisation which they assess and judge on the basis of what we assume is objective and verifiable hard data. Patient data can fall under that heading – if you have surveyed say 50%/2000 of those who have undergone a particular set of treatment in the past 12 months and have heard back from at least 10% of those people (it should be much higher) then you have hard quantitative data. (We assume that your questionnaire and statistical analysis are up to professional standards).
If you have as part of the preparation for a visit, gathered such local intelligence as is available from such sources as Healthwatch or the relevant Council committees, these data can form the basis for a semi-structured set of one on one meetings to explore the issues that emerge (and by cross-referencing your findings, test their validity). .
If you want only that the world should know that you came and did your best to meet the public, then go for the intensively marketed town hall event inviting all to come and for all to be listened to. To be ready to tell your story and give your views on the health and social care services are the only criteria for attendance. This is the ‘build it and they will come’ approach. But who are ‘they’?.
The Town Hall Mix
The town hall meeting attracts particular types of people – most commonly those who have a long and ongoing (and often unhappy) relationship with health services. Some of these people will have a missionary zeal which, as described above, can lead to them dominating conversations and distorting the listeners’ perceptions. Then there are the people who turn up either because of a professional or community role they hold – such as members of the Board of Healthwatch or a local charity. They are there in a representative role and may bring with them the views of other people gathered and analysed in more or less systematic ways. Currently, in a world of reconfiguration and change to the NHS there are also likely to be those who bring a strong political perspective to the meeting.
Going to the Mountain
We would like to suggest that a more diverse and nuanced range of views may be found in other places – places where people are unlikely to be there because they are driven by a particular personal passion about health services or health politics. Such bodies do exist. In Hackney there are a number of small scale, well-functioning local groups such a Parent Councils in schools, local park user groups and tenants associations which have a regular programme of meetings. Such groups provide a membership and a structure into which a well-managed engagement session can be built. Here you will find people who have some experience of working with external organisations but who nevertheless bring with them experience of using health and related services. Often the fact that this issue may not be at the heart of the work of their organisation will bring a new and different perspective.
Different Perspectives in Different Environments
In the case of health, such environments also offer something which most health engagement exercises fail to deliver: the input of relatively well people who do not have long term conditions but nevertheless use health services and the views and experiences of younger people, often with families who may well have caring roles in relation to other family members. Identifying such groups requires local knowledge, a bit of imagination and some effort – often they will not be closely networked into umbrella bodies such as local voluntary service bodies but they can be found.
One option for inspectors is to descend on say 10 individuals on a ward and ask them what is going on for them. Note down what they say as they say it. You will have collected information that is valuable because it gives an indicator of the emotional heat around the issues raised. It is not however a reliable basis for understanding the general patient experience. The risk is that the emotional force of the personal encounter of people in distress will colour the views of those in the inspection team who hear them. No matter that you have talked to very few on one day, the temptation for the inspection team is to use this as information that carries the same weight and import as an organised study over time.. By all means use the emotional fall-out as a clue to wider issues but the inspection team has more work to do before the anecdote becomes actionable. More usually the temptation is to use the patient anecdote as a hook for an inspector’s personal opinion. Use the anecdote as flavour not as fact.
Lesson 3: Respect local process and understand the difference between data you can trust and that you cannot.
To sum up:
|Result / Output||Technique||Comments||At the end of the day|
|Tick the box – been there, done that||Large meeting – undifferentiated attendees (general public)||Difficult to manage/ no clue as to quality/ significance of data gathered||What does any of this mean and we have no idea how this connects with the main focus of the inspection|
|Authoritative data on specific issues and experiences||Standard survey research with sample large enough to generate statistically significant data||Need a good sized sample – can you persuade the NHS institution to release contact details?||Lot of money for decent work – how long will it take for the ethics committee to agree to release names and addresses?|
|Issue Identification for report on local services||Use informed proxies eg relevant local organisations||Give them plenty of time to get their act together – they are vol orgs and may meet only 4 times a year||Will then have list to be x-tabbed against other intelligence and hospital based data|
|Performance data e.g waiting list targets||Management data collected in-house. Rare to survey user experience opinion||Sometimes a clinic will try this offering a photocopy for completion at the time and analysed with free tool||Only simple counts emerge eg. 36% said this but you will not know whether this 36% were young people under 30 because no x-tabbing is permitted without payment.|
|Pointers for issues of the day||Create or find and follow relevant twitter tags||Opportunistic and time-consuming to monitor over time||Twitter Universe reaction highly unpredictable – may strike it rich or hit dry well.|
In a second paper on consultative choices we consider different methods and what sort of outputs you can expect from them. Watch this space