Context

Improving the health status of individuals and populations is a central ambition of health-care systems in high-income countries, and the US Institute of Medicine has suggested that high-quality health-care delivery should be safe, effective, patient-centred, timely, efficient and equitable.¹ Berwick et al.² have noted the importance of patient experience of care as one of the suggested ‘triple aims’ of an advanced health-care system. A recent US report highlighted the important contribution that listening to, and acting on, patient feedback can potentially make to health-care improvement efforts.³

New developments within the English NHS highlight the embedding of public performance assessment within the regulation of the health-care system, including NHS England’s consultation on the production of general practitioner (GP) league tables⁴ and the Care Quality Commission’s (CQC) parallel development of a rating system for primary care.⁵ A transparent health-care system is regarded by policy-makers as essential to enable patients to make informed choices about the care that they receive⁶ and patient feedback on health-care services is now commonly gathered in the USA, Canada, Europe, Australia and China.

Efforts to improve quality of care in the NHS over the last 15 years have focused on providing prompt access to care (e.g. the time taken to see a GP or hospital waiting times) and on providing evidence-based clinical care [e.g. through the development of National Service Frameworks and the UK Quality and Outcomes Framework (QOF)⁷]. A direct link between patient feedback and quality improvement efforts was previously operationalised by including results arising from patient surveys as a component of the QOF.⁷ This performance management system provides financial incentives for GPs within the NHS to achieve agreed quality indicators covering areas including chronic disease management, practice organisation and additional services offered. With the introduction of the QOF it was possible, for the first time, to rank all practices according to their patient feedback and the results of surveys, aggregated at practice level, formed the basis of a pay-for-performance scheme between 2009 and 2011, when the UK government withdrew the pay-for-performance arrangements for patient experience.

Some of these policies have been highly effective. For example, associated with a wide range of quality improvement initiatives over a decade, there have been greater improvements in the UK for the clinical care of conditions such as heart disease and diabetes than in any other major developed country.⁸ Although relatively neglected in the early years of the millennium, patient experience of health care is now a high policy priority, and in 2008 the Next Stage Review suggested that:

. . . quality of care includes quality of caring. This means how personal care is – the compassion, dignity and respect with which patients are treated. It can only be improved by analysing and understanding patient’s satisfaction with their own experiences.

Department of Health, p. 47, emphasis in original.⁹ © Crown copyright 2008, contains public sector information licensed under the Open Government Licence v3.0

The review⁹ noted, however, that ‘[up until 2008] progress has been patchy, particularly on patient experience’ (p. 48) and announced the development of quality accounts for all NHS organisations in which ‘healthcare providers will be required to publish data . . . looking at safety, patient experience, and outcome’ (p. 51) (© Crown copyright 2008, contains public sector information licensed under the Open Government Licence v3.0).

Since 2008, therefore, there has been a major policy initiative to improve patient experience in the NHS. Most recently, the focus on patient experience has been enshrined in the NHS Outcomes Framework, which, in Domain 4, focuses on ensuring that ‘patients have a positive experience of care’¹⁰ (contains public sector information licensed under the Open Government Licence v3.0). In primary care, these policy initiatives and statements have been implemented primarily through the development and conduct of the GP Patient Survey,¹¹ first sent to 5.6 million patients in January 2009. The large sample size was intended to provide sufficient responses to characterise patient experience of primary care in all 8300 general practices in England. Detailed responses for individual practices were published on the NHS Choices website¹² and made available online and included information on access to GP services and interpersonal aspects of care, out-of-hours care and care planning. The questionnaire specifically included validated questions about interpersonal aspects of care based on questionnaires that the authors of the present report previously designed and on which we have previously reported.¹³ This large-scale survey is, of course, an expensive undertaking and its utility and impact need to be commensurable with this investment.

In seeking to achieve improvement in the quality of NHS services, gathering data is important both to inform the process of service development and innovation and to assess the impact of such changes in practice. It has been suggested that data to support such improvement initiatives need to be of sufficient quality to assess whether or not an innovation can be made to work, rather than being the more rigorous level of research data needed to assess whether or not an innovation works.¹⁴

Communication in the consultation has always been an important part of primary care and is closely linked to continuity of care. At the outset of this research, there had been many anecdotal accounts that GPs were more focused on meeting clinical targets identified on their computer screens than on the needs of the patient sitting in front of them. It seemed therefore an appropriate time to balance the focus on improving clinical care with a renewed focus on interpersonal care and communication in the consultation. The ability of patients to choose their own doctor is also important. Our research prior to commencing this programme showed that continuity of care had deteriorated since the introduction of the new GP contract in 2004¹⁵ and previous research had also highlighted that patients were less likely to report overall positive experiences if they were not able to choose a doctor whom they know.^16,17

Experience and satisfaction

Previous research has identified considerable confusion and overlap relating to the concepts of patient experience and satisfaction. The two concepts are closely linked, although at a simple level reports of experience relate to recounting or commenting on what actually happened during the course of a clinical encounter whereas reports of satisfaction focus on the patient’s or carer’s subjective evaluation of the encounter (i.e. asking for ‘ratings’ of care rather than simple ‘reports’ of care). Individual items in a survey may thus examine patients’ reports of their experience of care, whereas other items may explore patients’ evaluation of that care, with the linkage between report and evaluation/rating item pairs offering potential for the development of cut points in scales of performance.¹⁸ In practice, however, the terms are often used interchangeably and survey items designed as report items often have an evaluative component; for example, the question ‘Were you involved as much as you wanted to be in decisions about your care and treatment?’ from the NHS Inpatient Survey¹⁹ contains elements of both. Within the GP Patient Survey,¹¹ the instrument behind much of this programme of work, items often relate to ratings of care. For example, the communication questions ask patients to consider ‘how good’ the doctor was at providing various elements during a consultation, including giving enough time, involving in decisions about care and treating with care and concern.

Patient satisfaction may be seen as a multidimensional construct, focusing on the subjective experiences of patients, and related to their expectations of care and the perceived technical quality of the care provided.²⁰ Russell²¹ has recently summarised some of the problems associated with surveys of patient satisfaction with care, including problems with the validity and reliability of satisfaction survey instruments, the lack of a universal definition of the term ‘satisfaction’, the disinclination for patients to be critical of care received because of not wanting to jeopardise their treatment, satisfaction being determined by factors other than the actual health care received and the frequently non-specific nature of the findings arising from such surveys. In contrast, reports and surveys of patient experience may offer the potential to discriminate more effectively between practices than do reports of patient satisfaction,²² thus potentially offering greater external accountability of health-care providers, enhanced patient choice and the ability to improve the quality of care and measure the performance of the health-care system as a whole.²³

Patient experience matters

Patient experience is an important end point for NHS care in its own right. Patients consistently report that personal care is central to effective care and, in that context, the development and refinement of GPs’ interpersonal skills is a key priority.^23,24 It is noteworthy that many complaints regarding care centre not on technical and ‘clinical’ aspects of care, but on issues relating to interpersonal aspects of care and communication.^25,26 Good communication with patients is not just an end in its own right; it brings three important additional benefits.

First, our research²⁷ has confirmed earlier work which showed that patients balance a range of beliefs and concerns when making decisions about taking medicines. Adherence is related both to the quality and duration of the consultation and to the doctor’s ability to elicit and respect the patient’s concerns.^28–30 Better communication may lead to improved patient outcomes³¹ through, for example, improved blood pressure control in hypertensive patients.³²

Second, there is a close relationship between poor communication and serious medical error.³³ This is partly because not listening to the patient’s perspective may lead doctors to miss important clinical information and partly because patients react more negatively when things go wrong if communication has been poor during the clinical episode in question. A significant proportion of cases referred to medical defence societies have at their heart poor communication in the consultation³⁴ and improving communication with patients and engaging them more closely in their care is seen as key to improving patient safety.^35,36

Third, the increasing emphasis in the NHS on self-care and prevention demands good information and shared decision-making in the consultation. Our research shows that GPs and practice nurses are currently poorly prepared for roles in which they encourage patients to take greater responsibility for their own care³⁷ or their lifestyle choices.

Although intuitively of importance, enhanced patient experience of care also matters on account of an important range of other associations reported in the research literature, including improved safety-related outcomes,³⁸ improved self-reported health and well-being,³⁹ enhanced recovery,³¹ increased uptake of preventative health interventions^40,41 and reduced utilisation of health-care services including hospitalisation and emergency department visits.⁴²

Capturing patient experience of care

Although several approaches have been adopted to obtain information on patient experience of care – for example through the use of focus groups, patient participation groups, in-depth patient interviews, feedback booths placed in health-care settings and direct observation of patient experience⁴³ and the use of compliment and complaint cards to capture qualitative feedback – the only practical approach to capturing large-scale feedback with the intent of providing actionable information remains the use of surveys of patients. In primary care in England, this culture of feedback has been embedded into routine practice in several ways. Central among these is the use of structured patient feedback obtained through surveys of patients’ experience of care, at both national and practice levels.⁴⁴

Qualitative approaches may be judged to offer greater depth of feedback than quantitative approaches,⁴⁵ but such approaches are intensive in respect of data collection, although Locock et al.⁴⁶ have drawn on secondary analysis of a large national qualitative data archive to inform service improvements.

Newer forms of capturing feedback, such as the use of tablets and kiosks to capture real-time feedback (RTF), is an area of great current interest, but, as yet, these newer forms lack a strong evidence base from primary care. During the course of this research a report from a preliminary observational study⁴⁷ suggested that RTF offers potential in primary care settings and similar findings⁴⁸ have emerged from reports provided by patients with cancer attending oncology outpatient departments. Although there may be potential for the widespread use of real-time data capture of patient experience in primary care, the acceptability and feasibility of the approach in routine primary care is not known and nor is the nature of the feedback provided. Such an investigation needs detailed feasibility and pilot work using an experimental design of RTF of patient experience of primary care.

Large-scale surveys of NHS patients and staff have been in use since the mid-1990s, building on the experience of smaller-scale surveys conducted at local level or on the experience of surveys conducted for research purposes. Large-scale surveys of patient experiences of primary care were first introduced in 1998⁴⁹ with the express purpose of addressing issues relating to the quality of care and reducing inequalities in care by taking account of patients’ views in informing local service developments. Surveys of patients have been used extensively since the introduction of the UK QOF in 2004, when two questionnaires [General Practice Assessment Questionnaire (GPAQ)¹³ and CFEP⁵⁰] were ‘approved’ for use by the NHS and adopted as the basis of linking the pay of GPs to their participation in the patient survey programme.⁵¹

Such surveys may be administered in a variety of ways. In health-care contexts, paper-based surveys are most commonly used, although digital e-platforms are now commonly and widely used as a means of capturing information, most frequently using online processes. Computer-administered personal interviews⁵² and computer-administered telephone interviews may also be used, most commonly in research settings.

The NHS has established a major programme of surveys⁵³ developed for a wide range of settings. Several of these surveys focus on patient experience of care, emulating the suite of Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys introduced in the USA in 1995.⁵⁴

The content of primary care surveys of patient experience

Historically, the content of UK primary care surveys has evolved from the 1998 survey,⁵⁵ which covered a wide range of issues including primary care access and waiting times, GP–patient communication, patients’ views of GPs and practice nurses in terms of knowledge, courtesy and other personal aspects of care and the quality and range of services provided such as out-of-hours care and hospital referrals. The GP Patient Survey in 2008 developed and presented an expanded suite of items from the surveys of 2006 and 2007, which were focused almost exclusively on the accessibility of GP services; the 2008 survey focused on domains of care identified as being of importance to patients^56,57 including the accessibility of care, technical care, interpersonal care, patient centredness, continuity, outcomes and hotel aspects of care. More recently, the English NHS has outlined eight domains believed to be of critical importance in respect of patient experience. Overlapping with earlier thinking, these include respect for patient-centred values, information, communication and education, emotional support, physical comfort, continuity and access to care,⁵⁸ all being reflected, at least to some extent, in the ongoing GP Patient Survey programme.¹¹

Most recently, the Friends and Family Test has been introduced widely across the NHS, acting as a single-question proxy for patient experience based on the willingness of respondents to recommend their health-care provider to close acquaintances. The widespread use of the test has been accompanied by specific guidance on its implementation in practice^59,60 and research reports have recently started to emerge following the use of the test in hospital settings, in which concerns have been raised about the reliability of the test.^61,62 The test was rolled out to general practice settings in December 2014.

Out-of-hours services

Beyond the domains mentioned in the previous section, additional areas of enquiry incorporated in the 2008 version of the GP Patient Survey included out-of-hours care and care planning. Variation in patients’ experiences of out-of-hours care has been identified as an area of concern since 2000, with numerous influential reports considering the structures suitable for delivering out-of-hours care, as well as highlighting the variable experience of patients across the UK in respect of service delivery. In 2000, Dr David Carson reported on the structural aspects of out-of-hours care pertaining at the time and recommended an expanded role for NHS Direct as a facilitator of access to these GP-led services, proposing that patients should use a single telephone access point to enter the system.⁶³ Much less emphasis was placed on patients’ experience of out-of-hours care, although recommendations were made regarding the need to monitor patients’ experience of the developing service. The transfer of responsibility for out-of-hours care from GPs to primary care trusts (PCTs) was foreshadowed in a report by the House of Commons Health Committee,⁶⁴ which once again focused on structural and organisational issues relating to care provision. It was not until 2006,⁶⁵ following the publication of national quality requirements in respect of out-of-hours care in October 2004,⁶⁶ that patient experience of such services began to attract serious attention, with a recognition that, by 2006, although patient experience of out-of-hours services was generally ‘good’, one in five patients was dissatisfied with the service at that time. In addition, 40% of respondents in an independent survey of out-of-hours service users reported that the overall quality of the service was less than ‘good’.⁶⁵ The incorporation of six items in the 2008 GP Patient Survey with the intent of capturing information on aspects of out-of-hours GP services thus represented an extension of earlier versions of the questionnaire, recognising the growing importance of patient experience of care, and offered the potential to examine the experience of patients from various subgroups of the population and the potential to compare out-of-hours service providers in respect of their patients’ experience of care.

Measuring patient experience of care

The potential utility of questionnaires capturing patient feedback is, like other questionnaire-based feedback, dependent on the psychometric performance of the questionnaire in practice. Issues centring on the validity of the resulting data – whether or not the questionnaire items are measuring what is intended to be measured rather than some extraneous domain – underpin the reliability of inferences and conclusions that might be drawn following data collection. Validity and reliability themselves each consist of several elements and demonstrating validity of an assessment is generally regarded as having priority over demonstrating reliability.

Our previous research identified concerns expressed by doctors regarding the use of patient survey data for the purposes of providing individual feedback regarding a doctor’s performance.⁶⁷ Some of those concerns focused on the reliability and validity of the resulting data and on the conclusions being drawn regarding a doctor’s professional practice.

The validity of items within a questionnaire may be assessed in a number of ways, for example in exploring the pattern of item response using quantitative approaches such as factor analysis to investigate the latent variables identifiable within the theoretically related item responses. Qualitative approaches may also be used in the questionnaire design phase; for example, cognitive interviews were undertaken with patients from a range of sociodemographic backgrounds in the early stages of developing the GP Patient Survey.⁶⁸ Such interviews are designed to assess the interpretability and accessibility of the putative questionnaire items. Qualitative studies using cognitive interviewing or similar approaches may, however, be undertaken following respondents’ completion of questionnaires, seeking to explore the basis on which respondents are providing their evaluation. Such studies are unusual, but they potentially offer great value in exploring respondents’ insights and whether or not items presented are interpreted as originally intended.

Patients’ varying experiences of care

Our earlier research, and the research of others, has previously identified substantial variation between practices in patients’ reports of their experience of, and satisfaction with, care received and recent studies have also identified a range of patient experience being reported among doctors providing care in similar clinical specialties and settings.^69–73 This acted as the basis for the inclusion of patient feedback as an element required by UK regulatory authorities for the routine appraisal and revalidation of doctors. Despite these observations, few studies have examined the relationship between feedback on patient experience aggregated at practice level and the performance of individual doctors within practices, with one observational study⁷⁴ identifying a substantial range of performance among doctors from a sample of eight Scottish general practices; the authors noted a number of possible contributing factors that might have accounted for differences observed at doctor level, including the experience of the doctors themselves, as well as the doctors’ mental health and professional disillusionment.

Systematic differences in patients’ reports of their experience of care have also been reported to be related to the characteristics of patients themselves. Older patients, patients from white ethnic backgrounds, the better educated, the less deprived and those reporting a better health status have generally reported more favourable experiences of care than younger patients, those from minority ethnic groups, the less well educated and more deprived and those with a poorer health status. Similar differences have been reported across many health-care systems and have given rise to calls to take account of the characteristics of participating patients when considering the results of patient feedback on care. To date, however, such calls have generally not been heeded in practice, as the relative contribution of practice, doctor and patient to overall variation in feedback remains to be defined. Uncertainty regarding the need for, and effect of, such ‘case-mix adjustment’ remains a concern for doctors in their consideration of patient feedback.

Specifically in respect of variation in experience among patients from different ethnic backgrounds, previous analyses have identified variations in patient experience in relation to ethnic group, age and gender and have found an interaction between ethnicity and age for cancer referrals.^75,76 However, no studies to date have yet investigated such an interaction in respect of patients’ experience of communicating with their GP, for example investigating differences between older and younger patients, by gender and among patients representing a range of minority ethnic groups.

In addition, although communication between doctors and patients is a core component of patient experience,⁷⁷ and minority ethnic groups have reported lower patient experience scores for communication than the majority population,^75,78,79 such differences are not consistent for all minority ethnic groups. Previous analysis of patient experience data conducted by the authors highlighted that South Asian patients reported particularly negative experiences, including for waiting times for GP appointments, time spent waiting in surgeries for consultations to start and continuity of care.⁷⁵ However, such analyses have not been repeated using GP Patient Survey data.

A number of potential explanations have been suggested for the lower ratings provided by South Asian and other minority ethnic groups in respect of their experience of care. Broadly, these relate to whether South Asian patients (1) receive lower-quality care or (2) receive the same care but rate this more negatively.⁷⁵ For example, differences in the use of questionnaire response scales might lead to South Asian groups being less likely to endorse the most positive options when asked to evaluate a doctor’s communication skills.⁸⁰ Alternatively, there could be systematic variations in evaluations of consultations because South Asian respondents vary in their expectations of, or preferences for, care. However, recent evidence from the USA points to lower quality of care as the main driver of variations.⁸¹ Gaining understanding of why minority ethnic groups give relatively poor evaluations of their care is key to forming an effective response, as determining appropriate action is difficult until it is ascertained whether differences in evaluations relate to true differences in care or to variations in expectations, scale use and preferences. Exploring these observed differences between patients from various ethnic backgrounds is challenging using only observational, real-world data. More robust approaches are required, drawing on experimental designs in which some key elements of the consultation–interaction can be accounted for in the analysis, for example through the use of standardised consultations and video vignettes.^81,82

Using patient survey data to improve care

Although there is a belief, articulated in the Darzi Review,⁹ that patient surveys can be used to improve care, a systematic review from 2008⁸³ suggested that there is considerable uncertainty about how and whether or not this can actually be achieved. Several causal pathways for achieving improvements in provider performance through the release of publicly reported performance data have been proposed.^84–86 Some invoke market-like selection, claiming that patients will modify their choice of provider using publicly available data, such as that provided by patient experience websites.^84,87–89 Evidence to support this pathway is, however, weak.⁸⁵ A more likely mechanism driving performance improvement in response to the publication of performance data is health professionals’ concern for reputation, in which peer comparison motivates individuals and organisations to improve their care.^85,86

Furthermore, at the outset of this research, PCTs were poorly prepared to support and work with general practices to improve patient experience. In addition, the Darzi review⁹ had noted that progress in improving patient experience in the NHS had been slow, and our research had identified that some aspects of care, especially out-of-hours care^90,91 and continuity of care,¹⁵ may actually have worsened in recent years. In addition, as observed earlier, it had been noted that patients from minority ethnic communities consistently reported lower evaluations of the quality of primary care.⁷⁵ Although these problems had been clearly identified in published research, the research had provided less clarity about the meaning and interpretation of these findings and the best way to intervene to improve patient experience.

Irrespective of its potential to stimulate change, the publication of performance data is central to the openness and transparency that are seen as essential for a safe, equitable, patient-centred health-care system.⁹² Thus, regardless of any effect on quality improvement, such initiatives are likely to be here to stay.⁸⁵ In refining the information made public, it is important that performance data are accurate and relevant to all potential users. The US-based Robert Wood Johnson Foundation⁹³ has noted that, whilst there is a patient preference for information to be provided at the level of individual clinicians (and not at practice level), such information is only rarely available. Currently, there is some move towards publication of performance data from an organisational level to that of individual doctors. In the UK, for example, patients referred to the cardiology service at the University Hospital of South Manchester NHS Foundation Trust may go online to view both mortality and patient experience data for each cardiologist or cardiac surgeon.⁹⁴ However, within English primary care, the practice-level aggregation of data from the GP Patient Survey used to derive practice performance indicators potentially masks variation in performance among individual GPs, thereby inappropriately advantaging or disadvantaging particular doctors. Current indicators may consequently fail to provide users, providers or commissioners with an accurate assessment of performance within a practice.

Although intuitively simple, patient satisfaction is a complex concept⁹⁵ and patient questionnaire scores must be interpreted carefully. For example, practices need to understand if low ratings for communication reflect particular consultation behaviours or whether they are in fact the result of broader issues such as practice culture or the structure and availability of consultations and appointments.

Once the causes of low ratings have been better understood, interventions to improve care can then be designed. However, the current literature on the effects of feedback of patient assessments is insufficient in scope, quality and consistency to design effective interventions.⁸³ There are many reasons why simple feedback on patients’ experience of care is likely to have limited effects. Our research is designed to address these gaps in knowledge, enable managers, patients and professionals to have confidence in the meaning of patient assessments and provide effective interventions to improve care when problems are identified.

Summary

In summary, therefore, capturing patients’ experience of primary care is a current ambition of major importance in UK government health policy. Patient surveys, incorporating opportunities for people to comment on various aspects of their care, are a convenient means of capturing relevant information at scale. It is not clear, however, how health-care staff operating in practices respond to the resulting information. Previous experience suggests that staff may rationalise scores on the basis of concerns regarding the scientific properties of the survey, or uncertainty regarding the implications arising from providing care in their particular circumstances, for example taking account of the sociodemographic mix of respondents. On a similar vein, it remains unclear the extent to which overall practice performance, based on aggregated patient feedback, might relate to the performance of individual doctors within the practice. It is also unclear whether or not patients provide reliable evaluations of care – and the extent to which such evaluations might vary according to the sociodemographic characteristics of respondents. New modes of capturing patients’ experiences of their care have become available in recent years, but to date it is not clear whether or not novel, technology-based approaches can be successfully implemented in routine primary care settings, nor the extent to which any resulting data might reflect the results of the wider population.

In recent years, care provided by out-of-hours GP services has been a particular area of interest for the NHS and has been the subject of national audit and standard setting. However, it is not clear whether or not patients’ reports of their experience of out-of-hours care are valid and reliable. Neither is it clear the extent to which factors relating to the structure and organisation of such care might be associated with systematic differences in patients’ reports of their care. Furthermore, as for in-hours care, it is not clear how staff providing out-of-hours care might respond to patient feedback and how service managers might utilise such information in the planning and design of services aimed at being responsive to the needs of NHS patients.

Aims of the programme

This programme had seven aims:

1. to understand how general practices respond to low patient survey scores, testing a range of approaches that could be used to improve patients’ experience of care
2. to estimate the extent to which aggregation of scores to practice level in the national study masks differences between individual doctors
3. to investigate how patients’ ratings on questions in the GP Patient Survey relate to actual behaviour by GPs in consultations
4. to understand better patients’ responses to questions on communication and seeing a doctor of their choice
5. to understand the reasons why minority ethnic groups, especially South Asian respondents, give lower scores on patient surveys than the white British population
6. to carry out an exploratory randomised controlled trial (RCT) of an intervention to improve patient experience, using tools developed in earlier parts of the programme
7. to investigate how the results of the GP Patient Survey can be used to improve patients’ experience of out-of-hours care.

In presenting our work, we report our research and findings under three major themes: (1) understanding patient experience data, (2) understanding patient experience in minority ethnic groups and (3) using data on patient experience for quality improvement. These are outlined in brief in the following sections. The relationships between individual studies and the three themes are shown in Figure 1, which also outlines methods and participants.

FIGURE 1

The IMPROVE (improving patient experience in primary care) programme themes and studies contained within workstreams: (a) understanding patient experience data; (b) understanding patient experience in minority ethnic groups; and (c) using data on patient (more...)

During the course of the programme we conducted empirical studies across a number of general practices and out-of-hours providers. General practices were initially recruited to take part in a suite of studies (presented in Chapters 7–9) in which we conducted a patient experience survey at the level of individual GPs, gave feedback from this survey to both the practice and the individual doctors (see Chapter 9) and, for some practices, conducted focus groups with practice staff and interviews with GPs. Sampling was initially designed around the survey study: practices were sampled on the basis of location (two study areas, the South West and North London/East of England, covering both urban and rural settings), performance on the GP Patient Survey, practice size and area-level deprivation. Once the survey was completed, a number of practices were purposively sampled to take part in focus groups with staff (see Chapter 7) and interviews with GPs (see Chapter 8) and additional filming of consultations (see Chapter 3). Out-of-hours providers were recruited from across England. We worked with up to 11 providers in varying workstreams (see Chapter 11). We additionally completed multiple analyses of GP Patient Survey data¹¹ (see Chapters 4, 5 and 11) and, for an experimental vignette study, collected data from members of the general public (see Chapter 6).

Patient and public involvement

Our programme of work was supported by two study advisory groups: one was based in Cambridge and provided support across all streams of work except for the out-of-hours research and the other was based in Exeter and was convened specifically to provide input to the out-of-hours workstreams. In this section we briefly outline the formation and working of these groups over the course of the programme.