Healthcare workers and medical centres are increasingly requested to report data about their results of medical treatment. Governments call hospitals to account for their results in order to plan new healthcare structures, and to support new regulations in medical systems. Insurance companies are highly interested in the results of medical treatments to enable cost-effectiveness of care and benchmarking between different providers. Centres with the “best care” for the “lowest price” are popular in the financial departments of healthcare agencies. Newspapers and magazines regularly fill their columns with comparisons of hospital performances regarding patient safety and satisfaction. And last, but not least, patients themselves want to know and have the right to know whether their medical centre and their doctor deliver good quality of care.
Several systems have been developed to evaluate quality of care, sometimes with far-reaching consequences. In the UK, general practitioners' remuneration is now directly linked to the scores attained in the Quality and Outcomes Framework 1. The success of such an approach depends, in part, on designing a robust and clinically meaningful set of indicators. Outcome measures regarding the results of treatment are often attractive at first glance, but their interpretation can be complicated. Patients can be pleased to know that their doctor has the highest lung function results for chronic obstructive pulmonary disease or asthma patients in the country, but in most cases they cannot judge the meaning of such information. What is the meaning of a forced expiratory volume in one second (FEV1) value of 80% of predicted? Is such a number clinically relevant? Is it reliable? Is the number corrected for all sorts of influences, such as age, comorbidity and case mix? For proper evaluation of quality of care, more information is needed. In two studies, Donabedian 2, 3 advocates the evaluation of the three indicators “structure, process and outcome”.
Structure indicators point to the background of the care-delivery system. For example, is care for cystic fibrosis (CF) delivered by a dedicated team, consisting of a pulmonary physician, a physiotherapist, a dietician and a social worker? Is a microbiologist and a pharmacist involved in the care for these patients? Is there a specialised nurse available for all patients with asthma? Process indicators deal with the organisation of care. Does the hospital use a protocol for lung function measurements and microbiological evaluations? Are there clear rules for segregation of patients with methicillin-resistant Staphylococcus aureus? Finally, outcome parameters at the end are, often numerical, results of treatment: e.g. lung function results, numbers of complications or mortality data. Public discussions often deal with outcome parameters. They can be helpful to identify insufficiency of care, but they are seldom helpful in identifing the causes of healthcare failure. Correct and feasible directing healthcare systems ask for structure and, mainly, process indicators, rather than outcome measures 4, 5.
In the current issue of the European Respiratory Journal, Stern et al. 6 describe the results of the German Cystic Fibrosis Quality Assessment (CFQA) project. In this project, data from 6,835 CF patients in 93 German CF centres over a period >10 yrs are documented. These data allow the comparison of centres and different patient groups, such as males and females. Stern et al. 6 also provide so-called benchmarking diagrams, which show considerable differences in outcome between centres. For example, the percentage of patients with an FEV1 value >80% pred ranged 20–100%. The percentage of patients with a body mass index >19 kg·m−2 ranged ∼15–100%. Data about lung function and nutritional status seem to be relevant indicators, because in the database they were clearly related to mortality rates. The authors argue that the system facilitates centres to “learn of the best”. The CFQA project is an important step in quality management and improvement of CF care. The North American Cystic Fibrosis Foundation has run a national database for many years, and in other countries CF databases are already being run or are in progress 7. From these databases important scientific information has been published annually. However, as a method for improvement of quality of care, many of these databases have shortcomings inherent to outcome indicators.
Stern et al. 6 mention that some centres deliver their data late, while others do not deliver their data at all. Although the CFQA covers data from 6,835 patients over >10 yrs, in 2005 the records of only 4,551 patients were available. After subtracting patients who died, as well as those who did not give informed consent (n = 130), the annual return rate for 2005 was 73.4%. Such numbers raise important questions about reliability of the data and about the effects of case mix. Do all centres report all data about all their patients? Centres can easily influence their results in the benchmarking diagrams by selective reporting or nonreporting of outcomes. Incomplete data from changing populations also hamper interpretation of longitudinal data. Improvements of mean age and survival, for example, can merely point to the addition of adult patients with mild disease to the database, rather than a real improvement in prognosis. Reliable data for quality management asks for complete datasets, standardisation of sampling moments and procedures, and for regular audits of the centres by the registration committee. It has to be absolutely sure whether the best is really the centre with the best care and not the centre with the most insufficient administrator. Broad publications of outcome indicators from imperfect databases can have undesirable consequences for the confidence of patients and for the finances of the centres. Therefore, professionals in healthcare have the utmost responsibility to organise transparent and reliable databases. The supply of data by centres should probably be obligatory rather than voluntary. Incorporation of structure and process indicators in the reports can be extremely helpful in improving the quality of quality reports.
Nevertheless, the CFQA should be an important stimulus for other countries to set up national databases. They can stimulate critical self-reflection of the centres and provide answers to questions from the centre directors and CF teams. What are the characteristics of our CF population and how do they differ from others? How can we explain these differences and how can we improve factors in which we do not reach the level of others? Are our diagnostic and therapeutic treatment protocols effective with regard to patient outcome compared with others? Without such a critical self-appraisal, centres run the risk for undeserved self-satisfaction resulting in poor or, at the best, modest quality of care.
Further improvement of prognosis in cystic fibrosis patients will largely depend on outstanding care in outstanding centres. In such a setting, databases will be used as part of an ongoing cycle of quality assessment and quality improvement. All centres deserve a structure in which all processes lead to maximal outcomes. Such centres may not perform with the “lowest price”, but they will definitely provide “best care” to their patients.
- © ERS Journals Ltd