HL Halliday

Abstract

It is very difficult for practising paediatricians to keep up to date as there are an average of 19 new articles appearing in paediatric journals every day. The practice of Evidence-Based Medicine allows data from clinical trials to be synthesised into easily readable Systematic Reviews such as those in the Cochrane Library. In many areas of paediatrics, especially neonatology and paediatric surgery most clinical decisions are evidence-based but for community paediatrics it is clear that more research and audit is needed. Professor James H Hutchison would I am sure, be pleased with our progress to embrace Evidence-Based Paediatrics but we could still do better.

Introduction

It is a great honour to be asked to deliver the 11th Annual James H Hutchison Memorial Lecture to members of the Hong Kong Paediatric Society.

Professor James Hutchison (known as Hutch to those who knew him well) was a doyen of British paediatrics through the 1960s to the late 1970s before spending three years here in Hong Kong as Professor of Paediatrics at the University of Hong Kong. I well remember in 1972, as a young SHO in paediatrics travelling to Glasgow to sit my Diploma in Child Health examination, and coming up against the great man in the clinicals.

For me the second edition of his textbook "Practical Paediatric Problems" proved a godsend, even if all contained therein was not exactly 100% acclaimed by the rest of the paediatric world. The Preface to the first edition¹ in 1964 contained these two important sentences: "In this book I have tried to deal with the problems of the paediatrician and the family doctor", and "The considerable space devoted to the neonatal period reflects the fact that 70% of the total infant mortality in the UK now falls within the first 28 days of life".

I feel sympathy with both these points of view. Our training of undergraduates in paediatrics should introduce them to problems that they are likely to encounter in family practice. The vast majority of care of children in our country occurs in the community and not in hospital. Secondly, I make no apology for emphasising the importance of the newborn and neonatal care within the specialty of paediatrics, but of course I am biased.

Evidence-based Paediatrics - Introduction and Definition

Tonight I want to talk to you about Evidence-Based Paediatrics, mindful of the fact that my friend Professor Jack Sinclair from McMaster University in Canada delivered the 6th James H Hutchison Memorial Lecture in 1995 on "Evidence-based Paediatric Practice". I hope that, whilst the message has not changed, I will not repeat exactly what he had to say five years ago. Since we are both neonatologists there is a risk of some overlap. Being a neonatologist, like Professor Hutchison in his day, I hope you will forgive me if I use some examples from neonatology to highlight certain points during my lecture. I should start by defining Evidence-based Medicine. In 1996 David Sackett, formerly of McMaster University but now at Oxford, provided this definition.2 "Evidencebased medicine is the conscientious and judicious use of current best evidence from clinical care research in the management of individual patients".

Art and Science

A good model for evidence-based clinical decisions is one of three interlocking rings: research evidence, clinical expertise and patient preferences. This implies that both Science and Art are needed to practise evidence-based care. The Art of Medicine is in applying evidence-based knowledge to individual patients. It involves the tailoring of proven techniques to suit individual needs. However, a third component of any clinical decision about care involves the patients themselves, or as in our case in paediatrics, the parents or guardians. Patient or parent preferences become very important for making ethical decisions relating to withdrawal of care, management of serious congenital abnormalities or relapse in acute leukaemia, for example.

The importance of combining Art and Science was recently reiterated by Naylor in 1995:³"To paraphrase Osler, let us agree that good clinical medicine will always blend the art of uncertainty with the science of probability. Let us also hope that the blend can be weighted towards science whenever and wherever sound evidence is brought to light".

Hippocrates - 460 to 370 BC

The importance of Art and Science in Medicine is not a recent development. Hippocrates (Figure 1), who lived from 460 to 370 BC, was the first physician to treat medicine as a practical study rather than a speculative philosophy. He realised the importance of experience and observation, and many of his writings consist of a series of very apt short statements or aphorisms.⁴

Figure 1 Statue of Hippocrates (460 to 370 BC) in Museum on Kos

Hippocrates' first aphorism was: "Life is short, the art long-lived, the chance soon gone, experience deceptive and judgement difficult". Here, I believe that Hippocrates was stating the importance of the Art but warning that experience can be misleading without some scientific observations to back it up.

He also said: "In medicine we must pay attention not to plausible theorising but to experience and reason together.... I agree that theorising is to be approved, provided that it is based on facts, and systematically makes its deductions from what is observed.... But conclusions drawn from unaided reason can hardly be serviceable; only those drawn from observed fact". Hippocrates probably laid the foundations of modern Evidence-Based Medicine as he taught his students under the great plane tree on the Greek island of Kos (Figure 2).

Figure 2 The Great Plane Tree in the centre of Kos under which Hippocrates is alleged to have taught his students

Another aphorism of Hippocrates relates to neonatal medicine: "A seven months child survives whereas an 8 months child has never been known to survive". Either dating of pregnancy was a major problem in ancient Greece or by observation Hippocrates was the first to have discovered that pulmonary surfactant is produced in two surges during fetal development so that lung maturity can in some cases be achieved at 30 to 32 weeks but in other babies at 36 weeks severe and fatal respiratory distress occurs.

Galen - 138 to 201 AD

In stark contrast Galen, who lived from 138 to 201 AD, had a negative influence on the development of Evidence-Based Medicine for over 16 centuries with his dogmatic, authoritarian approach to care:⁵"All who drink of this remedy recover in a short time, except those whom it does not help, who all die. Therefore it is obvious that it fails only in incurable cases".

The 1940s to 1960s

Let me give you three further examples of failure to adopt evidence-based care in more recent times:⁵

The introduction of unlimited oxygen treatment for preterm babies in the 1940s on the basis of uncontrolled observations led to an epidemic of blindness due to retinopathy of prematurity, formerly called retrolental fibroplasia. By 1953, approximately 10,000 children in the US had been blinded. A randomised controlled trial of 568 infants carried out over one year rapidly brought the epidemic to an end. Lawyers and judges denounced this study during the later malpractice trials because of the procedure of assigning oxygen treatments at random. No word of criticism was expressed against the 12 years of informal experimentation when physician prescribed treatments led to 10,000 blind children.

The Grey Baby Syndrome arose when an antibiotic, chloramphenicol, used to treat adults was given to neonates, assuming that they had similar metabolic pathways and were really just "scaled down adults". In 1959, a controlled trial ended the grey baby syndrome epidemic. This trial was attacked in the US Senate because treatments were assigned at random.

Clearly lawyers, politicians and the public were not ready to embrace the introduction of Evidence-Based Medicine in the 1950s.

In 1964, The New York Times announced the end of respiratory distress syndrome with this headline: "Fatal Baby Disease Cured", just five years after Avery and Mead had shown that surfactant deficiency was its cause but before any successful surfactant replacement trials had been performed.

"Respiratory distress syndrome has been cured by giving Epsom salt enemas to 28 premature babies in five Louisville hospitals. RDS must be due to too much water in the body and as it tries to escape through the lungs the babies suffocate and choke. It is not yet certain that this theory is correct but it is certain that the treatment works as all 28 babies who were suffocating, dramatically improved and were normal in one hour or less". We didn't need to develop surfactant treatment after all! Twelve years after this headline when I went to work in Cleveland, Ohio as a Research Fellow we used to have babies transferred from small peripheral hospitals when they had failed to respond to the Epsom salt enemas.

I could give you many more examples of interventions adopted into clinical practice without adequate evidence of efficacy or safety. What is the solution to this problem? It is essential to critically evaluate any intervention that we undertake in paediatric practice. We must obtain evidence of benefit and to do this we need trials.

The Randomised Controlled Trial

Randomised controlled trials (RCTs) provide the strongest evidence which is graded as level I (Table 1). Then, controlled trials without randomisation, followed by cohort or case-control studies, multiple time series or dramatic uncontrolled studies and finally opinions of so called experts or descriptive epidemiology. The RCT is thus the gold standard for assessing the effects of any medical intervention.

However, the RCT had its origin not in medicine but in agricultural research. In the 1920s a statistician called RA Fisher applied this methodology for studies evaluating the effects of fertiliser and insecticide treatment on crop yields. In 1936 Austin Bradford Hill, another doyen of statistics, encouraged the use of randomised trials in medicine by writing a series of articles in the Lancet. In 1946 the Medical Research Council funded clinical trials of streptomycin for tuberculosis and in 1954 the Salk polio vaccine was tested in a very large randomised trial in the US. From 1970 onwards cancer chemotherapy, including childhood leukaemia, has been very effectively tested in a series of randomised trials worldwide. Apart from the pioneering work of Professor Bill Silverman in the 1950s randomised trials in neonatal/perinatal medicine have only gained popularity since the 1970s and 1980s.

Sample Sizes

In paediatrics and perinatal medicine interventions may have relatively small but nevertheless important clinical effects. For this reason large sample sizes are needed to detect these effects on outcome and this can only be achieved by conducting multicentre randomised controlled trials. Ideally long-term follow-up and economic evaluations should be included in these studies from the outset but funding is often difficult to obtain. For this reason there is a tendency for a number of smaller trials, often single centre, to be performed to investigate potentially important interventions and a large definitive trial is not supported. This is where systematic reviews of existing evidence are important and the technique of metaanalysis was developed to synthesise the available information from multiple small trials.

The Origin of the Cochrane Collaboration

In 1972 Archie Cochrane, a Community Physician in the UK, argued that because resources were limited they should be used to provide health care interventions that had been shown to be effective in properly controlled research. Archie Cochrane was especially angry that obstetricians had failed to evaluate outcomes related to place of birth - home versus hospital, and electronic fetal heart rate monitoring during labour. His thoughts were further developed and answered by Iain Chalmers and others who in the late 1980s produced the Oxford Database of Perinatal Trials which grew into the present day Cochrane Collaboration, well known to you all. The Cochrane Collaboration is made up of a number of Review Groups and a number of Fields. There is a Child Health Field and a Neonatal Review Group of which Professor Jack Sinclair is one of the Senior Editors and I am the European Co-ordinator.

Systematic Reviews in Paediatrics

Jack Sinclair has written extensively about Systematic Reviews in Neonatology⁶ but what he said is also applicable to Paediatrics in general. In 1997 he said: "The practice of evidence-based neonatology requires the synthesis of up-to-date, valid evidence concerning the efficacy and safety of treatments, and timely dissemination of that evidence to practitioners. (This is an important point to which I will return later). For therapeutic manoeuvres, effects of only moderate size can be clinically very important and, to be able to detect such modest but important effects, all the available studies must be considered. Avoidance of bias is essential, so assessments of benefits and risks of therapy must be based primarily on evidence from randomised, controlled trials."

What can Systematic Reviews be used for? The results are important for providers, consumers, policymakers and funders of health care; and to aid investigators and funders of research in identifying priorities for new research. Indeed the Medical Research Council (MRC) in the UK insists that a Systematic Review has been undertaken before any application for funding will be considered.

The Methodology of a Systematic Review

This can be divided into five parts.6 First, one must specify the objectives of the review; frame the question or questions that need to answered. The next step is to try to identify and select all the relevant studies; here the Cochrane Database of Randomised Trials, Medline and Embase will all be useful but even with them important published or unpublished trials can be missed. The third step is to assess the validity of each of the apparently relevant studies. This should be done systematically looking for potential bias during recruitment, in providing the intervention and in assessing the outcomes. After all the reliable studies have been selected it is necessary to combine the results to give precise numerical estimates of effects of the intervention. This is called meta-analysis and the measures of effect obtained are usually: odds ratio, relative risk, risk difference or number needed to treat. It is customary to calculate a range of confidence for each value and this is usually the 95% confidence interval. From these results conclusions can be drawn concerning both efficacy and safety of the intervention.

The Cochrane Neonatal Review Group

This is now one of the largest in the Cochrane Collaboration. The number of Systematic Reviews has been increasing rapidly since 1997 and there are now almost 100 and about 50 protocols which represent reviews in preparation. All reviews are regularly updated to provide the latest evidence on which to base therapeutic decisions in neonatology. Every Paediatric Department should have ready access to this CD-ROM.

In neonatology there are many examples of evidencebased treatments; three of the most important are: prenatal corticosteroids, postnatal surfactant and prophylactic indomethacin.

Perinatal Corticosteroid Therapy

The results of a meta-analysis of trials of prenatal corticosteroids in preterm labour are shown in Table 2. There are 15 trials but some outcomes are reported in only four or six trials.⁷ RR stands for relative risk and CI for confidence interval. NNT is number needed to treat. In 15 trials any prenatal steroid treatment reduced the risk of developing respiratory distress syndrome by about 35% with a 95% confidence interval of about 30% to 45% reduction. Prenatal steroids reduced the need for surfactant treatment. They also reduced the risks of intraventricular haemorrhage, necrotising enterocolitis and neonatal death. There was no effect on stillbirth rate.

Recently there has been concern about the safety of postnatal corticosteroids and the technique of meta-analysis can help to provide the balance between beneficial effects and adverse effects (Table 3). Corticosteroids given moderately early (between seven and 14 days) reduce neonatal mortality;⁸ the number needed to treat (NNT) is 16 with 95% confidence interval (CI) of nine to 50. Another benefits of postnatal corticosteroids is reduction in chronic lung disease (CLD) defined as oxygen dependency at either 28 days or 36 weeks post-menstrual age. The adverse effects of corticosteroids are hyperglycaemia, hypertension, gastro-intestinal bleeding and hypertrophic cardiomyopathy (Table 3).

I mentioned concerns about longterm adverse effects in three recent studies (one still unpublished). Table 4 shows a meta-analysis of longterm outcome of babies enrolled in early (<96 hours) postnatal corticosteroid trials.⁹ The rate of cerebral palsy is increased more than two fold (relative risk 2.32, 95% CI 1.48 to 3.65) with NNT of eight (95% CI five to 17). Death or cerebral palsy, developmental delay and abnormal neurological examination are also increased, each with a NNT of seven. This leaves the neonatologist with a dilemma - use of postnatal corticosteroids may improve survival but increase the risk of cerebral palsy.¹⁰

Is Neonatal Intensive Care Evidence-Based? This was a question posed by Dr Pamela Cairns, a Belfast graduate working as a Research Fellow with Jack Sinclair in 1998 in a study published as an abstract in Pediatric Research.11 Pamela Cairns examined numerous interventions in the Neonatal Intensive Care Unit at McMaster University Medical Centre. She accepted three levels of supporting evidence and assessed each intervention according to these criteria: level I - the intervention was based upon good RCT evidence, either a large single trial or a good meta-analysis of many smaller trials; level II - an intervention with convincing nonexperimental evidence such that an RCT would be unethical or unjustified, for example use of antibiotics for bacteraemia or meningitis; and level III - an intervention without substantial supporting evidence.

Pamela Cairns divided the interventions into primary (based upon diagnosis on admission) and secondary (based upon problems arising later during the hospital stay). Of the primary interventions 34% were level I and 62% level II making a total of 96% evidence-based and only 4% without supporting evidence of benefit. The corresponding figures for the secondary interventions were almost as impressive with 86% being evidence-based and only 14% not. The authors felt that the particularly high rate of evidence-based primary interventions was due to the majority of them being for respiratory distress, managed of course with surfactant replacement therapy, a well tested treatment.

We wanted to look at this further and one of my Research Fellows, Dr Anna Curley examined the hospital charts of 80 very low birth weight infants admitted to our Neonatal Unit in 1998.¹² She used the three levels of supporting evidence previously described. In order to determine the status of each intervention she searched through Medline, the Cochrane Database, RCTs register, hand searching of paediatric journals and finally expert opinion from textbooks and neonatologists. Once the evidence had been obtained three neonatologists came to a consensus about which level to ascribe to each intervention.

The classification of an intervention could vary depending upon the clinical setting, for example postnatal dexamethasone to wean seriously ill babies from the ventilator was classed as level I whereas its use in spontaneously breathing infants was classed as level III.

There were 943 interventions in these 80 very low birth weight infants. As predicted by Pamela Cairns, a large proportion - 50% were for respiratory problems. 15% were for haematological, 10% each for gastroenterological and cardiovascular, 5% ophthalmological, 3% infection, 1% neurological and 6% others including metabolic and trauma.

Overall 59% of interventions were level I and 33% were level II giving a total of 92% of decisions being evidencebased, very similar figures to those of Pamela Cairns.

However when we examined the results according to system there was some variation and perhaps surprisingly respiratory interventions were not the most evidencebased. Ophthalmology was 100% evidence-based at level I, a reflection of the usefulness of screening for retinopathy and treating stage III plus disease with either cryotherapy or laser therapy. Cardiovascular with 89% level I was probably the next best although haematology, infection and others also totalled 100% by having high level II supporting evidence. Neurology and respiratory came out at the bottom but both still achieved at least 80% level I and II combined.

This is better than other areas of paediatrics where similar studies have been performed. Paediatric surgery had 77% level I and II evidence, but community paediatrics had a rather poor 39%.

For internal medicine, Michaud et al in 1998 showed that 84% of therapeutic decisions were equivalent to level I and II evidence-based.13 So in comparison those of us practising neonatology can take credit for attempting to critically evaluate most of our therapeutic interventions. But there is still room for improvement provided we remember the words of Maimonides who said: "Teach thy tongue to say I do not know and thou shalt progress".

This is critically important to allow us to approach our patients and their parents when we ask for consent to enrol infants and children in the randomised controlled trials needed to improve our knowledge. I now want to turn to the equally important question of how to get research into practice.

Implementing Research Findings

In 1998 Garner realised that in order to implement research findings and change practices it was essential to "Disseminate the findings of systematic reviews to policy makers, health professionals and consumers".¹⁴

Effecting change in many areas of life is not easy and one may be tempted to use the carrot or the stick. Sometimes it is necessary to use both in a balanced way.

Changing clinical practice has been scientifically studied and Systematic Reviews exist to show that four interventions are effective: outreach visits, use of local opinion leaders, audit with feedback and patient mediated interventions.¹⁵

Outreach visits include presentations and roadshows from well known experts. Drug companies have known for many years how effective these are. In addition to visiting experts, local opinion leaders who are respected are also important. Audit is sometimes tiresome but it is effective and I suppose represents a bit of the stick rather than the carrot. It is also important to include patients, nurses as well as doctors and managers who have to find the money to implement the changes. Resources are always needed to implement research findings initially. Sometimes there are savings to be made in the long-term but managers and politicians need to be aware that money and resources are needed up front if change is to be successfully introduced.

To summarise the steps needed to put research findings into practice:

First, obtain the evidence - which I outlined in the first part of my talk. Second, develop guidelines based upon the evidence, hopefully level I or II, and thirdly obtain a stamp of approval from a College or national body. To disseminate the information locally, opinion leaders with local credibility are needed and managers and politicians need to provide resources to effect the change. If patients are involved success is increased. Good communication is essential. The carrot in the form of incentives and the stick in the form of audit help to sustain the change.

Conclusion

I hope that I have been able to convince you of the importance of Evidence-Based Medicine or Evidence- Based Paediatrics in an age when resources need to be directed towards medical interventions that are both effective and safe. Clinicians find it very difficult to keep up to date when one considers that in paediatrics there are on average 19 new articles appearing in the journals every day. The only way to overcome this information overload is to have access to Systematic Reviews, such as those in the Cochrane Library, where data are synthesised into a readily understandable form to guide clinical practice. Perhaps paediatricians as with all other physicians need to replace SCIO (I know) with QUAERO (I seek).

References

1. Hutchison JH. Practical paediatric problems. London: Lloyd-Luke, 1964.

2. Sackett DL, Rosenberg WM, Gray JAM, Haynes BR, Richardson SW. Evidence based medicine, what it is and what it isn't. BMJ 1996;312:71-2.

3. Naylor CD. Grey zones of clinical practice: some limits to evidence-based medicine. Lancet 1995;345:840-2.

4. Petersen WF. Hippocratic wisdom. A modern appreciation of ancient scientific achievement. Springfield, Illinois: Charles C Thomas, 1946.

5. Silverman WA. Retrolental fibroplasia: A modern parable. New York: Grune and Stratton, 1980.

6. Sinclair JC, Bracken MB, Horbar JD, Soll RF. Introduction to neonatal systematic reviews. Pediatrics 1997;100:892-5.

7. Crowley P, Chalmers I, Keirse MJNC. The effects of corticosteroid administration before preterm delivery: an overview of the evidence from controlled trials. Br J Obstet Gynecol 1990;97:11-25.

8. Halliday HL, Ehrenkranz RA. Moderately early (7-14 days) postnatal corticosteroids for preventing chronic lung disease in preterm infants (Cochrane Review). In: The Cochrane Library. Oxford: Update Software, Issue 2, 2000.

9. Halliday HL, Ehrenkranz RA. Early (<96 hours) postnatal corticosteroids for preventing chronic lung disease in preterm infants (Cochrane Review). In: The Cochrane Library. Oxford: Update Software, Issue 2, 2000.

10. Halliday HL. Perinatal corticosteroid treatment - helpful or harmful. Arch Perinat Med 2000;6:7-9.

11. Cairns PA, Cunningham K, Sinclair JC. Is neonatal intensive care evidence based? Pediatr Res 1998;43:168A.

12. Curley AE, Tubman TRJ, Halliday HL. Care of the very low birthweight infant: is it evidence-based? An Esp Pediatr 2000;52:554-60.

13. Michaud G, McGowan JL, van der Jagt R, Wells G, Tugwell P. Are therapeutic decisions supported by evidence from health care research? Arch Intern Med 1998;158:1665-8.

14. Garner P, Kale R, Dickson R, Dans T, Salinas R. Implementing research findings in developing countries. BMJ 1998; 317:531-5.

15. Bauchner H, Simpson L. Specific issues related to developing, disseminating, and implementing pediatric practice guidelines for physicians, patients, families, and other stakeholders. Health Serv Res 1998;33:1161-77.