Lifelong learning
Vol. 24 No 2 | Winter 2022
Feature
Audit and the science of improvement
Dr Steven Lillis
MBChB, FRNZCGP, FAcadMEd, MGP, PhD; Medical Adviser, Medical Council of New Zealand

While most continuing professional development programs have a component of audit in them, its also fair to say that the audit creates a range of emotions, from despair (not another thing to do) to curiosity (I wonder what I can improve on). Further, there is some scepticism in the medical education literature about the effectiveness of audit in continuing professional development.1 A Cochrane review on the effects of audit on clinical practice found small but potentially important improvements in clinical practice.2The authors did make what I think are crucial observations; the effectiveness is dependant on baseline performance and on how the feedback is given. This provides very useful guidance. Almost all of us as health professionals take great pride in doing our job well. Many of us will develop an area of interest (as a general practitioner, my area of interest is musculoskeletal medicine) and feel very comfortable in that area. If we are being honest with ourselves, we all have areas in which we are far less comfortable but often quietly ignore them. Yet these may well be the areas where we should be focusing our attention.

Evidence-based medicine is a cornerstone of medical practice. It doesn’t dictate what we do but should always inform what we do. In much the same way, evidence-based education should be a cornerstone of education and continuing professional development. It can be defined as ‘The conscientious and judicious application of best evidence to the design and delivery of medical education’. So, what does the evidence about audit reveal? An enlightening systematic review paper explored the evidence about how good we are at recognising our own learning needs.3 The surprising answer is not very good. We rely on external information to monitor our standards; peer groups, meetings with colleagues and case discussions all provide opportunities where we might think ‘I wasn’t aware of that, and I think I should have been.’ Audit also provides an external view on our knowledge and standards, but in a more formal way. It allows comparison of practice to a known and recognised standard. This is a critical part of choosing what to audit. There should be a recognised standard against which to judge ourselves. Audit with feedback is also significantly more effective for less complex professional behaviours than more complex. ‘Discharge summaries need to be sent within 24 hours’ is much more amenable to change than ‘patient experience of the service needs improving’.

Improving practice as a result of undertaking an audit also requires that the feedback is structured correctly. In much the same way as providing feedback on clinical encounters, there are ways to maximise the effect of audit feedback.4 Being aware that the person or team receiving feedback is in a vulnerable position is crucial and therefore addressing the barriers to receiving feedback becomes an essential part of the process. The generally accepted rules of feedback also apply to audit; it should be specific, focused on the task and not the person or team, given as close to the audit as possible and be objective and observational rather than subjective and value driven. It should also nurture reflection-in-action.5

A final point to make about audit is that there should be ways to make change. Publicly funded healthcare is notoriously underfunded, yet making change commonly takes resources and commitment beyond what an individual or team can do or take responsibility for. It becomes important to think at the proposal stage of an audit ‘What might the results be and what changes are possible if the audit indicates change should be made?’ This also avoids the pitfall of time spent on audit as simply a method of satisfying CPD requirements and positions the audit as something from which change will likely happen.

A particularly useful role of audit over recent years has been to look at issues of equity in healthcare. These audits have drawn attention to disparities in health outcomes for Māori in Aotearoa as well as Aboriginal peoples and Torres Strait Islanders in Australia. Audit in this area has proved a valuable tool not only to reveal avoidable inequities but also to monitor progress. Because data on ethnicity is relatively easy to collect, comparison groups are already built into data collection in many electronic records.

Audits vary widely in structure, function and complexity. No single audit tool will suffice. Using audit tools previously developed does make the task of designing audit much easier and many only need minor adaptation. The Queensland government maintains a useful site with a variety of audit tools and some well written ‘how to do an audit’ information.6

An old but still influential literature review on audit in medical practice found that training in method of audit, protected time to undertake the work, adequate resources and a clear structure of what will happen were important factors for success.7 Problems that can emerge and that can derail the process were diminished clinical ownership, litigation fears, professional isolation and power struggles. When done correctly, there was better communication amongst colleagues and between healthcare teams, improved patient care, better administration and increased professional satisfaction. These are worthy goals.

 

References

  1. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA. 1995;274(9):700-5.
  2. Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews. 2012;6. Art. No.: CD000259.
  3. Davis DA, Mazmanian PE, Fordis M, et al. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006;296(9):1094-102.
  4. Foy R, Skrypak M, Alderson S, et al. Revitalising audit and feedback to improve patient care. BMJ. 2020;368.
  5. Archer JC. State of the science in health professional education: effective feedback. Med Educ. 2010;44(1):101-8.
  6. Queensland Government. Audit tools for National Safety and Quality Health Service Standards. 2019. Available from: clinicalexcellence.qld.gov.au/resources/audit-tools-national-safety-and-quality-health-service-standards
  7. Johnston G, Crombie IK, Davies HT, et al. Reviewing audit: barriers and facilitating factors for effective clinical audit. Qual Health Care. 2000;9(1):23-36.

Leave a Reply

Your email address will not be published. Required fields are marked *