The Australian Safety and Efficacy Register of New Interventional Procedures – Surgical (ASERNIP-S) is the only health technology assessment group in the world that focuses purely on surgical procedures. During the 17 years since its inception, this group has developed a strong international profile through the production of more than 100 reports on evidence-based surgery, surgical technologies and audit. In addition, ASERNIP-S has pioneered the introduction of national audits into the surgical community. The ASERNIP-S project has grown from a small staff of two, in 1998, to a Division of Research, Audit and Academic Surgery with close to 50 employees.

Initially, all reports produced by ASERNIP-S were systematic reviews; however, the range and scope of reports has changed over the years to include rapid reviews, technology overviews, and assessment of new and emerging surgical technologies identified by horizon scanning and input into clinical practice guidelines.

Background

The environment needed for the establishment of ASERNIP-S was created in 1996, following the release of a report by the Australian Health Ministers’ Advisory Committee on Quality in Australian Health Care. The report stated that:

‘…improving safety and quality of care should be a central concern for all those in the health care system: policy-makers, managers and health practitioners. The current drive for efficiency must be matched by a drive for safety and quality.’¹

The report also promoted evidence-based practice and the use of practice guidelines.

In 1998, the ASERNIP-S was established under the auspices of the Royal Australasian College of Surgeons (RACS). The aim was to use evidence-based methods to evaluate and document the safety and efficacy of new surgical technologies and techniques as they were introduced and preferably before they were widely accepted into the Australian healthcare system. Funding was provided by the Federal Government of Australia, which ran initially as a pilot.

In the late 1990s there was considerable interest in the emerging field of evidence-based medicine in Australia and internationally. The Australian Government established a government body known initially as the Medicare Services Advisory Committee (later changed to the Medical Services Advisory Committee [MSAC]). The principal role of MSAC is to advise the Australian Minister for Health and Ageing on evidence relating to the safety, efficacy and cost-effectiveness of new medical technologies and procedures. This advice informs Australian Government decisions about public funding for new and in some cases existing medical procedures.

During the first year, ASERNIP-S undertook to provide evidence for the Australian Government by producing systematic reviews of ten new surgical procedures. Systematic reviews are the most appropriate and best-validated tool of health technology assessment organisations. They provide critical summaries through a thorough assessment of relevant articles in the world literature. The best systematic reviews are produced when the results of several good-quality randomised controlled trials (RCTs) are available. The process of meta-analysis enables the results of several similar studies to be combined to produce an overall estimate of the combined results.

Typically, for several reasons however, few RCTs are carried out in the area of surgery. For example, the individual circumstances and needs of a patient may mean that he or she is not a suitable candidate to undergo a comparative treatment. In addition, the aim of ASERNIP-S is to review procedures before they are widely accepted into the community to identify any potential risks at an early stage; this often means that little high-level evidence (such as RCTs) on the procedure is available yet in the published work.

Reporting

It was quickly realised that wide dissemination of information from ASERNIP-S assessments, both locally and internationally, was needed to achieve quality improvements in surgical care. A website was created and information posted as it became available. A study of web hits showed increased interest each year in the information made available. However, serious consideration was also given to making dissemination a more active process, ensuring that information was directed to relevant individuals or groups.

The research community was informed of proposed, current and completed projects through the database of the International Network of Agencies for Health Technology Assessment. Additional effort has been put into writing articles on each systematic review for publication in the peer-reviewed literature to inform the wider surgical community, both in Australia and internationally. Articles are also submitted to newsletters or organisations for surgeons and consumers. The website has also undergone considerable changes over the last 17 years.

Horizon scanning

The ability to provide timely reviews of new surgical technologies was recognised as a difficult problem. Attempting to review a procedure or technology too soon means that the amount of information available is extremely limited, with the associated likelihood that the procedure may never be brought into practice. Leaving the process too late will mean that a procedure becomes so entrenched in the system that healthcare practitioners will find it harder to revert to the older procedure and may have already invested heavily (both financially and in training time) in the new procedure or technology.

In 2000, to circumvent this problem and to inform surgeons and policymakers of new techniques about to affect the Australian healthcare system, ASERNIP-S developed a horizon scanning process for the detection of new and emerging technologies. This was extremely successful and has also undergone many changes in the succeeding years. Since that time we have become a member of the Australian and New Zealand Horizon Scanning Network (ANZHSN), which is an initiative of MSAC. The ANZHSN is overseen by the Health Policy Advisory Committee on Technology, a subcommittee of MSAC, and was established to provide advance notice of significant new and emerging technologies, but is not confined to surgery and includes diagnostics and other areas of medicine.

This new collaboration led to some changes in the process for assessing new surgical procedures. Surgical procedures detected through the horizon scanning process are assessed by horizon scanning technology prioritising summaries and horizon-scanning reports.² Surgical procedures that have more widespread use in Australia, or are already established into routine surgical practice, are assessed by rapid systematic reviews, systematic reviews or technology overviews.

Horizon scanning technology prioritising summary

Horizon scanning technology prioritising summaries are short (approximately five to ten pages) documents that provide readers with a background on a particular technology or technique and present the evidence available pertaining to the safety and efficacy of that technology. This can be used for further deciding if the procedure should be further assessed, monitored or if no further assessment is required.

When a procedure or technology is considered to be of substantial impact and have a considerable evidence base, a more detailed assessment, in the form of a horizon scanning report, will be undertaken.

Horizon scanning report

These reports are comprehensive (approximately 30–50 pages) documents that assess surgical technologies or techniques that are new or emerging into Australian healthcare.

Systematic reviews

Systematic reviews assess surgical procedures through the use of a clearly formulated question using systematic and explicit methods to identify, critically appraise and summarise relevant studies (published and unpublished) according to predetermined criteria.^{3 4} Reported outcomes can be synthesised either quantitatively or narratively or can include meta-analysis to statistically analyse and summarise the results of the included studies. Systematic reviews are fundamental tools for decision-making by health professionals, consumers and policy-makers, as they provide conclusions based on research evidence.

Rapid reviews

A rapid systematic review is an evidence-based assessment in which the methodology of the systematic review has been limited to shorten the timeline for completion. They are produced in response to an immediate need for a systematic summary and appraisal of available published reports for a new or emerging surgical procedure.⁵ This urgency may arise if the uptake of a new technique or technology appears to be inappropriate; given the evidence available at the time (in other words, it may be diffusing too quickly or too slowly). Alternatively, there may be uncertainty or controversy regarding the clinical or cost-effectiveness of a new procedure, or there may be significant concerns regarding its safety or indications for use in particular populations. Rapid reviews use the same methodology as full systematic reviews, but may restrict the types of studies considered (for example, by only including comparative studies and not case series) to produce the review in a shorter time period than a full systematic review.

Funding

As with any not-for-profit, non-government project dependent on external financial support, funding has been a concern for ASERNIP-S. During the early years, the Australian Government Department of Health and Ageing supported the entire project, enabling systematic reviews, auditing and horizon scanning to be undertaken. Later, separate funding was obtained for some auditing activities. Similarly, funding has been provided separately for the different types of reviews. MSAC directly funds their reviews and additional funding was obtained for the horizon scanning project. ASERNIP-S retains some flexibility in choice of reviews with separate government funding.

Some work has been commissioned through other sources, for example, the National Institute for Health and Care Excellence in the UK. The Australian National Institute of Clinical Studies commissioned ASERNIP-S to assess the effectiveness of different strategies for increasing the uptake of prophylaxis for venous thromboembolism in hospitalised patients. The National Breast Cancer Centre also commissioned a review on intraoperative radiation therapy. ASERNIP-S has also worked on the National Health and Medical Research Council (NHMRC) Guideline Assessment Register to help developers of clinical practice guidelines achieve the requirements set out by the NHMRC for evidence-based guidelines. ASERNIP-S assisted the Australasian Paediatric Endocrinology Group to produce NHMRC-endorsed guidelines on type 1 diabetes.

Audits

Since the inception of ASERNIP-S, it has been envisaged that auditing would be an integral component of the program. Recommendations could be made following systematic reviews for further trials or audits where necessary. However, only a small number of audits were initiated following the review process. Although small in scope, these audits helped to establish protocols that would be of use in later audits.

The first audit established following a review was for laparoscopic live-donor nephrectomy. Information was collected from Australian and New Zealand surgeons between 1999 and 2003. The pooled data identified that Australasian surgical practice compared favourably with worldwide practice, both in terms of efficacy and safety for donors in the short term.

The MSAC has provided separate funding for two additional audits that were commissioned following inconclusive results from MSAC reviews. It was not considered feasible to conduct randomised controlled trials for the procedures; hence audits were considered the appropriate vehicle. One audit of endovascular aneurysm repair ran for eight years and provided information on the mid- to long-term outcomes of the procedure, which helped inform the government funding decision. Two-year funding was provided towards the end of the audit by Cook Australia to enable the collection of long-term follow-up data.

The National Breast Cancer Audit (NBCA) was an audit that was started by the Breast Section of RACS, but later came under the management of ASERNIP-S.⁶ This audit was different to the research audits resulting from inconclusive reviews. Its aim was to improve surgical care of women with breast cancer through the review of audit data and the establishment of quality standards of treatment, and is still in operation today.

As a result of the expertise gained in this area, RACS has established the binational audits of surgical mortality, now including gynaecology deaths⁷, and the trainee logbooks program. It is hoped that the logbooks program will form the basis in the future for a full practice audit for Fellows.

Conclusion

ASERNIP-S has proved to be a world leader in the evaluation of new surgical techniques and technologies. By focusing on surgery, the ASERNIP-S is unique among the international health technology assessment organisations. The strong support of the Fellows of RACS has been, and continues to be, essential to the success of this important work. The role of evidence-based medicine is not often well understood by the general public or even those working in the healthcare sector. However, for surgery, it is vital that techniques and technologies being introduced into practice come under vigorous and public assessment.

The key challenge for this highly successful research group is to achieve long-term funding so that we can provide a stable work environment for our very talented group of researchers and highly skilled support staff. Appropriate funding is vital to ensure the continuation of this high-level and necessary service to the Australian healthcare sector.

1. Australian Health Ministers’ Advisory Council. The Final Report of the Taskforce on Quality in Australian Health Care. Commonwealth of Australia, Canberra: AGPS, 1996.
2. Maddern GJ, Middleton PF, Tooher R, Babidge WJ. Evaluating new surgical techniques in Australia: the Australian Safety and Efficacy Register of New Interventional Procedures-Surgical experience. Surg Clin North Am 2006; 86(1): 115-28.
3. Maddern GJ, Middleton PF, Tooher R, Babidge WJ. Evaluating new surgical techniques in Australia: the Australian Safety and Efficacy Register of New Interventional Procedures-Surgical experience. Surg Clin North Am 2006; 86(1): 115-28.
4. Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097.
5. Maddern GJ, Middleton PF, Tooher R, Babidge WJ. Evaluating new surgical techniques in Australia: the Australian Safety and Efficacy Register of New Interventional Procedures-Surgical experience. Surg Clin North Am 2006; 86(1): 115-28.
6. Boult M, Cuncins-Hearn A, Tyson S et al. National Breast Cancer Audit: Establishing a web-based data system. ANZ J Surg 2005; 75: 844-847.
7. Raju RS, Guy GS, Majid AS, Babidge W, Maddern GJ. The Australian and New Zealand Audit of Surgical Mortality – Birth, deaths and carriage. Ann Surg 2015; 261(2): 304-8.