Evaluation of Varian’s SmartAdapt for clinical use in radiation therapy for patients with thoracic lesions

Abstract Introduction Deformable image registration (DIR) is a required tool in any adaptive radiotherapy program to help account for anatomical changes that occur during a multifraction treatment. SmartAdapt is a DIR tool from Varian incorporated within the eclipse treatment planning system, that can be used for contour propagation and transfer of PET, MRI, or computed tomography (CT) data. The purpose of this work is to evaluate the registration and contour propagation accuracy of SmartAdapt for thoracic CT studies using the guidelines from AAPM TG 132. Methods To evaluate the registration accuracy of SmartAdapt the mean target registration error (TRE) was measured for ten landmarked 4DCT images from the https://www.dir‐labs.com/ which included 300 landmarks matching the inspiration and expiration phase images. To further characterize the registration accuracy, the magnitude of deformation for each 4DCT was measured and compared against the mean TRE for each study. Contour propagation accuracy was evaluated using 22 randomly selected lung cancer cases from our center where there was either a replan, or the patient was treated for a new lesion within the lung. Contours evaluated included the right and left lung, esophagus, spinal canal, heart and the GTV and the results were quantified using the DICE similarity coefficient. Results The mean TRE from all ten cases was 1.89 mm, the maximum mean TRE per case was 3.8 mm from case #8, which also had the most landmark pairs with displacements >2 cm. For contour propagation accuracy, the DICE coefficient results for left lung, right lung, heart, esophagus, and spinal canal were 0.93, 0.94, 0.90, 0.61, and 0.82 respectively. Conclusion The results from our study demonstrate that for thoracic images SmartAdapt in most cases will be accurate to below 2 mm in registration error unless there is deformation greater than 2 cm.


| INTRODUCTION
Deformable image registration (DIR) is at the heart of a robust adaptive radiotherapy program. Deformable image registration is needed to manage inter-and intrafractional changes of patient anatomy relative to their treatment plans. Examples of these changes include breathing, weight changes, surgeries, disease progression/regression, or simply those caused by variations in imaging setup/position (i.e., arms up vs arms down). Anatomical variation is the major reason why the delivered dose can never be exactly equal to the planned dose. Using a DIR tool in radiation therapy helps achieve a better understanding of the total delivered dose to a patient. For example, trying to adapt a treatment plan to the changes in the target volume as the treatment course progresses, DIR allows two anatomies to be linked together deformably allowing the contours and dose to be transferred from one computed tomography (CT) study to another.
The goal of any DIR algorithm is to produce a deformation vector field (DVF), mapping voxels from a source image to voxels in a target image. There are many different approaches to produce a DVF and are typically defined by three components namely image similarity metric, regularization, and optimization. Image similarity metrics are used by an algorithm to determine how "correct" the registration is at any step. Regularization is how the algorithm produces a realistic DVF based on desired properties, for example conservation of mass or continuity. Optimization is the algorithms approach to combine regularization and image similarity metric to reach an optimal DVF quickly. The characteristics of all of these components define a DIR algorithm and will define its accuracy and efficacy for different imaging modalities and anatomical sites depending on image contrast and deformation type and magnitude. 1,2 SmartAdapt (V 13.6) is a DIR tool available to Eclipse TM treatment planning system (Varian Palo Alto, CA). It is understood that SmartAdapt is based on an accelerated demons algorithm, 3 which uses the gradients in image intensity values to drive the registration. 4 Driven by image intensity, SmartAdapt can perform DIR on CT, CT-PET, and MRI and propagate contours between different datasets. However, the dose deformation feature, is not currently supported in the software. Varian's other DIR software, Velocity™, uses a B-spline driven deformable image registration and provides the dose deformation tool. SmartAdapt software is usually included in Eclipse TPS by default and is of lower cost compared to Velocity.
As such, the prospects of using SmartAdapt are attractive to Eclipse TM users for both contour propagation and possibly dose deformation for clinics with limited resources.
The Task Group 132 of the AAPM 5 has provided guidelines for understanding DIR tools and recommends commissioning, quality assurance, and quality control methods for the clinical use of image registration processes. As per the guidelines, any DIR tool needs to be commissioned before clinical implementation allowing physicists to better understand the fundamental components of the employed DIR algorithm. TG132 suggests primarily a quantitative evaluation of DIR tools through the use of predetermined landmarks to calculate the target registration error (TRE). That report also recommends an independent evaluation of the quality of registration for identifiable features such as organ contours.
There are a number of studies evaluating SmartAdapt for different sites including head and neck, 3,6 cervical, 7 and prostate. 8 Also, there is a report on the evaluation of SmartAdapt using a thoracic phantom. 9 Those studies can be classified as feature-based validations since they are comparing manmade contours to automatically propagated contours from SmartAdapt. It should be noted that the validity of contour propagation does not automatically imply the validity of the algorithm for other purposes. An essential element in DIR validation is the evaluation of the interior of contoured volumes using anatomical landmarks. This is of importance when, for example, PET data are registered/deformed to planning CT images or if dose deformation is required. Currently there is no known study performing landmark or contour evaluation of SmartAdapt for thoracic images. In this study we aim to evaluate SmartAdapt using landmarks and contours for thoracic images using TG132 recommendations.
This will establish a baseline to provide evidence to support SmartAdapt for CT-CT registration, contour propagation, CT to PET-CT deformation as well as for dose deformation.
Thoracic region was chosen for this study due to its unique type of motion and image contrast. Indeed, lung tissue can exhibit large amounts of deformation caused by inflation and deflation within one respiratory cycle. Also, lung cancer patients may undergo plural effusion changing their anatomy, while lung lesions may also exhibit rapid changes during a course of radiotherapy. Thoracic images also exhibit high contrast between bones, soft tissue, and lung providing an excellent contrast to drive the registration algorithm. With all these unique characteristics, DIR has many applications in thoracic CT studies including accounting for tumor volume or anatomy changes during a course of radiation therapy.

| MATERIALS AND METHODS
Based on the recommendations of the TG132 report, a two-step evaluation including (a) a landmark deformation analysis and (b) an independent assessment of contour propagation by SmartAdapt are studied.
These CT datasets are from patients as part of there treatment planning for thoracic malignancies (lung and esophagus) no other selection criteria was considered. 10,11 This dataset included all of the available 4DCT datasets from dir-labs and was recommended by TG-132 for testing the target registration error (TRE) of DIR algorithms.
Here, LI is the inspiration phase landmark, LE is the expiration phase landmark, and d is the landmark in the expiration phase produced by the DIR. The mean TRE is calculated by averaging the TRE over all 300 landmarks for each case and is called TRE mean . As per AAPM TG132 criteria, the TRE mean should be smaller than the smallest voxel dimension (2 mm) 10,11 for all ten cases included in this cohort. In addition, TG132 recommends that TRE mean be less than 2mm and the max TRE be less than 5mm, specifically in case number 6.
Mean landmark displacement, D LM , quantifies the magnitude of deformation for each case and is calculated using Eq.
is an indication of large anatomical deformation.

2.B | Contour propagation accuracy
In this section, we evaluate SmartAdapt's DIR algorithm by comparing its contour propagation accuracy to human-made contours in  The D LM is presented in Fig. 3 for all ten cases. It can be seen that cases 4, 6, 7, and 8 show the largest D LM , about 10 mm on average.
However, the TRE mean is large for cases 7 and 8. To investigate this indepth, all 3000 landmarks were extracted from the ten cases and sorted into ten equally spaced bins based on their D LM . The TRE mean was then calculated for each bin and displayed in Fig. 4.
As expected, TRE mean increases with the displacement magnitude but the increase is not linear but rather an exponential behavior can be seen. Figure 5 is breaking down the distribution of landmark displacements per case and cases 4 and 6 have fewer landmarks with displacements above 23 mm. However, cases 7 and 8 have the largest number of landmarks with displacement >23 mm which may partially explain the higher TRE mean in cases 7 and 8.

3.B | Contour propagation accuracy
The DICE similarity coefficients for GTV, esophagus, spinal canal, heart, and left and right lungs are averaged over all patients and presented in Table 1. For comparison, contour propagation subsequent to a rigid registration is also shown in that table. As expected, a rigid registration produces smaller DICE than DIR for all contours.
Using the modified criteria SmartAdapt performed well for the lungs and heart with an average DICE above 0.90, and spinal canal having an average DICE above 0.80. SmartAdapt DIR did not perform as well for esophagus as shown by an average DICE value of 0.61. This low performance for the esophagus is partially explained by its high relative surface area but additionally, by the embedded uncertainty for contouring the esophagus among different observers. 13 GTV also showed a low DICE with a mean value of 0.72 which will be discussed in more detail later in the discussion.

| DISCUSSION
The aim of this work was to determine the clinical viability of the SmartAdapt DIR tool for thoracic CT images. Evaluation was performed using the AAPM TG132, which recommends an average The dataset used in this study for landmark evaluation came from the https://www.dir-lab.com/ created from the work from Castillo et al. 10,11 There are many different articles using the same dataset to test a variety of DIR algorithms including new research DIR algorithms. 14,15 For this study we compared SmartAdapt only against commercially available algorithms and did not compare it to those research algorithms to keep our conclusions clinically practical. A study by Kadoya et al. 16    CT or exhibit large interobserver variability will likely perform poorly using SmartAdapt's DIR.

| CONCLUSION
SmartAdapt is a DIR tool available to many Eclipse users, but currently lacks thorough evaluation of its registration accuracy for the clinical use of contour propagation and deformably registering PET or MRI data. The goal of this study was to provide evidence for the future clinical use of SmartAdapt using the TG-132 recommendations with thoracic CT studies. The landmark analysis measured an average TRE of under 2mm (recommended by TG132) for eight of ten cases, but not for the two cases experiencing large amounts of deformation above 20 mm. Contour propagation using SmartAdapt was within TG-132 recommendations for lung, heart, and spinal cord contours but not for the esophagus or GTV where manual adjustments may be required. Overall SmartAdapt has shown to be a competent DIR tool for thoracic images within a radiotherapy clinic and should be considered for routine clinical use. Given the favorable results from SmartAdapt in this study, our future work will explore using SmartAdapt for deformable dose transformations within the Eclipse treatment planning system.

ACKNOWLEDGMENTS
We acknowledgment the group responsible for dir-labs.com as there lung 4DCT datasets made this work possible.

CONF LICT OF I NTEREST
The authors have no conflict of interest to report for this study.