Remote dosimetric auditing of clinical trials: The need for vendor specific models to convert images to dose

Abstract Introduction A previous pilot study has demonstrated the feasibility of a novel image‐based approach for remote dosimetric auditing of clinical trials. The approach uses a model to convert in‐air acquired intensity modulated radiotherapy (IMRT) images to delivered dose inside a virtual phantom. The model was developed using images from an electronic portal imaging device (EPID) on a Varian linear accelerator. It was tuned using beam profiles and field size factors (FSFs) of a series of square fields measured in water tank. This work investigates the need for vendor specific conversion models for image‐based auditing. The EPID measured profile and FSF data for Varian (vendor 1) and Elekta (vendor 2) systems are compared along with the performance of the existing Varian model (VM) and a new Elekta model (EM) for a series of audit IMRT fields measured on vendor 2 systems. Materials and methods The EPID measured beam profile and FSF data were studied for the two vendors to quantify and understand their relevant dosimetric differences. Then, an EM was developed converting EPID to dose in the virtual water phantom using a vendor 2 water tank data and images from corresponding EPID. The VM and EM were compared for predicting vendor 2 measured dose in water tank. Then, the performance of the new EM was compared to the VM for auditing of 54 IMRT fields from four vendor 2 facilities. Statistical significance of using vendor specific models was determined. Results Observed dosimetry differences between the two vendors suggested developing an EM would be beneficial. The EM performed better than VM for vendor 2 square and IMRT fields. The IMRT audit gamma pass rates were (99.8 ± 0.5)%, (98.6 ± 2.3)% and (97.0 ± 3.0)% at respectively 3%/3 mm, 3%/2 mm and 2%/2 mm with improvements at most fields compared with using the VM. For the pilot audit, the difference between gamma results of the two vendors was reduced when using vendor specific models (VM: P < 0.0001, vendor specific models: P = 0.0025). Conclusion A new model was derived to convert images from vendor 2 EPIDs to dose for remote auditing vendor 2 deliveries. Using vendor specific models is recommended to remotely audit systems from different vendors, however, the improvements found were not major.


| INTRODUCTION
Quality assurance (QA) is an essential procedure to assess accuracy of relevant parameters in radiotherapy 1 while an external audit is recommended to assess consistency of local QA and effectiveness of delivery and measurement systems. 2 The importance of external audits is emphasized in radiotherapy clinical trials where a consistent accuracy is essential. [3][4][5] Conventional audits are performed by sitevisits or postal methods, which can be expensive and/or labor intensive. [6][7][8] Some virtual methods have been explored to reduce the audit cost using in-house QA methods. 9 Recently a novel approach was introduced to remotely assess intensity modulated radiotherapy (IMRT) deliveries using pre-treatment images from electronic portal imaging devices (EPIDs). The method was known as the Virtual Epid Standard Phantom Audit (VESPA) and designed for dosimetric auditing of clinical trials at remote facilities. The VESPA utilized an in-house software for analysis and provided a relatively consistent detection system for data acquisition. 10 Participating facilities were provided with CT data sets of the virtual water phantoms and transferred prostate and head and neck IMRT treatment plans onto these to calculate dose in their local treatment planning system (TPS). They electronically sent their images and planned dose to the auditing site for assessment.
The in-house software of the VESPA back-projects in-air acquired images from EPIDs into virtual water phantoms and converts the signals to dose at 10 cm depth within the phantoms. 11,12 The conversion is performed based on a model developed by King et al. at Calvary Mater Newcastle Hospital (CMNH). The software input includes a machine specific file, a beam model file and DICOM images and doses. The machine specific file refines the input and adapts it to each machine/delivery system using the facility calibration images. This file includes parameters defining central axis coordinate on the EPID and EPID-linac sag correction. Another software input is the beam model file referred to here as the Varian model (VM). The VM is not adjusted for each facility. It has been developed using aS1000 EPID acquired images from a Varian linac deliveries (vendor 1) of series of square fields. The beam profiles and field size factors (FSFs) of the deliveries were also measured in water tank and used for the VM optimization. The VM has been extensively benchmarked and used for vendor 1 in-house QA.
Six facilities took part in a pilot study of the remote based auditing method. Three of the facilities acquired data from Varian delivery and measurement systems (vendor 1) and three from Elekta (vendor 2). 13 The pilot study used the VM for both vendors but applied primary vendor differences to the machine specific file. Differences in the detector size and resolution were applied; vendor 1: aS1000 EPIDs with 40 × 30 cm 2 active area, that is, 1024 × 768 image resolution with 0.039 cm pixel resolution and, vendor 2: iViewGT EPIDs with 41 × 41 cm 2 active area, that is, 1024 × 1024 image resolution with 0.040 cm pixel resolution. 14 Moreover, prior to analysis, acquired images at 160 cm source to detector distance (SDD) from vendor 2 were resampled to 100 cm. The ".HIS" format images acquired from iViewGT EPIDs were also converted to DICOM in consistent with the software input requirement. In spite of the applied differences to each machine file, slightly lower gamma pass rates were observed in the auditing results from vendor 2. The vendor 2 systems also demonstrated a different field size response for reconstructed dose at the phantom isocentre compared with those from vendor 1. These all could be due to the differences of relevant dosimetry characteristics between the two vendors. Ignoring the differences can result in significant uncertainties in the audit outcome. 15 Accordingly, this research studies relevant dosimetric variations between the two vendors and corresponding dose conversion models. Then, it investigates whether using vendor specific models could make the audit results independent from the vendors. This study develops a model (EM) to convert images from EPID to dose inside the virtual phantom for vendor 2 deliveries. Then, the EM performance is compared with the VM for measured water tank data from vendor 2 deliveries. The EM is used for remote auditing of 54 IMRT fields from four vendor 2 facilities.
Statistical study of the auditing results determines whether a vendor specific model is required for auditing of each vendor. This work will facilitate implementation of this new and efficient auditing procedure using a remote EPID based dosimetry with improved sensitivity.

2.A | Dosimetry
A series of square field beams, 3 × 3, 4 × 4, 6 × 6, 10 × 10, 15 × 15, 20 × 20, and 25 × 25 cm 2 , were delivered by a vendor 1 and a vendor 2 linac and, in-air images were acquired by respectively an aS1000 and iViewGT EPID. The profiles and FSFs were acquired from the image signals to evaluate the differences of relevant dosimetric parameters between the two vendors. Note, the profiles and FSFs were later used for modeling signal to dose. The profiles were obtained from the pixel data in the crossplane through the central axis. The profiles penumbras were defined to quantify the profile differences. The penumbra widths were defined as the distance between 80% and 20% of the maximum dose for each side of the profile relative to central axis. The FSFs were directly extracted from the mean pixel value of the central 11 × 11 pixels of the image signals and, the difference between FSFs of the vendors was quanti-

2.B | Modeling
Following the method of King et al., 11 which was used to develop a vendor 1 model (VM), a vendor 2 model (EM) was developed to convert images to dose onto the virtual phantom. Images from an iViewGT EPID and a vendor 2 measured dose in water tank (WT) were acquired. The images were acquired in-air from delivery of series of square field beams, 3 × 3, 4 × 4, 6 × 6, 10 × 10, 15 × 15, 20 × 20, and 25 × 25 cm 2 . The water tank data were measured at 10 cm depth and used to optimize the model parameters. The water tank data were acquired at 100 cm SDD using a small cylindrical ionization chamber of CC01 for small field sizes, that is, 3 × 3, 4 × 4, 6 × 6 cm 2 , and a CC13 for the large field sizes, that is, 10 × 10, 15 × 15, 20 × 20, and 25 × 25 cm 2 . All images were acquired at 160 cm SSD and resampled to 100 cm SSD using interpolation. The images were truncated at about 1 cm of the detector edge to avoid the edge artefacts. As the images were found noisier than those from aS1000 EPIDs, an adaptive "wiener2" filter in MATLAB was used to reduce the image noise and its impact on the model convolution function. The "wiener2" low pass filters the images that have been degraded by a constant power additive noise. It uses a pixel wise adaptive method based on statistics estimated from a local neighborhood of each pixel. 18 An initial trial EM could not consistently predict the FSFs for the four facilities. After investigation, an averaged FSF from the TPSs of the four facilities was used as the reference FSF for modeling purposes, see Supporting information.
The EM model accuracy was quantified via calculating discrepancy between the image and water tank dose for the profiles and FSFs where "nfield" was number of dose measurements/points. Furthermore, percentage differences were calculated for the EM dose

2.C | Auditing
The EM was used to convert pre-treatment images from IMRT deliveries, a post-prostatectomy (PP) and a head and neck (HN) plan, to dose for four vendor 2 facilities. Details of these plans and the audit procedures are detailed elsewhere. 10,13 Each facility delivered (7-9) IMRT fields per patient plan. For each field, the converted EPID dose was compared to corresponding TPS dose. The comparisons were performed by an in-house developed gamma function at three different criteria, 3%/3 mm, 3%/2 mm, and 2%/2 mm. The EM performance was compared with the VM performance for the IMRT audits at 1%/1 mm gamma criteria. Finally, a statistical study was conducted on the pilot audit including facilities from both vendors to compare performance of the vendor specific models and VM solely applied to all facilities.  and, for C 2 and C 3 . However, a relatively large discrepancy was observed in penumbras of all facilities at the very large field, that is, 20 × 20 cm 2 . The subplot in Fig. 2(a) shows percentage difference for the 10 × 10 cm 2 profiles. The largest difference was observed for C 3 and the smallest for C 2 . Relatively similar trend was observed for other field sizes (not plotted). Figure 2(    EM and WT dose profiles. The Fig. 4 Figure 6 and Table 1

| DISCUSSION
The VESPA auditing procedure is designed as an inexpensive and efficient auditing procedure that can be performed remotely with the time for the central site physicist generally being 2-3 h to assess the results. The audit requires time from the local physicists to produce the IMRT verification plans and deliver the beams to the EPID, however, all other auditing methods require local personnel time.
The VESPA also does not include any equipment or transport costs.
The studies on the method has been conducted on two vendors using one vendor verified model (VM) to convert the image signal to dose inside the phantom. Investigation for the need for vendor specific models makes the audit reliable over different vendors.
Studies on relevant EPID measured dosimetric parameters showed differences between the two vendors. The discrepancy increased between the vendors' profiles at the very small/large field sizes, 3 × 3 and 20 × 20 cm 2 . The smaller penumbras observed for vendor 1 profiles indicate sharper profiles of corresponding images which may result in increasing the VM accuracy. The small penumbras for vendor 1 could be due to the proximity of the collimating system to F I G . 5. Auditing results of a post-prostatectomy (PP) and a head and neck (HN) plan from four vendor 2 facilities, C 1 , C 2 , C 3, and C 4 , using the EM for analysis. Each facility has delivered (7-9) IMRT fields per treatment sites, totally 54 fields. The results include gamma pass rates and corresponding mean gammas for each patient plan.
the machine isocenter. For the FSFs of the two vendors, the discrepancy was increased by field size which was in accordance with the previous observations in the pilot audit. The FSF differences between the vendors could be due to differences in either EPID scatter or head scatter beam as the EPID signals incorporate both effects.
The study on vendor 2 facilities showed some inconsistencies in their dosimetric parameters. The C 3 signals showed largest discrepancy with C 1 signals at profiles, penumbras and FSFs. The C 2 showed the minimum differences with the C 1 profile but for penumbras and, the C 4 showed the closest values to C 1 penumbras.
However, the FSF influence seems more important than the profiles impact for the model accuracy since the FSFs are used in optimizing four out of six model parameters while two parameters are tuned by profiles. A comparison between Figs. 1 and 2 shows larger inter-vendor discrepancy (vendor 1 and vendor 2) than intra-vendor variations (C 1 , C 2 , C 3 and C 4 ) for both parameters. This is in accordance with a report from Cozzi et al. 19  The new EM and the VM were used to convert dose for deliveries from respectively vendor 2 and vendor 1 facilities in a study. The deliveries were also analysed using only VM for both vendors. Statistical studies of the two scenarios demonstrated a minor improvement when using vendor specific models (P = 0.0025) than the VM (P < 0.0001). Vendor dependency of the auditing results reduced when using vendor specific models (EM for vendor 2 and VM for vendor 1). However, mean gammas for vendor 2 were still larger than for vendor 1. This could be due to the impact of other variables such as facility TPS types which were not considered in this study.

| CONCLUSION
Observed differences in relevant dosimetry parameters between vendor 1 and vendor 2 suggested using vendor specific models, to convert signal to dose onto the virtual phantoms, could account for dosimetry differences between the vendors. By developing a new model (EM) and using vendor specific models, the EM for vendor 2 and VM for vendor 1, the audit difference reduced between two vendors. The audit accuracy was improved and using vendor specific models was advised for future audits. The remote audit approach provides a highly automated method with significantly reduced cost.

ACKNOWLEDG EMENTS
The authors are grateful for the assistance of the many physicists and therapists at the remote centers who planned the benchmark cases and measured EPID data. Funding has been provided from the

CONFLI CT OF INTEREST
It is represented and warranted that, as at the date of this declaration, there is not any actual or perceived conflict of interest, or potential conflict of interest.

R E F E R E N C E S SUPPORTING INFORMATION
Additional supporting information may be found online in the Supporting Information section at the end of the article.     The Clinac FSF is a TPS data used for the VM modeling.
MIRI ET AL.