Error sensitivity of a log file analysis tool compared with a helical diode array dosimeter for VMAT delivery quality assurance

Abstract Purpose Integrating log file analysis with LINACWatch® (LW) into clinical routine as part of the quality assurance (QA) process could be a time‐saving strategy that does not compromise on quality. The purpose is to determine the error sensitivity of log file analysis using LINACWatch® compared with a measurement device (ArcCHECK®, AC) for VMAT delivery QA. Materials and methods Multi‐leaf collimator (MLC) errors, collimator angle errors, MLC shift errors and dose errors were inserted to analyze error detection sensitivity. A total of 36 plans were manipulated with different magnitudes of errors. The gamma index protocols for AC were 3%/3 mm/Global and 2%/2 mm/Global, as well as 2%/2 mm/Global, and 1.5%/1.5 mm/Global for LW. Additionally, deviations of the collimator and monitor units between TPS and log file were calculated as RMS values. A 0.125 cm3 ionization chamber was used to independently examine the effect on dose. Results The sensitivity for AC was 20.4% and 49.6% vs 63.0% and 86.5% for LW, depending on the analysis protocol. For MLC opening and closing errors, the detection rate was 19.0% and 47.7% for AC vs 50.5% and 75.5% for LW. For MLC shift errors, it was 29.6% and 66.7% for AC vs 66.7% and 83.3% for LW. AC could detect 25.0% and 44.4% of all collimator errors. Log file analysis detected all collimator errors using 1° detection level. 13.2% and 42.4% of all dose errors were detected by AC vs 59.0% and 92.4% for LW using gamma analysis. Using RMS value, all dose errors were detected by LW (1% detection level). Conclusion The results of this study clearly show that log file analysis is an excellent complement to phantom‐based delivery QA of VMAT plans. We recommend a 1.5%/1.5 mm/Global criteria for log file‐based gamma calculations. Log file analysis was implemented successfully in our clinical routine for VMAT delivery QA.


| INTRODUCTION
Volumetric modulated arc therapy (VMAT) is widely spread in modern radiotherapy and was first mentioned in literature in 1995. 1 The gantry continuously moves around the patient, and the field shape as well as the dose output is changed during the radiation process.
The field is continuously shaped by a multi-leaf collimator (MLC).
The positions of the gantry, the collimator, all leaves and the dose output at any time of the beam delivery are decisive factors for the treatment plan quality. Furthermore, it must be assured that the patient, especially their tumor region, is positioned correctly. This complex radiation technique therefore requires time-consuming pretreatment quality assurance (QA) procedures for all patient plans.
Common methods of pre-treatment delivery QA are two-or three-dimensional array detectors, such as MapCHECK ® (Sun Nuclear, Melbourne, FL), ArcCHECK ® (Sun Nuclear, Melbourne, FL) or Delta 2 (Scandidos, Uppsala, Sweden). These measurement arrays with diodes or ionization chambers are a time-consuming approach to check complex VMAT plans 3,4 and cannot be used during patient treatment.
An alternative method for delivery QA is integrating log file analysis into clinical routine. 2,5 Log files are automatically generated by the linear accelerator (linac) and contain all relevant plan data. Log files are considered as a practical and time-saving method as part of patient QA. Haga et al. showed that the calculated dose distribution of the treatment planning system (TPS) plan agreed well with the recalculated log file using 2%/2 mm gamma index criteria. 6 When using log file analysis, machine QA becomes more essential, because all dose relevant parameters in the log file must be checked. 7 Using only log file analysis for delivery QA without any other physical measurements is thoroughly discussed in the literature. 8 Numerous delivery QA methods have already been compared with each other. [9][10][11] However, the comparison of the sensitivity of 4 Hz log files of an Elekta Synergy linac with the sensitivity of a standard three-dimensional (3D) phantom for VMAT delivery QA has not yet been reported in literature. The aim of this study is to compare log file analysis (LINACWatch ® , Qualiformed) with a standard 3D phantom (ArcCHECK ® , Sun Nuclear) and ionization chamber using different kinds of treatment plans containing MLC, dose, and collimator errors.     2 Gy was used. The 12 head and neck plans were divided into four radiation plans for patients being treated solely on the left side (two to three arcs between 180°and 220°), four plans on the right side (two to three arcs between 220°and 340°) and four plans for patients being treated with radiotherapy on both sides (two to three arcs between 280°and 360°). For all head and neck plans the fraction dose was between 1.8 and 2 Gy. Additionally, the 12 SBRT plans were divided into 6 thoracic and 6 head metastasis plans. The fractional dose varied between 4 and 10 Gy and the planning target volume was between 100 and 200 cm 3 . The SBRT radiotherapy plans were composed of one to two arcs between 180°and 360°.

2.C | Statistical analysis and dosimetric comparison
Every plan was irradiated on the ArcCHECK phantom and the corresponding, simultaneously acquired log file was sent to LINACWatch.
Thus, the same plan was examined by both delivery QA systems.
Sensitivity was defined as the ratio of properly detected incorrect plans to all incorrect plans according to formula (1). For specificity, plans free from any error (reference plans) were taken into consideration according to formula (2). sensitivity % ð Þ¼ number of detected non À error À free plans total number of non À error À free plans Á 100 (1) specificity % ð Þ¼ number of detected reference plans total number of reference plans Á 100 (2) Two different protocols for ArcCHECK ® and two different protocols for LINACWatch ® were used to detect the incorrect plans. AC is widely used, and standard gamma criteria (see Table 1) can be found in in the literature and were applied for AC evaluation. 10,11 For LW, no standard gamma criteria have been established yet in the literature. It was important for us that reference plans could be detected as such (specificity = 100%). Due to the nature of the gamma value calculation, applying the same criteria to different systems will not lead to comparable results. Therefore, different gamma criteria are required. We researched suitable gamma criteria with LW prior to this study, which led to stricter gamma criteria as well as stricter acceptance limits for LW (see Table 1).
For LW, three evaluation levels were involved for detecting nonerror-free plans. First, the calculated fluence of the log file and the fluence of the TPS were compared using the gamma index method. 12 The second evaluation level was the root-mean-square The ionization chamber dose difference is standardized to the measured dose of the error-free plan [see formula (3) The statistical evaluation was performed using SPSS (IBM, NY, USA), for the evaluation of the sensitivity, specificity and the dose of the ionization chamber. A p-value smaller than 0.05 was defined as statistically significant. 4.0. GUI. Calculated fluence of treatment planning system (left); Calculated fluence of log file (middle); Fluence comparison using 2%/2 mm gamma criteria (right).  Table 2 shows the sensitivity of all 540 non-error-free plans, depending on the analysis protocol and measurement device.
In general, the results indicate that for tighter gamma criteria, the sensitivity of the system increases. If the gamma criteria are too low, the specificity decreases. Specificity shows no significant differences between all used protocols. All error free plans were detected as such, indicating precise delivery of linac.
The stricter gamma index for AC (2%/2 mm) and LW (1.5%/ 1.5 mm) with a detection level of >90% and >95% does not ensure finding all MLC positioning, dose or collimator positioning errors used in this study (see Table 1).
LW is more sensitive in detecting the implemented errors intro-

3.B | Individual analysis: prostate -H&N -SBRT
In general, steeper curves indicate higher sensitivity of the protocol or system. The figures show the response of the gamma value of the respective system to implemented errors.   Figure 3 shows the mean gamma passing rate depending on an MLC shift error. The sensitivity of all systems for detecting an MLC shift error is lower in prostate and H&N than in SBRT plans. SBRT behavior differs from prostate and H&N, because of its smaller field size. Therefore, the impact of implemented errors is higher in SBRT. LW is more sensitive in detecting an MLC shift error than AC using the prescribed protocols.

3.B.4 | Dose errors
LW shows a higher sensitivity than AC device (see Fig. 5). LW (1.5%/1.5 mm) could detect all dose errors from AE2% with the fluence gamma criterion and additionally, the same errors with the RMS value of the monitor units set to 1% limit (see Fig. 6).

| DISCUSSION
In our study, all reference plans were correctly detected as plans without any error for all protocols used. Both examined gamma criteria for each system (AC and LW) were applicable for clinical use in terms of specificity. Gamma values of reference plans were higher for LW than for AC, which led to stricter gamma criteria as well as stricter acceptance limits (e.g. LW 1.5%/1.5 mm had a higher passing rate for the reference plans than AC 2%/2 mm). For less strict criteria applied on LW, the sensitivity would have declined, and the specificity would have remained constant. Consequently, less strict criteria for LW would not lead to any advantage. This is the opposite for AC: A stricter criteria led to a decline in specificity.
There is a strong correlation between the gamma passing rate and the magnitude of the errors in the treatment plans. The passing rate responded stronger to errors in case of log file analysis. For all treatment sites (prostate, H&N and SBRT) these results clearly show that log file analysis is more sensitive than AC measurements regardless of the applied protocol used in this study.
The agreement for both the reference and the non-error-free plans for AC gamma passing rate is consistent with previous research not using log files. 3,[9][10][11]13,14   The essential strength of our investigation is that all measurements were performed with AC, LW and ionization chamber at the same time. Log files were generated during AC measurements of all plans. Further strengths are the use of different treatment sites (prostate, H&N and SBRT) and various fractional doses.
These methods could be further be refined by also considering the high-resolution log files (25 Hz instead of 4 Hz). Fluence without CT data does not illustrate the clinical consequences of an irradiation error, because dose recalculation is not performed. Therefore, no TPS data (beam modeling, beam data) or plan quality is checked.
More accurate calculation algorithms for log file analysis, such as an independent Monte Carlo (in combination with CT), could increase the efficiency of detecting clinically relevant errors. Furthermore, it enables finding different kinds of TPS errors, especially beam data or beam modeling.

| CONCLUSION
Our conclusion is that log file analysis is an excellent tool for delivery QA with Elekta linacs of VMAT plans. LW is very sensitive to detect small delivery errors. We recommend using LW with 1.5%/ 1.5 mm global for the gamma calculation delivery QA. We also recommend using RMS limits of 1°for collimator position and 1% for dose errors. Log file analysis is an outstanding complement to phantom-based delivery QA, which, consequently, we integrated successfully into our clinical routine.

ACKNOWLEDG MENTS
The authors thank the physicists at the Institute of Medical Physics (Academic Teaching Hospital Feldkirch) for useful discussions and for comments that improved this manuscript.

CONFLI CT OF INTEREST
The authors have no relevant conflict of interest to disclose.