Long‐term experience of MPC across multiple TrueBeam linacs: MPC concordance with conventional QC and sensitivity to real‐world faults

Abstract Machine Performance Check (MPC) is an automated Quality Control (QC) tool that is integrated into the TrueBeam and Halcyon linear accelerators (Linacs), utilizing the imaging systems to verify the Linac beam and geometry. This work compares the concordance of daily MPC results with conventional QC tests over a 3‐year period for eight Linacs in order to assess the sensitivity of MPC in detecting faults. The MPC output measurements were compared with the monthly ionization chamber measurements for 6 and 10 MV photon beams and 6, 9, 12, 16, and 18 MeV electron beams. All 6 MV Beam and Geometry (6MVBG) MPC test failures were analyzed to determine the failure rate and the number of true and false negative results, using the conventional QC record as the reference. The concordance between conventional QC test failures and MPC test failures was investigated. The mean agreement across 1933 MPC output and monthly comparison chamber measurements for all beam energies was 0.2%, with 97.8% within 1.5%, and a maximum difference of 2.9%. Of the 5000–6000 MPC individual test parameter results for the 6MVBG test, the highest failure rate was BeamOutputChange (0.5%), then BeamCenterShift (0.3%), and was ≤ 0.1% for the remaining parameters. There were 50 true negative and 27 false negative out of tolerance MPC results, with false negatives resolved by repeating MPC or by independent measurement. The analysis of conventional QC failures demonstrated that MPC detected all failures, except occasions when MPC reported output within tolerance, a result of the MPC–chamber response variation. The variation in MPC output versus chamber measurement indicates MPC is appropriate for daily output constancy but not for the measurement of absolute output. The comparison of the 6MVBG results and conventional records provides evidence that MPC is a sensitive method of performing beam and mechanical checks in a clinical setting.


| INTRODUCTION
The Quality Control (QC) programs of linear accelerators (Linacs) in the clinic generally follow the frequency, testing methods, and tolerances in published national guidance. 1,2 In recent years, automated QC programs utilizing the electronic portal imaging device have been proposed, 3,4 and Linac vendors have begun to incorporate QC applications into the Linac platforms. One such application is Machine There have been a number of publications evaluating and validating the accuracy of MPC 5-10 since it was first introduced by Varian in 2015. The testing of MPC to date has been via two methods: either comparing constancy over time against another established measurement method [5][6][7][8][9] or via the introduction of deliberate errors. [6][7][8]10 Methods for introducing deliberate errors included intentionally incorrectly calibrating a Linac parameter, for example, the Linac beam symmetry or MLC position calibration, [6][7][8] or by introducing solid water into the beam path, 10 or by applying known motions to the MPC phantom using a rotating/linear motion stage. 10 The results of this testing of deliberate errors in mechanical and imaging parameters demonstrated sub-mm and sub-degree accuracy. [6][7][8]10 With regard to beam characteristics, MPC was able to detect a deliberate steering error in the 6 MV beam center to within 0.2 mm of the IC PROFILER (Sun Nuclear Corporation, Melbourne, FL, USA) ionization chamber array measurement, and the uniformity agreement with symmetry from the IC PROFILER was within 0.9% for 6 and 10 MV flattened and flattening filter free (FFF) beams. 6 This testing demonstrates that MPC is capable of detecting beam and mechanical faults to a level that is appropriate for QC, comfortably less than the 1-2 mm and 2-3% tolerances in TG142 1 and IPEM report 81v2. 2 MPC results have been compared over time against the results from other established measurement methods, over periods of 3 weeks 5 up to 1 year. 9 Of these, the longest datasets are Barnes and Greer, 6 which compared MPC on a single Linac over a 5-month period for both flattened and FFF 6 and 10 MV beams, and Binny et al 9 who report results for six TrueBeams over periods ranging 4.5-12 months (average 7.5 months) for 6 and 10 MV photon beams and 6,9,12, and 16 MeV electron beams. Barnes and Greer 6 showed that the agreement between Farmer ionization chamber and MPC output measurements was 0.6% over the 5 months. Binny et al 9 demonstrated that mean output variations were within ± 0.5% compared with Farmer ionization chamber and ± 1.5% compared with Daily QA3 (Sun Nuclear Corporation, Melbourne, FL, USA) dose constancy results, respectively.
This work aimed to evaluate MPC over a much longer period of time than has been reported previously, reporting on 3 yr of using MPC daily across eight Linacs, which equates to over 20 Linac years.
We report the output stability for 6 and 10 MV photon beams, and 6, 9, 12, 16, and 18 MeV electron beams. In addition, rather than testing the sensitivity of MPC to deliberate errors, that has been reported previously, we instead evaluate the ability of MPC to detect real-world QC faults over the 3-year period by comparison with conventional QC testing records. We assess the sensitivity of MPC to detect faults in the clinical setting and therefore the suitability of MPC as a routine QC tool.

2.A | MPC and Conventional QC
MPC was run daily at morning run-up on each of the eight Varian TrueBeams in the Author's department, all running TrueBeam platform software version 2.5. The Linac fleet comprised seven True-Beams, and one TrueBeam Stx, named G1 to G6 on the main site and Q1 and Q2 at the satellite center. All Linacs were equipped with the aS1200 portal imager, with the TrueBeams equipped with Millennium 120 MLC, and the TrueBeam STx with the HD120 MLC, which were reinitialized monthly in servicing prior to QC. All Linacs can deliver 6 and 10 MV flattened photon beams, and 6, 9, 12, 16, and 18 MeV electron beams, apart from one (G4) which has 6 and 10 MV only.
Additionally, the DailyQA3 (Sun Nuclear Corporation, Melbourne, FL, USA) was used to perform weekly measurements on all beams.
The device utilizes a number of ionization chambers and diode detectors to measure output constancy, flatness, symmetry, energy, and radiation field size. 11 All the parameters tested by MPC and the tolerances are outlined in Table I MPC was baselined in the commissioning period of each Linac once the beam setup was optimized based on Beamscan (PTW, Freiburg, Germany) water tank measurements. There is a function in the MPC application to allow subsequent re-baselining, required due to drift in MV panel response over time. This will simultaneously set new baseline values for the output, beam uniformity, and beam center. Therefore, on occasions where MPC was re-baselined to correct the drift in output constancy versus ionization chamber measurement, the beam flatness and symmetry and beam center position were confirmed via conventional QC checks. Ideally, the beam setup would be optimal prior to re-baselining in order to minimize the systematic errors in the MPC measurement. Due to the pressures of our busy Clinical department, a more practical approach was adopted, whereby we required the conventional QC flatness and symmetry to be less than 2%, and required the conventional QC relating to the beam center position to be within tolerance. The flatness and symmetry level of 2% were set tighter than our suspension level tolerance of 3% (Table I), to reduce the magnitude of any potential systematic error; in the worst-case scenario, if the symmetry were just less than 2% when baselined, the true symmetry could reach 4% before MPC recorded an out of tolerance result. On implementation of MPC, we regarded these potential systematic errors as Previous authors 6,9 have implemented a 1% tolerance on the grounds that the MPC threshold is set in the application at 2%, thus ensuring that daily output is within the TG142 1 tolerance of 3%. We extended the tolerance slightly on implementation to 1.5% on any given month, and 1% over three consecutive months. We accepted a slightly wider potential discrepancy in the daily MPC reported output, in order to reduce the frequency of MPC calibrations, and required the ionization chamber-MPC output difference to be consistent over several months to avoid incorrectly baselining on an outlying result. We justified this slight increase when MPC was implemented in our department, based on knowledge of the tolerance of ± 5% for output constancy in UK guidance 12 that was in place at the time (now superseded by report 81v2 2 which recommends a reduced tolerance of 3%), and the range in tolerances for daily output constancy devices of between ± 1% and ± 5% reported in use in a UK survey of Linac QC. 13 In order to further evaluate MPC output constancy measurements, we have compared our data with data from a UK national audit of daily 6 MV output measurements. 14  Farmer field chambers has also been included to indicate the stability of the Farmer ionization chambers.      There were a further 13 occasions where the measurement of output with an alternative device (either Daily QA3 or ionization chamber) was within tolerance, demonstrating that the MPC output result was a false negative. This occurs due to allowing a difference of up to 1.5% between MPC and ionization chamber measurements when the MPC output calibration is checked during monthly QC.

3.B | Analysis of MPC records
The remaining six false negatives are all out of tolerance BeamCen-treShift results for the G5 Linac. Here, the conventional QC indicated no issues with the beam alignment: The crosshair walkout and light field jaw settings were < 0.5 mm and < 1 mm, respectively, and were consistent with previous months, and the Radiation field size QC indicated no issues with beam position. Therefore, MPC was subsequently re-baselined.

3.D | Estimates of sensitivity and specificity
The estimates of the sensitivity, specificity, and NPV for the 6 MV Beam group (output, uniformity, and center shift) and for the 6 MV Geometry group are in Table V In general, we found the agreement between MPC and ionization chamber to be good, with 97.8% of all measurements within 1.5%. It was found that the least stable energy was 6 MeV, with a slightly lower proportion of MPC measurements within 1.5% (95.2%) compared with the other energies and a wider standard deviation of 0.7%. This is possibly due to the inherent instability of this beam energy, our local experience is that 6 MeV has the least stable unservo'd dose rate, requiring more regular tuning by our engineers.
Although we found the MPC-ionization chamber agreement for 6 MeV to be less consistent than the other energies, it was considered acceptable locally.
We report larger maximum differences between MPC and ionization chamber outputs than have been reported previously, 6,9 with maximum % differences of 2.7% for 6 MeV and 2.9% for 12 MeV and a maximum difference of 2% for the remaining energies, Table II.
This will be partly due to allowing a tolerance of up to 1.5% in the MPC to ionization chamber output comparison measurement compared with the tighter 1% tolerance. 6,9 The larger difference may also be attributed to investigating the output stability over a longer time frame and on a larger number of Linacs. Barnes  The maximum dose differences of up to 2.9%, and the proportion of measurements within 1%, 1.5 %, and 2% (Table II) demonstrate that MPC is suitable for a daily output constancy check, but not for the measurement of absolute output. We would advocate the use of an independent check device, such as the Daily QA3 on a weekly basis, in accordance with previous recommendations, 9 and a periodic ionization chamber check of MPC calibration. 6,9 The % difference in the monthly MPC and ion chamber outputs,  Table II. Since the panel sensitivity reduces over time, this difference will tend to be positive. From Fig. 3, we were able to estimate the reduction in MV panel sensitivity to be 0.5-1% per year, which is consistent with the drift of 0.5% over 5 months in Barnes and Greer. 6 In addition, there are data in the literature for the long-term stability of the Varian portal imagers with respect to output constancy where this has been studied independent of MPC. These generally indicate no or minimal long-term drift, contrary to our findings, and those of Barnes and Greer. 6 Sun et al 4 report no observed drift for the aS1000 portal imager over 6 months, although it is possible that this is in agreement with our findings, since we would estimate a change in sensitivity of 0.25-0.5% over this period, while they report the portal imager output constancy measurement to be within 0.5% compared against ion chamber. King et al 18 report that the change in aS500 portal imager response is less than 0.5% for 6 and 18 MV output constancy on three Linacs over 3 yr, and Greer and Barnes 19 report the variation in aS500 portal imager response to have a standard deviation of 0.4% for 6 and 18 MV output constancy over 7 months. These observed differences may be due to differences between the aS1200 and aS500 panels. We also found there to be a slight variability in the change in panel sensitivity between machines, likely due to varying MV panel usage for each Linac. Overall, the panel demonstrates sufficient long-term stability to be useful for daily monitoring of output.

4.E | "Hybrid QC"
There is a growing trend in the published literature of MPC replacing conventional QC. Barnes  to the variation we have experienced between MPC and ionization chamber output, in line with that reported elsewhere. 9 We are replacing many of the conventional geometric monthly QC tests with the daily 6 MV Beam and Geometry MPC test. The corresponding monthly conventional tests are now carried out on an annual basis, and in response to specific faults and repairs, acting as an ongoing independent check of MPC. We have adopted the term "Hybrid QC" for this approach, a hybrid of conventional and automated QC methods. This Hybrid QC method utilizes the benefits of automated QC of reduced operator error, increased accuracy, and time savings, but avoids complete reliance on automation. This approach could be adopted by other centers using Varian TrueBeam Linacs, freeing up treatment time and resources so the medical physicist can focus on other areas like development activities. We would advocate thorough commissioning of MPC prior to using it in place of conventional QC tests.

| CONCLUSIONS
The variation in output as measured by MPC versus ionization chamber measurement indicates that MPC is appropriate as a daily output constancy check, but cannot replace monthly ionization chamber output measurements. Our comparison of the MPC and conventional QATrack+ records has provided evidence that MPC is a robust and sensitive method of performing beam and mechanical checks in a clinical setting. There were a small number of false negative results reported by MPC, and we would advocate the use of independent methods, such as use of the Daily QA3 device, to quickly resolve these when they occur. We are in the process of re-evaluating the frequency of our monthly geometric conventional QC tests with a view to reducing the frequency due to our confidence in MPC.

CONFLI CT OF INTEREST
The authors have no relevant conflicts of interest to disclose.