Determination of machine‐specific tolerances using statistical process control analysis of long‐term uniform scanning proton machine QA results

Abstract Purpose The purpose of this study was twofold: (a) report the long‐term monthly quality assurance (QA) dosimetry results of the uniform scanning beam delivery system, and (b) derive the machine‐specific tolerances based on the statistic process control (SPC) methodology and compare them against the AAPM TG224 recommended tolerances. Methods The Oklahoma Proton Center has four treatment rooms (TR1, TR2, TR3, and TR4) with a cyclotron and a universal nozzle. Monthly QA dosimetry results of four treatment rooms over a period of 6 yr (Feb 2014–Jan 2020) were retrieved from the QA database. The dosimetry parameters included dose output, range, flatness, and symmetry. The monthly QA results were analyzed using the SPC method, which included individuals and moving range (I‐MR) chart. The upper control limit (UCL) and lower control limit (LCL) were set at 3σ above and below the mean value, respectively. Results The mean difference in dose output was −0.3% (2σ = ±0.9% and 3σ = ±1.3%) in TR1, 0% (2σ = ±1.4% and 3σ = ±2.1%) in TR2, −0.2% (2σ = ±1.0% and 3σ = ±1.6%) in TR3, and −0.5% (2σ = ±0.9% and 3σ = ±1.3%) in TR4. The mean flatness and symmetry differences of all beams among the four treatment rooms were within ±1.0%. The 3σ for the flatness difference ranged from ±0.5% to ±1.2%. The 3σ for the symmetry difference ranged from ±0.4% to ±1.4%. The SPC analysis showed that the 3σ for range 10 cm (R10), R16, and R22 were within ±1 mm, whereas the 3σ for R28 exceeded ±1 mm in two rooms (3σ = ±1.9 mm in TR2 and 3σ = ±1.3 mm in TR3). Conclusion The 3σ of the dose output, flatness, and symmetry differences in all four rooms were comparable to the TG224 tolerance (±2%). For the uniform scanning system, if the measured range is compared against the requested range, it may not always be possible to achieve the range difference within ±1 mm (TG224) for all the ranges.


| INTRODUCTION
Recently, task group 224 (TG224) of the American Association of Physicists in Medicine (AAPM) released the report on quality assurance (QA) of the proton machine. 1 At present, proton centers across the world primarily use three delivery techniques: double scattering (DS), uniform scanning (US), and pencil beam scanning (PBS). The TG224 report recommends QA tests to be performed daily, weekly, monthly, and annual. Additionally, the TG224 states that "recommended tolerance limits for each of the recommended QA checks are tabulated, and are based on the literature and consensus data from the clinical proton experience of the task group members." 1 The TG224 tolerances can be used as guidelines for the machine QA, but due to variability in technologies and beam delivery systems among different proton therapy vendors, it is critical to determine the machine-specific tolerances limits for the TG224 recommended parameters. 1 Statistical process control (SPC) is one of the methods that can be used to determine the machine-specific tolerance limit. The application of SPC to control charts allows users to assess the temporal stability of each test parameter and determine whether the various parameters of the system are in statistical control. [2][3][4][5][6] Literature [2][3][4] has reported using control limits at AE3σ for detecting meaningful changes in system performance. The SPC helps in monitoring the process using the control charts, which are used to distinguish between the common and special cause variations. [2][3][4] Several studies [2][3][4][5][6][7][8][9][10][11] highlighted the importance of using SPC analysis in the radiotherapy department. For instance, Binny et al. 2,7 used the SPC to evaluate the beam output and symmetry of the linear accelerators. Shiraishi et al. 11 and Stanley et al. 5,6 assessed the stability of image quality parameters using SPC. The authors observed the application of SPC in conventional photon therapy, but the literature on SPC analysis in proton therapy is limited. Rah et al. 3 demonstrated the feasibility of SPC for patient-specific QA in DS proton therapy. Rana et al. 4 applied SPC to their daily QA results in PBS proton therapy. Both proton studies 3,4 were published prior to the publication of TG224. To date, there is no literature reporting the long-term machine performance of the US proton delivery system. Also, SPC analysis of TG224 recommended monthly QA dosimetry of the US delivery mode is not available in the literature.
In the current study, the authors sought to (a) report the longterm monthly QA dosimetry results of the US beam delivery system, and (b) derive the machine-specific tolerances based on the SPC methodology and compare them against the AAPM TG224 recommended tolerances.

2.A | Beam delivery system
The Oklahoma Proton Center has four treatment rooms (TR1, TR2, TR3, and TR4) with a cyclotron and a universal nozzle (IBA, Louvainla-Neuve, Belgium). The TR1 has a fixed horizontal beamline, TR2 and TR3 have two beamlines (30°and 90°), and TR4 is a full gantry. A detailed description of the IBA universal nozzle has been provided in the published studies. 12,13 In brief, the high-energy proton beam is widened by the first scatter in the nozzle. The proton beam is downgraded to lower energy as it passes through the range modulator wheel. The beam is then scanned by horizontal and vertical scanning magnets in the nozzle such that a uniform dose is delivered for a rectangular scanning area. After passing through the ionization chambers, the proton beam exits the nozzle and passes through the snouts. The snouts for our delivery system are extendable. Apertures and range compensators are attached to the snout for clinical treatment.

2.B | Monthly QA dosimetry tests and detectors
TG224 recommends four dosimetry tests for the monthly QA of US proton delivery. 1 Tolerance for dose output, field flatness, and field symmetry is set at AE2% relative to the baseline, whereas tolerance for the distal range is AE1 mm. 1 The monthly QA program at our center includes all four TG224 recommended parameters. Dose output  The monthly QA results were then analyzed using the SPC method, which included individuals (I) and moving range (MR) chart. 3,4,7 The control I chart has a central line represented by the mean value (X), whereas the upper control limit (UCL) and lower control limit (LCL) are set at 3σ above and below the mean value, respectively. 3,4,7 If the measured data are within the UCL and LCL, the process is considered to be within control. If the measured data are outside the UCL and LCL, the process is said to be out of control. The UCL and LCL for the I chart are calculated using the following formula: where, k = number of data points and MR = moving range.

3.B | Range
The ranges were evaluated for 10, 16, 22, and 28 cm. In the given treatment room, the mean difference for R10 was higher compared to the mean differences for other ranges. The evaluation of R10 among four rooms showed that TR4 produced the smallest mean difference of 0.6 mm (2σ = AE0.3 mm and 3σ = AE0.4 mm) and TR2 produced the largest mean difference of 1.0 mm (2σ = AE0.2 mm and 3σ = AE0.3 mm). Among four treatment rooms, the mean difference ranged from −0.2 to 0.4 mm for R16, from 0.1 to 0.5 mm for R22, and from −0.3 to 0.2 mm for R28. In general, the 3σ value increased with an increase in range (i.e., from R10 to R28).

3.C | Flatness and symmetry
The mean flatness and symmetry differences of all beams among the four treatment rooms were within AE1.0%. The 3σ for the flatness difference ranged from AE0.5% to AE1.2%. The 3σ for the symmetry difference ranged from AE0.4% to AE1.4%.

| DISCUSSION
The complexity of the proton beam delivery system demands rigorous QA to monitor the system performance. As there is an increasing interest in using proton therapy for cancer treatment, new vendors are joining the proton therapy market over the last decade. The current dose output tolerance at our institution is AE2%, which is also a recommended value by the TG224. It was observed that the variation in dose output was worst in TR2, whereas the 2σ and 3σ results among TR1, TR3, and TR4 were in better agreement with each other. The 3σ of dose output in TR2 was slightly higher (AE2.1%) than the TG224 recommendation (AE2.0%). Overall, the 3σ results for the dose output in all four rooms are comparable to the TG224 tolerance. The authors observed less variation in range in TR1 and TR4 than in TR2 and TR3. In TR1, the UCL and LCL ranged from 0.6 to 1.2 mm and from −0.9 to 0.6 mm, respectively. In TR4, the UCL and LCL ranged from 0.8 to 1.2 mm and from −0.7 to 0.2 mm, respectively. When comparing range deviations for the other two rooms, TR3 performed slightly better than in TR2. In TR3, the UCL and LCL ranged from 0.4 to 1.3 mm and from −1.6 to 0.6 mm, respectively. In TR2, the UCL and LCL ranged from 1.0 to 2.1 mm and from −1.7 to 0.7 mm, respectively. These results clearly suggest that it may not always be possible to achieve the range difference (i.e., without comparing against the baseline values) within AE1 mm for all ranges in our proton system; however, if the range difference is outside AE1.5 mm, the vendor is requested to fix the failing range as a part of the service agreement.
Based on the monthly QA results in the current study, the authors observed that the universal tolerance for a given metric may not always be applicable for all beams in different treatment rooms.  New proton centers are employing a PBS delivery system only.
Although our study is based on the US beam delivery, the statistics from the long-term results of our QA program can be valuable experimental information to proton centers that are employing US technique to treat proton therapy patients. The upgrade from the DS/US to PBS involves a massive financial cost and logistical challenges. The patients will continue to receive treatment in the existing proton centers that employ the US beam delivery technique.
Additionally, as these existing US proton centers continue to age, the authors believe that the rigorous QA is essential to ensure optimal performance of the proton system. The authors believe that the statistical results presented in the current study will encourage other proton centers to report the QA tolerances of their proton machines from different vendors. As more proton centers start evaluating the performance of their system using SPC, the proton community will have an opportunity to further refine the proton machine QA tolerances.

| CONCLUSION
The SPC analysis of dosimetry measurements over a period of 6 yr was performed to assess whether various dosimetry parameters of our USPT system were in statistical control. The authors demonstrated that the 2σ and 3σ values could be used as the warning and action level tolerances, respectively, for the monthly proton machine QA. The 3σ of the dose output, flatness, and symmetry differences in all four rooms were comparable to the TG224 tolerance (AE2%).
For the range, the TG224 recommends AE1 mm tolerance (not relative to the baseline values). For the USPT system, if the measured F I G . 6. Histograms showing the difference in dose output, range, flatness, and symmetry of all monthly quality assurance beams in four treatment rooms.