Evaluation of Elekta Agility multi‐leaf collimator performance using statistical process control tools

Abstract Purpose To evaluate the performance and stability of Elekta Agility multi‐leaf collimator (MLC) leaf positioning using a daily, automated quality control (QC) test based on megavoltage (MV) images in combination with statistical process control tools, and identify special causes of variations in performance. Methods Leaf positions were collected daily for 13 Elekta linear accelerators over 11‐37 months using the automated QC test, which analyzes 23 MV images to determine the location of MLC leaves relative to radiation isocenter. Leaf positioning stability was assessed using individual and moving range control charts. Specification levels of ±0.5, ±1, and ±1.5 mm were tested to determine positional accuracy. The durations between out‐of‐control and out‐of‐specification events were determined. Peaks in out‐of‐control leaf positions were identified and correlated to servicing events recorded for the whole duration of data collection. Results Mean leaf position error was −0.01 mm (range −1.3–1.6). Data stayed within ±1 mm specification for 457 days on average (range 3–838) and within ±1.5 mm for the entire date range. Measurements stayed within ±0.5 mm for 1 day on average (range 0–17); however, our MLC leaves were not calibrated to this level of accuracy. Leaf position varied little over time, as confirmed by tight individual (mean ±0.19 mm, range 0.09–0.43) and moving range (mean 0.23 mm, range 0.10–0.53) control limits. Due to sporadic out‐of‐control events, the mean in‐control duration was 2.8 days (range 1–28.5). A number of factors were found to contribute to leaf position errors and out‐of‐control behavior, including servicing events, beam spot motion, and image artifacts. Conclusions The Elekta Agility MLC model was found to perform with high stability, as evidenced by the tight control limits. The in‐specification durations support the current recommendation of monthly MLC QC tests with a ±1 mm tolerance. Future work is on‐going to determine if performance can be optimized further using high‐frequency QC test results to drive recalibration frequency.


| INTRODUCTION
Accurate delivery of conformal and intensity modulated radiation treatments (IMRT) is highly dependent on multi-leaf collimator (MLC) leaf positioning accuracy. This is especially important for stereotactic body radiation therapy (SBRT), where high doses of radiation are delivered to targets with small setup margins, which are on the order of mechanical machine specifications. [1][2][3][4][5] In order to ensure accuracy and precision of MLC leaf positioning, routine MLC quality control (QC) testing is recommended to be performed weekly using visual inspection of matched segments and monthly quantitative testing using a procedure such as a picket fence test, with a tolerance of 1 mm. 1 However, it is possible that a higher frequency of quantitative testing, in combination with a high accuracy test, could enable MLC units to perform to a tighter tolerance. Currently, accurate MLC testing is time-consuming, which limits the feasibility of higher frequency testing in a clinical setting.
Streamlining quantitative MLC QC and the results analysis using automated tools enables performance assessment and is the first step toward MLC performance optimization.
A previous study introduced an automated QC test for MLC leaf positioning accuracy, 6 which was performed three to four times per week on two units to assess the performance of the Elekta MLCi and MLCi2 Elekta MLC models. The purpose of this work was to apply this test daily, along with statistical process control tools, to evaluate the long-term performance and stability of the Elekta Agility MLC model. Control charts were used to characterize normal behavior, in order to detect abnormal or special cause variations. A number of factors were investigated to determine the cause of variation in MLC leaf position, including leaf positioning mechanism, servicing events, and beam steering. The impact of image artifacts on MLC leaf position was also assessed, as the automated QC test in this work uses megavoltage (MV) imaging to measure leaf positions. This study will provide the groundwork needed to implement prospective, automated MLC QC and control-chart based analysis in order to optimize MLC performance.

| ME TH ODS
Leaf positions were collected daily for 13 Elekta units over  (average 22) months using the automated QC test, which analyzes 23 MV images to determine the location of MLC leaves relative to the radiation isocenter. 6 First, the location of the radiation isocenter is detected using 9 MV images of a 4 × 4 cm 2 field acquired at various collimator angles. Then the relative panel and collimator angular alignment is estimated from the position of leaf pairs extending into a 20 × 20 cm 2 radiation field. Finally, leaf positions are measured for five different pickets (4 × 24 cm 2 fields) located at five nominal leafbank positions. Picket 1 consists of Y1 leaves at nominal −60 mm and Y2 leaves at 100 mm. Images for each picket are acquired at collimator 0°and 180°to capture all MLC leaves in the MV imager field-of-view. A 6 MV beam is used for all images. In total, extending the imaging panel, running the beam, collecting the images, and performing the analysis takes about 7-8 minutes and requires no user intervention after starting the first beam. More details about the leaf measurement procedure can be found in the work of Létourneau et al. 6 A pair of individual and moving range control charts 7 was produced for each leaf to assess long-term leaf positioning reproducibility and stability of the system. Individual control limits correspond to approximately three times the standard deviation of measured leaf positions on either side of the mean leaf position. Moving range is computed as the absolute value of the difference in a measurement value from the measurement prior. The lower moving range control limit is 0, and the upper limit is 3.268 times the mean moving range, which corresponds to three times the population standard deviation.
This means there is a 0.27% chance of observing a measurement outside of these control limits due to normal variations in the process. While the individual control chart highlights the fluctuations in the measured statistic's mean (i.e., leaf position) over time, the moving range control chart detects changes in the process variance and emphasizes the rate of change of leaf positions as a function of time.$dummy$Together, these charts are used to demonstrate whether a process variation is in or out of control.$dummy$Control limits were computed using MATLAB, and were recomputed following MLC recalibration.$dummy$MLC recalibration was performed using the vendor's recalibration procedure, which takes about 60 minutes.
Leaf position error was computed as actual position minus nominal, where a negative error indicates the leaf extended further than the nominal position. Specification levels of ±0.5, ±1, and ±1.5 mm were tested to determine the MLC system's positional accuracy. The mean and range of duration between out-of-control and out-of-specification events (i.e., measurements falling outside of control and specification limits, respectively) were determined.
Beam spot motion could affect measured leaf positions by shifting the projection of the MLC on the imager by ray lines originating from the beam spot. Although the MLC leaves would not actually be moving in space, the radiation field edges defined by the MLC would be shifted. In order to assess whether beam spot motion was affecting leaf position errors, daily differences (current minus prior leaf position) were computed for each measured leaf position, and averaged over all leaves in each bank. A Pearson's correlation coefficient was computed between Y1 and Y2 mean daily differences for each picket of each unit. A negative correlation would indicate that both Y1 and Y2 leaves were appearing to shift in the same direction on a day-to-day basis, which would occur if the beam itself was moving.  behavior, whereas individual out-of-control events demonstrate a continuous out-of-control state. Large peaks in out-of-control leaf positions were identified as having amplitude greater than the mean plus 1 standard deviation in the number of out-of-control leaf positions. Servicing events were recorded for each unit and plotted against the identified peaks in out-of-control points to assess correlation. Servicing events that fell within ±1 day of peaks were noted, except for events that were related to linear accelerator sub-systems such as the treatment couch or the kV cone-beam CT system, which were ignored. A number of factors, including image ghosting artifacts and actual individual leaf motion were investigated in order to determine the cause of any abnormal behavior. An example correlation plot for picket 5 of one unit is shown in Fig. 2. The non-zero mean daily differences, in addition to the small standard deviations over all leaves in each bank (as indicated by error bars), demonstrate that these daily shifts are similar for all leaves in a bank. In addition, the magnitudes of some of these daily shifts were greater than the moving range control limits, and thus would register as out-of-control behavior. leaf 64 on images for a few consecutive days and confirmed that the leaf motion and the test results were consistent. Four other units also featured one or two especially noisy leaves. An example of individual and moving range control charts for a well-behaved leaf is shown in Fig. 3 and featured distinct shifts in leaf position following recalibration, as well as occasional out-of-control points, which were often related to servicing events (Table 1). For example, the shift at measurement 83 in Fig. 3 occurred following replacement of the monitoring ion chamber. Servicing events and machine faults that corresponded to peaks in the number of moving range out-of-control leaf positions are listed in Table 1. Figure 4 shows a plot of the number of moving range out-of-control leaf positions, peaks in out-of-control that were identified using the 1 SD threshold, as well as peaks that corresponded to servicing events for the same unit shown in  Day-to-day differences in measured multi-leaf collimator position for the Y2 bank vs the Y1 bank, for picket 5 of one unit, averaged over all 80 leaves in each bank. Error bars indicate the standard deviation over all 80 Y2 leaves for that measurement point. Y1 and Y2 moving range upper control limits, averaged over all leaves, are displayed for reference as yellow vertical and purple horizontal lines, respectively. The very strong, significant negative correlation (R = −0.98), along with tight error bars, indicate that Y1 and Y2 leaves appear to shift in the same direction on a day-to-day basis, and this trend is common to most leaves. Some of these apparent shifts were greater in magnitude than the moving range control limits, and thus were registered as out-of-control points. . This additional noise in leaf position for picket 1 was due to an image ghosting artifact from the extended leaf field collected prior to picket 1 leaf measurements (see Fig. 8).

| DISCUSSION
The length of the observation period, the daily test frequency and the number of linacs included in this study enable a thorough However, more unexpectedly, other peaks in out-of-control events were related to beam adjustments, replacement of key parts such as the electron gun, monitoring ion chamber, thyratron and magnetron, adjustments to other head components, and general maintenance. MLC, multi-leaf collimator; MV, megavoltage. Servicing events that occurred the day before, day of, or day after a large peak in out-of-control points are listed.
F I G . 4. Number of moving range out-ofcontrol points, summed over all 160 leaves and all five pickets, plotted over time.
Peaks in out-of-control (blue circle) were identified as having an amplitude greater than the mean +1 standard deviation in the number of out-of-control leaf positions.
Yellow circles indicate peaks that corresponded to servicing events displayed above the plot and indicated by green lines (i.e., servicing events that occurred either the measurement day before, same day or day after the peak).
also result from a difference in image pixel scaling factor between the daily MLC QC tests and the image-based procedure used by the manufacturer for MLC calibration. The strong, significant negative correlations between Y1 and Y2 mean daily differences in leaf position could be explained by beam spot motion, since Y1 and Y2 leaves appear to move in the same direction on average. In addition, the small standard deviations over leaves in a leaf bank demonstrate that these trends were common to most, if not all leaves. This is consistent with the beam spot motion theory, since any motion would cause the projection of all Y1 and Y2 leaves on the portal imager to appear to move in the same direction from 1 day to the next. For most pickets on most units, the magnitudes of some mean daily differences were even greater than the moving range control limits. In fact, when mean daily differences were plot- Since leaf positions are extracted from portal images, they can be impacted by image artifact. Picket 1 images featured a ghosting pattern from the prior extended leaf field (Fig. 8), which resulted in noisier leaf positions over time, larger control limits, and leaf position errors that demonstrated a periodical trend across leaves (Fig. 1).
We have been investigating methods to reduce the ghosting, including changing the order in which images are acquired, or allowing a time delay in between the extended leaf and picket 1 fields. When employing these strategies, the ghosting was reduced and picket 1 leaf patterns were more similar to other pickets. This test also revealed variation between MLC leaves within a single bank, and in particular a few leaves were found to be exceptionally noisy. After further investigation, it was determined that the noisy leaf measurements were in fact real, and these few leaves were moving more than others. While the positioning accuracy and stability for some noisy leaves improved after MLC recalibration, other leaves then became noisy. The MLC test was clearly able to detect unstable leaves, but the ability to improve positioning performance for all leaves seems limited to the replacement of the linac leaf control electronics.
Finally, we believe the test was less accurate at detecting the position of leaves 1 and 80. This was demonstrated by greater leaf position errors and larger control limits. In-specification durations were improved by omitting these leaves from analysis (see Fig. 7).
The combination of penumbra on two adjacent edges of the leaf could have reduced the accuracy of leaf edge detection at the leaf tip. It is possible that bringing the jaw in to cover the outer edge of these leaves could reduce the second edge penumbra and improve accuracy of detection.
The current recommended tolerance for monthly MLC leaf positioning accuracy is ±1 mm. 1 1 The accuracy of leaf position measurement at other gantry angles will be explored in future work.
Although initially utilized mainly in the manufacturing industry, the use of statistical process control tools for monitoring and characterizing process performance is slowly becoming more common in the field of radiation therapy. Studies have used these tools to evaluate patientspecific IMRT QA and monitor unit verification, [8][9][10][11] output and beam flatness/symmetry measurements, 12 electron spectra from linear accelerators, 13 and MLC QA. 6 Some groups even advocate to replace traditional specification-based QA with control-chart driven quality management. 14 While we have demonstrated a retrospective use of statistical process control tools to analyze this large amount of data and we have been able to relate some servicing events with MLC out-ofcontrol behavior, there are still a lot of observed out-of-control events that remained unexplained. Linac servicing events in our institutions are recorded manually in a service database. We believe that the use of control charts to detect change in MLC leaf position performance should be coupled to a more exhaustive method to record servicing events in order to facilitate the investigation of out-of-control behavior.
In addition, automated recording of machine operating parameters such as beam steering parameters and MLC optical chain parameters could help understanding changes in MLC performance observed during QC.
The combination of recording servicing and machine operation parameters, along with the daily MLC QC test analyzed with control charts, could lead to optimized MLC leaf position performance and inform the user on when MLC recalibration is required.
(a) (b) F I G . 8. (a) Example picket 1 image, demonstrating ghosting of the prior field's extended leaf pattern, which impacted leaf position results. Red crosses indicate leaf edge/position, as identified by the automated quality control software. Blue arrows indicate a few example locations where a positive leaf error (i.e., shift right) might be erroneously measured for the Y1 picket due to the presence of the ghosting pattern. The following picket 2 image (b), which does not feature this ghosting pattern, is shown for comparison.

| CONCLUSION
The Elekta Agility MLC model was found to perform with high stability, as evidenced by the tight control limits. The in-specification durations support the current recommendation of monthly MLC QC tests with a ±1 mm tolerance in order to detect potential drifts in performance and apply appropriate corrective action. Multiple factors were found to influence leaf positioning accuracy, including beam spot motion, leaf gain calibration, drifting leaves, and image artifacts. In particular, out-of-control leaf positions were often correlated to servicing events, indicating that certain types of servicing events may require subsequent MLC calibration. Future work is ongoing to determine if Agility performance can be optimized further using high-frequency QC test results to drive recalibration frequency.

ACKNOWLEDGMENTS
We thank the radiation therapists for performing the daily measurements, Kyle Foster for helpful discussions and contributing to analysis, Nurul Amin and Bern Norrlinger for their help with implementing the QC test, Meaghen Shiha for her help investigating MLC leaf position errors, and Matthew Bergshoeff for assisting with data export.

CONF LICT OF I NTEREST
No conflict of interest.