A systematic evaluation of the error detection abilities of a new diode transmission detector

Abstract Transmission detectors meant to measure every beam delivered on a linear accelerator are now becoming available for monitoring the quality of the dose distribution delivered to the patient daily. The purpose of this work is to present results from a systematic evaluation of the error detection capabilities of one such detector, the Delta4 Discover. Existing patient treatment plans were modified through in‐house‐developed software to mimic various delivery errors that have been observed in the past. Errors included shifts in multileaf collimator leaf positions, changing the beam energy from what was planned, and a simulation of what would happen if the secondary collimator jaws did not track with the leaves as they moved. The study was done for simple 3D plans, static gantry intensity modulated radiation therapy plans as well as dynamic arc and volumetric modulated arc therapy (VMAT) plans. Baseline plans were delivered with both the Discover device and the Delta4 Phantom+ to establish baseline gamma pass rates. Modified plans were then delivered using the Discover only and the predicted change in gamma pass rate, as well as the detected leaf positions were evaluated. Leaf deviations as small as 0.5 mm for a static three‐dimensional field were detected, with this detection limit growing to 1 mm with more complex delivery modalities such as VMAT. The gamma pass rates dropped noticeably once the intentional leaf error introduced was greater than the distance‐to‐agreement criterion. The unit also demonstrated the desired drop in gamma pass rates of at least 20% when jaw tracking was intentionally disabled and when an incorrect energy was used for the delivery. With its ability to find errors intentionally introduced into delivered plans, the Discover shows promise of being a valuable, independent error detection tool that should serve to detect delivery errors that can occur during radiotherapy treatment.


| INTRODUCTION
In recent years, the complexity of radiotherapy treatment has increased considerably. Not only are modulated delivery techniques producing extremely complex dose distributions that are highly conformal to the target, but there has also been a simultaneous increase in dependence on complex imaging modalities to ensure the required accuracy of alignment to the target. [1][2][3][4][5] Studies have shown that such increases in complexity can result in higher risks of errors occurring. [6][7][8] A relatively recent series of high profile newspaper articles highlighted some of the worst of these errors. 9 This has led our field towards a heightened awareness of and attention towards strategies of mitigation for such mistakes. Currently, significant effort is being focused on benefitting from lessons learned in other industries to better understand and eliminate errors at their source. 10,11 Efforts include application of principles of process engineering, including failure mode effects analysis 12,13 and fault tree analysis, 14,15 to identify the highest impact fault modes, along with efforts to organize incident learning systems to collect, study, and learn from errors. 16,17 Some approaches lead to detailed checklists, but research has shown that a risk of "checklist fatigue" occurs when such lists grow to be too long, thus, defeating their primary purpose. 18 Other groups have chosen to focus their efforts on the development of automated approaches, using software and/or hardware, to automatically identify potentially dangerous discrepancies. [19][20][21][22][23][24][25][26][27][28] Some software approaches evaluate plan quality to ensure that planners did not inadvertently fail to achieve important quality metrics. 29 Another approach utilizes radio-frequency identification technology and associated software to ensure that required patient setup devices are both present and in the correct location and orientation. 30 Another category of automated equipment/software strives to measure the daily delivered dose distribution either by evaluating the entrance [21][22][23][25][26][27]31,32 or exit fluence. [33][34][35] One of the disadvantages of using exit dosimetry is that the exit dose depends on both machine performance and patient geometry.
While devices that measure entrance fluence cannot predict changes in dose delivered to the patient stemming from changes in patient geometry, detailed knowledge of the fluence delivered by the linear accelerator can be used to calculate the dose to the patient using the patient's geometry. The Discover device 23 is a hardware device that is designed to measure the daily delivered entrance fluence and is shown in Fig. 1 mounted on a linear accelerator. The detector is made up of 4040 diode detectors separated by 2.5 mm along the multileaf collimator (MLC) motion direction. The diodes are separated by 5 mm perpendicular to the direction of motion of the MLC leaves. The area covered is 19.5 × 25 cm when projected to isocenter level. The detector is attached to the head of the linac, allowing the treatment beam to pass through with relatively little attenuation (~1%). The associated software compares the fluence pattern, as it is delivered to the patient, to the intended distribution and, as such, serves as a near real-time, automated error detection system. The clinical value of such a system is inherently tied to its sensitivity, as well as any limitations in its ability to detect errors. This work systematically evaluated the ability of the Discover system to detect multiple delivery errors that were intentionally introduced into treatment plans, and we report on the sensitivity with which the device detected such errors and any limitations in error detection ability.

| MATERIALS AND METHODS
As a fluence measurement device, the Discover (ScandiDos, Uppsala, Sweden) is meant to be used during every treatment to monitor the fidelity of the dose distribution delivered. There are two different ways that the Discover can be used: (a) by itself or (b) in conjunction with a phantom during a separate quality assurance (QA) session.
When used by itself, the use of the Discover is referred to as "Express Measure" and the device can provide information on the location of the MLC leaves, as well as the gantry and collimator. However, while the device is measuring fluence, there is presently not a way to convert these measurements to a dose distribution. This is because the signal level measured varies with the full geometric characteristics of the aperture used which, in effect, determines the scatter conditions.  These modified plans were then delivered through the Discover device to evaluate the fidelity of delivered plan quality so that the sensitivity of the device at detecting the errors could be established.

2.A | Single static field
As the simplest case to investigate, a single static field was created as a base plan. In the coronal plane the field represented a triangular shape as depicted in Fig. 2

2.A.1 | Single-field intensity-modulated radiation therapy (IMRT)dynamic delivery
To slightly increase plan complexity, a single IMRT field from a clinical plan was randomly chosen. The original clinical field was planned using the 6 MV beam and utilized dynamic MLC leaf motion (dMLC) with 130 control points using the jaw-tracking feature of Eclipse.
The originally planned field was then modified by shifting all MLC leaves by 0.25, 0.5, 1, 2, and 5 mm for each segment, analogous to what was depicted in Fig. 2(b). Other errors simulated from the baseline plan included changing the beam energy and disabling jaw tracking for the field.

2.A.2 | Single-field intensity-modulated radiation therapy (IMRT)step-and-shoot (SS)
The segments for the same field as in Section 2.1.1 was recalculated for delivery using step-and-shoot mode with jaw tracking. The resulting plan used 18 control points. This new baseline plan was also modified by shifting all leaves by 0.25, 0.5, 1, 2, and 5 mm and by disabling jaw tracking. Gamma pass rates (2%, 2 mm) were used to compare the measurements, with the pass rate from the baseline plan delivered to the phantom as the baseline value.
All modified SS-IMRT plans were delivered twiceonce using the Discover alone and once with the D4+ alone. This allowed for a direct comparison of the Synthesis-predicted change in gamma pass rate due to introduced errors to that from the actual dose distribution measured by the D4+ for the same plan.

2.B | Dynamic conformal arc (DCA)
As a next level of plan complexity, a dynamic conformal arc DCA was created. The arc conformed to a cylindrical-shaped target so that the MLC shape changed from a rectangle to a circle as the arc proceeded. The base plan used a half arc from 90°to 270°and used the 6 MV energy, along with jaw tracking. This plan was modified to shift the MLC leaves by 0.25, 0.5, 1, 2, and 5 mm, disable jaw tracking and introducing gantry position errors of 1°, 2°, 5°, and 10°. Two additional plans with combined errors were also created where the MLC leaves were shifted by 1 mm and the gantry by 1°and 2°.

2.C | Volumetric-modulated arc therapy (VMAT)
A baseline single-arc VMAT plan was created using an arc from 90°t o 270°with the 6MV energy and jaw tracking. This plan was modi-     F I G . 4. Measured leaf motion (left) and gamma pass rate (right) for plan modifications that did not include intentional leaf movement. Notice that the gamma pass rate drops significantly for instances of incorrect beam/energy. Interestingly, the 1 cm retracted × jaw apparently did not result in a significant enough change in measured fluence to trigger a noticeable drop in the gamma pass rate.

3.B | Single IMRT field
F I G . 5. Measured leaf motion (left) and gamma pass rate (right) for modified dynamic MLC leaf motion intensity modulated radiation therapy plans. Gamma pass rates fall dramatically once the introduced error exceeds the DTA criterion. While the detected leaf position correctly shows almost no shift when a wrong energy is used or jaw tracking is applied, the much lower predicted gamma pass rate shows the presence of an error. SARKAR ET AL.

| DISCUSSION
The results shown in Fig. 3 confirm that the Discover unit can detect submillimeter changes in leaf positions. Of note was the fact that even a single leaf being incorrectly positioned was correctly identified by the system. For static beams, with the maximum detected leaf position deviation being within 0.5 mm of the actual error introduced, the results confirm that this device can fulfill the TG-142 36 requirement of detecting leaf position repeatability to within 1 mm.
As for IMRT plans, the report requires 95% of error counts to be <3.5 mm. While the software does offer statistics on average and maximum leaf deviations, the current implementation of the software does not directly provide this information, the distribution of leaf deviations is provided so that this information can be deduced. Figure 4 shows the importance of using the Discover in synthesis mode rather than as express measure only. As previously explained, the Discover unit cannot convert the measured fluence to dose without initially being synchronized with measurements from the Delta4+phantom. Therefore, while the device correctly measured that the leaves were correctly positioned when a different energy was used, there would be no way for it to detect that the fluence measured does not correlate with the dose distribution expected. In that case, the field would pass QA and no flag would be raised.
However, when initially synchronized in Synthesis Mode, the Discover device can then correctly interpret that the fluence delivered was not correct for subsequent measurements using the device by itself.
For IMRT measurements, the average leaf deviation measured  While the D4+-measured gammas generally agree with the synthesispredicted gammas (middle panel), the one exception is for changes in collimator position which is not identified by the synthesis predictions.
The maximum detected deviations were much higher than expected when gantry angle deviations were introduced. For instance, these deviations went as high as 6 mm when a 10°gantry shift was included for DCA plans. The corresponding value was 25 mm for VMAT plans.
These large leaf position errors are attributable to leaf pairs that are only part of the field for a short time and then move out of the field, under the secondary jaws. Due to the synchronization challenges, these large motions are captured as leaf position errors.
For VMAT plans, the gamma pass rates for dose distributions measured with the D4+ generally followed the same trend as synthesis-predicted values, with the actual dose measurements having pass rates that were generally higher, just as was observed for SS plans. The one big difference was for plans with collimator changes.
In general, the predicted gamma pass rates stayed almost constant, even when collimator angles were changed by as much as 10°. This was clearly detected as a delivery error when the doses were measured with the D4+. This behavior is due to the fact that the Discover rotates with the collimator such that any collimator rotation results in no change to the fluence measured. The Discover, however, does have a gyroscope that will measure the collimator angle.
For the plan variations tested here, the collimator location was correctly measured within 1°and the measurement software allows the user to set a pass/fail criterion for the field that includes any observed collimator deviation.

| CONCLUSION
For the Discover system to detect all errors simulated here, including plans delivered using the wrong energy, or failure of the jaws to be positioned correctly, the Discover should be used in Synthesis Mode.
When used this way, average leaf motion differences detected by the Discover system were within 0.5 mm of the known introduced error for all plan types and MLC gamma value distributions showed high consistency between errors introduced in the plan and the criteria used for the gamma calculations. As such, the Discover device promises to be an effective, near real-time detection system for capturing potentially disastrous dose delivery errors. However, for those clinics that do not possess the D4+ as well as the Discover, the device can still be used in "Express Measure" mode and the device can still be used to verify the correct position of the MLC leaves and gantry and collimator for each beam.

CONF LICT OF I NTEREST
Drs. Sarkar, Szegedi, and Salter have been reimbursed for meeting registration and travel costs by Scandidos Inc.