Evaluation of a commercial Monte Carlo dose calculation algorithm for electron treatment planning

Abstract The RayStation treatment planning system implements a Monte Carlo (MC) algorithm for electron dose calculations. For a TrueBeam accelerator, beam modeling was performed for four electron energies (6, 9, 12, and 15 MeV), and the dose calculation accuracy was tested for a range of geometries. The suite of validation tests included those tests recommended by AAPM's Medical Physics Practice Guideline 5.a, but extended beyond these tests in order to validate the MC algorithm in more challenging geometries. For MPPG 5.a testing, calculation accuracy was evaluated for square cutouts of various sizes, two custom cutout shapes, oblique incidence, and heterogenous media (cork). In general, agreement between ion chamber measurements and RayStation dose calculations was excellent and well within suggested tolerance limits. However, this testing did reveal calculation errors for the output of small cutouts. Of the 312 output factors evaluated for square cutouts, 20 (6.4%) were outside of 3% and 5 (1.6%) were outside of 5%, with these larger errors generally being for the smallest cutout sizes within a given applicator. Adjustment of beam modeling parameters did not fix these calculation errors, nor does the planning software allow the user to input correction factors as a function of field size. Additional validation tests included several complex phantom geometries (triangular nose phantom, lung phantom, curved breast phantom, and cortical bone phantom), designed to test the ability of the algorithm to handle high density heterogeneities and irregular surface contours. In comparison to measurements with radiochromic film, RayStation showed good agreement, with an average of 89.3% pixels passing for gamma analysis (3%/3mm) across four phantom geometries. The MC algorithm was able to accurately handle the presence of irregular surface contours (curved cylindrical phantom and a triangular nose phantom), as well as heterogeneities (cork and cortical bone).

and bones, extended source to surface distance (SSDs), oblique incidence, and small irregular fields. 3,4 In comparison to photon treatment planning, electron Monte Carlo (MC) dose calculation algorithms are more computationally efficient, requiring fewer histories to achieve a given statistical uncertainty. The relative speed and improved accuracy of electron MC algorithms has allowed their introduction into commercial treatment planning systems (TPS) in recent years.
The RayStation TPS implements a MC dose calculation algorithm for electron treatment planning that uses the VMC++ Monte Carlo code for in-patient energy transport and scoring. 5 VMC++ is a voxel-based Monte Carlo code that has been incorporated into several commercial planning systems, including Oncentra MasterPlan® and CMS XiO®. The RayStation implementation of this MC algorithm models several beamline components, including the jaws, MLCs, electron applicator scraper layers, and the electron cutout, in the generation of phase space data which is then fed into the VMC++ code in order to perform dose calculations in the patient geometry.
Several guidance documents exist to aid medical physicists in developing tests to validate and commission an electron MC dose calculation algorithm for treatment planning purposes. The AAPM Medical Physics Practice Guideline 5.a (MPPG 5.a) recommends validation tests for electron beams, including comparison of calculated vs. measured dose distributions for standard cutouts, custom cutouts at standard and extended SSDS, oblique incidence, and inhomogeneous phantom geometries. 6 Though the suite of tests recommended by MPPG 5.a represents the minimum testing that should be performed to commission an electron dose calculation algorithm, the tests are typically performed using simple geometries (e.g., water tank and a cork slab phantom) that do not fully explore the accuracy of more advanced Monte Carlo algorithms. AAPM Task Group Report No. 105 addresses issues specifically associated with Monte Carlo-based treatment planning algorithms; this task group report states that beam model validation should include measurements in heterogeneous phantom geometries, similar to those reported in the Electron Collaborative Working Group report. 1,7,8 The more complex validation phantom geometries used by the Electron Collaborative Working Group include irregular surface contours (e.g., nose-shaped phantoms, stepped-surface phantoms) as well as internal 3D heterogeneities (e.g., bone and air cavities). Cygler et al. (2003) evaluated the VMC++ code using phantom geometries of varying degrees of complexity, including 1D (slab), 2D (rib), and 3D (small cylindrical) heterogeneities, as well as a complex phantom geometry designed to mimic the trachea and the spine. 9 Though studies have been done to quantify the accuracy of elec-

2.A | RayStation beam modeling
RayStation's electron dose calculation algorithm utilizes phase space data in order to perform Monte Carlo dose calculations. The electron beam within the linear accelerator is modeled using a "source phase space" at the level of the secondary scattering foils. The source phase space electron particles are then propagated through the beamline components in the linear accelerator (jaws, MLCs, electron applicator, and electron cutout) in order to generate an "exit phase space" which is then used for Monte Carlo transport through the patient geometry; the exit phase space is defined at the level of the patient-specific cutout in the applicator. Therefore, electron beam modeling in RayStation requires the user to enter machine-specific information in order to model the accelerator beamline for exit phase space generation, including the thickness and position of the MLCs and jaws, information about the electron applicator scraper layers, energy and applicator-specific jaw settings, primary and secondary scattering foil locations, and cutout thickness. Modeling of the source phase space requires the measurement of in-air data (without an applicator in place), including in-air relative output factors and in-air profiles for various field sizes at 70 and 90 cm SSD.
Additional required measurement data for beam modeling includes in-water depth dose curves measured with and without electron applicators in place, in-water profiles at two depths for each energy and applicator combination, and absolute calibration data (dose per MU) for each energy and electron applicator combination. In-water measurements were used to optimize electron spectrum parameters in the beam model, as well as to fine-tune source phase space parameters.
All beam modeling data were acquired for a TrueBeam linear accelerator for 6, 9, 12, and 15 MeV electron beams using a 3D scanning water tank (Blue Phantom 2 , IBA Dosimetry, Schwarzenbruck, Germany). Profiles, depth dose curves, and in-air output factors were measured using an electron field diode detector (IBA Dosimetry), while the absolute dose measurements were performed using a 0.04cc ion chamber (IBA Dosimetry). ing, all profile and depth dose data were acquired using a 3D scanning water tank and a 0.04 cc ion chamber, and all RayStation dose calculations were performed using a 2 mm dose grid and 500,000 histories per cm 2 (resulting in a relative uncertainty of <1.0%). Point dose measurements were evaluated using dose difference criteria between calculated and measured dose, while profile data were compared using 1D gamma analysis (3% global dose difference and 3mm distance-to-agreement criteria).
Medical Physics Practice Guideline 5.a recommends that output factors for all electron applicators with standard square cutout sizes for each energy be calculated in order to confirm the correct behavior of output as a function of field size and energy. To perform this comparison, output factors were measured using a 0.04 cc ion chamber and compared to RayStation-calculated output factors for a range of square cutout sizes in each electron applicator for 100, 105, and 110 cm SSD.
In addition to square cutouts, MPPG 5.a also recommends testing two custom cutout shapes at standard and extended SSDs in order to verify the accuracy of the calculated isodose distribution as well as the system's ability to handle changes in SSD (MPPG 5.a test 8.1). For this test, two clinically relevant cutout shapes were chosen -a small circular cutout approximately 3 cm in diameter and a larger cutout approximately 6 cm × 20 cm in dimension. Point dose measurements were performed near d max for each electron energy. Additionally, depth dose curves and inline and crossline profile data were measured at two depths per electron energy (d max and R 50 ) for both 100 and 105 cm SSD. The measurement geometry was reproduced to calculate dose in RayStation for these two custom cutouts.
To evaluate the accuracy of the dose calculation in the presence of beam obliquity (MPPG5.a test 8.2), measured profile data were compared to calculated dose distributions for beams of oblique incidence (20°gantry rotation, standard 10 cm × 10 cm cutout, 105 cm SSD).
Depth dose curves, as well as inline and crossline profiles at d max and R 50 were measured and compared to RayStation calculations.
Lastly, the accuracy in the presence of heterogeneous materials was evaluated using a cork slab phantom geometry with various thicknesses of cork for different electron energies (MPPG 5.a test 8.3). Measurements were acquired downstream of the cork heterogeneity using a parallel plate ion chamber for a 10 cm × 10 cm reference cutout and 100 cm SSD. Point dose measurements were evaluated at d max , and the distance-to-agreement between measured and calculated R 50 was also evaluated.

2.C | Complex phantom validation
Four complex phantom geometries were used to test the electron Monte Carlo algorithm in RayStation beyond the testing current recommended by MPPG 5.a. These four complex phantom geometries were designed to mimic clinical cases and to be a more challenging test of the algorithm's ability to handle heterogeneities and irregular surface contours. All four phantoms were made in-house.
• Nose phantom: This phantom is composed entirely of solid water with a triangular piece to mimic the geometry of a nose • Breast phantom: This is a cylindrical solid water phantom with a diameter of 15.6 cm. The phantom has halves that are used to sandwich radiochromic film in an edge-on orientation [ Fig. 1(c)].
There is no heterogeneous material in this phantom aside from the air pockets, but it was chosen to somewhat mimic the curvature of a breast for an enface electron beam treatment.
•  Figure 2 shows the difference between measured output factors and RayStation-calculated output factors for 100, 105, and 110 cm SSD. Of the 312 output factors evaluated, 20 (6.4%) were outside of 3% and 5 (1.6%) were outside of 5%, with these larger errors generally being for the smallest cutout sizes within a given applicator. Table 2 shows the results of gamma analysis comparisons for all electron energies for the four complex validation phantoms. For the energy with the worst agreement for a given phantom Fig. 3 shows the overlay between the film-measured IDLs and the RayStation-calculated IDLs. Based on these results, the RayStation electron Monte Carlo algorithm is able to accurately calculate the hot spots due to the triangular nose geometry [ Fig. 3(a)], changes in range due to bone [ Fig. 3(b)], the effect of a curved patient surface [ Fig. 3(d)], and penumbra broadening in lung [ Fig. 3(c)]. The average agreement for gamma analysis across all phantoms and electron energies was 89.3% pixels passing for 3%/3 mm criteria.

3.C.2 | Prescription methods
Our procedures for electron treatment planning in the Pinnacle plan-

3.C.3 | Density overrides
The RayStation planning system does not take into account the density of structures outside of the external contour in the dose calculation unless the structure is designated as a special ROI type (Fixation, Support, or Bolus). The case shown in Fig. 4 contains a wire over the lumpectomy scar that was present at the time of the simulation scan but will not be present during treatment. Therefore the wire was contoured and overwritten to a density of air for both the Pinnacle and RayStation plans. For cases without bolus, our clinical practice allows the wire structure to either overlap with the external contour and be overwritten to air density (as was done for this particular treatment plan), or to be edited out of the external contour, both options that result in the density of the wire being set to zero. However, for cases with bolus, our clinical practice is to edit the wire out of the external contour, which removes the need to perform a density override of the wire and also makes the bolus created in the planning system conform to the patient contour better (i.e., less puckering). It should be noted that though the wire was accurately contoured and its ROI was overwritten to air density in T A B L E 1 Summary of MPPG 5.a validation results comparing measurements against RayStation dose calculations for all electron energies (6,9,12,and 15 MeV The calculated crossline profiles for the small custom cutout were wider than measured for one of our TrueBeam machines. However, when this validation test was repeated on a matched TrueBeam linear accelerator, the profile agreement was excellent with passing rates > 95%. Measurement error or a small mismatch between the size of the cutout in the planning system vs. the physical cutout is suspected.

3.C.4 | Secondary MU calculations
Because of the transition to volume-based prescriptions, our procedures for performing secondary MU calculations, which were previously based on an IDL, also needed to be changed. Several comparisons between RayStation MU and Mobius3D MU were performed for clinical cases. To perform this comparison, a calculation point was placed at the depth of dose maximum in the RayStation plan in order to obtain a %IDL for the secondary MU calculation. Table 4 shows that several of these cases showed greater than 5% differences in MU calculated with Mobius3D vs.
RayStation. These differences arise from the different calculation F I G . 2. Percent error between RayStation-calculated output factors and measured output factors for various square cutout sizes, applicator sizes, electron energies, and source to surface distances. Errors > 3% but < 5% are highlighted in yellow, and those exceeding 5% are highlighted in red.
geometries in RayStation, which is able to handle irregular surface contours, heterogeneities, and oblique incidence, and Mobius3D, which performs a calculation in a homogeneous water phantom with normal beam incidence. In order to isolate differences due to electron output for a patient-specific cutout, a quality assurance plan was created for each patient plan in RayStation. For this quality assurance (QA) plan, the patient-specific cutout, beam energy, collimator angle, and SSD were used for a dose calculation using a water phantom and a normally incident electron beam. The dose calculated from this QA plan was then used to perform the secondary MU comparison. The use of this water phantom QA plan greatly improved the MU agreement, as expected and summarized in water phantom dose calculation using the patient-specific cutout was incorporated into our secondary MU calculation procedures in order highlight differences in output rather than differences in the sophistication of the primary calculation vs. the secondary calculation.