Evaluation of technical performance of optical surface imaging system using conventional and novel stereotactic radiosurgery algorithms

Abstract The Catalyst HD (C‐RAD Positioning AB, Uppsala, Sweden) optical surface imaging (OSI) system is able to manage interfractional patient positioning, intrafractional motion monitoring, and non‐contact respiratory gating without x‐ray exposure for radiation therapy. In recent years, a novel high‐precision surface registration algorithm for stereotactic radiosurgery (SRS algorithm) has been released. This study aimed to evaluate the technical performance of the OSI system using rigid phantoms, by comparing the conventional and SRS algorithms. To determine the system’s technical performance, isocenter displacements were calculated by surface image registration via the OSI system using head, thorax, and pelvis rigid phantoms. The reproducibility of positioning was evaluated by the mean value calculated by repeating the registration 10 times, without moving each phantom. The accuracy of positioning was evaluated by the mean value of the residual error, where the 10 offset values given to each phantom were subtracted from the isocenter displacement values. The stability of motion monitoring was evaluated by measuring isocenter drift during 20 min and averaging it over 10 measurements. For the head phantom, all tests were compared with the mask types and algorithms. As a result, for all sites and both algorithms, the reproducibility, accuracy, and stability for translation and rotation were <0.1 mm and <0.1°, <1.0 mm and <1.0°, and <0.1 mm and <0.1°, respectively. In particular, the SRS algorithm had a small absolute error and standard deviation of calculated isocenter displacement, and a significantly higher reproducibility and accuracy than the conventional algorithm (P < 0.01). There was no difference in the stability between the algorithms (P = 0.0280). The SRS algorithm was found to be suitable for the treatment of rigid body sites with less deformation and small area, such as the head and face.


2.A | OSI system
Catalyst HD was used as the OSI system in this study. The OSI system consists of three scanner units arranged at angles of approximately 120°to each other, to enable continuous surface detection of a patient, even if the gantry hides the patient from one of the three scanners. Each scanner unit emits visible blue light with a wavelength of 405 nm to the patient, and the projected light illuminates the patient. The reflected light is captured by the charge-coupled device cameras. The frame rate of surface capture is 200 frames per second, and the maximal scan volume has a width, length, and height of 800, 1300, and 700 mm, respectively. The scan volume is manually adjustable through a software interface. When the scan volume is changed, the reference surface can be cropped to a smaller volume than the scan volume. In the conventional algorithm, the captured surface is registered with the cropped reference surface by using a non-rigid iterative closest point (ICP) algorithm. 6,7 The registration identifies position errors between the captured surface and the reference surface in six degrees of freedom (DOF): three representing translations and three representing rotations.
The SRS algorithm for head SRS and SRT calculates isocenter displacements through semi-non-rigid ICP registration. In addition, the SRS algorithm captures a scanned image of 640 × 480 pixels from one camera unit of the OSI system as the full field of view, whereas the conventional algorithm captures an image with a resolution of 320 × 240 pixels. The surface image of the SRS algorithm consists of up to 921 600 points and that of the conventional algorithm consists of up to 230 400 points. Therefore, the spatial resolution of the SRS algorithm is approximately four times higher than that of the conventional algorithm.

2.B | Technical performance tests of OSI system
As shown in Fig. 1, the following anthropomorphic rigid phantoms were used to test the reproducibility, accuracy, and stability of the Matsumoto, Nagano, Japan) were prepared.
All the technical performance tests were performed using Elekta Infinity (Elekta AB, Stockholm, Sweden), equipped with a six DOF HexaPOD system and an XVI system for cone-beam computed tomography (CBCT). The gantry angle of the Linac was 0°, and the collimators were completely closed. The ambient room lighting conditions matched typical treatment conditions, although no effect on system accuracy is expected. The orientation was set according to the Elekta Synergy coordinate system and the arrows indicate the positive directions of the coordinate system for translations and rotations (Fig. 2). The phantoms were scanned using a CT system, Aquilion LB (Canon Medical Systems Co., Otawara, Tochigi, Japan), and the CT images of the phantoms were transferred to a treatment planning system, Monaco (Elekta AB, Stockholm, Sweden). A plan was built for each phantom on the Monaco system, and the plan isocenter was located at the center of each phantom. Both the plan and CT images were transferred to the XVI system to rectify the initial position of each phantom.
To determine the reproducibility and accuracy of positioning and stability of motion monitoring of the OSI system, the isocenter displacement was measured. The isocenter displacement was a result of the registration between the reference surface and live surface of the phantoms. The reproducibility, accuracy, and stability values were compared with the phantom types. In addition, they were KOJIMA ET AL.
| 59 compared with the algorithms and the mask types, including no mask for the head phantom.

2.B.1 | Reproducibility of positioning
Reproducibility tests were performed in the positioning mode (cPosition application) of the Catalyst HD system. Each of the head, thorax, and pelvis phantoms were positioned at the isocenter by using CBCT and HexaPOD. Then, each phantom was scanned with the three cameras of Catalyst HD system. The initially scanned surface image was used as a reference surface for reproducibility, and sequentially scanned surface images were registered with the reference image as live images. The scan volume for the live image covered the volume of the entire phantom, and all surface data of the reference surface were also for registration (Fig. 3). The phantoms were not moved during the test, and they were scanned 10 times.
The isocenter displacement was calculated immediately after each scan. The means and standard deviations (SDs) of the displacement in six DOFs were calculated. These values were evaluated to determine the reproducibility of the OSI system for interfractional positioning.
The head phantom was immobilized with each of the two masks, and the effect of the masks on the SRS algorithm was compared with that on the conventional algorithm. The settings of the OSI system for the tests are shown in Table 1. The gain and integration time (IT) were set at 300% and 4000 µs for the conventional algorithm and 100% and 18000 µs for the SRS algorithm, respectively.

2.B.2 | Accuracy of positioning
Accuracy tests were performed in the positioning mode (cPosition application) of the Catalyst HD system. Each of the head, thorax, and pelvis phantoms were positioned at the isocenter, and the position was rectified using CBCT and HexaPOD. Position correction was continued until the residual errors of translation and rotation were close to 0.0 mm and 0.0°, respectively. 8,9 As in the previous test, each phantom was scanned using the three cameras of the Catalyst HD system and used as a reference surface for the accuracy test. The setup of the scan volume and reference surface data were similar to that used in the reproducibility test (Fig. 3). An offset of 15.0 mm or less for translation and 2.0°or less for rotation were applied to the phantom position by using CBCT and HexaPOD ( Table 2). The phantoms were scanned, and the captured phantom surface was registered using the reference surface to calculate the isocenter displacement. These procedures were repeated 10 times, while applying the defined offset to the phantoms for each test. The F I G . 1. The phantoms and masks for the technical performance test.
offset was applied to the head phantom inside the masks. The residual error of correction by Catalyst HD system was calculated by subtracting the offset value from the isocenter displacement (Fig. 4).
The means and SDs of the residual error were evaluated to determine the accuracy of the OSI system for interfractional positioning.
The effect of the masks and the difference in accuracy between the two algorithms were investigated for the head phantom. The settings of the OSI system used in this test are shown in Table 1.
The gain and IT were identical to those in the reproducibility test.

2.B.3 | Stability of motion monitoring
Stability tests were performed in the monitoring mode (cMotion application) of the Catalyst HD system. Each phantom was positioned in the same way as in the reproducibility test. Then, the application of the Catalyst HD system was switched from the positioning to the monitoring mode. The phantoms were re-scanned using Catalyst HD system and used as a reference surface for stability. The setups of the scan volume and reference surface data were also identical to those in the other tests (Fig. 3). Each phantom was continuously monitored for 20 min to determine the deviation of isocenter displacement. The monitoring was repeated 10 times, and the means and SDs of the deviation of isocenter displacement in the six DOFs were calculated. These values, which represent the drift of the isocenter location, were evaluated to determine the stability of the OSI system for intrafractional motion monitoring. The head phantom, masks, and settings, including gain and IT, were the same as those in the other tests.

2.C | Statistical analysis
Multivariate analysis of variance was used for the statistical analysis.
The objective variable was the absolute value of isocenter displacement or residual error and the independent variables were the types of the phantoms, algorithms, and mask types, including no mask. The causal relationship between these variable was tested using Wilks' lambda test with JMP version 13.2.0 (SAS Institute Inc., Cary, NC, USA). The P < 0.01 was considered to be statistically significant. F I G . 2. The geometry and coordinate system for the technical performance test. The coordinate system for the technical performance tests was the same as the Elekta Synergy coordinate system: the arrows indicate the positive directions for translation and rotation.

3.A | Reproducibility of positioning
The results of the reproducibility test for interfractional positioning are shown in Table 3. All the mean values for translation were <0.1 mm, and those for rotation were <0.1°. There was no significant difference in the reproducibility between the phantom types for the conventional algorithm (P = 0.2845). However, the reproducibility of the head phantom for the SRS algorithm was significantly higher than that of the other phantoms for the conventional algorithm (P < 0.01).
Regarding the head phantom, the results of reproducibility showed no significant difference among the tests with no mask, F I G . 3. Scan volume and reference surface data for the technical performance test. The yellow boxes depict the scan volume and the reference surfaces are represented in pink.  The mean absolute error of reproducibility for the conventional and SRS algorithms is shown in Fig. 5(a). The SRS algorithm had a small SD and mean absolute error. Thus, the SRS algorithm has a higher reproducibility than the conventional algorithm. The results for the head phantom showed no significant difference for tests conducted with no mask, the white mask, and the black mask for the conventional algorithm (P = 0.0381); however, a significant difference was observed for the SRS algorithm (P < 0.01).
In particular, for the SRS algorithm, the variation in the isocenter displacement for the white mask was greater than those for without a mask and the black mask for all values of translation and rotation.
There was also a significant difference between the two algorithms (P < 0.01). The mean absolute error of accuracy for the conventional and SRS algorithms is shown in Fig. 5(b). The SRS algorithm had a small mean absolute error. Thus the SRS algorithm had a higher accuracy than the conventional algorithm.

3.C | Stability of motion monitoring
The results of the stability of intrafractional monitoring are shown in Table 5. All the mean values for translation were <0.1 mm, and those for rotation were <0.1°. There was a significant difference in the stability between the phantom types (P < 0.01).
The stability for the head phantom showed no significant difference among the tests conducted with no mask, the white mask, and the black mask for the conventional (P = 0.6940) and SRS (P = 0.1639) algorithms. The mean absolute error of stability for the conventional and SRS algorithms is shown in Fig. 5(c). The stability was found not to be significantly affected by the algorithms (P = 0.0280).
F I G . 5. The results of reproducibility, accuracy, and stability for the head phantom using the conventional and SRS algorithms. These results show the mean absolute error of isocenter displacement or residual error.
The reproducibility, accuracy, and stability of the Catalyst HD system were investigated using rigid phantoms. The reproducibility for translation was <0.1 mm and that for rotation was <0.1°, for all sites and both algorithms. A similar study, using the Sentinel system, 2 reported that the reproducibility for translation was <0.5 mm and that for the rotation was <0.1°. The Sentinel system acquires surface images by using multiple line scans with a laser, while the Catalyst HD system has three scanner units and acquires surface images with a wide coverage by using one entire projection with a light emitting diode.
In this study, the differences in surface scanning methods and surface image coverage led to higher reproducibility for the Catalyst HD system than for the Sentinel system. In addition, the nominal reproducibility of Catalyst HD system, which C-RAD published on the company website, was 0.2 mm, 10 which is comparable to our results.
The accuracy for translation was <1.0 mm and that for rotation was <1.0°, for all sites and both algorithms. Several previous studies using phantoms 2,3,11 reported that the accuracy was <1.0 mm and the nominal accuracy of Catalyst HD system, which C-RAD published on the company website, was ≤0.5 mm for a rigid body, 10 which is comparable to our results. The accuracy for the pelvis phantom was significantly worse in the longitudinal direction. A previous clinical study 5 reported that the accuracy for the pelvis was 0.1, 1.8, and 1.4 mm in the lateral, longitudinal, and vertical directions, respectively, which is comparable to our results.
We believe that the additional inaccuracy along the longitudinal direction was primarily because the surface of the pelvis phantom was more cylindrically symmetric than that of the other phantoms.
In other words, the isometry of the edges of the source surface images in the non-rigid ICP registration was not maintained correctly along the longitudinal direction. 7 Therefore, the penalties of registration calculation owing to the translations along the longitudinal axis were not as accurate as those along the other axes. As an alternative approach, we believe that the size of the surface area perpendicular to the longitudinal axis may affect the accuracy of registration calculation. Therefore, three-dimensional points, comprising the reference surface for accuracy, were plotted on a two-dimensional coronal, axial, and sagittal plane, perpendicular to the axis of the vertical, longitudinal, and lateral direction, respectively. A boundary around the points was computed. The area inside the boundary of each plane was calculated for the head, thorax, and pelvis phantoms (Fig. 6).
The pelvis phantom had a smaller area value on the axial plane than the other phantoms. The absolute area value on the axial plane was 14.1 and 32.5 cm 2 for the head and thorax phantoms, respectively, The stability for translation was <0.1 mm and that for rotation was <0.1°for all sites and both algorithms. The nominal stability of Catalyst HD system, which C-RAD published on the company website, was ≤0.5 mm 10 ; this is comparable to our results. Although the Catalyst HD system may have sufficient stability to detect patient motions in high-precision radiation therapy, such as intensity modulated radiation therapy and SRS, its stability may deteriorate when a gantry and an imaging device obstruct its scanner unit and when a Linac couch rotates. 12 Further, the ability to detect the isocenter displacement in motion monitoring (i.e., the accuracy of motion monitoring) was inferred to be comparable to the positioning accuracy results obtained under the same settings for the Catalyst HD system.
With respect to the monitoring mode, this was because the surface scanned by Catalyst HD system was always used as the reference surface for the registration.
The SRS algorithm demonstrated better reproducibility and accuracy than the conventional algorithm for the head phantom. This was because the reconstructed surface image obtained using the SRS algorithm had a higher image resolution. Moreover, the uncertainty of the registration calculation was reduced because of an increase in the number of corresponding point pairs between the live surface and the reference surface. In contrast, the increase in the number of their points for the registration may take longer computation time. Therefore, it is better for the SRS algorithm to choose less deformable and smaller areas of anatomical sites for registration.
Another aspect is that the white mask might be recognized as a part of the head, which may lead to errors in calculating the F I G . 6. Projected reference surface of the phantoms onto coronal, axial, and sagittal planes. S denotes the area value of the projected phantom in each plane image.
isocenter displacement, owing to the misalignment between the mask itself and the inside of the mask. In particular, the SRS algorithm using semi-non-rigid registration, which is not good for dealing with deformation, may result in larger errors than the conventional algorithm using non-rigid registration. Therefore, for treatment to the head and face using the SRS algorithm, the black mask that is undetectable by Catalyst HD system may be more suitable than the white mask. In addition, a previous study 13 reported that the difference in the positioning accuracy between the SRS with an opentype mask using another OSI system, and the SRS with a closedtype mask using the infrared tracking system was ≤1.0 mm for translation and ≤1.0°for rotation. The accuracy of this study is comparable to, or better than, that of the previous clinical study 13 for SRS with an open-type white mask using another hardware (AlignRT system). We suggest that an open-type black mask should be worn when using the SRS algorithm for the head and face during clinical use.
A limitation of this study is that the results presented in this paper were obtained using rigid body phantoms. The results may not fully reflect the technical performance of this system in a clinical situation. Several uncertainties, including patient's body shape changes, respiratory movement, and the reference surface size (scan volume) and type (external contour of a treatment plan or scan by the OSI system) will probably affect the technical performance of the system in clinical use. In particular, some uncertainties related to the contouring process (e.g., differences in size of voxel size, number of contour composition points, and operators) may further affect the technical performance when using the external contour of the treatment plan as a reference surface. The sensitivity and technical performance tests in this study were performed using the Catalyst HD system only. OSI systems released from other companies may not achieve the same results.

| CONCLUSION
In this study, we evaluated the reproducibility and accuracy of interfractional patient positioning and the stability of intrafractional motion monitoring. The reproducibility for translation and rotation was <0.1 mm and <0.1°, respectively; the accuracy was <1.0 mm and <1.0°, respectively; and the stability was <0.1 mm and <0.1°, respectively. In particular, the SRS algorithm had a significantly higher reproducibility and accuracy than the conventional algorithm (P < 0.01), and a small absolute error and SD of calculated isocenter displacement. There was no difference in the stability between the algorithms (P = 0.0280). The SRS algorithm was found to be suitable for the treatment of rigid body sites with less deformation and small area, such as the head and face.