Characterization of a new physical phantom for testing rigid and deformable image registration

Abstract The purpose of this study was to describe a new user‐friendly, low‐cost phantom that was developed to test the accuracy of rigid and deformable image registration (DIR) systems and to demonstrate the functional efficacy of the new phantom. The phantom was constructed out of acrylic and includes a variety of inserts that simulate different tissue shapes and properties. It can simulate deformations and location changes in patient anatomy by changing the rotations of both the phantom and the inserts. CT scans of this phantom were obtained and used to test the rigid and deformable registration accuracy of the Velocity software. Eight rotation and translation scenarios were used to test the rigid registration accuracy, and 11 deformation scenarios were used to test the DIR accuracy. The mean rotation accuracies in the X‐Y (axial) and X‐Z (coronal) planes were 0.50° and 0.13°, respectively. The mean translation accuracy was 1 mm in both the X and Y direction and was tested in soft tissue and bone. The DIR accuracies for soft tissue and bone were 0.93 (mean Dice similarity coefficient), 8.3 and 4.5 mm (mean Hausdouff distance), 0.95 and 0.79 mm (mean distance), and 1.13 and 1.12 (mean volume ratio) for soft tissue content (DTE oil) and bone, respectively. The new phantom has a simple design and can be constructed at a low cost. This phantom will allow DIR systems to be effectively and efficiently verified to ensure system performance.

late different tissue shapes and properties. It can simulate deformations and location changes in patient anatomy by changing the rotations of both the phantom and the inserts. CT scans of this phantom were obtained and used to test the rigid and deformable registration accuracy of the Velocity software. Eight rotation and translation scenarios were used to test the rigid registration accuracy, and 11 deformation scenarios were used to test the DIR accuracy. The mean rotation accuracies in the X-Y (axial) and X-Z (coronal) planes were 0.50°and 0.13°, respectively. The mean translation accuracy was 1 mm in both the X and Y direction and was tested in soft tissue and bone. The DIR accuracies for soft tissue and bone were 0.93 (mean Dice similarity coefficient), 8.3 and 4.5 mm (mean Hausdouff distance), 0.95 and 0.79 mm (mean distance), and 1.13 and 1.12 (mean volume ratio) for soft tissue content (DTE oil) and bone, respectively. The new phantom has a simple design and can be constructed at a low cost. This phantom will allow DIR systems to be effectively and efficiently verified to ensure system performance.   10 This phantom used optical markers to measure deformation from the coordinated information extracted from an optical camera via inhouse software. To generate deformation, pressure was applied to the back of the phantom. The authors did not intend for their model to be used as an end-user phantom since the use of the phantom and characterization of the deformation require a considerable amount of time and expertise. Kirby et al. developed a pelvic phantom that used rubber, mineral-filled plastic, and nylon bolts to simulate a real patient. 11 However, the phantom required special knowledge and materials that are not available to all DIR users.
Other anthropomorphic phantoms have been developed for quality assurance, 12, 13 but they have not had the ability to simulate a variety of tissue deformations and location changes. A validation method based on physician-drawn structure contours or physician-picked anatomical landmarks has also been widely used. 14 However, this approach is time-consuming and inevitably suffers from interobserver and intraobserver variability. Recently, a new virtual phantom was published in the American Association of Physics in Medicine (AAPM) task group 132 report, which can be downloaded online, to test DIR accuracy. 15 This virtual phantom has several limitations. It uses image offset instead of physical phantom movement to test the rotation, translation, orientation accuracy. Therefore, the end-to-end test of accurate data representation, image transfer, and integrity verification between image acquisition devices and image registration systems cannot be performed. The phantom also uses fixed insets and shapes of which a contour deformation cannot be simulated and the test of DIR accuracy cannot be simultaneously performed under rotation, translation, and deformation conditions. In addition, the DIR test used an anthropomorphic pelvis phantom, which has limited image contrast. The high (lung) and low (brain) contrast subjects were not included and cannot be validated under such clinical conditions. Although there are numerous methods to independently validate DIR systems, all of them demand a great deal of resources and time.
The purpose of this study was to design and develop a userfriendly, low-cost physical phantom that would be capable of testing rigid and DIR accuracy in a streamlined and seamless fashion (provisional patent application, Attorney Docket No.: UTSC.P1357US.P1).
The phantom was constructed using a variety of inserts simulating different shapes and properties of human tissue. These inserts can be arranged to simulate rigid or deformable changes in the patient anatomy as compared with its reference position. This phantom has labeled dimensions, which facilitates quantitative measurements for accuracy tests of both rigid and deformable registration. It can be imaged with CT, MRI, and PET/SPECT scanners to test DIR accuracy of multiple imaging modalities. Users can also use different materials in the inserts to test the DIR accuracy in a wide variety of clinical situations using both high-and low-contrast media.

2.A | Phantom design
The design of the phantom (called Wuphantom, named after the designer) is shown in Fig. 1(a). The main body of the phantom is fabricated from acrylic plastic with a density of 1.02 g/ml, which is slightly heavier than water. A phantom holder was constructed to allow tilting and rotation [ Fig. 1(b)]. There are two round insert slots Each of the inserts can also be rotated to simulate contour changes in both shape and location compared to the reference circle, which is usually used as the reference. Each insert cavity can be independently filled with solid or liquid materials with different densities, simulating different types of tissue in patients. A smaller cavity was constructed on the right side of the phantom to accommodate commercially available RMI inserts (Gammex, Inc.) with known densities [ Fig. 2(a) and 2(d)] and test the location changes of different types of tissue with known electron densities. We performed quality assurance on the phantom construction. The phantom rotation and tiling angle were within 0.12°of accuracy.  For the X direction movement test, the large insert (DTE oil) and

2.B | Phantom image acquisition
small insert (bone) were rotated from 0°to 180°relative to the reference position. This produced a distance displacement of 20 mm in the X direction for both density inserts used [ Fig. 3(b)]. For testing movement in the Y direction, the large insert was rotated from 270°t o 90°, and the small insert was rotated from 225°to 45°, simulating displacements of 20 and 14.1 mm, respectively, in the Y direction.
The images were first roughly registered using manual alignment by shifting and rotating the secondary image. A region of interest that encompassed the whole phantom was drawn. The Velocity rigid registration process was used to align the two image sets.

2.C.2 | Deformed image registration test
The secondary images for the DIR accuracy tests were acquired by replacing and rotating the circular and oval-and tree-shaped inserts.
This was done to simulate tissue deformation from a circular shape to another circular shape or an irregular shape (oval or tree). Rotat- bone deformation. We used the rigid and deformable multipass tool in the Velocity software program to perform the DIR process for all the selected secondary image sets. All of the corresponding contours were propagated into the secondary image sets.

2.D | Accuracy of DIR
The accuracy of the DIR of a contour can be characterized by three major factors: the conformity index (also called the Dice coefficient index), the maximum surface distance (also called the Hausdouff distance), and the volume ratio. The conformity index is defined as the ratio of twice the overlap of two structures over the sum of their volumes. This method is widely used in DIR comparisons. 16 The conformity index ranges from 0 to 1, denoting the degree of a perfect match between the two structures: where a and b are points belonging to sets A and B, respectively.
The volume ratio is defined as the ratio of the deformed volume to the reference volume:  The Velocity DIR system uses a B-spline algorithm and mutual information-based registration. We also performed DIR tests for materials of different densities, which are listed in Table 4. We selected water, dish soap, and oatmeal to test contours that have a similar density (i.e., dish soap [density = 1.03 g/ml] to DTE oil [density = 0.95 g/ml]) on DIR registration, simulating tumor shrinkage or softening during the course of the radiation treatment. We also tested low-density material deformation (i.e., oatmeal to oatmeal [density = 0.56 g/ml]) for simulating lung tissue deformation. These results are shown in Table 5.

| DISCUSSION
To the best of our knowledge, all of the physical phantoms used for DIR testing have either lacked quantitative testing capability or It has been reported that the accuracy of DIR algorithms has been tested inconsistently. 17-21 A multi-institution study was conducted to provide a consistent and direct comparison of the various algorithms and the performance of different systems. 22 The report indicated that there were large discrepancies in shifts and the DIR accuracy was equivalent to the voxel size. Our goal in this study was to design a phantom that could be used by any cancer institution that uses DIR in the clinic, either for commissioning or for quality assurance after DIR software upgrades. The phantom is made of acrylic and uses the existing plugs from an RMI phantom, which are available at most cancer treatment centers. It also requires less crafting, making it favorable for mass production at a low cost. We believe it has potential as a prototype phantom for national accreditation purposes to standardize the performance evaluation of all DIR systems across the country. There are limitations in the current study. Although the preliminary test results indicate that the Velocity adaptive re-contouring tool provides reasonably good estimates of contours generated from the original CT image set, the inserts were filled with uniform contents (DTE oil, dish soap, oatmeal, and bone) and contoured accordingly. Actual patient contours will have varied CT# within the volume (i.e., the clinical target volume encompassing the gross tumor and surrounding tissue). As a result, the inserts will need to be tested with mixed contents in the future. Our test used a standard head and neck CT scan protocol for both the reference and secondary images. It should be pointed out that the registration results for both rigid and deformable modalities were not fully evaluated when the scan protocol changed (i.e., noise value changes due to auto mAs use or slice thickness changes). Cone beam CT has been used as a main imaging tool for future online adaptive therapy. The accuracy of cone beam CT vs conventional CT registration would need to be fully studied prior to clinical use. Further work needs to be performed in this area.

| CONCLUSION S
We have developed a physical phantom that can provide a complete end-to-end evaluation of the accuracy of rigid and DIR system. The phantom has defined dimensions and a variety of inserts that can change the shape and contents, simulating different tissue characteristics. The phantom has a low cost; thus, it is widely accessible to clinics throughout the country and world.

ACKNOWLEDG MENT
We thank Ann Sutton and the Department of Scientific Publications at the MD Anderson Cancer Center for their efforts and editorial assistance.

CONFLI CT OF INTEREST
The authors declared no conflict of interest.