Signal and contrast to noise ratio evaluation of fluoroscopic loops for interventional fluoroscope quality control

Abstract Modern fluoroscopes pose a challenge for the clinical physicist for annual testing and continued upkeep. These fluoroscopes are critical to providing care to patients for complex interventions, and continue to evolve in automated image quality adjustments. Few tools in software or hardware currently exist to assist the physicist or technologist in gauging fluoroscope constancy or readiness for procedures. Many modalities such as mammography, computed tomography or even magnetic resonance imaging are much more evolved with respect to testing or quality control. In this work we sought to provide simple reproducible tools and methods for spot evaluating or continued quality testing of interventional fluoroscopes.


| INTRODUCTION
Complex fluoroscopically guided interventions (FGIs) have become common in many interventional radiology departments. Quality control (QC) is a necessary and appropriate activity to gauge the readiness of the fluoroscopes used in these procedures. The American College of Radiology (ACR) as well as the American Association of Physicists in Medicine recommend a QC program for these devices to ensure accurate and consistent patient care. 1 However, currently there is no agreed upon objective metric to be used to ascertain changes made to dose and image quality on the images seen during routine testing, QC, or the For Presentation images employed during clinical use. Additionally, fluoroscopes used in FGIs continue to evolve and become more complicated. The National Electrical Manufacturers Association has addressed some of the complexities of performing annual inspections and image quality adjustments by establishing a standard for medical imaging manufacturers to provide a manual operating mode on fluoroscopes to accomplish these tasks. 2 In this work, we sought to identify simple and reproducible metrics to be used for both periodic QC and continuous operating levels to ensure that the fluoroscope is ready to be used in FGI procedures, and to gauge changes made to organ programs or dose that could affect image quality.
Signal to noise ratio (SNR) and contrast to noise ratio (CNR) are widely used in other x-ray imaging modalities such as computed tomography (CT) and mammography as QC standards. In CT, CNR is used as a pass/fail criterion for ACR CT accreditation. 3 In mammography, some manufacturers use either SNR, CNR, or both to set weekly pass/fail limits to monitor performance of the system, as well as pass/ fail criteria as part of the new ACR Digital Mammography Accreditation Program. 4 These pass/fail standards are helpful to the technologists not only for accreditation purposes but also to remove the ambiguity of modality readiness within the quality program. If SNR or CNR do not meet the minimum threshold, the unit or program being used is deemed unfit for patient use. Tapiovaara  Objects Ltd, North Yorkshire, UK) and a solid state radiation detector to track air kerma and image quality. 6 Both of these methods required direct analysis by a medical physicist. Now, with the ubiquitous role of neutral archives in modern interventional radiology departments, it is possible to automate much of the image routing and enable image quality tracking across a fleet of fluoroscopes using a centralized database. The centralized database provides a "digital workbench" whereby phantom images acquired daily by the technologists are processed using automated image analysis. The QC data are immediately available to be reviewed and recorded by the technologist.
We begin an initial endeavor into continuous monitoring of image quality in fluoroscopy, via the use of SNR and CNR as metrics for QC and operational readiness, as part of an ongoing in-house QC program. Technologists perform daily QC using a customized phantom, and after months of data collection, we begin to investigate pass/fail criteria for SNR and CNR. The benefits of such a program include engagement of the technologists, in concert with the physicist, in the quality program, as well as aiding the staff in operational readiness of the fluoroscope.

2.A | X-ray fluoroscopes and phantoms
Quality control and testing was performed in two high-patient-volume interventional departments at our institution. The Interventional Radiology (IR) department specializes in peripheral angiography and has five Siemens fluoroscopes (Siemens Healthineers, Forchheim, Germany); one Artis Zeego, two Axiom Artis, and two Artis Q single plane systems. The Interventional Neuroradiology (INR) department has two Siemens Artis Zee Bi-plane systems, which are predominantly used for neurovascular work. All rooms in both departments have the ability to "Store Fluoro," which is a feature used to save the last fluoro scene as a series or loop that would otherwise not be captured. To achieve steady state dosimetric parameters (kVp, mA, spectral filter), the stored fluoro loops were only saved after multiple activations of the fluoro pedal, a method often described as "double clutching," to allow a possible spectral filter or other parameters to change if necessary due to increased or different load via our phantoms. Lastly, all fluoro loops were acquired for 5 s, to acquire enough frames for an average to be determined, as well as to allow the system kV or mA to reach steady state within the loop. Three separate and unique phantoms were used for testing based on availability and ease of use. Each is described in more detail to follow.
The first phantom, called the slab phantom, consisted of 35.5 cm × 43.2 cm × 2.54 cm polymethyl methacrylate (PMMA) sheets that can be stacked at various thicknesses on the patient couch representing varying patient sizes, similar to the method described in TG-125. 7 While this phantom was not one singular phantom, collectively the slabs were only used as an initial proof of concept for SNR and CNR testing methodology. A rectangular tin foil swatch (50 mm × 50 mm × 0.05 mm thick) was employed as a target for CNR measurements and utilized as a surrogate for an iodinated contrast agent, similar to the method described by Kotre et al. 8 The tin swatch (Goodfellow Corporation, Coraopolis, PA, USA) was placed directly on top at the center of the first 2.54 cm slab of PMMA ( Fig. 1) and imaged with clinically used programs, at 7.5 pulses per second (pps), at maximum Source To Image Distance (SID), and with the table top at the Interventional Reference Point.
Field of View (FOV) was kept constant at 43 cm. Slabs were subsequently added and the resulting phantom was imaged for each added slab, and for each added slab the resulting loop was stored ( Fig. 2). At 22.9 cm of PMMA, both low dose and high dose fluoro programs were also tested. Although the table pad is typically in place for patient imaging, and removal would alter the beam characteristics, to ease slab placement and balance, the table pad was removed to provide a flat working surface for initial testing.
The second phantom used for testing was a 25.4 cm × 25.4 cm × 7.62 cm custom-built patient equivalent Contrast-Detail (CD) phantom consisting of 1.59 mm sheet of copper with a 6.35 mm aluminum sheet sandwiched in additional sheets of PMMA (Fig. 3). The design of this phantom allows for it to be easily carried from room to room. Although this phantom has detail objects inside, there is a space at the center of the phantom, which is free of detail objects and provides a uniform area well suited for SNR measurements. This phantom is currently employed as part of an ongoing daily QC program in our IR department that requires technologists to fluoro the phantom and subjectively count the CD objects or holes. SNR was calculated using the middle, uniform part of this phantom. The phantom was placed on the table pad in the same position each day and centered under fluoroscopy, with a SID set to 100 cm, a nominal FOV set at 32 cm, and with the table raised to a height so that the phantom just met the receptor (Fig. 4). This phantom and setup was used for SNR measurements in all of the IR rooms.
The last phantom, called the CNR phantom, employed in testing closely resembled the CD phantom, except that the detail section containing holes was replaced with uniform polycarbonate and a tin swatch identical to the previously described slab phantom and was centered and affixed within the middle of the phantom. This phantom was used for both SNR and CNR measurements in the INR rooms. A similar setup as described above for the CD phantom was used.  Figure 5 shows a single frame from one of the fluoro runs. The red hatched areas are the ROIs used for the SNR and CNR measurements. The SNR for each frame was computed via eq. (1): where: SNR f is the frame average SNR, X BG is the average background signal (pixel value) in a ROI, and σ BG is the standard deviation in the same background ROI.
The CNR for each frame in the loop was determined via eq. (2): where: CNR f is the frame average CNR, X BG is the average background signal in an ROI, and σ BG is the standard deviation in the  A Kirsch filter was then applied to find the edges of the contrast holes and an erosion filter was then applied to reduce edge noise.
Using a priori knowledge of the contrast hole objects, the largest contrast hole was found using a circle detection algorithm. The 12 mm center ROI was then geometrically placed based from the center of the largest contrast hole. The analysis was repeated for each of the last 15 valid frames of fluoroscopy data, similar to the method used for the slab phantom analysis. The ROI in the center of the phantom was used to determine a frame SNR, and then repeated over all frames and averaged to yield one representative SNR for the entire loop.
Daily QC was also performed over a 5 month period in the INR department using the previously described CNR phantom. As was done in the IR department, the dedicated QC program was built on each fluoroscope. The fluoroscopy loops were acquired in the same way as above with the CD phantom, by technologists pressing the pedal for five seconds using the same setup and dose program. Similar to the SNR measurements, to reduce the noise, the images were ini-  (IR-18). Table 2 shows a summary of the statistics for all five IR rooms, indicating kVp, mA, SNR, and values above and below the UCL and LCL.
Initially, the daily QC data for CNR for each frame of the fluoro loop were only determined from the ROIs in the center circle (Fig. 6 Section 3.A also showed the relationship between SNR and CNR with respect to a surrogate patient load (PMMA). Figure 9 indicates that even with a large phantom thickness (40.64 cm thickness), the SNR for the system does not appear to decline sharply, but rather decreases slowly. Since the SNR does not abruptly or significantly change, we would anticipate the image quality should follow as the patients get thicker. However, this testing was under one set of F I G . 1 1 . Signal to noise ratio (SNR) and contrast to noise ratio (CNR) using fixed 22.86 cm polymethyl methacrylate (PMMA) slabs with different fluoro dose levels. Note the change in both SNR and CNR for low and high dose fluoro modes where the dose is lower or higher than the normal fluoro dose setting. From the low to norm setting, the dose rate increased 5 mGy/min, however the kVp decreased by 3, and the copper spectral filter decreased 0.6 mm, indicating that the beam quality was significantly higher for the low fluoro setting compared to the normal setting, which reduced the SNR from low to norm at this elevated thickness of PMMA. conditions, and could be compared to a second set of conditions to determine if there are more optimal settings for differing patient thicknesses.
The CNR results presented in Fig. 10 show a slightly different pattern with respect to SNR. The CNR of the tin iodine surrogate, shows a significant decline with thickness of PMMA, and a complete loss of low contrast resolution at 33 cm. The reason for the increase in CNR at 35.6 cm and greater is both due to a spectral filter change and subsequent change in kVp. This test may provide insight into the fidelity of a particular fluoro program and perhaps where the iodinated contrast agent is most or least effective for a given set of parameters. Again, these settings could be compared to another program for image quality optimization purposes. Figure 11 shows the relationships between SNR and CNR for three different fluoro programs for one thickness of PMMA. Often clinicians will use the low dose setting, with the intent to lower the dose rate on fluoroscopy. Table 1 shows that for these conditions, and with an increase in kVp and lower dose compared to the normal setting while the SNR is 73 for the low dose program, the CNR is quite low compared with normal and high fluoro settings. Having both SNR and CNR attributes for differing settings on the fluoroscope may aid staff in choice of program selection but also allow the clinical physicist to make changes and compare "A vs. B" and better understand quantitatively the tradeoffs associated with the change.
Lastly, the initial testing of the slab phantoms was under optimal conditions and only as proof of concept. Time and care were taken for slab placement as well as removal of the was key, and even though the support pad altered the X-ray beam with respect to initial slab phantom testing, haste before starting the patients took priority when moving from the slab phantom to the CD and CNR phantom daily QC and hence, the support pad was left on the table.   for all data for each of the rooms, was less than 4%, and only one fluoroscope (IR-18) had more than one instance of data above for UCL or below the LCL.  The limitations of this study are that there was only one fluoroscope vendor available for these tests, and a small sample of models from that vendor. Future work will include at least two more vendors and several more models to fully understand SNR and CNR utility and clinical impact in fluoroscopy. Automated and observerindependent QC of units used during FGIs was performed in highpatient volume IR and INR departments. Minimal technologist effort and change in workflow were needed to regularly monitor system performance and readiness of the system for the day. These data allow for room-specific SNR thresholds to be established and used as a criterion for providing immediate feedback on whether the system is operating at an expected level.

| CONCLUSION S
With the ever increasing complexity of fluoroscopes, coupled with proprietary dose and image quality software algorithms from the vendors, robust and straightforward methods for QC and image quality metrics are needed. We have shown the utility in simple SNR and CNR metrics and how they can be used during fluoroscopy daily QC or during performance testing with very simple patient equivalent phantoms.

ACKNOWLEDG MENTS
The authors thank Lee Sumpter, Steve Haug, Bob Growden and Ben Heydari, the QA Techs in Interventional and Neuro Radiology for diligently taking the data. The authors also thank Klaus T A B L E 3 Shows the results from CNR/SNR data for four planes of two biplane fluoroscopes over a 4 month period. Wintersteiger, Field Service Engineer from Siemens Healthineers, for programming the equipment and Gary Manninen for his database expertise.

CONF LICT OF I NTEREST
The authors have no relevant conflict of interest.