Real-time respiratory motion compensated roadmaps for hepatic arterial interventions

Abstract

Purpose

During hepatic arterial interventions, catheter or guidewire position is determined by referencing or overlaying a previously acquired static vessel roadmap. Respiratory motion leads to significant discrepancies between the true position and configuration of the hepatic arteries and the roadmap, which makes navigation and accurate catheter placement more challenging and time consuming. The purpose of this work was to develop a dynamic respiratory motion compensated device guidance system and evaluate the accuracy and real-time performance in an in vivo porcine liver model.

Methods

The proposed device navigation system estimates a respiratory motion model for the hepatic vasculature from prenavigational X-ray image sequences acquired under free-breathing conditions with and without contrast enhancement. During device navigation, the respiratory state is tracked based on live fluoroscopic images and then used to estimate vessel deformation based on the previously determined motion model. Additionally, guidewires and catheters are segmented from the fluoroscopic images using a deep learning approach. The vessel and device information are combined and shown in a real-time display. Two different display modes are evaluated within this work: (1) a compensated roadmap display, where the vessel roadmap is shown moving with the respiratory motion; (2) an inverse compensated device display, where the device representation is compensated for respiratory motion and overlaid on a static roadmap. A porcine study including seven animals was performed to evaluate the accuracy and real-time performance of the system. In each pig, a guidewire and microcatheter with a radiopaque marker were navigated to distal branches of the hepatic arteries under fluoroscopic guidance. Motion compensated displays were generated showing real-time overlays of the vessel roadmap and intravascular devices. The accuracy of the motion model was estimated by comparing the estimated vessel motion to the motion of the X-ray visible marker.

Results

The median (minimum, maximum) error across animals was 1.08 mm (0.92 mm, 1.87 mm). Across different respiratory states and vessel branch levels, the odds of the guidewire tip being shown in the correct vessel branch were significantly higher (odds ratio = 3.12, p < 0.0001) for motion compensated displays compared to a noncompensated display (median probabilities of 86 and 69%, respectively). The average processing time per frame was 17 ms.

Conclusions

The proposed respiratory motion compensated device guidance system increased the accuracy of the displayed device position relative to the hepatic vasculature. Additionally, the provided display modes combine both vessel and device information and do not require the mental integration of different displays by the physician. The processing times were well within the range of conventional clinical frame rates.

1 INTRODUCTION

Hepatic arterial interventions, such as transarterial embolization, chemoembolization, and radioembolization, are minimally invasive treatment options, and the current standard of care for many patients with hepatocellular carcinoma or liver metastases.^1-3 During liver embolizations, a catheter is navigated to a feeding artery of the tumor to deliver embolic particles in order to decrease blood supply to the tumor and increase dwell time of added agents such as radionuclides or chemotherapeutic drugs. Commonly, the catheter and guidewire (device) are navigated under fluoroscopic image guidance, which provides good device visibility. Digital subtraction angiography can be performed using a breath-hold and iodinated contrast injection to provide a reference image of the vasculature. The reference image is typically shown side-by-side with live fluoroscopic images. The position of the device within the vasculature is determined by comparing the device shape to the configuration of the arterial branches in the reference image. This can be a challenging and time-consuming task that requires observing and mentally integrating the information from multiple displays while performing the navigation. Roadmapping is a technique which aims to combine the device and vessel information by subtracting a previously acquired contrast-enhanced X-ray image (without the device) from live noncontrast fluoroscopic images of the device. In the absence of patient motion, this approach eliminates anatomical structures in the background and provides a negative (bright pixel values) mask of the static vasculature with a real-time representation (dark pixel values) of the device. This technique is often used for procedures in the head and neck or the extremities, where the effect of respiratory and cardiac motion is negligible. The use of subtraction-based roadmapping is limited for procedures in the thorax and abdomen where the target organ is subject to significant respiratory motion (e.g., liver), which causes subtraction artifacts. In those instances, a side-by-side view with a static roadmap is often utilized. This requires the radiologist to mentally integrate both displays while navigating and is less accurate since the position of the vasculature is constantly changing, making the static vessel roadmap a poor representation of the true vessel position.

Previously published literature on respiratory motion compensation can be categorized into techniques for (i) respiratory signal extraction, (ii) motion modeling, and (iii) visualization. Additionally, previous work can be distinguished by the type of application and evaluation method. Respiratory signal extraction is the process of estimating the respiratory state of a patient at a particular point in time. It is used in external beam radiotherapy,^4-7 gated 3D imaging,^8-10 as well as interventional device guidance.^11-19 Traditional techniques often include external devices such as optical tracking of surface markers,^{5-7, 20} respiratory belts,⁵ and ultrasonic diaphragm tracking.^{8, 21, 22} While external respiratory motion tracking techniques have shown to be successful, the disruption of the clinical workflow and requirement for additional devices and a more complex setup has driven the research of image (fluoroscopy)-based respiratory tracking techniques. Using the motion of an intravascular device, such as a guidewire or catheter placed in a target vessel as an indicator of respiratory motion, has been suggested.^{11, 12, 19} Ambrosini et al.¹⁹ directly registered the device centerline to forward projected centerlines of a 3D roadmap using a shape similarity metric. Orozco et al.¹² and Atasoy et al.¹¹ both tracked the location of the device compared to previous image frames to estimate respiratory motion. While the intravascular device provides a high-contrast target, which can be easily tracked using fluoroscopic imaging, the main challenges for these approaches are that only localized motion information around the device can be estimated, and it can be difficult to distinguish between device motion caused by respiration versus device navigation by the radiologist. To address the latter, Atasoy et al. tracked device motion using a template from a manually selected region of interest, which is only updated if considerable changes in device shape occur.¹¹ Anatomical landmarks and soft tissue features have also been used for respiratory motion tracking from fluoroscopic images.^{4, 13-15, 17, 18} While tracking of soft tissue motion has been done successfully for coronary roadmapping applications,^{13, 18} the direct translation to liver procedures is challenging due to a lack of distinct features in the soft tissue of the liver. Similarly, Berbeco et al.⁴ used intensity changes in a region of interest in the center of the lung to detect changes in tissue density due to the compression of the lung tissue during breathing. This approach is generally not feasible for liver applications due to the limited field of view (FoV) above the diaphragm. Dimensionality reduction using variants of principal component analysis (PCA) has also been used successfully to extract respiratory motion signals in cardiovascular interventions^{23, 24} and could potentially be translated to other types of procedures. For fluoroscopic imaging in the liver, diaphragm tracking is commonly used for respiratory signal extraction, which can be achieved through registration of manually annotated patches around the diaphragm^{14, 17} or tracking of edge features.¹⁵ In a previous feasibility study, we proposed a simpler technique for diaphragm tracking for 3D navigation during TIPS procedures using the vertical intensity-weighted centroid of an image.¹⁶ In the present work, the centroid technique is applied to 2D motion compensated roadmapping and thoroughly evaluated.

Motion modeling of respiratory motion has primarily been performed using translation only,^{11, 13, 17, 20} rigid,¹⁹ or affine transforms,^{12, 14-16, 18} however, more flexible models, such as motion vector fields,⁸ have also been investigated. Different strategies for the estimation of the motion model have been proposed, including directly inferring vessel motion (VM) from the motion of structures visible in noncontrast X-ray images such as soft tissue,^{13, 18} anatomical landmarks,¹⁷ or intravascular devices.^{11, 12, 19} While anatomical landmarks and intravascular devices provide localized motion information, soft tissue tracking may not be feasible for liver applications due to the lack of distinct soft tissue features. King et al.¹⁴ used a series of respiratory-gated 3D MRI acquisitions to generate a patient-specific motion model of the vasculature. Differences in patient position between MRI and interventional imaging and the time between acquisitions, however, might cause discrepancies between the motion model and actual respiratory motion. The technique proposed in this article generates a patient-specific vascular motion model directly from noncontrast and contrast-enhanced X-ray sequences acquired in the interventional suite at the time of the procedure, thus, eliminating the need for additional modalities and reducing the risk of discrepancies in the motion model compared to the actual breathing motion.

The visualization of motion compensated roadmaps has not been the focus of many previous publications. Most vendors of interventional C-arm systems provide tools to overlay 3D roadmaps onto 2D X-ray displays.^25-27 However, navigating devices using a constantly moving roadmap can be challenging and the simultaneous display of respiratory motion and navigational device motion may be distracting to physicians. Removing motion in X-ray image sequences is used to improve the visibility of stents^{28, 29} using the CLEARstent technology (Siemens Healthineers, Forchheim, Germany), where adjacent image frames are registered based on two markers attached to the ends of the stent and subsequently averaged. This approach blurs anatomical background but increases stent visibility through noise reduction. The proposed method offers a motion-compensated display mode, where the device shape extracted from live fluoroscopic images is deformed to compensate for the vascular motion, such that it can be correctly displayed in a static roadmap.

The purpose of this work was to develop a real-time image-based guidance system, which provides motion-compensated overlays of vasculature and intravascular devices without subtraction artifacts and good device visibility using only interventional C-arm imaging. The contributions of this work to the existing literature on respiratory motion compensation include (i) a robust and computationally inexpensive method for image-based respiratory state tracking, which does not require manual input, (ii) a technique to calculate respiratory motion models from free-breathing noncontrast and contrast- enhanced X-ray sequences, which can be easily integrated into the interventional workflow, and (iii) a visualization technique which applies the inverse VM to the device such that it can be displayed correctly in a static vessel roadmap. Furthermore, most previous studies were evaluated through retrospective analysis of clinical or phantom images, while the proposed system was implemented in real-time and evaluated in prospective animal studies, where it was used for device navigation. To this end, the following hypotheses were investigated: (H1) the respiratory motion of the liver vasculature can be accurately modeled based on prenavigational contrast-enhanced X-ray image sequences, (H2) motion-compensation of the vessel roadmap or the device representation in the display increases the number of frames where the device is shown in the correct vessel branch, and (H3) the image processing and display can be performed in real-time with clinically relevant frame rates.

2 MATERIALS AND METHODS

The presented techniques aim to provide an image display for catheter and guidewire (device) navigation that includes both real-time vessel and device information. The vessel morphology and a model describing the deformation of the vasculature due to respiratory motion are extracted from a prenavigational, contrast-enhanced X-ray image sequence acquired under free-breathing conditions. Subsequently, during device navigation, the current respiratory state is determined from the live fluoroscopic image and the corresponding deformation is extracted from the motion model. This allows for display of a motion-compensated roadmap of the vasculature at any point in time. The device is segmented from live fluoroscopic images using a deep learning approach and overlayed on the motion-compensated roadmap.

2.1 Respiratory motion model and tracking

Figure 1 shows an overview of the respiratory motion modeling and motion compensated device navigation approach. The purpose of the motion model is to establish a mapping function between respiratory state and vessel deformation. The motion model is generated from two X-ray image sequences acquired prior to device navigation. Non-contrast (mask, ) and contrast-enhanced (fill, ) sequences, where represents the acquisition time of an image frame, are both acquired under free-breathing for the duration of at least one respiratory cycle. Within this work, each image frame is assumed to be log-transformed and is defined on a regular 2D grid represented by coordinates x and y, where y is the dimension in cranial-caudal direction. The dynamic vasculature during free breathing is extracted from the fill images by subtracting a corresponding mask at time , which minimizes the mean squared error (MSE).

(1)

where

(2)

FIGURE 1

Flow chart describing the prenavigational generation of a motion model (left) and live image processing (right)

To build the motion model, an end-expiratory image frame is selected and registered to all remaining frames using an affine transformation. The selection of is based on two criteria: (i) the respiratory state (see Equation 4) and (ii) good contrast-enhancement of the vasculature. This is determined by averaging all subtracted image frames whose respiratory state is smaller than the fifth-percentile (approximately end expiration) of all respiratory states. The MSE between the averaged image and all subtracted end-expiratory frames is then calculated and the frame that minimizes the MSE is chosen as .

The registration is performed by minimizing the MSE (see Equation 3) using a gradient descent approach (maximum number of iterations = 100, initial step size = 0.00625, relaxation factor = 0.5, gradient tolerance = , and step tolerance = ) with three pyramid levels. At each level, the registration is performed on downsampled versions of the images, such that the image resolution doubles after each level and the final level is performed on the original image resolution (960 × 960 pixels). This approach yields a transformation matrix for each point in time representing the deformation of the vasculature between the current frame and end expiration.

(3)

To complete the prenavigational motion model, the respiratory state is calculated for each image frame based on the position of the diaphragm in the noncontrast images. Specifically, the average pixel intensities of each image row and subsequently the center of mass in vertical direction is calculated. The center of mass is then used as a surrogate for the respiratory state, as its position changes with the ratio of bright to dark pixels in the FoV above and below the diaphragm, respectively. This approach requires the diaphragm to be in the FoV. However, this is generally not a limitation for procedures in the liver. Mathematically the respiratory state at time t can be written as:

(4)

Within the scope of this work, a 2D affine transform is used to describe the deformation of the vasculature, where each element of the transformation matrix can be described by a linear function of the respiratory state . We further know that the transform for the end-expiration respiratory state corresponding to is the identity matrix. The linear function to calculate the transformation matrix can, thus, be written in vector form as:

(5)

where and represent the respiratory states at maximum inspiration and expiration, respectively, and are the free parameters describing the slope of the linear function for each element of the affine transform. The parameters are estimated by minimizing the mean squared difference between the estimated transforms from the registration step to the transformation matrix calculated based on the respiratory state for the same frame.

(6)

In practice, the solution to Equation 6 is obtained analytically using the Moore–Penrose inverse.³⁰ During device navigation, the respiratory state is calculated for each live image frame using Equation 4 by replacing with . The corresponding transformation matrix describing the deformation of the vasculature for the current respiratory state is then calculated based on Equation 5.

2.2 Device segmentation from live images

During the device navigation, fluoroscopic imaging is performed, which provides information on the position and shape of endovascular devices such as guidewires and catheters. The device tracking is performed using a deep-learning segmentation approach as described in Wagner et al.,³¹ which uses a modified version of the SegNet architecture³² as encoder-decoder convolutional neural network. The network was trained on a combination of manually annotated fluoroscopic images from pig studies as well as hybrid images created by superimposing simulated guidewires on real X-ray images.³¹ The training of the network was performed using the MATLAB 2019a Deep Learning Toolbox (MathWorks, Natick, MA, USA). To improve the performance of the real-time prototype, an inference only version of the final trained network was implemented using CUDA 10.0 and CUDNN version 7.6.5 (Nvidia, Santa Clara, CA, USA). The binary mask output of the network representing the device segmentation was further processed to reduce the effect of device thickening as a result of the upsampling operations performed in the encoder-decoder network, which could reduce visibility of the vessel roadmap. Therefore, a topology preserving thinning algorithm³³ was applied to the binary mask followed by a Gaussian filter and soft thresholding operation. The thinning step reduces the device width to a single pixel, while the subsequent filtering and thresholding steps increase the width to a constant thickness. For the purpose of this work, a 7 × 7 pixel Gaussian filter kernel with a standard deviation pixels and a threshold of was used. All parameters were chosen empirically to provide good visibility of the device while avoiding occlusion of small vessels by the device. The final image representing the segmented device is therefore

(7)

where represents the convolutional neural network and subsequent topology preserving thinning.

2.3 Real-time navigation display

Within this work, two display modes are evaluated: a compensated roadmap display (CRD) and an inverse compensated device display (ICDD). The CRD shows a moving vessel roadmap overlaid with the real-time image of the intravascular device, where both device and respiratory state are estimated from live fluoroscopic images. To generate the CRD, the transformation is applied to the vessel roadmap and the segmented guidewire image is superimposed on the transformed roadmap. The displayed image frame can, thus, be calculated as:

(8)

where denotes a color overlay of B onto A. For the ICDD, the estimated vessel deformation is inverted and applied to the device, which is then superimposed on a static roadmap. The displayed image frame can be described as:

(9)

While both approaches compensate for respiratory motion, ICDD completely removes motion caused by respiration from both the vasculature and endovascular devices creating a less distracting display for navigation.

2.4 Porcine study

Approval for this study was received from the local Institutional Animal Care and Use Committee (IACUC). A porcine study including seven female pigs with an average age of 6 months and weight of 52.1 ± 2.0 kg was performed by an interventional radiologist with over 10 years of experience in hepatic arterial interventions. Femoral arterial access was obtained, and a 5Fr catheter was placed in the common hepatic artery under fluoroscopic guidance. Two nonsubtracted prenavigational X-ray image sequences were acquired for the duration of one full breathing cycle under continuous ventilator respiration. An anteroposterior viewing angle was chosen as working angle for all experiments. The first sequence was a noncontrast 2D acquisition with a frame rate of 15 frames per second (fps) and an isotropic pixel resolution of 0.308 mm. The second sequence was a contrast-enhanced acquisition with the same imaging parameters and the following injection protocol: Iohexol 300 mgI/mL contrast injected for 5 s with a flow rate of 2.5 mL/s, followed by a 2 s saline chase at the same rate to reduce the overall amount of contrast required.³⁴ A respiratory motion model was established based on these prenavigational acquisitions as described in the previous section. To obtain a reference of the true VM, a wire with an X-ray visible marker near the distal tip was inserted into a 2.8Fr torqueable microcatheter (Direxion Hi-Flo, Boston Scientific, Marlborough, MA, USA). The microcatheter was placed in three to six distal branches of the hepatic arteries under fluoroscopic image guidance. After placement of the microcatheter, an approximately 12 s fluoroscopic image sequence was acquired under continuous ventilation while keeping the microcatheter in place. The marker position was manually annotated in each image frame and the displacement of the marker position served as reference of the true motion of the current vessel branch. Finally, a 0.035″ guidewire (Glidewire, Terumo, Shibuya City, Tokyo, Japan) was used to select distal branches of the hepatic arteries under fluoroscopic imaging using the same frame rate and pixel resolution as for the prenavigational sequences to evaluate whether the wire was shown in the correct vessel branches.

2.5 Evaluation and statistical methods

2.5.1 Respiratory state estimation robustness

In some cases, the physician might want to change the zoom level or the position of the patient table interprocedurally, which would cause a change in the FoV. To determine the robustness of the respiratory state estimation with respect to changes in the FoV, seven noncontrast X-ray image sequences were analyzed. The respiratory state was calculated for the full FoV (960 × 960 pixel) as well as 12 regions of interest (RoI) (512 × 512 pixel) distributed evenly such that the diaphragm was still visible in all RoIs. Matching RoIs were used for the generation of the motion model and the respiratory signal extraction in the device navigation sequences. In practice, valid RoIs could be determined by performing rigid registration between the first image frame after changing the FoV and the most similar frame of the initial noncontrast sequence. The largest overlapping region between the two images can then be used as RoI to recalculate the motion model and perform respiratory signal extraction on all future frames. The mean absolute error (MAE) between the respiratory states calculated on the full FoV and each of the RoIs was used as a measure of robustness. Additionally, the robustness with regards to changes in image quality (e.g., due to a change in acquisition protocol) was evaluated by comparing respiratory states calculated on cine quality and fluoroscopy image sequences without contrast. Since cine and fluoroscopy images were acquired at different points in time, the respiratory state curves cannot be compared directly. Instead, the intervals containing the majority of respiratory states for a single X-ray sequence were determined by calculating the first-percentile and 99th-percentile. The intervals from corresponding cine and fluoroscopy sequences were then compared by calculating the MAE.

2.5.2 Accuracy of VM estimation (H1)

A total of 23 2D X-ray sequences with the radiopaque marker in a distal vessel branch were acquired. The device was held in place such that changes in the marker's location were only due to respiratory motion. Each sequence containing a single marker in a single vessel branch was analyzed by manually annotating the marker centroid in each frame and the corresponding marker motion (MM) vector was calculated as the difference between the current marker centroid and the position at respiratory state = 0 (end expiration). The estimated VM was calculated for each frame using the motion model and the centroid based respiratory state for a single point in the same vessel where the marker is located. The accuracy of the estimated VM was quantified as the Euclidean distance on the (x,y) grid for each frame, where

(10)

To estimate the average Euclidean distance between coordinate points, an intercept-only linear mixed model (LMM) was fit to the data where the measurement cluster (machine sequence nested within individual swine) was modeled as a random effect using the “lme4” package.³⁵ To calculate the MSE of the VM Estimation relative to Marker-tracking, separate linear models were fit for the x- (left to right) and y-coordinates (superior to inferior [SI]) for each individual swine.

2.5.3 Device display accuracy (H2)

A total of 41 2D X-ray sequences (20 413 frames in total) were acquired where guidewires were placed in different distal hepatic arterial branches and motion-compensated displays were generated using a real-time prototype. A manual frame by frame evaluation was performed to determine whether the wire tip was displayed in the correct vessel branches using CRD, ICDD, or a display without motion compensation. This was measured for each image sequence, in terms of the percentage of frames, the tip was displayed in the correct vessel branch within each sequence. To account for cases where the device tip is shown outside the vessel boundaries but close enough to be interpreted correctly by the radiologist, the device tip was considered “inside” if it was within 2 pixels of the vessel boundary. To assess the accuracy of the displayed device relative to the vessel roadmap, the probability of the wire tip being shown inside the vasculature (ratio between number of frames with the wire tip inside the vasculature and the total number of frames in the sequence) was calculated. A generalized linear mixed model (GLMM) was fit to the data to compare the probabilities across motion-compensated and uncompensated display types. The model adjusted for the effect of arterial branch level (a six-level factor for 1st, 2nd, 3rd, etc… order branches), which is correlated to vessel thickness, and respiratory state (end expiration, mid inspiration, and end inspiration), whereas the measurement cluster (machine sequence nested within individual swine) was modeled as a random effect. An overview of the number of sequences contributing to the 18 combinations (6 branch levels × 3 respiratory state ranges) is shown in Figure 2. Subsequent 95% confidence intervals were estimated via parametrically bootstrapping (3000 iterations) for the LMM, while the GLMM confidence intervals were estimated via profiled likelihood. All statistical calculations were done using R (V 3.6.2). A p-value less than 0.05 was considered statistically significant.

FIGURE 2

Overview of the number of image sequences (maximum 41) contributing to each combination of vessel branch level and respiratory state for the evaluation of the display accuracy

2.5.4 Real-time performance (H3)

The suitability of the system for real-time processing was evaluated by measuring processing times for individual time frames during live image guidance as well as the total time required to establish the prenavigational motion model.

3 RESULTS

A motion-model was established for each animal based on the two prenavigational image sequences with a median of 163 noncontrast and 266 contrast-enhanced frames. Figure 3 shows examples of the respiratory state measured over a noncontrast X-ray sequence, where the respiratory cycles can be easily identified as each peak corresponds to an inspiratory phase. The MAE (average ± standard deviation) between the respiratory state calculated on the full FoV and the individual RoIs was corresponding to an average VM estimation error of less than 4%. When comparing respiratory states calculated on cine and fluoroscopy quality image, the MAE was . Note, the reported MAEs (as the respiratory state itself) are unitless values, where an error of 1.0 would correspond to the difference between end- expiratory and end-inspiratory state. Motion compensated displays (CRD and ICDD) were generated for all fluoroscopic image sequences showing the microcatheter with the radiopaque marker or the guidewire. Figure 4a and b shows examples of the CRD and ICDD display, where the uncompensated vessel roadmap and guidewire visualization, respectively, were overlaid as reference. Videos of the two display modes are provided as Supporting information Video S1 (CRD) and Video S2 (ICDD). Figure 5a and b shows all wire tip positions for two different navigational image sequences with and without motion compensation overlaid on roadmaps acquired at end expiration. The maximum observed VM for each pig in SI direction was between 10 and 21 mm at the detector plane.

FIGURE 3

Example of the estimated respiratory state over a 15 s mask sequence

FIGURE 4

Comparison between compensated roadmap display (CRD) and inverse compensated device display (ICDD) showing the same X-ray frame with different visualization techniques: (a) Compensated roadmap display (CRD) showing the corrected roadmap in white with device position in blue and uncompensated roadmap overlaid in red. (b) Inverse compensated device display (ICDD) with motion compensated wire in blue and uncompensated wire in red shown in the wrong vessel branch. In both the CRD and ICDD, the device is correctly shown in the right medial hepatic artery

FIGURE 5

Wire tip positions during navigation for two fluoroscopic image sequences without motion compensation (red) and with motion compensation (blue) overlaid on a roadmap acquired at end-expiration. Without motion compensation, the wire tip periodically moves in and out of the vessel, while it stays inside of the vessel roadmap (left medial hepatic artery) with motion compensation. The effect can be seen more clearly in horizontal vessels since the primary motion is in cranial-caudal direction

3.1 Accuracy of VM estimation (H1)

The average Euclidean distance between the VM estimation relative to the marker tracking results on the (x,y) grid (estimated via the LMM) was 1.23 mm (95% CI = (0.95, 1.51)). Across animals, the three-point range (minimum, median, maximum) of Euclidean distances was 0.92, 1.08, and 1.87 mm. For the x-coordinate at the individual swine level, the three-point range of MSE was 0.19, 0.53, and 3.63 mm. For the y-coordinate, the three-point range of the MSE was 0.78, 1.24, and 2.37 mm. An example case showing the worst-case scenario is shown in Figure 6, where the reference measurement is degraded due to the stiffness of the catheter.

FIGURE 6

Left: subtraction of end-expiratory and end-inspiratory X-ray image frames with radiopaque marker. The yellow circles denote the maker positions in each frame. Right: subtraction of end-expiratory and end-inspiratory contrast enhanced frames X-ray frames showing the position of the corresponding vessels. The yellow dots showing the corresponding marker positions are displayed outside of the vessel roadmap due to the deformation caused by the catheter stiffness. The reference motion vector (displayed as yellow arrow) shows that the catheter is moving more distal into the vessel during end-inspiratory frames causing an increased lateral component compared to the estimated motion vector (red arrow), which is more indicative of the true VM in this case

3.1.1 Device display accuracy (H2)

Figure 5 shows the wire tip positions from all time frames with and without motion compensation for two navigational sequences. The GLMM fit to the data suggested significant variation between CRD, ICDD, and noncompensated display with respect to the probability of the image containing the wire tip in the vessel roadmap. The odds of the device being shown inside the vessel roadmap were significantly higher for both CRD (odds ratio [OR]) = 3.12, 95% CI = (2.96, 3.30), p <0 .0001) and ICDD (OR = 3.12, 95% CI = (2.96, 3.29), p < 0.0001) compared to the display without motion compensation, where OR was calculated as the ratio between the odds for ICDD/CDD and the noncompensated display. The displays with motion compensation had a higher probability of containing the wire tip in the vessel roadmap in 17 out of 18 combinations of respiratory state and vessel branch level. The median (minimum, maximum) probability of containing the wire tip was 86% (46%, 96%) for CRD, 86% (45%, 96%) for ICDD, and 69% (14%, 96%) without motion compensation, across the 18 combinations. Figure 7 shows the individual results for all combinations of respiratory state and vessel branch level for CRD, ICDD, and nonmotion-compensated displays.

FIGURE 7

Estimated probability of the image containing the wire tip, within 2 pixels, along with 95% confidence intervals, for different respiratory states (end-expiration, end-inspiration, and mid-inspiration) and different vessel branch levels with decreasing vessel diameters (1 = largest, 6 = smallest). Large differences between noncompensated and motion compensated modes were observed especially for end- and mid-inspiration frames

3.1.2 Real-time performance (H3)

The generation time of the prenavigational motion models ranged from 7.71 s (for 18 fill image frames) to 109.55 s (for 81 fill frames). The average processing time per frame to generate the prenavigational model was 0.76 ± 0.35 s. During device navigation, the processing time for the estimation of the respiratory state and the generation of the motion compensated display was 17.23 ± 1.70 ms enabling frame rates of up to 58 fps with a delay approximately equal to the processing time for one image frame (∼17.23 ms).

4 DISCUSSION

Our study evaluated the accuracy and real-time performance of a novel motion compensated device navigation technique, which provides real-time image information on both vessel and intravascular navigation device position during hepatic arterial procedures. This was achieved by generating a VM model based on prenavigational mask and fill image sequences, which was then used to infer VM from a shift of the center of mass in the live images. The accuracy of the estimated VM was compared to the motion of a microcatheter with an opaque marker, where the median error was on the order of the microcatheter diameter (∼1 mm). While the proposed approach provided on average very accurate VM estimation, a few outliers were observed with errors up to 3.63 mm. The two primary sources of error were (i) the stiffness of the catheter and wire with radiopaque marker, which caused deformation of the vasculature and (ii) cases where the catheter and VM differed slightly because the catheter was not physically attached to the surrounding vessel. In the worst-case scenario shown in Figure 6, the marker is pushed further distally into the vessel during the inspiratory phase causing an artificial lateral component of the reference motion and an overestimation of the associated error of the estimated VM. Since the catheter with marker used for motion estimation is much stiffer than the guidewire used for navigation, this effect is expected to be less prominent in the evaluation of the display accuracy but can sometimes cause the wire to be shown slightly outside the vessel. Other potential sources for errors are the segmentation of the wire and the estimation of the respiratory state. However, previous work on deep-learning-based device segmentation has shown highly accurate and robust segmentation results.³¹ For extracting respiratory signal, we employed a simple, easy-to-implement real-time method. Other approaches to signal extraction have been investigated^{4, 11-15, 17-19, 23, 24} and could be applied to the proposed system. However, the smoothness of the estimated respiratory state curves (as shown in Figure 3) suggests that the respiratory state is tracked robustly. While small errors and outliers are unavoidable, the proposed approach generally shows high accuracy and is expected to provide a similar experience as conventional roadmapping in neurointerventional applications in the head, where respiratory motion can be neglected. Two different display modes were evaluated where motion compensation was applied either to the vessel roadmap (CRD) or the endovascular device (ICDD). In both cases, the probability of the wire being shown correctly within the vessel roadmap was significantly higher than without motion compensation. The largest difference was observed for level 3 to level 5 vessel branches, which approximate the primary clinical targets for the delivery of embolic particles. Previous research has shown that the perceived motion of a target is influenced by other moving objects in the visual field and may lead to a bias in the perceived heading direction.^36-38 The ICDD technique provides the advantage of a less distracting display with only the device moving. In practice, a combination of both views, for example, using a side-by-side view of the ICDD and the moving roadmap could be used to retain information on respiratory motion during the procedure. Overall, the motion compensated displays provide three advantages over conventional fluoroscopic image guidance: (1) the display integrates both vessel and device information and does not require the mental integration of two separate displays as it would be the case for side-by-side displays of a static roadmap and fluoroscopy, (2) the approach avoids subtraction artifacts due to respiratory motion, which would be present in uncompensated conventional roadmapping displays, and (3) the device is shown in the correct vessel branch regardless of the respiratory state, whereas conventional static roadmaps only provide correct vessel information during the phase of respiration the roadmap was acquired in.

The processing time for generating the prenavigational motion model was less than 2 min for all cases, which would allow the use of the technique for device navigation in hepatic arterial interventions without interrupting clinical workflow. The processing of the live image data and visualization of the motion compensated display required less than 20 ms, which allows frame rates of up to 58 fps, while the typical frame rate for these procedures in clinical practice is usually at or below 15 fps. The experienced delay using the motion compensated display, which can be approximated by the processing time of one time frame (∼17.23 ms), was well below the experimentally determined threshold for telesurgery (∼300 ms).³⁹ Therefore, the proposed real-time navigation system is not expected to cause hand-eye coordination problems.

Compared to conventional fluoroscopic image guidance, the proposed techniques show both the vasculature and device in a single display, where real-time motion compensation is used to adjust for respiratory motion. Specifically, the proposed static roadmap display with motion compensated device (ICDD) provides a similar experience to conventional roadmapping techniques used for procedures in the head and neck or the extremities, where respiratory motion is absent. The total amount of contrast medium injected during preprocedural imaging (12.5 ml) was comparable to injection protocols for conventional DSA acquisitions in the hepatic arteries (12 to 25 mL) reported in literature.^{40, 41}

The proposed clinical workflow for the proposed system includes an initial setup, where the best C-arm angulation, field of view, and imaging parameters are selected and used throughout the remainder of the procedure. This initial setup is already part of conventional X-ray fluoroscopy guided liver procedures and would not require major changes to existing workflows. While the proposed device navigation system was generally designed to be used with constant imaging parameters throughout the procedure, smaller changes in the FoV due to changes in the table position or zoom level could be compensated for. In these cases, the respiratory state could be recalculated using smaller RoIs, which are visible in both the initial setup scans as well as the live fluoroscopy images after changing imaging parameters. The evaluation of the robustness of the respiratory state suggests average VM estimation errors of less than 4% for RoIs compared to the full FoV. It should be noted that slight changes in the image characteristics due to the divergent nature of cone beam were not included in this evaluation and could affect the accuracy if the table is moved. However, the effects are expected to be minimal for small table movements. Changes in the C-arm angulation would require repeating the initial scans to create a new 2D roadmap and recalculate the motion model. The proposed approach also showed robust respiratory signal extraction despite intraprocedural changes to the dose level. The end-expiration and end-inspiration respiratory states calculated on cine and fluoroscopy quality images differed by less than 5%. Since, cine and fluoroscopy sequences were acquired at different points in time, the difference could also partially be explained by changes in the tidal volume. However, the evaluation provides an upper bound of the expected error due to changes in image quality.

One of the limitations of the proposed technique is that stiff microcatheters or guidewires may cause a deformation of the vessels which is not accounted for in the VM model, and this might cause the device to be displayed outside of the vascular lumen. However, this is also observed in conventional roadmapping techniques without respiratory motion and does not impact the usability of the approach since the device is usually displayed parallel to the correct vessel branch slightly outside of the vascular lumen. Additionally, the navigation to smaller arterial branches often requires more flexible devices which cause less deformation and could further increase the number of frames where the device is shown inside of the vessel roadmap. Another potential limitation is the affine transformation model used to describe VM. While the motion of the larger vessels is generally well represented by the affine model, it is possible that in some cases especially the motion of smaller vessels cannot be completely described. However, more complex transformations, such as motion vector fields, could be easily substituted in the proposed system to allow for a more flexible motion model. Finally, the study was performed using a normal porcine model with continuous ventilation, which does not perfectly replicate, but closely approximates, human hepatic arterial anatomy and respiratory motion.⁴² Minor differences in the accuracy of the motion estimation may be expected for free-breathing or sedated patients. Since the proposed technique does not rely on constant tidal volumes or breathing frequency, the algorithm is expected to be able to compensate for small changes in the respiratory cycle. However, large changes related to the type of breathing (abdominal vs chest breathing) may cause errors in the VM estimation primarily in the lateral and anteroposterior direction. SI motion is generally well described by changes in the diaphragm position used for respiratory motion tracking. In practice, the risk of large changes in breathing during the procedure could be reduced using verbal coaching.⁴³

5 CONCLUSIONS

In conclusion, our study shows that the proposed respiratory motion compensated device guidance system allows for simultaneous visualization of the vasculature and device, where the device location relative to the vascular roadmap remains accurate in the presence of respiratory motion. Furthermore, the required processing times allow running the system at clinically relevant frame rates with delays considerably lower than previously reported thresholds for real-time applications. If translated to clinical practice, this may reduce operator-associated errors in device navigation and decrease procedure times.

ACKNOWLEDGMENTS

Research reported in this publication was supported by the National Institute of Biomedical Imaging and Bioengineering under award number R21EB024553. Funding was also received from the National Cancer Institute under award number F30CA250408, from the National Institute of General Medical Sciences under award number T32GM140935 and the Clinical and Translational Science Award (CTSA) program, through the NIH National Center for Advancing Translational Sciences (NCATS), grant UL1TR002373. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The concepts presented in this paper are based on research and are not commercially available.

CONFLICT OF INTEREST

The authors have no relevant conflict of interest to disclose.

The data that support the findings of this study are available from the corresponding author upon reasonable request.