Nonrigid registration of medical image based on adaptive local structure tensor and normalized mutual information

Abstract Nonrigid registration of medical images is especially critical in clinical treatment. Mutual information is a popular similarity measure for medical image registration; however, only the intensity statistical characteristics of the global consistency of image are considered in MI, and the spatial information is ignored. In this paper, a novel intensity‐based similarity measure combining normalized mutual information with spatial information for nonrigid medical image registration is proposed. The different parameters of Gaussian filtering are defined according to the regional variance, the adaptive Gaussian filtering is introduced into the local structure tensor. Then, the obtained adaptive local structure tensor is used to extract the spatial information and define the weighting function. Finally, normalized mutual information is distributed to each pixel, and the discrete normalized mutual information is multiplied with a weighting term to obtain a new measure. The novel measure fully considers the spatial information of the image neighborhood, gives the location of the strong spatial information a larger weight, and the registration of the strong gradient regions has a priority over the small gradient regions. The simulated brain image with single‐modality and multimodality are used for registration validation experiments. The results show that the new similarity measure improves the registration accuracy and robustness compared with the classical registration algorithm, reduces the risk of falling into local extremes during the registration process.

significant advantages in reflecting soft tissue information, but is not sensitive to calcified regions and subject to geometric distortion due to magnetic interference. PET can clearly observe the metabolism of various organs, but due to the low pixel resolution, the structure and regional contour of the organs cannot be clearly found. Due to the different imaging principles of various medical devices, the medical image information is greatly different. The single-modality image provides unilateral information for the doctor's clinical diagnosis, but in order to obtain more complete and complementary image information, it is necessary to fuse various types of information so that doctors can make more accurate and reliable diagnosis.
Currently, the similarity measures of nonrigid medical image registration are mainly divided into feature-based method and intensitybased method 1 ; the former usually uses some points, lines, area, and edges of the image as feature information, the registration method has advantages of small computational cost and high efficiency, but is not easy to calculate automatically, the image registration effect is also easily affected by factors such as the level of the operator and the accuracy of feature point selection. The intensity-based method directly uses the intensity information of image, which avoids the error in image registration caused by the feature extraction process.
For simplicity, intensity-based similarity measures are typically used instead of feature-based similarity measures. In the field of intensitybased method, similarity measures can be divided into two categories: one method directly uses the intensity of the image pixel which contains the sum of squared distances 2 (SSD) and the gradient difference 3 (GD). Another method which based on information theory mainly considers the intensity distribution of pixel points and uses statistical entropy strategy. Mutual information 4 (MI) and normalized mutual information 5 (NMI) belong to this category. Among them, the method based on the mutual information is the most widely used, but it only considers the intensity distribution of image, ignoring the spatial and geometric information of image completely.
Pluim et al. 6 proposed a similarity measure called gradient mutual information (GMI) by combining MI with gradient, it improved robustness of registration. It is not an extended measure of MI, but just the addition of the multiplicative factor todescribe the neighborhood information. Russakoff 7 proposed the regional mutual information (RMI), which calculates the MI of two images in the local range, can reflect the intensity distribution of local image and obtain better robustness than traditional MI similarity measures. However, it needs to calculate the entropy of probability distribution with higher dimensional. Loeckx 8 proposed the conditional mutual information similarity measures, which uses spatial information as an additional channel to calculate the joint probability distribution. The algorithm improves the registration accuracy, but the corresponding anatomical structures are further spatially separated, some structural features are ignored. Rivaz 9 proposed α-mutual information called self-similarity α-MI (SeSaMI), which makes full use of the local feature structure of the image to enhance the robustness of MI. Luan 10 proposed the qualitative measure of mutual information, which adds the utility coefficient to the traditional calculation of MI, the calculation process is complicated and difficult to practical application. Hossny 11 proposed a local structural mutual information registration method which divides the image space into multiple spaces to estimate MI independently, and uses the weighted sum of MI as the similarity measure, local structural similarity index is preserved. Wang Jun 12 proposed a B-spline and regional mutual information (BRMI) registration method. The image is regarded as the distribution of multidimensional points, each point represents a pixel and its neighboring point pixels, by calculating the information entropy of multidimensional points, the local mutual information of the two images is obtained. The method effectively improves the accuracy of registration but the efficiency is reduced.
Qu Jiahui 13 proposed a hyperspectral image fusion algorithm using structure tensor, which introduces structure tensor to extract the spatial details of enhanced PAN images. Experiments show that the spatial information extracted by the horizontal gradient and vertical gradient only retains the edge information of the original image, but the spatial information obtained by the structural tensor method contains a large number of edges and structural information. James M. Sloan 14 describes a novel structural image descriptor for image registration called the fractionally anisotropic structural tensor representation (FASTR). It does not depend on voxel intensities absolutely, and is insensitive to the image which has a slowly varying intensity inhomogeneity. The results show that FASTR would produce more accurate results than MI towards the images with distinct intensity inhomogeneity. However, the proposed similarity measure can only be used for rigid medical image registration with simple deformations such as translation, rotation or scaling. In fact, because medical images have local correspondence missing and complex nonlinear deformations, as well as the irregular physiological movements of the organs, nonrigid registration is necessary and can fully describe the spatial relationship between images. In this paper, a new similarity measure based on adaptive local structure tensor and NMI is proposed, in which spatial information, geometric information and mutual information are combined to improve the similarity measure. The nonrigid registration with large deformation is considered.
The contributions of this paper have following four aspects: 1. In order to reflect the local structure information, adaptive Gaussian filtering is introduced into the local structure tensor, and the parameters of Gaussian kernel function are defined according to the regional variance of image, which can better protect the image details.
2. Discrete NMI is defined according to the contribution of pixel points to the total similarity measure, which is beneficial to the combination with spatial information.
3. The spatial information extracted from the adaptive local structure tensor is used to customize the weighting function. It multiplies with the discrete NMI to obtain a new measure function called adaptive local structure tensor-normalized mutual information (ALST-NMI).
4. The algorithm in this paper is used to register the brain images with single-modality and multimodality, the accuracy and effectiveness of registration are both improved.

2.A | The registration framework
Medical image registration mainly includes four modules: transformation model, similarity measure, optimization algorithm, and interpolation algorithm. First, the appropriate transformation model is selected according to the specific application, and determines the spatial transformation method of the floating image. Second, a similarity measure is defined to measure the degree of similarity between the reference image and floating image after transformation. In this way, it is judged whether two images have been correctly matched. Third, an interpolation algorithm is used to assign the intensity value to pixels in the image, and the current similarity measure is obtained by comparison with the reference image to determine whether the next round of optimization is needed to update the parameters. Finally, a specific optimization algorithm is used to search for the best transformation result continuously until the similarity of the two images is maximized.
The flow diagram is shown in Fig. 1. In this paper, B-spline is selected as the transformation model, ALST-NMI is the similarity measure, and the steepest gradient descent method is used as the optimization algorithm for the registration experiment. In this paper, the cubic B-spline is selected as the transformation model. The two-dimensional (2D) cubic B-spline transformation can be expressed as follows: the next rounding, and B m (u) represents the m-th cubic B-spline basis function.

2.B.2 | Optimization algorithm
The purpose of image registration is to find a spatial transformation to make corresponding points of the different images reach the same spatial position and anatomical position. The registration problem is transformed into the optimal solution of the cost function, then the optimal transformation parameters are found to minimize the cost function.

Reference image
where the cost Φ function is a negative ALST-NMI similarity measure function, g is a nonrigid transformation function, and the symbol represents a compound operation of f F and function g.
The optimization strategy refers to the search optimization process in which the spatial transformation parameters are continuously adjusted in the image registration, so that the similarity measure is maximized and the images are aligned as much as possible. An applicable optimization strategy can not only improve the computational efficiency of the algorithm, but also obtain more accurate optimization results. Therefore, the steepest gradient descent method is used to iterative update the parameter value along the direction of the gradient descent, as shown in the eq. (4).
where a k is the search step size and rΦ(μ k ) is the gradient of the cost function.

2.C | ALST-NMI similarity measure
Normalized mutual information measure is a commonly used similarity measure, which can accurately represent the similarity among intensity images. It can also effectively solves the problem that overlap region in the image affects the registration accuracy in MI registration process, and ensure the effectiveness. However, the registration algorithm based on the NMI does not consider the spatial information of the image, resulting in low accuracy even misleading registration. Gradient can effectively describe local structure information and is used to estimate local geometry information of image widely 15 . However, the gradient is sensitive to noise, and the positive and negative gradients are similar on both sides of the edge in the image, which will causes the gradient information to counteract in the smoothing algorithm. The local structure tensor does not produce the problem abovementioned, and can also still be extracted under the condition of the local gradient loss. At the same time, it [16][17][18] directly uses the image intensity matrix to perform operations, which can effectively preserve the structural and gradient information of the image pixel, provide a more meaningful description than the gradient information. Therefore, this paper proposes a new similarity measure which combines NMI and local structure tensor.

2.C.1 | Discrete normalized mutual information
The NMI can be expressed as: NMI ¼ HðRÞ þ HðFÞ HðR; FÞ where R and F represent the reference image and floating image namely, H(R) and H(F) denote the information entropy of R and F, and H(R,F) is the joint information entropy of two images.
Normalized mutual information is distributed to each pixel, discrete NMI can be defined by the contribution of pixel points to the total similarity measure: HðR; FÞ (6) where N 1 and N 2 is the number of pixels accumulatively used in the reference image and floating image, x represents the position of pixel in f R and f F , and P R (x) and P F (x) are the marginal probabilities of the x.

2.C.2 | Adaptive local structure tensor
The local structure tensor of each point is shown in eq. (7): In order to protect the image details and denoise, the adaptive Gaussian filtering is introduced into local structure tensor. First, the local structure is judged as a consistent region or a region based on edge and corner. If it belongs to the region based on edge and corner, the degree of dispersion in the region is relatively large, the variance of the corresponding pixel value tends to be large; if it belongs to the consistent region, the degree of dispersion in the region is relatively small, and the variance of the corresponding pixel value is small. Therefore, we calculate the regional variance, first, then the variance of Gaussian kernel function and the Gaussian template are adaptively selected according to the regional variance. The larger the regional variance is, the Gaussian kernel function with smaller variance and smaller Gaussian template is selected; the smaller the regional variance is, the Gaussian kernel function with larger variance and larger Gaussian template is selected.
The formula for regional variance is: where D is the regional variance and x ij represents the point in the S region 2.C.3 | ALST-NMI similarity measure The singular value decomposition is performed on local structure tensor to obtain non-negative eigenvalues which is denoted as λ 1 , λ 2 (λ 1 ≥ λ 2 ≥ 0); λ 1 and λ 2 reflect the value of energy change in the direction of its corresponding feature vector. In the consistent region of the image, the intensity value changes a little or almost unchanges, that is, λ 1 ≈ λ 2 ≈ 0; in the boundary region, the intensity value across the edge varies greatly, that is, λ 1 ≥ λ 2 ≈ 0; the intensity value varies in all directions at the corners, that is,

| 103
In order to measure the role of each pixel in the image geometry, the following local structure descriptors are extracted from the local structure tensor, the trace C 1 = λ 1 + λ 2 of the local structure tensor is used to describe the strength of the local variation, C 2 = (λ 1 − λ 2 ) 2 is defined to characterize the consistency of the local structure, and the scale vector C(x) =[C 1 C 2 ] is defined to represent the structural information adjacent to the pixel, which has better image structure expression ability. Then the weighting function is defined as follows: where θ is a constant, BðxÞ ¼ 1 þ ð rf R ðxÞ j jrf F ðxÞ j j Þ 1=4 , rf R (x) and rf F (x), respectively, represent gradient vectors of the reference image and the floating image at x pixel points.
The choice of B(x) mainly considers the balance between the strong gradient region and the small gradient region, it has a large value at the strong gradient position and a relatively small value at the small gradient position. The final similarity measure can be expressed as:  MSE, mean-square error; SSIM, structural similarity index; CC, correlation coefficient; NMI, normalized mutual information; NJ, negative Jacobian; GD, gradient difference; SSD, sum of squared distances; BRMI, B-spline and regional mutual information; ALST-NMI, adaptive local structure tensor-normalized mutual information.

3.A | Single-modality experiment and results
In order to validate the robustness and accuracy of ALST-NMI measure, the method is applied for nonrigid medical image registration and compared with SSD, NMI, BRMI and LST-NMI measure. All the experiments are conducted on a set of real brain images from the brain web with size of 353 × 354. The reference image is shown in Fig. 2(a), the floating image is shown in Fig. 2(b) and 2(b) is obtained by forming the

3.A.1 | Registration performance evaluation
In order to objectively evaluate the results of registration, this paper uses mean square error, structural similarity index, NMI, and correlation coefficient to quantitatively evaluate the performance of registration.

Mean square error
where R(i, j) and F(i, j) represents pixel in the reference image and the floating image, respectively, m × n represent the resolution of the image.

Structural similarity index
where μ R, μ F, σ R , σ F , and σ RF represents the mean, variance, and covariance of the images R and F, respectively. C 1 = (k 1 L) 2 and (c) normalized mutual information; (d) B-spline and regional mutual information; (e) local structure tensor-normalized mutual information; (f) adaptive local structure tensor-normalized mutual information. C 2 = (k 2 L) 2 are constants used to maintain stability, and L is the dynamic range of the pixel value, k 1 = 0.01, k 2 = 0.03.

Mutual information
MIðR; FÞ ¼ HðRÞ þ HðFÞ À HðR; FÞ (13) where H(R) and H(F), respectively, denote the information entropy of reference image R and float image F, and H(R,F) is the joint entropy of two images.

Correlation coefficient
where Rði; jÞ and Fði; jÞ represents the mean of pixel in the reference image and the floating image, namely.

Jacobian determinant
Whether the displacement vector field (DVF) has the ability to keep the image topology unchanged can be measured by Jacobian determinant of the DVF. The value of the Jacobian determinant is larger than zero, indicating that the DVF has the ability to maintain the topology unchanged, the percentage of pixels with negative Jacobian determinant values is expressed as NJ (negative Jacobian).
Equation (15)   Compared with the B-spline + NMI, the mean square error decreases by 84.87%, the structural similarity increased by 11.6%, the NMI increased by 11.13%, and the correlation coefficient increased by 5.77%, the accuracy of image registration is improved, and the effectiveness of the ALST-NMI measure is proved. Figure 4 shows the histogram of the displacement error distribution of the single-modality registration. Compared with other methods, the registration accuracy of the proposed method is significantly improved, 94.7% of the pixels have been effectively registered, and the percentage of displacement errors of each length in the pixel points has decreased. There is no excessive displacement error, and the maximum error is only 0.6 cm, the error larger than 0.2 cm is less than 0.4% of the total pixels.

3.B | Multimodality experiment and results
In order to validate the registration results of ALST-NMI measure on multimodality images, MRI and CT brain images were selected for   (Fig. 6). Figure 7 shows the histogram of the displacement error distribution of the multimodality registration. Compared with the GD method, the ALST-NMI measure has obvious improvement for large displacement error, the pixels with error of 1 cm decreases from 2.88% to 0.22%. Compared with the LST-NMI method, the mean error is significantly reduced, which proves the validity of adaptive local structure tensor. 66.75% of the pixels have been effectively registered, and the error larger than 0.5 cm is less than 2% of the total pixels.

| CONCLUSION
In this paper, a similarity measure based on adaptive local structure tensor and NMI is proposed for nonrigid medical image registration with large deformation, in which intensity information and spatial information are both considered. ALST-NMI similarity measure uses a weighting function to balance the registration of strong gradient regions and small gradient regions, the experiments show that the Bspline + ALST-NMI can effectively improve the image registration accuracy. However, since the NMI and the local structure tensor of the discrete points need to traverse each pixel, the registration efficiency is reduced to a certain extent. It is necessary that how to achieve higher registration precision and computational efficiency simultaneously. Research of Abnormal Grain Conditions Detection using Radio

ACKNOWLEDG MENTS
Tomographic Imaging based on Channel State Information.

CONFLI CT OF INTEREST
The authors declare no conflict of interest.