Recent Patents in Signal Processing (February 2016) – Image registration

For our February 2016 issue, we cover recent patents dealing with image registration. The section below covers patents granted recently for various new image registration algorithms and devices, including applications of video registration for heads up displays, 2D/3D image registration and medical image analysis.

The disclosure no 9,247,133 pertains to devices, methods, and computer readable media for performing image registration. A few generalized steps may be used to carry out the image registration techniques described herein: 1) acquiring image data from an image sensor; 2) selecting a pair of overlapping image portions from the acquired image data for registration; 3) determining an area of "maximum energy" in one of the image portions being registered; 4) placing an image registration window over both image portions at the determined location of maximum energy; 5) registering the overlapping image portions using only the image data falling within the image registration windows; and 6) determining, according to one or more metrics, whether the image registration window should be shifted from a current location before registering subsequently acquired image portions.

The method introduced in patent no. 9,245,336 includes obtaining first volumetric image data, which is acquired at a first time, including a region of interest with a structural feature located at a first position. The method further includes obtaining second volumetric image data, which is acquired at a second different time, including the region of interest with the structural feature located at a second different position. The method further includes determining a registration transformation that registers the first and second volumetric image data such that the at least one structural feature in the first volumetric image data aligns with the at least one structural feature in the second volumetric image data. The registration transformation is based at least on a contour guided deformation registration. The method further includes generating a signal indicative of the registration transformation.

The invention no. 9,241,657 relates to a medical image registration method in which an actual patient's body or portion of a patient's body (22) is positionally registered to a patient image created using a medical body-imaging technique such as MR, CT or x-ray imaging, wherein in a medical navigation system, the actual spatial position of at least a part of the body or body portion (22) is assigned to a respective position in the image of said body or body portion (22), wherein the part of the body or body portion (22) comprises a registration area in the form of the surface of a substantially rigid body structure, in particular an inner structure, made of bone or cartilage. The invention also relates to a medical image registration device which comprises a surface scanner (15) for scanning a registration area comprising the surface of a substantially rigid body structure, in particular an inner structure, made of bone or cartilage.

Embodiments of the invention no. 9,240,046 provide a system and method that is able to automatically provide a starting point for 2D to 3D image registration, without relying on human recognition of features shown in the 2D image. This is achieved by pre-processing the 3D data to obtain synthetically generated 2D images of those parts of the 3D data volume which will be used for registration purposes. Many different synthetically generated 2D images of the or each part of the 3D volume are produced, each from a different possible viewing direction. Each of these synthetic images is then subject to a feature extraction process to extract characterizing feature data of the registration feature shown in the images. Once the feature extraction has been undertaken for each image, when registration is to be performed the real-time 2D image is processed by applying each of the sets of extracted features thereto, to try and identify which set best matches the registration features in the 2D image. For example, where a generalized Hough transform was used in the feature extraction, the R tables would be applied to the 2D image to obtain respective accumulation images. The accumulation images may then be ranked to identify which registration feature is shown in the 2-D image, and from which view direction. This gives the required information of which registration feature is being shown in the 2D image, and also the in-plane location and orientation. This information can then be used as a starting point for the 2D to 3D registration procedure

Patent no 9,229,230 proposes a system where video sources and inertial sensors are attached to a weapon and to goggles. A computer receives video images from the weapon- and goggles-mounted sources and inertial data from the sensors. The computer calculates a location for an image from the weapon-mounted source within an image from the goggles-mounted source using the inertial sensor data. The sensor-based location is checked (and possibly adjusted) based on a comparison of the images. A database contains information about real-world objects in a field of view of the goggles-mounted source, and is used to generate icons or other graphics concerning such objects.

The invention no. 9,214,009 provides an image registration device and method, an image segmentation device and method and a medical image apparatus. The image registration device includes: a rigid registration unit for performing a rigid registration of a first input image with a second input image, which is a reference, to obtain first deformation information; a first non-rigid registration unit for performing a first non-rigid registration of the first and second input images by taking the first deformation information as the initial deformation information to obtain second deformation information; and a second non-rigid registration unit for performing a second non-rigid registration of the first and second input images by taking the second deformation information as the initial deformation information to obtain third deformation information. By combining rigid registration with non-rigid registration, the present invention realizes a fast image registration.

Patent no. 9,208,560 proposes a method for registering images into a coordinate system, which includes the following steps: (a) performing image normalization on a source image and generating a normalized image; (b) retrieving at least one color-deconvoluted image from color-deconvoluting the source image; (c) determining at least one image feature from the at least one color-deconvoluted image; (d) comparing the at least one image feature of the at least one color-deconvoluted image with a target image, and generating a relative matching structural feature result; and (e) transforming the normalized image into a registered image according to the matching structural feature result.

In patent no. 9,208,560 a method of 3D-2D registration for medical imaging is proposed which includes the following steps: providing a first input interface for acquiring a three-dimensional image; providing a second input interface for acquiring a fixed two-dimensional image using an imaging system that includes a source and a detector and that has an unknown source-detector geometry; initializing image transformation parameters and source-detector geometry parameters; generating a reconstructed two-dimensional image from the three-dimensional image using the image transformation parameters and the source-detector geometry parameters; determining an image similarity metric between the fixed two-dimensional image and the reconstructed two-dimensional image; and updating the image transformation parameters and the source-detector geometry parameters using the image similarity metric, and a corresponding non-transitory computer-readable medium and apparatus.

An image registration apparatus from patent no 9,135,706 comprises: a features detector (34) configured to extract a two-dimensional set of features (36) from a two-dimensional image (30) and to extract a three-dimensional set of features (38) from a three-dimensional image (32); a projection processor (40) configured to project three-dimensional data into two-dimensional projection data; and a registration processor (46, 52) configured to (i) adjust parameters to register the two-dimensional set of features and the three-dimensional set of features projected by the projection processor using a projection geometry (42), and to (ii) use the adjusted parameters to register the two-dimensional image and the three-dimensional image projected by the projection processor using the projection geometry.

If you have an interesting patent to share when we next feature patents related to image registration, or if you are especially interested in a signal processing research field that you would want to be highlighted in this section, please send email to Csaba Benedek (benedek.csaba AT sztaki DOT mta DOT hu).

References

Number: 9,247,133
Title:   Image registration using sliding registration windows
Inventors: Doepke; Frank (San Jose, CA), Sachs; Todd S. (Palo Alto, CA), Hunter; Kevin L. (Sunnyvale, CA), Gatt; Alexis (Santa Clara, CA)
Issued:  January 26, 2016
Assignee: Apple Inc. (Cupertino, CA)

Number: 9,245,336
Title: Contour guided deformable image registration
Inventors: Mallya; Yogish (Bangalore, IN), Poston; Timothy (Bangalore, IN)
Issued:  January 26, 2016
Assignee: KONINKLIJKE PHILIPS N.V. (Eindhoven, NL)

Number: 9,241,657
Title: Medical image registration using a rigid inner body surface
Inventors: Vollmer; Fritz (Munich, DE), Hoever; Manuel (Munich, DE)
Issued:  January 26, 2016
Assignee: Brainlab AG (Feldkirchen, DE)

Number: 9,240,046
Title: Method and system to assist 2D-3D image registration
Inventors: Carrell; Tom (London, GB), Varnavas; Andreas (London, GB), Penney; Graeme (London, GB)
Issued:  January 19, 2016
Assignee: Cydar Limited (Cambridgeshire, GB)

Number: 9,229,230
Title: System and method for video image registration and/or providing supplemental data in a heads up display
Inventors: Scales; John Richard (Huntsville, AL), Rodgers; Michael Harris (Huntsville, AL)
Issued:  January 5, 2016
Assignee: Science Applications International Corporation (McLean, VA)

Number: 9,214,009
Title: Image registration device and method, image segmentation device and method and medical image apparatus
Inventors:  Wan; Jun (Beijing, CN), Wang; Yanli (Beijing, CN), Wang; Shaobin (Beijing, CN), Sugiyama; Atsuko (Otawara, JP)
Issued:  December 15, 2015
Assignee: Kabushiki Kaisha Toshiba (Minato-ku, JP), Toshiba Medical Systems Corporation (Otawara-shi, JP)

Number: 9,208,560
Title: Image registration method
Inventors:  Wang; Ching-Wei (Taipei, TW)
Issued:  December 8, 2015
Assignee: National Taiwan University of Science and Technology (Taipei, TW)

Number: 9,165,362
Title: 3D-2D image registration for medical imaging
Inventors: Siewerdsen; Jeffrey (Baltimore, MD), Otake; Yoshito (Baltimore, MD), Wolinsky; Jean-Paul (Towson, MD)
Issued:  October 20, 2015
Assignee: The Johns Hopkins University (Baltimore, MD)

Number: 9,135,706
Title: Features-based 2D-3D image registration
Inventors: Zagorchev; Lyubomir G. (Lebanon, NH), Manzke; Robert M. (Cambridge, MA), Chan; Raymond C. (Brookline, MA)
Issued:  September 15, 2015
Assignee: Koninklijke Philips N.V. (Eindhoven, NL)