Recent Patents in Signal Processing (April 2017) – Image Fusion

For our April 2017 issue, we cover recent patents dealing with image fusion, including new general image fusion methods, and specific applications such as image stabilization, and medical image analysis.

In patent no 9,558,543 an image fusion method and an image processing apparatus are provided. A first image is generated based on a first photographing parameter, and a second image is generated based on a second photographing parameter. A first pixel reference value of each of first pixels is calculated by using a self-define mask according to color components and a luminance component of the first pixels on the first image. A second pixel reference value of each of second pixels is calculated by using the self-define mask according to color components and a luminance component of the second pixels on the second image. A synthesizing reference map recording a plurality of synthesizing weights is obtained by comparing the first pixel reference value and the corresponding second pixel reference value. A fusion image is obtained by synthesizing the first image and the second image according to the synthesizing reference map.

Embodiments of the invention no. 9,501,852 provide a method and apparatus for image fusion. A method for image fusion according to the embodiments of the present invention comprises: obtaining multiple images for a same scene; and calculating a fused image of the multiple images based on Bayes analysis by using a kernel function.

Patent no. 9,478,028 introduces systems and methods for image registration, which include an image feature detection module (116) configured to identify internal landmarks of a first image (110). An image registration and transformation module (118) is configured to compute a registration transformation, using a processor, to register a second image (112) with the first image based on surface landmarks to result in a registered image. A landmark identification module (120) is configured to overlay the internal landmarks onto the second image using the registration transformation, encompass each of the overlaid landmarks within a virtual object to identify corresponding landmark pairs in the registered image, and register the second image with the first image using the registered image with the identified landmarks.

An image fusion system and method are provided in patent no. 9,369,612. The system includes a plurality of cameras configured to generate a plurality of images, respectively, and an image fusion unit configured to fuse the plurality of images into a single image.

In the embodiment no. 9,285,309, provided is an image fusion apparatus using a multi-spectral filter array sensor, the image fusion apparatus including a separation unit configured to separate an input image input through the multi-spectral filter array sensor into an RGB image and a near infrared (NIR) image, a color reconstruction unit configured to extract an RGB image in a visible band by removing an NIR value incident together with R, G and B values through the multi-spectral filter array sensor from the R, G and B values, and an image fusion unit configured to fuse the RGB image in the visible band with the NIR image, in which a fusion rate of the RGB image and the NIR image is based on a size of each of the R, G and B values of the RGB image in the visible band.

As introduced in the invention no. 9,269,140, a system can include a model to represent a volumetric deformation of a brain corresponding to brain tissue that has been displaced by at least one of disease, surgery or anatomical changes. A fusion engine can perform a coarse and/or fine fusion to align a first image of the brain with respect to a second image of the brain after a region of the brain has been displaced and to employ the deformation model to adjust one or more points on a displacement vector extending through a displaced region of the brain to compensate for spatial deformations that occur between the first and second image of the brain.

The hyperspectral detector systems and methods disclosed in patent no. 9,269,014, which include capturing a context image and a single-column spectral image that falls within the context image. The context and spectral images are then combined to form a fused image. Using the fused image, the spectral image is panned over the scene and within the context image to capture spectral signatures within the scene. The spectral signatures are compared to reference spectral signatures, and the locations of the one or more spectral signatures within the context image are marked. The systems and methods obviate the need to store and process large amounts of spectral data and allow for real-time display of the fused context image and spectral image, along with the marked locations of matched spectral signatures.

Systems, methods, and computer readable media to improve image stabilization operations are described in patent no. 9,262,684. Novel approaches for fusing non-reference images with a pre-selected reference frame in a set of commonly captured images are disclosed. The fusing approach may use a soft transition by using a weighted average for ghost/non-ghost pixels to avoid sudden transition between neighborhood and almost similar pixels. Additionally, the ghost/non-ghost decision can be made based on a set of neighboring pixels rather than independently for each pixel. An alternative approach may involve performing a multi-resolution decomposition of all the captured images, using temporal fusion, spatio-temporal fusion, or combinations thereof, at each level and combining the different levels to generate an output image.

If you have an interesting patent to share when we next feature patents related to image fusion, or if you are especially interested in a signal processing research field that you would want to be highlighted in this section, please send email to Csaba Benedek (benedek.csaba AT sztaki DOT mta DOT hu).

References

Number: 9,558,543
Title: Image fusion method and image processing apparatus
Inventors: Hsieh; Chen-Chiung (Taipei, TW), Wu; Wen-Hao (Taipei, TW)
Issued: January 31, 2017
Assignee: Tatung University (Taipei, TW), Tatung Company (Taipei, TW)

Number: 9,501,852
Title: Method and apparatus for image fusion
Inventors: Jiang; An (Beijing, CN), Cui; Feng (Beijing, CN), Xie; Qiwei (Beijing, CN)  
Issued: November 22, 2016
Assignee: Nokia Technologies Oy (Espoo, FI)

Number: 9,478,028
Title: Intelligent landmark selection to improve registration accuracy in multimodal image fusion
Inventors: Parthasarathy; Vijay (Tarrytown, NY), Lin; Muqing (Irvine, CA), Kruecker; Jochen (Washington, DC)
Issued: October 25, 2016
Assignee: Koninklijke Philips N.V. (Eindhoven, NL)

Number: 9,369,612
Title: Image fusion system and method
Inventors: Oh; Jae Yoon (Changwon-si, KR), Lee; Joon Sung (Changwon-si, KR), Park; Young Seo (Changwon-si, KR)
Issued: June 14, 2016
Assignee: Hanwha Techwin Co., Ltd. (Changwon-si, KR)

Number: 9,285,309
Title: Image fusion method and apparatus using multi-spectral filter array sensor
Inventors: Choi; Eun-Cheol (Changwon, KR)
Issued: March 15, 2016
Assignee: Hanwha Techwin Co., Ltd. (Changwon-si, KR)

Number: 9,269,140
Title: Image fusion with automated compensation for brain deformation
Inventors: Machado; Andre G. (Beachwood, OH)
Issued: February 23, 2016
Assignee: The Cleveland Clinic Foundation (Cleveland, OH)

Number: 9,269,014
Title: Hyperspectral detector systems and methods using context-image fusion
Inventors: Comstock, II; Lovell Elgin (Charlestown, NH), Desmarais; Leon J (Claremont, NH), Santman; Jeffry John (Keene, NH), Wigley; Peter Gerard (Corning, NY), Woodman; Patrick W (Marlborough, NH)
Issued: February 23, 2016
Assignee: Corning Incorporated (Corning, NY)

Number: 9,262,684
Title: Methods of image fusion for image stabilization
Inventors:  Tico; Marius (Mountain View, CA), Zhou; Jianping (Fremont, CA), Nariani Schulze; Anita (Los Altos, CA), Toft; Rolf (Palo Alto, CA), Hubel; Paul (Mountain View, CA), Sun; Wei (San Jose, CA)
Issued: February 16, 2016
Assignee: Apple Inc. (Cupertino, CA)