Recent Patents in Signal Processing (April 2016) – range image formation and processing

For our April 2016 issue, we cover recent patents dealing with range image formation and processing.

The disclosure no 9,241,127 intorduces an image processing method which includes the operations of receiving N.times.N raw data from an N.times.N sub pixel array including a plurality of first exposure pixels and second exposure pixels having different exposure times, respectively; obtaining a difference value between a value obtained by normalizing an interpolated value of the first exposure pixels and an interpolated value of the second exposure pixels for each channel in the N.times.N sub pixel array based on the N.times.N raw data; setting a mismatching value to a maximum value among difference values obtained for the respective channels; and setting a corrected value of a central pixel in the N.times.N sub pixel array based on the N.times.N raw data and based on the mismatching value and a threshold value.

In patent no. 9,204,113 an apparatus is presented comprising a scaling circuit, a luma circuit and a blending circuit. The scaling circuit may generate a plurality of scaled frames in response to a first plurality of frames generated by a sensor. The first plurality of frames may have a first exposure. The luma circuit may generate an average luminance value for each of a plurality of processed pixels in each of a second of the plurality of frames generated by the sensor. The second of the plurality of frames may have a second exposure and each of the average luminance values is calculated based on a plurality of neighboring pixels in a neighborhood of the processed pixel.

A method of patent no 9,167,167 is disclosed of a heterogeneous image sensor array comprising more than one image sensor structures that are interconnected. The final image sensor array image output for each image sensor structure or pixel is computed using single image sensor structure output data or output data from more than one image sensor structures. The heterogeneous array exhibits complexity, cost, power consumption, device yields and reliability benefits when compared to other image sensor array structures.

Patent no. 9,153,027 introduces a system, method, and computer program product are provided for performing fast, non-rigid registration for at least two images of a high-dynamic range image stack. The method includes the steps of generating a warped image based on a set of corresponding pixels, analyzing the warped image to detect unreliable pixels in the warped image, and generating a corrected pixel value for each unreliable pixel in the warped image. The set of corresponding pixels includes a plurality of pixels in a source image, each pixel in the plurality of pixels associated with a potential feature in the source image and paired with a corresponding pixel in a reference image that substantially matches the pixel in the source image.

In the invention no. 9,131,160 we get method for controlling an exposure duration of a high dynamic range image, including: consecutively generating a first high dynamic range image having a first exposure duration ratio and a second high dynamic range image having a second exposure duration ratio greater than the first exposure duration ratio; performing image quality evaluations on both the first high dynamic range image and the second high dynamic range image to obtain a first image quality and a second image quality the first exposure duration ratio and setting the first exposure duration ration as the optim, respectively; and determining whether the second image quality is better than the first image quality; if yes, generating a third high dynamic range image having a third exposure duration ratio greater than the second exposure duration ratio; if not, generating the third high dynamic range image having al exposure duration ratio.

According to patent no. 9,100,589 a computing device may capture a first image using a first exposure length. It may be determined that x pixels in the captured first image are properly-exposed and that y pixels in the captured first image are improperly-exposed. Based on the y improperly-exposed pixels, a second exposure length may be determined. A second image may be captured using the second exposure length. The second image may include y' pixels that correspond to the y improperly-exposed pixels in the first image. At least some of the x properly-exposed pixels from the first image may be combined with at least some of the y' pixels from the second image to form a composite image.

Following patent no. 9,064,313 technologies may provide for yielding a low dynamic range image. A logic architecture may be implemented to perform a global tone map operation on a high dynamic range image to yield a preview image. The logic architecture may also be implemented to perform a local tone map operation on a region of interest of a preview image to yield a low dynamic range image. Additionally, the logic architecture may be implemented to provide an image that includes a predetermined amount of image pixels that are not saturated. In addition, the logic architecture may be implemented to employ image view data to yield a low dynamic range image. Moreover, the logic architecture may be implemented to convert the region of interest into a region that includes the foveal region of a user. In one example, the tone map operation is performed entirely automatically and interactively.

Invention no. 9,025,862 introduces a method for matching the pixels (10-1, 10-2) of a first range image of a scene (18) as seen from a first point of sight (14) with pixels (12-1, 12-2) of a second range image of the scene as seen from a second point of sight (16) comprises the following steps: providing the first range image as a grid of source pixels (10), on which the scene is mapped in accordance with a first projection associated with the first point of sight, wherein each source pixel has a point in the scene projected thereon in accordance with the first projection and has associated therewith a range value determined for that point in the scene; providing a grid of target pixels (12) for the second range image and a second projection associated with the second point of sight; and for each one of the target pixels, a) determining which source pixel would have the same point (P.sub.1, P.sub.2) in the scene projected thereon in accordance with the first projection as the target pixel would have projected thereon in accordance with the second projection if the imaged scene were a planar surface at a certain surface distance from the first point of sight; b) determining a depth coordinate of the point in the scene that the source pixel determined in step a) actually has projected thereon in accordance with the first projection; c) if the depth coordinate is greater than a threshold, which is itself greater than the surface distance, repeating steps a), b) and c) with an increased surface distance at step a) and an increased threshold at step c), and else associating the target pixel with the source pixel determined in step a).

In patent no. 8,989,484 we find an apparatus and method for generating a High Dynamic Range (HDR) image from which a ghost blur is removed based on a multi-exposure fusion. The apparatus may include an HDR weight map calculation unit to calculate an HDR weight map for multiple exposure frames that are received, a ghost probability calculation unit to calculate a ghost probability for each image by verifying a ghost blur for the multiple exposure frames, an HDR weight map updating unit to update the calculated HDR weight map based on the calculated ghost probability, and a multi-scale blending unit to generate an HDR image by reflecting the updated HDR weight map to the multiple exposure frames.

If you have an interesting patent to share when we next feature patents related to range image processing, or if you are especially interested in a signal processing research field that you would want to be highlighted in this section, please send email to Csaba Benedek (benedek.csaba AT sztaki DOT mta DOT hu).

References

Number: 9,241,127
Title: Wide dynamic range image processing method and image signal processor using the same
Inventors: Sfaradi; Roee (Nes Ziona, IL), Eisenberg; Amit (Kiryat Ono, IL), Monzon; Oded (Modi'in-Maccabim-Re'ut, IL)
Issued:  January 19, 2016
Assignee: Samsung Electronics Co., Ltd. (Gyeonggi-Do, Kr)

Number: 9,204,113
Title:  Method and/or apparatus for implementing high dynamic range image processing in a video processing system
Inventors: Kwok; Wilson (Santa Clara, CA), Kohn; Leslie D. (Saratoga, CA)
Issued:  December 1, 2015
Assignee: Ambarella, Inc. (Santa Clara, CA)

Number: 9,167,167
Title: System and method for high dynamic range image sensor sensitive array
Inventors:  Davidovici; Sorin (Oceanport, NJ)
Issued:  October 20, 2015
Assignee: RJS Technology Inc. (Hollis, Nh)

Number: 9,153,027
Title: System, method, and computer program product for performing fast, non-rigid registration for high dynamic range image stacks
Inventors: Gallo; Orazio (Santa Cruz, CA), Pulli; Kari Antero (Palo Alto, CA), Hu; Jun (Sunnyvale, CA)
Issued: October 6, 2015
Assignee: NVIDIA Corporation (Santa Clara, CA)

Number: 9,131,160
Title: Method for controlling exposure time of high dynamic range image
Inventors: Lin; Keng-Sheng (New Taipei, TW), Cheng; Chih-Chi (New Taipei, TW), Li; Chung-Te (New Taipei, TW), Chen; Hsin-Yu (Tainan, TW), Yang; Wen-Chu (New Taipei, TW)
Issued:  September 8, 2015
Assignee: Quanta Computer Inc. (Tao Yuan Shien, Tw)

Number: 9,100,589
Title: Interleaved capture for high dynamic range image acquisition and synthesis
Inventors: Geiss; Ryan (Mountain View, CA)
Issued:  August 4, 2015
Assignee: Google Inc. (Mountain View, CA)

Number: 9,064,313
Title: Adaptive tone map to a region of interest to yield a low dynamic range image
Inventors: Seshadrinathan; Kalpana (Santa Clara, CA), Nestares; Oscar (San Jose, CA)
Issued:  June 23, 2015
Assignee: Intel Corporation (Santa Clara, CA)

Number: 9,025,862
Title: Range image pixel matching method
Inventors: Garcia Becerro; Frederic (Palafrugell, ES), Mirbach; Bruno (Konz, DE)
Issued:  May 5, 2015
Assignee: IEE International Electronics & Engineering S.A. (Luxembourg, LU)

Number: 8,989,484
Title:  Apparatus and method for generating high dynamic range image from which ghost blur is removed using multi-exposure fusion
Inventors:      Moon; Young Su (Seoul, KR), Tai; Yong Min (Gunpo-si, KR), Cho; Jung Uk (Hwaseong-si, KR), Lee; Shi Hwa (Seoul, KR)
Issued:  March 24, 2015
Assignee: Samsung Electronics Co., Ltd. (Suwon-Si, KR)