Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
Note that according to the Federal Circuit’s 2004 Superguide v. DirecTV decision, “at least one of
… and … “ requires at least one instance of each and every item listed. Claims 2, 13, 15 and 26 contain such limitations, however, the specification supports a disjunctive “or” interpretation (see paragraphs 57 and 65). For examination purposes, the limitations be interpreted to require at least one instance of any of the items listed.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 6, 13, 14, 16, 19, and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Georgescu et al. (US 20050074154 A1), (hereinafter, Georgescu) in view of Pertsel et al. (US 8189057 B2), (hereinafter, Pertsel).
Regarding claim 1, Georgescu teaches a method for processing image data, the method comprising:
determining, based on data from one or more sensors, a movement of an image capture device associated with a capture of a plurality of image frames; adjusting a position of at least one object in each of the plurality of image frames based on the movement of the image capture device (Georgescu, “In an embodiment of the present invention, local motion of the endocardial wall is tracked as well as global motion. Global motion may be present for a number of reasons. For example, the patient's breathing or movement of the Sensor by the technician may cause global motion. In order to accurately track the local motion of the endocardial wall, the global motion must be measured and compensated for in the cardiac image.”, pg. 3, paragraph 0029, lines 1-8, “In accordance with the present invention, global motion can be compensated for using optical flow-based methods. These methods are based on the fact that the most apparent difference between local motion and global motion is the movement of the tissues Surrounding the left ventricle. If there is no global motion, the Surrounding tissues do not contract or dilate like the left ventricle. However, if there is global motion, the whole image is affected by the global motion with certain translation and/or rotation in each pixel. In order to compensate for the global motion, movement in the region of tissue Surrounding the left ventricle is measured to achieve a compensation measurement.”, pg. 3, paragraph 0031, The global motion caused by movement of an ultrasound probe is compensated by estimating optical flow between frames to measure translation and rotation of surrounding tissue regions.);
determining a motion of the at least one object based on a difference in the adjusted position among the plurality of image frames (Georgescu, “The present invention is directed to a method for tracking local deformable motion of an object. An example where such a method would be utilized is for tracking the local motion of a myocardial wall to detect regional wall motion abnormalities in the heart.”, pg. 2, paragraph 0025, lines 1-5, After removal of the global motion, local deformable motion of objects can be accurately tracked from the compensated images.).
Georgescu does not teach selecting a value for at least one image capture parameter associated with the plurality of image frames based on the motion of the at least one object.
However, Pertsel teaches selecting a value for at least one image capture parameter associated with the plurality of image frames based on the motion of the at least one object (Pertsel, “In the step 73, data of N number of pre-capture images are used to calculate motion quantities for use in setting the exposure parameters, where N equals two or more, and can be five or more. As explained in detail below, any change in motion of the scene image relative to the camera's photosensor is detected and quantified by looking at changes in Successive pre-capture images, both globally (movement of the entire image) and locally (local movement within the image)... A next step 81 determines whether the exposure parameters automatically calculated in the step 75 are such that the motion quantities will not cause them to be altered. For example, if the exposure duration (shutter speed) is set by the step 75 to be below a certain threshold, then no further decrease of the exposure time to reduce motion blur should be done.”, column 6 and 7, lines 40-67 and 1-22, respectively, Global and local motion is determined for a set of images and used to adjust exposure settings to reduce motion blur.).
Georgescu teaches compensating for global motion for accurately detect and track local object motion (Georgescu, “The present invention is directed to a system and method for local deformable motion analysis, and more particularly, to a system and method for accurately tracking motion of an object isolating local motion of an object from global motion of an object.”, pg. 1, paragraph 0002). Pertsel teaches adjusting image capture parameters, such as exposure time, based on detecting object motion (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the local object motion detection of Georgescu to adjustments of image capture parameters based on object motion as taught by Pertsel (Pertsel, column 6 and 7, lines 40-67 and 1-2). The motivation for doing so would have been to adjust the exposure settings with respect to local object motion, thereby reducing motion blur in the image (as suggested by Pertsel, “It is often difficult for the user to hold a camera by hand during an exposure without imparting some degree of shake or jitter, particularly when the camera is very Small and light. As a result, the captured image may have a degree of overall motion blur that depends on the exposure time, the longer the time the more motion blur in the image. In addition, long exposures of a scene that is totally or partially moving can also result in motion blur in the captured image. An object moving fast across the scene, for example, may appear blurred in the image.”, column 2, lines 28-37). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Georgescu with Pertsel to obtain the invention as specified in claim 1.
Regarding claim 3, Georgescu in view of Pertsel teaches the method of claim 1. Georgescu in view of Pertsel, as presented above, does not teach wherein the movement of the image capturing device occurs during an exposure time corresponding to each of the plurality of image frames.
However, Pertsel further teaches wherein the movement of the image capturing device occurs during an exposure time corresponding to each of the plurality of image frames (Perstel, “It is often difficult for the user to hold a camera by hand during an exposure without imparting some degree of shake or jitter, particularly when the camera is very Small and light. As a result, the captured image may have a degree of overall motion blur that depends on the exposure time, the longer the time the more motion blur in the image.”, column 2, lines 28-33, “Rather than post-processing the acquired video data by taking image motion into account, however, the present invention monitors images of the scene in advance of taking the picture and then sets the exposure parameters to values that enhance the resulting image based on the amount of motion present. The processing calculates at least an optimal exposure time that can be used along with other exposure parameters to acquire data of an image... In FIG. 1, an example of a camera in which the present invention may be implemented is schematically shown, which may be a still camera or a video camera.”, column 4, lines 29-49, Images are taken using an optical camera to capture exposures during camera movement. The camera’s motion is detected and used to control exposure time for each frame.).
As presented above, Georgescu in view of Pertsel teaches capturing sequential image frames using an ultrasound probe and performing global motion compensation for movement of the image-capturing sensor based on a difference in frames. Georgescu in view of Pertsel does not teach compensating for movement of the image-capturing sensor that occurs during an exposure period, such as in an optical imaging system. Pertsel further teaches detecting movement of an optical camera during an exposure time (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the ultrasound imaging system of Georgescu in view of Pertsel by replacing the ultrasound probe with the optical camera as further taught by Pertsel (Pertsel, column 4, lines 29-49, see Fig. 1), thereby capturing and compensating for camera motion during an exposure time. The motivation for doing so would have been to extend the motion compensation approach of Georgescu in view of Pertsel to other imaging modalities, thereby increasing its practical applicability. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Georgescu in view of Pertsel with the further teachings of Pertsel above to obtain the invention as specified in claim 3.
Regarding claim 6, Georgescu in view of Pertsel teaches the method of claim 1, wherein determining the motion of the at least one object includes determining an optical flow between the plurality of image frames (Georgescu, “This method safely removes the regions 306, 310 inside the epicardium and leaves the Surrounding areas 308, 312 for measuring global motion. The next component of the method is to measure the motion within the selected region 308, 312. Optical flow estimation and fusion techniques are used to measure the motion of each pixel in the echocardiogram.”, pg. 3, paragraph 0033 and 0034, lines 17-19 and 1-4, respectively).
Regarding claim 13, Georgescu in view of Pertsel teaches the method of claim 1, wherein the at least one image capture parameter includes at least one of an exposure time and a gain (Pertsel, “However, in most situations the scene is not so brightly illuminated. Therefore, when the preliminary parameters calculated by the step 75 are not within optimum ranges, they are adjusted by a step 85 in order to optimize them for the amount of motion that was calculated by the step 73. Generally, if that
motion is high, the exposure time is reduced, with a corresponding increase in the size of the aperture and/or increase in the gain in order to maintain the same average image signal luminescence.”, column 7, lines 34-42).
Claim 14 corresponds to claim 1, with the addition of an apparatus comprising a memory and a processor configured to execute the method according to claim 1. Georgescu in view of Pertsel teaches the addition of an apparatus comprising a memory and a processor (Georgescu, “The information obtained by the sensor 102 is communicated to a processor 104 which may be a workstation or personal computer.”, pg. 2, paragraph 0027, lines 1-3) configured to execute the method according to claim 1. As indicated in the analysis of claim 1, Georgescu in view of Pertsel teaches all the limitations according to claim 1. Therefore, claim 14 is rejected for the same reasons of obviousness as claim 1.
Claim 16 corresponds to claim 3, with the addition of an apparatus comprising a memory and a processor configured to execute the method according to claim 3. Georgescu in view of Pertsel teaches the addition of an apparatus comprising a memory and a processor (see analysis of claim 14) configured to execute the method according to claim 3. As indicated in the analysis of claim 3, Georgescu in view of Pertsel with the further teachings of Pertsel teaches all the limitations according to claim 3. Therefore, claim 16 is rejected for the same reasons of obviousness as claim 3.
Claim 19 corresponds to claim 6, additionally reciting an apparatus comprising a memory and a processor configured to execute the method according to claim 6. Georgescu in view of Pertsel teaches the addition of an apparatus comprising a memory and a processor (see analysis of claim 14) configured to execute the method according to claim 6. As indicated in the analysis of claim 6, Georgescu in view of Pertsel teaches all the limitations according to claim 6. Therefore, claim 19 is rejected for the same reasons of obviousness as claim 6.
Claim 26 corresponds to claim 13, with the addition of an apparatus comprising a memory and a processor configured to execute the method according to claim 13. Georgescu in view of Pertsel teaches the addition of an apparatus comprising a memory and a processor (see analysis of claim 14) configured to execute the method according to claim 13. As indicated in the analysis of claim 13, Georgescu in view of Pertsel teaches all the limitations according to claim 13. Therefore, claim 26 is rejected for the same reasons of obviousness as claim 13.
Claims 2 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Georgescu et al. (US 20050074154 A1) in view of Pertsel et al. (US 8189057 B2) and further in view of Buyukozturk et al. (US 20180061063 A1), (hereinafter, Buyukozturk).
Regarding claim 2, Georgescu in view of Pertsel teaches the method of claim 1. Georgescu in view of Pertsel does not teach wherein the one or more sensors include at least one of a gyroscope, an accelerometer, a magnetometer, and an inertial measurement unit (IMU).
However, Buyukozturk teaches wherein the one or more sensors include at least one of a gyroscope, an accelerometer, a magnetometer, and an inertial measurement unit (IMU) (Buyukozturk, “Measuring the optical flow field of the scene can include using motion magnification . Measuring the optical flow field of the scene can include combining representations of local motions of a surface in the scene to produce a global motion signal… The camera can be part of a mobile device, and the measuring , determining , and calculating can occur in the mobile device . The mobile device can include an external sensor including at least one of an accelerometer, gyroscope, magnetometer, IMU, global positioning system (GPS) unit, or velocity meter.”, pg. 1, paragraphs 0009 and 0010, Various sensor can be used to measure and calculate global motion for a sequence of images.).
Georgescu in view of Pertsel teaches measuring a global motion for ultrasound images to perform motion compensation (Georgescu, “In an embodiment of the present invention, local
motion of the endocardial wall is tracked as well as global motion. Global motion may be present for a number of reasons. For example, the patient's breathing or movement of the Sensor by the technician may cause global motion.”, pg. 3, paragraph 0029, lines 1-5). Buyukozturk teaches using various sensors, including a gyroscope, an accelerometer, a magnetometer, and an inertial measurement unit, to measure global motion (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Georgescu in view of Pertsel to include any of the various sensors as taught by Buyukozturk (Buyukozturk, pg. 1, paragraphs 0009 and 0010) for measuring global motion. The motivation for doing so would have been to take a direct measurement of the imaging device’s motion using external sensors physically connected to the device, thereby improving the accuracy of the global motion estimation and compensation. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Georgescu in view of Pertsel with Buyukozturk to obtain the invention as specified in claim 2.
Claim 15 corresponds to claim 2, with the addition of an apparatus comprising a memory and a processor configured to execute the method according to claim 2. Georgescu in view of Pertsel and further in view of Buyukozturk teaches the addition of an apparatus comprising a memory and a processor (see analysis of claim 14) configured to execute the method according to claim 2. As indicated in the analysis of claim 2, Georgescu in view of Pertsel and further in view of Buyukozturk teaches all the limitations according to claim 2. Therefore, claim 15 is rejected for the same reasons of obviousness as claim 2.
Claims 4 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Georgescu et al. (US 20050074154 A1) in view of Pertsel et al. (US 8189057 B2) and further in view of Gao et al. (US 20210112188 A1), (hereinafter, Gao).
Regarding claim 4, Georgescu in view of Pertsel teach the method of claim 1. Georgescu in view of Pertsel does not teach further comprising: in response to determining that the movement of the image capturing device is greater than a threshold value, selecting a default value for the at least one image capture parameter.
However, Gao teaches in response to determining that the movement of the image capturing device is greater than a threshold value, selecting a default value for the at least one image capture parameter (Gao, “In general, multiple features may be combined to evaluate the extent to which environmental brightness measurements are caused by reflected light from the display, which may indicate a need to reduce the rate of exposure change… In further examples, sensor data from a gyroscope may be used to evaluate device motion in addition to or instead of camera image data. It may be advantageous to use both a gyroscope and image-based motion detection because
a gyroscope provides a measurement of global motion while image-based motion detection provides a measurement of local motion. If sensor data from a gyroscope is used, then a control system may refrain from decreasing the rate of exposure change when the gyroscope indicates a significant
amount of device motion (e.g., above a high threshold level).”, pg. 5, paragraphs 0053 and 0054, Global motion caused by movement of a camera is determined using a gyroscope. Thresholding is applied to decide if exposure changes should be reduced or left at the normal rate. When motion is determined to be above this threshold, the system keeps the default exposure-change behavior.).
Georgescu in view of Pertsel teaches estimating global and local motion to control exposure settings to reduce motion blur (Pertsel, column 6 and 7, lines 40-67 and 1-22, respectively). Gao teaches applying thresholding to global motion to control exposure settings (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the exposure control of Georgescu in view of Pertsel to additionally control exposure setting based on thresholding global motion as taught by Gao (Gao, pg. 5, paragraphs 0053 and 0054). The motivation for doing so would have been to maintain stable exposure control during large camera movements. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Georgescu in view of Pertsel with Gao to obtain the invention as specified in claim 4.
Claim 17 corresponds to claim 4, with the addition of an apparatus comprising a memory and a processor configured to execute the method according to claim 4. Georgescu in view of Pertsel and further in view of Gao teaches the addition of an apparatus comprising a memory and a processor (see analysis of claim 14) configured to execute the method according to claim 4. As indicated in the analysis of claim 4, Georgescu in view of Pertsel and further in view of Gao teaches all the limitations according to claim 4. Therefore, claim 17 is rejected for the same reasons of obviousness as claim 4.
Claims 5 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Georgescu et al. (US 20050074154 A1) in view of Pertsel et al. (US 8189057 B2) and further in view of Smith et al. (“Electronic image stabilization using optical flow with inertial fusion”, IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010), (hereinafter, Smith).
Regarding claim 5, Georgescu in view of Pertsel teach the method of claim 1. Georgescu in view of Pertsel does not teach wherein adjusting the position of the at least one object includes computing an electronic image stabilization (EIS) compensation.
However, Smith teaches wherein adjusting the position of the at least one object includes computing an electronic image stabilization (EIS) compensation (Smith, “This paper presents a novel EIS algorithm designed to operate in the presence of large image displacement, image blurring, and moving objects. Using the similarity motion model, the algorithm fuses pyramidal Lucas-Kanade optical flow using Shi-Tomasi good features with inertial measurement motion estimation by way of a discrete Kalman filter. Inertial measurement motion estimation is performed by summing angular displacements between frames of a MIDG II inertial measurement unit (IMU) and multiplying the angular displacements by a constant. The two motion estimates are then optimally fused using a nine-state discrete Kalman filter.”, pg. 1, 2nd column, 1st full paragraph, see Section III. Optical Flow with Inertial Fusion and Fig. 6).
Georgescu in view of Pertsel teaches performing global motion compensation using optical flow methods (Georgescu, “In accordance with the present invention, global motion can be compensated for using optical flow-based methods.”, pg. 3, paragraph 0031, lines 1-3). Smith teaches performing global motion compensation using an electronic image stabilization algorithm which combines optical flow with inertial measurements (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Georgescu in view of Pertsel by replacing the global motion compensation with the electronic image stabilization algorithm as taught by Smith (Smith, pg. 1, 2nd column, 1st full paragraph, see Section III. Optical Flow with Inertial Fusion and Fig. 6). The motivation for doing so would have been to reduce prediction error compared to optical flow alone (as suggested by Smith, “The novel algorithm presented in this paper, optical flow with inertial fusion, combines these two methods, and is capable of reduction in RMS error compared to 40% optical flow alone in the presence of moving objects.”, pg. 8, 2nd column, 3rd full paragraph, lines 6-9). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Georgescu in view of Pertsel with Smith to obtain the invention as specified in claim 5.
Claim 18 corresponds to claim 5, with the addition of an apparatus comprising a memory and a processor configured to execute the method according to claim 5. Georgescu in view of Pertsel and further in view of Smith teaches the addition of an apparatus comprising a memory and a processor (see analysis of claim 14) configured to execute the method according to claim 5. As indicated in the analysis of claim 5, Georgescu in view of Pertsel and further in view of Smith teaches all the limitations according to claim 5. Therefore, claim 18 is rejected for the same reasons of obviousness as claim 5.
Claims 7-11 and 20-24 are rejected under 35 U.S.C. 103 as being unpatentable over Georgescu et al. (US 20050074154 A1) in view of Pertsel et al. (US 8189057 B2) and further in view of Ichihashi et al. (US 20100119176 A1), (hereinafter, Ichihashi).
Regarding claim 7, Georgescu in view of Pertsel teaches the method of claim 1. Georgescu in view of Pertsel does not teach wherein determining the motion of the at least one object includes determining a motion mask between a first image frame and a second image frame from the plurality of image frames.
However, Ichihashi teaches wherein determining the motion of the at least one object includes determining a motion mask between a first image frame and a second image frame from the plurality of image frames (Ichihashi, “ In step S43, the mask generating unit 33 generates a motion mask using the input image supplied from the upsampling unit 31 and the estimation image supplied from the motion compensation unit 32. Thereafter, the mask generating unit 33 supplies the generated motion mask to the mixing unit 34 and the weight computing unit 43… In step S44, the mixing unit 34 mixes the input
image supplied from the upsampling unit 31 with the estimation image supplied from the motion compensation unit 32 using the motion mask supplied from the mask generating unit 33. That is, the mixing unit 34 selects one of the pixels of an image to be obtained from now (hereinafter also referred to as an ’SR mixing image’) as a pixel of interest. The mixing unit 34 then computes, using the pixel value of a pixel of the motion mask located at a position the same as that of the pixel of interest, a weight Wi (0s Wis1) of a pixel in the input image located at a position the same as that of the pixel of interest and a weight We (=1-Wi) of a pixel in the estimation image located at a position the same as that of the pixel of interest.”, pgs. 5 and 6, paragraphs 0095-0098, A motion mask is generated from motion estimation between images. This mask is used to generate blending weights for a super-resolution process.).
Georgescu in view of Pertsel teaches tracking local motion of an object using globally compensated images (Georgescu, “Motion from multiple back ground image regions is combined to measure the global motion in that image frame. The measured global motion in the object image regions is compensated for to measure local motion of the object, and the local motion of the object is tracked.”, pgs. 1 and 2, paragraph 0009, lines 8-13). Ichihashi teaches generating a motion mask which is used to compute blending weights for image super-resolution (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the local motion tracking of Georgescu in view of Pertsel to include a motion mask for image super-resolution as taught by Ichihashi (Ichihashi, pgs. 5 and 6, paragraphs 0095-0098). The motivation for doing so would have been to increase the resolution of the image based on motion of the object, thereby increasing the quality of the image (as suggested by Ichihashi, “That is, when a higher-resolution output image is acquired from the input image, the image quality of the entire area of the output image can be easily and reliably increased.”, pg. 10, paragraph 0177, lines 6-8). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Georgescu in view of Pertsel with Ichihashi to obtain the invention as specified in claim 7.
Regarding claim 8, Georgescu in view of Pertsel and further in view of Ichihashi teaches the method of claim 7, wherein the motion mask includes one or more motion vectors indicating a shift of at least one pixel between the first image frame and the second image frame (Ichihashi, “That is, the motion compensation unit 32 detects a motion vector of each of the pixels of the SR image using the input image and the SR image. Thereafter, the motion compensation unit 32 performs motion compensation on the SR image using the detected motion vectors and generates an estimation image.… In step S43, the mask generating unit 33 generates a motion mask using the input image supplied from the upsampling unit 31 and the estimation image Supplied from the
motion compensation unit 32.”, pg. 5, paragraphs 0094 and 0095, A motion vector is determined and used to generate an estimation image. This estimation image is compared with the input image to generate the motion mask.).
Regarding claim 9, Georgescu in view of Pertsel and further in view of Ichihashi teaches the method of claim 8, further comprising: determining a weighted table that includes one or more weight values corresponding to at least a portion of the one or more motion vectors (Ichihashi, “the mixing unit 34 selects one of the pixels of an image to be obtained from now (hereinafter also referred to as an “SR mixing image') as a pixel of interest. The mixing unit 34 then computes, using the pixel value of a pixel of the motion mask located at a position the same as that of the pixel of interest, a weight Wi (0s Wis1) of a pixel in the input image located at a position the same as that of the pixel of interest and a weight We (=1-Wi) of a pixel in the estimation image located at a position the same as that of the pixel of interest.”, pg. 6, paragraph 0098, The motion vectors are used to determine the motion mask. From this, weights can be determined on a per-pixel basis in order to blend in the super-resolution process.).
Regarding claim 10, Georgescu in view of Pertsel and further in view of Ichihashi teaches the method of claim 9, wherein the one or more weight values are selected based on a central region in the plurality of image frames (Ichihashi, “the mixing unit 34 selects one of the pixels of an image to be obtained from now (hereinafter also referred to as an “SR mixing image') as a pixel of interest. The mixing unit 34 then computes, using the pixel value of a pixel of the motion mask located at a position the same as that of the pixel of interest, a weight Wi (0s Wis1) of a pixel in the input image located at a position the same as that of the pixel of interest and a weight We (=1-Wi) of a pixel in the estimation image located at a position the same as that of the pixel of interest.”, pg. 6, paragraph 0098, Weights are selected based on matching pixel points across the images. This includes selecting weights for central pixels of the image.).
Regarding claim 11, Georgescu in view of Pertsel and further in view of Ichihashi teaches the method of claim 9, wherein the one or more weight values are selected based on a region of interest in the plurality of image frames (Ichihashi, “Accordingly, for example, as shown in FIG. 12, the weight computing unit 43 increases the weight Wh of an enhancement image for a pixel of an area R11 of a moving subject for which the advantage of Super-resolution processing is not obtained and a pixel of an area R12 including a diagonal line for which the edge enhancement processing is
effective… For example, the pixel value of a pixel of a motion mask indicates the level of motion of the subject. Accordingly, it is determined that a pixel having a pixel value larger than or equal to a predetermined threshold value is a pixel of the area of the moving subject. Thus, the weight Wh of a pixel of the enhancement image located at a position the same as that of the pixel is further increased.”, pg. 10, paragraphs 0172-0174, see Fig. 12, Weights can be selected and adjusted according to objects of interest.).
Claim 20 corresponds to claim 7, with the addition of an apparatus comprising a memory and a processor configured to execute the method according to claim 7. Georgescu in view of Pertsel and further in view of Ichihashi teaches the addition of an apparatus comprising a memory and a processor (see analysis of claim 14) configured to execute the method according to claim 7. As indicated in the analysis of claim 7, Georgescu in view of Pertsel and further in view of Ichihashi teaches all the limitations according to claim 7. Therefore, claim 20 is rejected for the same reasons of obviousness as claim 7.
Claim 21 corresponds to claim 8, with the addition of an apparatus comprising a memory and a processor configured to execute the method according to claim 8. Georgescu in view of Pertsel and further in view of Ichihashi teaches the addition of an apparatus comprising a memory and a processor (see analysis of claim 14) configured to execute the method according to claim 8. As indicated in the analysis of claim 8, Georgescu in view of Pertsel and further in view of Ichihashi teaches all the limitations according to claim 8. Therefore, claim 21 is rejected for the same reasons of obviousness as claim 8.
Claim 22 corresponds to claim 9, with the addition of an apparatus comprising a memory and a processor configured to execute the method according to claim 9. Georgescu in view of Pertsel and further in view of Ichihashi teaches the addition of an apparatus comprising a memory and a processor (see analysis of claim 14) configured to execute the method according to claim 9. As indicated in the analysis of claim 9, Georgescu in view of Pertsel and further in view of Ichihashi teaches all the limitations according to claim 9. Therefore, claim 22 is rejected for the same reasons of obviousness as claim 9.
Claim 23 corresponds to claim 10, with the addition of an apparatus comprising a memory and a processor configured to execute the method according to claim 10. Georgescu in view of Pertsel and further in view of Ichihashi teaches the addition of an apparatus comprising a memory and a processor (see analysis of claim 14) configured to execute the method according to claim 10. As indicated in the analysis of claim 10, Georgescu in view of Pertsel and further in view of Ichihashi teaches all the limitations according to claim 10. Therefore, claim 23 is rejected for the same reasons of obviousness as claim 10.
Claim 24 corresponds to claim 11, with the addition of an apparatus comprising a memory and a processor configured to execute the method according to claim 11. Georgescu in view of Pertsel and further in view of Ichihashi teaches the addition of an apparatus comprising a memory and a processor (see analysis of claim 14) configured to execute the method according to claim 11. As indicated in the analysis of claim 11, Georgescu in view of Pertsel and further in view of Ichihashi teaches all the limitations according to claim 11. Therefore, claim 24 is rejected for the same reasons of obviousness as claim 11.
Claims 12 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Georgescu et al. (US 20050074154 A1) in view of Pertsel et al. (US 8189057 B2) and further in view of Ichihashi et al. (US 20100119176 A1) and Hamamoto et al. (JP 2011155582 A), (hereinafter, Hamamoto).
Regarding claim 12, Georgescu in view of Pertsel and further in view of Ichihashi teaches the method of claim 11. Georgescu in view of Pertsel and further in view of Ichihashi does not teach wherein a portion of the one or more weight values corresponding to an area outside the region of interest is set to zero.
However, Hamamoto teaches wherein a portion of the one or more weight values corresponding to an area outside the region of interest is set to zero (Hamamoto, “as shown in FIG. 7A, when the size of the main subject area T11 is equal to or larger than the threshold value, the weight of the motion vector detected in the small area including 20 the background area may be set to zero. Similarly, as shown in FIG. 7B, when the size of the main subject region T12 is smaller than the threshold, the weight of the motion vector detected in the small region including the main subject region T12 may be set to zero.”, pg. 16, lines 19-23, Weights for background regions are set to zero when the size of a subject meets or exceeds a threshold value.).
Georgescu in view of Pertsel and further in view of Ichihashi teaches selecting and adjusting weights for objects in images (Ichihashi, “Accordingly, for example, as shown in FIG. 12, the weight computing unit 43 increases the weight Wh of an enhancement image for a pixel of an area R11 of a moving subject… ”, pg. 10, paragraphs 0172-0174, see Fig. 12). Hamamoto teaches setting background region weights to zero based on a subject size thresholding (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the weights selection and adjustment of Georgescu in view of Pertsel and further in view of Ichihashi to include the subject size thresholding as taught by Hamamoto (Hamamoto, pg. 16, lines 19-23), thereby setting background region weights to zero. The motivation for doing so would have been to filter out redundant weights corresponding to background regions, thereby increasing processing speed. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Georgescu in view of Pertsel and further in view of Ichihashi with Hamamoto to obtain the invention as specified in claim 12.
Claim 25 corresponds to claim 12, with the addition of an apparatus comprising a memory and a processor configured to execute the method according to claim 12. Georgescu in view of Pertsel and further in view of Ichihashi and Hamamoto teaches the addition of an apparatus comprising a memory and a processor (see analysis of claim 14) configured to execute the method according to claim 12. As indicated in the analysis of claim 12, Georgescu in view of Pertsel and further in view of Ichihashi and Hamamoto teaches all the limitations according to claim 12. Therefore, claim 25 is rejected for the same reasons of obviousness as claim 12.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CONNOR LEVI HANSEN whose telephone number is (703)756-5533. The examiner can normally be reached Monday-Friday 9:00-5:00 (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CONNOR L HANSEN/Examiner, Art Unit 2672
/GANDHI THIRUGNANAM/Primary Examiner, Art Unit 2672
Prosecution Insights