DETAILED ACTION
Claims 3 and 4 have been cancelled.
Claims 1, 2, 5, 6 and 11-15 are currently pending.
The previous rejections to claims 1, 2, 5, 6 and 11-15 under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, is withdrawn due to Applicant’s amendment.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/24/26 has been entered.
Response to Arguments
Applicant's arguments filed 4/24/26 have been fully considered but they are not persuasive.
The Applicant argues on pages 9 and 10 of the response in essence that: Wong discloses, at [0168] that "[a]n optimizer can be used (e.g. an energy-based gradient descent optimizer, a linear programming optimizer, etc.) to adjust the vertices of the mesh, optimizing for feature alignment between images and other constraints." However, Wong does not disclose or suggest "wherein the gradient descent is applied by altering a zoom of the second image," as recited in claim 1.
Wong discloses using a gradient descent optimizer to adjust the vertices of the mesh (paragraph 168). Adjusting the vertices of the max distorts the image (paragraph 150), which will result in the scale of the image being altered.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 5, 6 and 11-15 are rejected under 35 U.S.C. 103 as being unpatentable over Watts et al. US Publication 2013/0169844 (hereafter “Watts”) and Wong et al. US Publication 2021/0004933 (hereafter “Wong”).
Referring to claims 1, 14 and 15, Watts discloses an image processing apparatus comprising:
a first image sensor;
a second image sensor located in a different position from a position of the first image sensor (paragraph 45, In some embodiments, two images may be captured during operation 601 by different cameras or, more specifically, different optical lenses provided on the same device. These images may be referred to as stereo images. In some embodiments, the two cameras are separated by between about 30 millimeters and 150 millimeters);
at least one memory configured to store instructions; and
at least one processor configured to execute the instructions to:
obtain a first image captured by the first image sensor;
obtain a second image captured by the second image sensor (paragraph 45, In some embodiments, two images may be captured during operation 601 by different cameras or, more specifically, different optical lenses provided on the same device);
determine a rough region of interest (RROI) in the first image (paragraph 51, Operation 603 involves detecting the object in each initial image);
determine a geometric transformation that maps a position of an RROI of the second image corresponding to the RROI of the first image to a position of the RROI of the first image (paragraph 52, Operation 604 involves determining an object center line of the object in each initial image as described above with reference to FIGS. 4A and 4B. In some embodiments, other alignment and/or scaling techniques may be used during operation 604);
determine a boundary of a region of interest (ROI) in the first image corresponding to the RROI of the first image, based on the geometric transformation (paragraph 53, In operation 606, the foreground portion may be separated from the background portion);
obtain a third image by applying the geometric transformation to the second image to map the RROI of the second image onto an image plane of the first image (paragraph 52, The method continues with cross-fading the two initial images along the object center line thereby yielding a combined image during operation 605);
determine a static region between the first image and the third image (paragraph 50, The stereo disparity may be used during detecting operation 602 to determine proximity of each pixel or patch in the stereo images to the camera and therefore to identify at least the background portion of the image); and
determine the boundary of the ROI of the first image based on the static region, wherein the static region includes pixels in the first image (paragraph 50, The stereo disparity may be used during detecting operation 602 to determine proximity of each pixel or patch in the stereo images to the camera and therefore to identify at least the background portion of the image) (paragraph 47, In general, a parallax is a displacement or difference in the apparent position of an object viewed along two different lines of sight. It may be represented by the angle or semi-angle of inclination between those two lines. Nearby objects have a larger parallax than more distant objects when observed from different positions, which allows using the parallax values to determine distances and separate foreground and background portions of an image).
Watts does not disclose expressly using a gradient descent.
Wong discloses wherein the at least one processor is further configured to execute the instructions to:
determine a parameter of the geometric transformation using a gradient descent that decreases an error between a position of the feature point of the RROI of the first image and a transformed position of the feature point of the RROI of the second image according to the geometric transformation (paragraph 168, An optimizer can be used (e.g. an energy-based gradient descent optimizer, a linear programming optimizer, etc.) to adjust the vertices of the mesh, optimizing for feature alignment between images and other constraints),
wherein the gradient descent is applied by altering a zoom of the second image (paragraph 150, Locally deforming the mesh preferably includes adjusting the vertices of the mesh to locally distort one image).
At the time of the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to use a parameter of the geometric transformation using a gradient descent. The motivation for doing so would have been to increase the accuracy of aligning images. Therefore, it would have been obvious to combine Wong with Watts to obtain the invention as specified in claims 1, 14 and 15.
Referring to claim 2, Watts discloses wherein the geometric transformation includes at least one of translation, scaling, or perspective change (paragraph 52, Operation 604 involves determining an object center line of the object in each initial image as described above with reference to FIGS. 4A and 4B. In some embodiments, other alignment and/or scaling techniques may be used during operation 604).
Referring to claim 5, Watts discloses wherein the parameter of the geometric transformation includes at least one of a translation parameter, a scaling parameter, or a perspective change parameter (paragraph 52, Operation 604 involves determining an object center line of the object in each initial image as described above with reference to FIGS. 4A and 4B. In some embodiments, other alignment and/or scaling techniques may be used during operation 604).
Referring to claim 6, Watts discloses wherein the perspective change parameter is defined based on a relative position of the first image sensor and the second image sensor (paragraph 50, The stereo disparity may be used during detecting operation 602 to determine proximity of each pixel or patch in the stereo images to the camera and therefore to identify at least the background portion of the image).
Referring to claim 11, Watts discloses wherein the at least one processor is further configured to execute the instructions to:
determine a feature point of a background of the first image based on the boundary of the ROI of the first image (paragraph 42, separate object center lines may be identified for different objects, e.g., objects on the foreground and objects on the background);
determine a transformed position of a feature point of a background of the second image corresponding to the feature point of the background of the first image, according to the geometric transformation (paragraph 52, Operation 604 involves determining an object center line of the object in each initial image as described above with reference to FIGS. 4A and 4B. In some embodiments, other alignment and/or scaling techniques may be used during operation 604);
determine a parameter of an image effect for the feature point of the background of the first image, based on a difference between a position of the feature point of the background of the first image and the transformed position of the feature point of the background of the second image (paragraph 9, In some embodiments, the combined background portion is blurred using one or more techniques, such as a circular blurring and a Gaussian blurring. The combined background portion may be blurred adaptively. The adaptive aspect may depend on differences in positions of object center lines on the foreground and background portions. Additional differences in these positions may drive more blurring of the background); and
generate an output image by applying the image effect on the first image, based on the parameter of the image effect (paragraph 9, In some embodiments, the combined background portion is blurred using one or more techniques, such as a circular blurring and a Gaussian blurring. The combined background portion may be blurred adaptively. The adaptive aspect may depend on differences in positions of object center lines on the foreground and background portions. Additional differences in these positions may drive more blurring of the background).
Referring to claim 12, Watts discloses wherein the image effect includes blurring, and the at least one processor is further configured to execute the instructions to:
determine a blur level for the feature point of the background of the first image based on the difference between the position of the feature point of the background of the first image and the transformed position of the feature point of the background of the second image (paragraph 9, In some embodiments, the combined background portion is blurred using one or more techniques, such as a circular blurring and a Gaussian blurring. The combined background portion may be blurred adaptively. The adaptive aspect may depend on differences in positions of object center lines on the foreground and background portions. Additional differences in these positions may drive more blurring of the background).
Referring to claim 13, Watts discloses wherein the image effect includes emulating 3-dimensional (3D) parallax, and the at least one processor is further configured to execute the instructions to:
generate a 3D parallax image based on the first image and the second image by determining a distance between an object corresponding to the ROI of the first image and an object corresponding to the feature point of the background of the first image as higher based on the difference between the position of the feature point of the background of the first image and the transformed position of the feature point of the background of the second image being larger (paragraph 63, However, images captured with cameras 722 and 724 may not be stereo images from which stereo disparity may be determined. Still detection of at least the foreground portion of the stereo images may be performed during operation 726. Various techniques that do not require stereo disparity may be used, such as motion parallax) (paragraph 47, The motion parallax may be used for video images. It is a depth cue that results from a relative motion of objects captured in the image and the capturing device. In general, a parallax is a displacement or difference in the apparent position of an object viewed along two different lines of sight. It may be represented by the angle or semi-angle of inclination between those two lines. Nearby objects have a larger parallax than more distant objects when observed from different positions, which allows using the parallax values to determine distances and separate foreground and background portions of an image [showing that in applying motion parallax to the combined image requires determining distances between the object and background]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PETER K HUNTSINGER whose telephone number is (571)272-7435. The examiner can normally be reached Monday - Friday 8:30 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Q Tieu can be reached at 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER K HUNTSINGER/Primary Examiner, Art Unit 2682
Prosecution Insights