DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 10-12 and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Hou (CN 109410211 A1) (all citations to English-language machine translation included with this office action).
Regarding claim 1:
Hou teaches: an image generation method performed by a computer device (the image generation method described in relation to Fig. 4 at pages 27-30, the image generation including segmentation of a target object in an image (page 27), and obtain a plurality of processed target object images (page 30), which are synthesized with preset background image to obtain image samples (page 31)) , and comprising:
obtaining a target depth image, the target depth image including a target object in a real scene (page 22, “the electronic device can simultaneously capture the target object through the color camera and the depth camera, so that the corresponding depth image [target depth image] can be obtained while obtaining the image to be divided”), and each pixel point in the target depth image having a depth value (page 22, depth images have values assigned to pixels);
segmenting the target depth image based on the depth value corresponding to each pixel point in the target depth image, to obtain a target object template image, the target object template image comprising a plurality of pixel points corresponding to the target object in the target depth image (page 29, segment the target object in the image, the pixel range of the target object can be determined. The segmented target object in the image corresponds to the target object template);
obtaining M background images corresponding to a target scene, the target scene being a scene set associated with an image processing model (page, 31, 35, associated with a convolutional neural network model), and M being an integer greater than or equal to 1 (pages 31, 35, obtaining a preset background image, here M = 1); and
superimposing the target object template image on the M background images, to generate M target scene images, the target scene images being configured for training the image processing model (pages 30, 31, 35, combining a plurality of processed target object images with preset background image to obtain a plurality of image samples, which can be input with a corresponding label into a convolutional neural network for parameter training).
It would have been obvious for one of ordinary skill in the art to have further modified the applied reference(-s), in view of same, to have obtained the above, and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
The prior art included each element recited in claim 1, although not necessarily in a single embodiment, with the only difference being between the claimed element and the prior art being the lack of actual combination of certain elements in a single prior art embodiment, as mapped and described above.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 10:
Hou teaches: the image generation method according to claim 1, wherein the obtaining a target depth image comprises: obtaining a first depth image (page 22);
performing target detection on the first depth image (see mapping in claim1), and
determining the first depth image as the target depth image if the first depth image comprises the target object (mapping to claim 1; also this claim limitation is broad and more so labels the images, with no distinguishing features.
It would have been obvious for one of ordinary skill in the art as of the effective filing date of Applicant’s claims to have further modified the applied reference(-s) in view of same to have obtained the above, motivated to have practical target objects likely relevant to surroundings.
Regarding claim 11:
Hou teaches: the image generation method according to claim 1, wherein the obtaining a target depth image comprises: obtaining the target depth image captured by a depth camera in the real scene, the real scene comprising the target object and a real background (page 22, depth image of target object obtained with color and depth camera, combined with “Background technique” at page 1, which describes a real scene with target object).
It would have been obvious for one of ordinary skill in the art as of the effective filing date of Applicant’s claims to have further modified the applied reference(-s) in view of same to have obtained the above, motivated to have practical target objects likely relevant to surroundings.
Regarding claim 12: see also claim 1.
Hou teaches: a computer device, comprising: a memory, a transceiver, and a processor; the memory being configured to store a plurality of computer programs; the processor being configured to execute the plurality of computer programs in the memory to perform an image generation method including.
The method of claim 12 corresponds to that of claim 1; the same rationale for rejection applies.
Regarding claim 17: see claim 10.
These claims are similar; the same rationale for rejection applies.
Regarding claim 18: see claim 11.
These claims are similar; the same rationale for rejection applies.
Regarding claim 19: see also claim 1.
Hou teaches: a non-transitory computer-readable storage medium, comprising a plurality of computer programs, wherein the plurality of computer programs, when executed by a processor of a computer device, cause the computer device to perform an image generation method including:
The method of claim 19 corresponds to that of claim 1; the same rationale for rejection applies.
Regarding claim 20: see claim 11.
These claims are similar; the same rationale for rejection applies.
Claim(s) 2, 6, 7, 8, 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Hou in view of Mu (CN 112036284 A1) (citations to English language machine translation included with this office action).
Regarding claim 2:
The applied reference(-s) to claim 1 do not proactive teach claim 2. Consider the following.
In analogous art, Mu teaches: the image generation method according to claim 1, wherein the segmenting the target depth image based on the depth value corresponding to each pixel point in the target depth image, to obtain a target object template image comprises:
performing binarization processing on the target depth image based on the depth value corresponding to each pixel point in the target depth image, to obtain a target object mask image (page 24, first paragraph); and
segmenting the target depth image based on the target object mask image to obtain the target object template image (pages 24-25).
It would have been obvious for one of ordinary skill in the art as of the effective filing date of Applicant’s claims to have further modified the applied reference(-s) in view of same to have obtained the above, motivated to make use of known image processing to segment and locate desired image features.
Regarding claim 6:
Mu teaches: the image generation method according to claim 1, wherein the segmenting the target depth image based on the depth value corresponding to each pixel point in the target depth image, to obtain a target object template image comprises: performing image masking on the target depth image based on the depth value corresponding to each pixel point in the target depth image, to obtain a plurality of pixel points of an image corresponding to the target object (page 24); and
generating the target object template image based on the plurality of pixel points of the image corresponding to the target object (page 24).
It would have been obvious for one of ordinary skill in the art as of the effective filing date of Applicant’s claims to have further modified the applied reference(-s) in view of same to have obtained the above, and incorporated the teachings of Mu into the methods of Hou, motivated to make use of known image processing to segment and locate desired image features.
Regarding claim 7:
Mu teaches: the image generation method according to claim 1, wherein the segmenting the target depth image based on the depth value corresponding to each pixel point in the target depth image, to obtain a target object template image comprises: obtaining K depth values corresponding to K pixel points in the target depth image (page 24, here, pixel points are related to a preset area);
calculating an average depth value of the target depth image based on the K depth values (page 24, depth values of target pixels in the preset area are averaged);
determining L target object pixel points from the K pixel points based on the average depth value and the K depth values (pages 24-25, the target object pixel points can be the updated pixel points determined based on the average depth and K depth values); and
segmenting the target depth image based on the L target object pixel points to obtain the target object template image (page 25).
It would have been obvious for one of ordinary skill in the art as of the effective filing date of Applicant’s claims to have further modified the applied reference(-s) in view of same to have obtained the above, motivated to make use of known image processing to segment and locate desired image features.
Regarding claim 8:
Mu teaches: the image generation method according to claim 7, wherein the determining L target object pixel points from the K pixel points based on the average depth value and the K depth values comprises: determining, from the K pixel points, the L target object pixel points whose depth values are less than the average depth value (page 25, pixel points outside the effective depth range include those less than the average).
It would have been obvious for one of ordinary skill in the art as of the effective filing date of Applicant’s claims to have further modified the applied reference(-s) in view of same to have obtained the above, motivated to make use of known image processing to segment and locate desired image features.
Regarding claim 13: see claim 2.
These claims are similar; the same rationale for rejection applies.
Regarding claim 14: see claim 6.
These claims are similar; the same rationale for rejection applies.
Regarding claim 15: see claim 7.
These claims are similar; the same rationale for rejection applies.
Claim(s) 9 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Hou in view of Agrawal (U.S. Patent App. Pub. No. 2018/0174299).
Regarding claim 9:
It would have been obvious for one of ordinary skill in the art to have combined and modified the applied reference(-s), in view of same, to have obtained: the image generation method according to claim 1, wherein the superimposing the target object template image on the M background images, to generate M target scene images comprises: resizing the target object template image for M times, to generate target object template images of M different sizes, the M different sizes being all less than sizes of the M background images; and
respectively overlaying the target object template images of the M different sizes on the M background images to generate M training images, and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
Agrwal teaches that it is known to resize images to fit within a desired area, or have a better fit (para. 20). Modifying Hou, such that the target object template is resized to better fit with the overlaying to the background image, as per Applicant’s claim 9, is taught/suggested by the prior art, and would have been obvious and predictable to one of ordinary skill.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 16: see claim 9.
These claims are similar; the same rationale for rejection applies.
Allowable Subject Matter
Claims 3-5 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 3 recites: the image generation method according to claim 2, wherein the performing binarization processing on the target depth image, to obtain a target object mask image comprises: performing binarization processing on the target depth image to obtain a pixel coefficient corresponding to each pixel point in the target depth image; and
generating the target object mask image based on a pixel coefficient corresponding to the target object in the target depth image.
The following is a statement of reasons for the indication of allowable subject matter: the closest prior art to the above features of claim 3 is included with this office action, and of these, the following references are of significance:
CN113111857A Human body posture estimation method based on multi-modal information fusion
The instant reference is related to human body posture estimation using deep learning (see page 2).
A human pose estimation method of the instant reference has the following steps (see pages 2-3):
Step 1: Use the RGBD camera to collect RGB and Depth image pairs, and perform background segmentation operations on the RGB and Depth images respectively;
Step 2: Let the coordinates of a certain pixel of the RGB image be XR(i,j), and the coordinate of the corresponding depth map pixel is XD(i,j), and generate a mask map according to the resolution of the depth map.
Step 3: Design a controllable threshold δ according to the complexity of the scene, and perform the following binarization operations on the mask map:
Step 4. Do dot multiplication of the binarized mask image with the RGB and Depth images to suppress most of the background information in the RGB and Depth images
Step 5. Use the RGB and Depth images with the background information segmented in step 4 as the input of the multi-modal network.
During training, the instant reference teaches an alpha coefficient that controls the peak range of pixel values (see page 8). The coefficient can be used a s a shrinkage factor to control expansion of peak values. Id.
Imran, S., Long, Y., Liu, X., & Morris, D. (2019, June). Depth coefficients for depth completion. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 12438-12447). IEEE.
The instant reference teaches a new representation for depth called Depth Coefficients (DC), which enables convolutions to more easily avoid inter-object depth mixing. (Abstract).
Regarding “depth mixing” (combining multiple images—often with different depth information—into a single output image), and the avoidance of inter-object depth mixing, the instant reference teaches the use of depth coefficients as follows:
PNG
media_image1.png
664
548
media_image1.png
Greyscale
However, the above references, while the first reference describes related topics to Applicant’s claims (binarization, mask, segmentation), and a coefficient value, the first reference does not teach the specific combination of features as recited in Applicant’s claim 3, and instead the coefficient of the first reference is related to controlling peak value during training, but not in relation to generating a target object mask image corresponding to a target object in a depth image, as per Applicant’s claim 3. And while
the second reference also teaches depth coefficients, the second reference does not refer to or make mention of binarization or masks.
Neither reference, alone or in varied combination, would have rendered obvious the above features of Applicant’s claim 3 as it relates to obtaining a pixel coefficient corresponding to each pixel point in the target depth image; and generating the target object mask image based on a pixel coefficient corresponding to the target object in the target depth image.
Claims 4 and 5 depend from claim 3 and inherit the allowability of claim 3.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US-20140072205-A1 systems and methods for generating depth data, utilizing a first image and a second image which are captured from different viewpoints, including a depth data generation unit which determines, for each segment, a disparity value of the segment, based on the disparity value of the representative pixel included in the segment to generate depth data indicative of depths corresponding to the plural segments
US-20180020188-A1 Techniques described herein are related to a system, medium, and method of depth data filling of shadows for image processing.
US-20210158554-A1 Artificial intelligence for generating a depth map
US-20230386057-A1 an electronic apparatus for obtaining depth information from a two-dimensional (2D) image and an image processing method thereof
US-20210343030-A1 acquiring an input image comprising a document portion; and performing image segmentation on the input image to form a binary input image that distinguishes the document portion from the remaining portion of the input image.
* * * * *
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Sarah Lhymn whose telephone number is (571)270-0632. The examiner can normally be reached M-F, 9:00 AM to 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Sarah Lhymn
Primary Examiner
Art Unit 2613
/Sarah Lhymn/Primary Examiner, Art Unit 2613
Prosecution Insights