DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant's claim for foreign priority based on an application filed in Japan on 25 October 2023 and 04 October 2024. It is noted, however, that applicant has not filed a certified copy of the JP2023-183528 and JP2024-175265 applications as required by 37 CFR 1.55.
Allowable Subject Matter
Claim 10 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 13-17 are allowed.
The following is a statement of reasons for the indication of allowable subject matter:
With regards to claim 10, several of the features of this claim were known in the art as evidenced by Strandborg et al (US PG Pub. No. 2024/0169494), which discloses generating a defocus map including a defocus amount of the partial region based on a depth value of the partial region and a photographing parameter of the camera at: ¶ [0028](“[A]pplying an extended depth-of-field correction to at least one image segment of the given image that is out of focus, …, based on optical depths in at least one segment of the reconstructed depth map corresponding to the at least one image segment of the given image”) ; ¶ [0091]-[0093](“[T]he EDOF correction could also be applied by utilizing… a defocus map estimation technique…); ¶ [0089](“The depth-of-field may be determined based on at least one of: a focal length of the lens of the given camera, a distance between the object and the given camera, a pixel size, an aperture size, a transmission characteristic of an aperture, a number of apertures (i.e., in case of the given camera having multiple apertures).”); ¶¶ [0136]-[0137]; see, also: ¶ [0061]; ¶ [0076]. However, Strandborg does not disclose obtaining an inference result for a subject in a photographed image based on a learned model learned using the defocus map generated by the information processing apparatus according to claim 1.
With regards to claims 13, 16 and 17, several of the features of these claims were known in the art as evidenced by Smith et al (Chinese Pub. No. CN 112567287 A) which discloses arranging a three dimensional model of a subject in a virtual three dimensional space, and arranging three dimensional models of a first camera and a second camera at intervals so that optical axes of the first camera and the second camera are parallel to each other at pp. 27-28 (“the optical axis 1140a of the eye camera 1160a parallel to the optical axis 1140b of the eye camera 1160b”); p. 21, pars. 1-2; but Smith does not disclose a third camera. Smith discloses determining at least a first region around the subject in a double-eye image in which the subject is captured, the double-eye image being rendered based on a first photographing field of view of the first camera at p. 9 (“[T]he virtual depth of the point…determined by double-eye depth clue of convergence and stereoscopic vision… [T]he three-dimensional object rendering to the environment of the user, so that the virtual object looks similar to the real world object.”); p.25. Smith discloses generating defocus information including a defocus amount of a partial region of the first region based on parallax information of a partial region of a second region corresponding to the first region in a left-eye image in which the subject is captured, the left-eye image being rendered based on a second photographing field of view of the second camera and a partial region of a third region corresponding to the first region in a right-eye image in which the subject is captured, the righteye image being rendered based on a third photographing field of view of the third camera at p. 9 (“Different image features are focused for scenes located on different depth planes or on the basis of different image feature defocusing on viewing different depth planes”); p. 25 (“[T]he depth map information… Such information can be calculated according to the process referred to as three-dimensional (Stereo) 940, wherein the depth information is determined using techniques such as triangulation or flight time sensing.”); p. 32. But, Smith does not disclose the defocus information comprises a defocus map.
With regards to claims 14-15, these claims depend from claim 13 and therefore incorporate the features of thar claim that were found allowable.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 7, 9 and 11-12 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Strandborg et al (US PG Pub. No. 2024/0169494).
With regards to claim 1, Strandborg discloses a processor and memory coupled to the processor storing instructions for performing the steps of its system at ¶ [0038]; ¶ [0134] and FIG. 1.
Strandborg discloses arranging a three-dimensional model of a subject and a camera in a virtual three-dimensional space at: ¶ [0028](“[U]tilising the 3D model of the real-world environment to generate a reconstructed depth map from a perspective of the given camera pose…”); ¶ [0043]; ¶¶ [0136]-[0137].
Strandborg discloses generating a depth map including at least a depth value of a partial region of a region around the subject, based on an image (“image of the real-world”) in which the subject appears at: ¶ [0028](“[R]eceiving a given image of the real-world environment captured using a given camera… [U]tilising the 3D model of the real-world environment to generate a reconstructed depth map from a perspective of the given camera pose…”); ¶ [0043](“surfaces of objects or their parts present in the real-world environment”); ¶¶ [0058]-[0059](“The reconstructed depth map represents depth information … pertaining to objects or their parts present in the real-world environment from the perspective of the given camera pose, said depth information being generated using the 3D model.”) Strandborg further discloses the image (“image of the real-world”) being rendered based on a photographing field of view of the camera, and distance information corresponding to the photographing field of view at: ¶ [0028](“[R]eceiving a given image of the real-world environment captured using a given camera… utilising the 3D model of the real-world environment to generate a reconstructed depth map from a perspective of the given camera pose”); ¶ [0059](“[W]hen utilising the 3D model to generate the reconstructed depth map, the at least one server is configured to employ at least one data processing algorithm… [T]he at least one data processing algorithm is at least one of: an image synthesis algorithm (such as an RGB-D image synthesis algorithm), a view synthesis algorithm, a rendering algorithm.”); ¶¶ [0136]-[0137].
Strandborg discloses generating a defocus map including a defocus amount of the partial region based on a depth value of the partial region and a photographing parameter of the camera at: ¶ [0028](“[A]pplying an extended depth-of-field correction to at least one image segment of the given image that is out of focus, …, based on optical depths in at least one segment of the reconstructed depth map corresponding to the at least one image segment of the given image”) ; ¶ [0091]-[0093](“[T]he EDOF correction could also be applied by utilizing… a defocus map estimation technique…); ¶ [0089](“The depth-of-field may be determined based on at least one of: a focal length of the lens of the given camera, a distance between the object and the given camera, a pixel size, an aperture size, a transmission characteristic of an aperture, a number of apertures (i.e., in case of the given camera having multiple apertures).”); ¶¶ [0136]-[0137]; see, also: ¶ [0061]; ¶ [0076].
With regards to claim 7, Strandborg implicitly discloses the partial region has a size covering at least a part of a face of the subject at ¶ [0043] when it discloses that its "object" refers to a human. One of ordinary skill in the art would infer that an image of a human would include an image of a face, and that the face would have a depth comprising a partial region.
With regards to claim 9, Strandborg inherently discloses the instructions, when executed by the at least one processor, further cause the at least one processor to store the image and the defocus map in association with each other at ¶¶ [0091]-[0093] because in order to defocus the image using the defocus map necessarily requires storing, at least temporarily in RAM, the defocus map and image in association with each other.
With regards to claim 11, the steps performed by the method of this claim are anticipated by Strandborg for the same reasons as were provided in the discussion of claim 1, which recites an apparatus configured to perform these same steps.
With regards to claim 12, the steps stored in the computer readable medium of this claim are anticipated by Strandborg for the same reasons as were provided in the discussion of claim 1, which recites an apparatus configured to perform these same steps.
(Continued on next page)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2-4 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Strandborg et al (US PG Pub. No. 2024/0169494) in view of Zhang (US PG Pub. No. 20210390665).
With regards to claim 2, Strandborg discloses generating a defocus map including a defocus amount at: ¶ [0028]; ¶ [0089]; ¶ [0091]-[0093]; ¶¶ [0136]-[0137]; see, also: ¶ [0061]; ¶ [0076]. But, Strandborg does not specify the defocus amount of the partial region is a value obtained by subtracting a distance to a focal plane of the camera from the depth value of the partial region. However, this limitation was known in the art:
Zhang discloses a defocus amount obtained by subtracting a distance to a focal plane of the camera from the depth value at ¶ [0075] and TABLE 6 (“defocus [u,v] = (depth [u, v] − FOCAL_PLANE_DEPTH)”). At the time of filing of the present application, it would have been obvious to a person of ordinary skill in the art to calculate defocus by subtracting a distance to a focal plane of the camera from the depth value as taught by Zhang, as a substitute for the manner of calculating defocus taught by Strandborg. This combination is a simple substitution of one known element for another to obtain predictable results. The prior art contained a method, taught by Strandborg, which differed from the claimed method by the substitution of the manner of calculating defocus. Calculating defocus by subtracting a distance to a focal plane of the camera from the depth value, and its functions were known in the art as evidenced by the Zhang reference. One of ordinary skill in the art could have substituted the manner of calculating defocus into the method taught by Strandborg and the results would have been predictable; to wit, defocus would be obtained.
With regards to claim 3, Zhang discloses the at least one processor adjusts the subtracted value according to a magnitude of a photographing parameter (e.g., “blur radius”) of the camera at ¶ [0075] and TABLE 6; see, also: ¶ [0035]; ¶¶ [0052]-[0053]. The motivation for the combination is the same as previously presented.
With regards to claim 4, Zhang discloses the at least one processor determines the defocus amount of the partial region based on information in which a diaphragm value corresponding to a predetermined lens of the camera and a depth value of the partial region are associated with each other at ¶¶ [0075]-[0077]; in particular, ¶ [0077]. The motivation for the combination is the same as previously presented.
With regards to claim 8, Zhang discloses the photographing parameter of the camera is a diaphragm value at ¶¶ [0075]-[0077]; in particular, ¶ [0077]. The motivation for the combination is the same as previously presented.
Claims 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Strandborg et al (US PG Pub. No. 2024/0169494) in view of Sugimoto et al (Japanese Pub. No. JP 2015128252 A)
With regards to claim 5, Strandborg discloses generating a depth map including at least a depth value of a partial region of a region around the subject, based on an image (“image of the real-world”) in which the subject appears at: ¶ [0028]; ¶ [0043](“surfaces of objects or their parts present in the real-world environment”); ¶¶ [0058]-[0059]; ¶¶ [0136]-[0137]. Strandborg does not specify the depth value of the partial region is an average value. However, this limitation was known in the art:
Sugimoto discloses the depth value of the partial region is an average value at p. 11 of the English translation. At the time of the filing of the present application, it would have been obvious to a person of ordinary skill in the art to use a mean or mode depth for a region as a depth, as taught by Sugimoto, when generating a depth map according to the method disclosed by Strandborg. The motivation for doing so comes from the prior art wherein, one of ordinary skill in the art would infer as a matter of common sense, the benefits of using mean or mode depth averages would include increased computational efficiency. Therefore, it would have been obvious to combine Sugimoto with Strandborg to obtain the invention specified in this claim.
With regards to claim 6, Sugimoto discloses the depth value of the partial region is a most frequent value at p. 11 of the English translation. The motivation for the combination is the same as previously presented.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID F DUNPHY whose telephone number is (571)270-1230. The examiner can normally be reached 9 am - 5 pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID F DUNPHY/Primary Examiner, Art Unit 2673
Prosecution Insights