DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of claims 1-20 are pending below.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/4/2024 and 4/9/2025 was filed and considered. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by ONUMA (US 2024/0119668).
Claim 1:
ONUMA (US 2024/0119668) anticipated the following subject matter:
An image processing apparatus comprising:
at least one memory storing instructions; and at least one processor that executes the stored instructions that cause the at least one processor to (figure 9 and 0076):
extract a first subject image from a first image captured by a first imaging unit (figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, first image from plurality of images and first imaging unit/image capturing unit);
extract a second subject image from a second image captured by a second imaging unit figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, second image from plurality of images and second imaging unit/image capturing unit);
estimate a three-dimensional shape of the first subject based on the first subject image extracted (figure 1 and 0016 detail three-dimensional shape estimate unit 3 of the object);
estimate a second subject state based on the second subject image extracted (figure 1 and 0016 detail three-dimensional shape estimate unit 3 of the object);
acquire a predictive image predicted to be acquired in a case where the first subject is captured by the second imaging unit, based on an estimation result of the three-dimensional shape of the first subject and an estimation result of the second subject state (figure 1 and 0016-0017, where 0017 detail estimating 3D shape from different views from first and second imaging unit); and
determine whether the first subject and the second subject are an identical subject by comparing the predictive image with the second subject image extracted (figure 5 and 0045-0053 detail extracted shapes input with object position with comparison using overlapping, previous/current time, identifiers, range of object).
Claim 2:
The image processing apparatus according to claim 1, wherein the estimation result of the second subject state includes orientation information for the second subject and image quality parameter information for the second subject (0019 and 0022 detail imaging unit data above generated with parameter such as position, orientation and optical characteristic, sound).
Claim 3:
The image processing apparatus according to claim 1, wherein the estimation result of the second subject state includes the orientation information for the second subject, and wherein the instructions cause the at least one processor to:
acquire a background image of the predictive image based on a position of the second subject in the second image, and acquire a foreground image of the predictive image by changing a viewpoint to the three-dimensional shape of the first subject based on the orientation information for the second subject (0019 detail parameters including position, orientation, sound collection, background mode, texture, including virtual viewpoint from the captured images and 3D shape of the object (foreground image/object)).
Claim 4:
The image processing apparatus according to claim 1, wherein the estimation result of the second subject state includes the image quality parameter information for the second subject, and wherein the instructions cause the at least one processor to: change the predictive image based on the image quality parameter of the second subject (0035 detail filtering of low, average and high frequency (finer spatial resolution for small structure and fine texture)).
Claim 5:
The image processing apparatus according to claim 1, wherein the estimation result of the three-dimensional shape of the first subject includes information about a position and a color of the first subject in a three-dimensional space (figure 1 and 0016 detail object position calculation unit 13, and 0032 detail display through user GUI with text and color).
Claim 6:
The image processing apparatus according to claim 1, wherein the instructions cause the at least one processor to:
determine whether the first subject and the second subject are an identical subject based on a distance between feature vectors acquired by inputting the predictive image (figure 5 and 0045-0053 detail extracted shapes input with object position with comparison using overlapping, previous/current time, identifiers, range/distance of object) and the second subject image to a network model (0069-0071 detail external storage device 906 use for image data for processing target, where external storage device is network local or internet).
Claim 7:
The image processing apparatus according to claim 1, wherein the instructions cause the at least one processor to: determine whether the first subject and the second subject are an identical subject based on an output result acquired by inputting a difference between the predictive image and the second subject image to a network model (figure 5 and 0045*0053 and 0069-0071 detail comparing with parameters as well as using external image data for same processing of target/object).
Claim 8:
The image processing apparatus according to claim 1, wherein an imaging range of the first imaging unit does not overlap with an imaging range of the second imaging unit (figure 5 and 0045-0053 detail extracted shapes input with object position with comparison using overlapping, previous/current time, identifiers, range/distance of object).
Claim 9:
The image processing apparatus according to claim 1, wherein a joint position of the second subject is estimated based on the second subject image, and wherein the instructions cause the at least one processor to:
generate the predictive image by changing a joint position of the three-dimensional shape of the first subject based on a joint position estimation result of the second subject (0055-0059, paragraph 0055 detail 3D shape estimate with clipping of the shape by mean of height such as knees and waist (joint position) up to top surface of structure/object).
Claim 10:
The image processing apparatus according to claim 1, wherein the instructions cause the at least one processor to:
detect an occluded area for the second subject; and exclude a region corresponding to the occluded area in the predictive image from one or more comparison targets (above detail predictive and comparing, where 0046-0048 detail overlapping of objects position and time, where determine extracted shape with no identifier at time of completion is non-tracking target (occluded area of subject, exclude)).
Claim 11:
The image processing apparatus according to claim 10, wherein the instructions cause the at least one processor to:
estimate the joint position of the second subject and a likelihood of the joint position based on the second subject image (0057 detail joints estimated by predetermined height for knees and waist of each object from plurality of images); and determine the occluded area for the second subject based on an estimation result of the joint position likelihood of the second subject (above detail estimating of 3D shape and its data and parameter, where 0046-0048 detail overlapping of objects position and time, where determine extracted shape with no identifier at time of completion is non-tracking target (occluded area of subject, exclude)).
Claim 12:
An image processing apparatus comprising:
at least one memory storing instructions; and at least one processor that executes the stored instructions that cause the at least one processor to (figure 9 and 0076):
extract a first subject image from a first image captured by a first imaging unit (figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, first image from plurality of images and first imaging unit/image capturing unit);
extract a second subject image from a second image captured by a second imaging unit (figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, second image from plurality of images and second imaging unit/image capturing unit);
estimate a three-dimensional shape of the first subject based on the first subject image extracted (figure 1 and 0016 detail three-dimensional shape estimate unit 3 of the object);
estimate, in a case where the first subject is captured by the second imaging unit, a first subject state in a captured image based on the first subject image (figure 1 and 0016-0017, where 0017 detail estimating 3D shape from different views from first and second imaging unit);
acquire a predictive image predicted to be acquired, in a case where the first subject is captured by the second imaging unit, based on an estimation result of the three-dimensional shape of the first subject and an estimation result of the first subject state (figure 1 and 0016-0017, where 0017 detail estimating (predictive) 3D shape from different views from first and second imaging unit); and
determine whether the first subject and the second subject are an identical subject by comparing the predictive image with the second subject image (figure 5 and 0045-0053 detail extracted shapes input with object position with comparison using overlapping, previous/current time, identifiers, range of object).
Claim 13:
The image processing apparatus according to claim 12, wherein the instructions cause the at least one processor to:
estimate a plurality of candidates for the first subject state in a case where the first subject is captured by the second imaging unit (figure 2A and 0029 detail shape extraction for each object (estimating candidates)); and generate a plurality of the predictive images according to the plurality of candidates (figure 1 and 0016-0017, where 0017 detail estimating 3D shape from different views from first and second imaging unit).
Claim 14:
The image processing apparatus according to claim 12, wherein the instructions cause the at least one processor to:
predict a joint position of the first subject to be captured by the second imaging unit; and generate the predictive image by changing the joint position of the three-dimensional shape of the first subject based on the joint position predicted (0055-0059, paragraph 0055 detail 3D shape estimate with clipping of the shape by mean of height such as knees and waist (joint position) up to top surface of structure/object).
Claim 15:
The image processing apparatus according to claim 14, wherein the instructions cause the at least one processor to: store a joint position of the second subject captured by the second imaging unit; and predict the joint position of the first subject in the predictive image based on the joint position of the second subject stored (0028-0029 detail estimating joint position by height, where 0029 detail using predetermined (stored) height).
Claim 16:
An image processing apparatus comprising:
at least one memory storing instructions; and at least one processor that executes the stored instructions that cause the at least one processor to (figure 9 and 0076):
extract a first subject image from a first image captured by a first imaging unit (figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, first image from plurality of images and first imaging unit/image capturing unit);
extract a second subject image from a second image captured by a second imaging unit (figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, second image from plurality of images and second imaging unit/image capturing unit);
estimate a second subject state based on the second subject image extracted (figure 1 and 0016 detail three-dimensional shape estimate unit 3 of the object);
acquire a predictive image predicted to be acquired, in a case where the first subject is captured by the second imaging unit, based on the first subject image extracted and an estimation result of the second subject state (figure 1 and 0016-0017, where 0017 detail estimating 3D shape from different views from first and second imaging unit); and
determine whether the first subject and the second subject are an identical subject by comparing the predictive image with the second subject image extracted (figure 5 and 0045-0053 detail extracted shapes input with object position with comparison using overlapping, previous/current time, identifiers, range of object).
Clam 17:
An image processing method (figure 5 teaches flowchart/method) comprising:
extracting, as a first extraction, a first subject from a first image captured by a first imaging unit (figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, first image from plurality of images and first imaging unit/image capturing unit);
extracting, as a second extraction, a second subject from a second image captured by a second imaging unit (figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, second image from plurality of images and second imaging unit/image capturing unit);
estimating, as a first estimation, a three-dimensional shape of the first subject based on a first extraction result as a result of the extracting as the first extraction (figure 1 and 0016 detail three-dimensional shape estimate unit 3 of the object);
estimating, as a second estimation, a second subject state based on a second extraction result as a result of the extracting as the second extraction (figure 1 and 0016 detail three-dimensional shape estimate unit 3 of the object);
generating a predictive image in a case where the first subject is captured by the second imaging unit based on an estimation result of the three-dimensional shape of the first subject and an estimation result of the second subject state (figure 1 and 0016-0017, where 0017 detail estimating 3D shape from different views from first and second imaging unit); and
identifying an identity of the first subject and the second subject by comparing the predictive image with the second subject extracted in the extracting as the second extraction (figure 5 and 0045-0053 detail extracted shapes input with object position with comparison using overlapping, previous/current time, identifiers, range of object).
Claim 18:
An image processing method (figure 5 teaches flowchart/method) comprising:
extracting, as a first extraction, a first subject image from a first image captured by a first imaging unit (figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, first image from plurality of images and first imaging unit/image capturing unit);
extracting, as a second extraction, a second subject image from a second image captured by a second imaging unit (figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, second image from plurality of images and second imaging unit/image capturing unit);
estimating, as a first estimation, a three-dimensional shape of the first subject based on the first subject image extracted (figure 1 and 0016 detail three-dimensional shape estimate unit 3 of the object);
estimating, as a second estimation, a first subject state in a case where the first subject is captured by the second imaging unit based on the first subject image extracted (figure 1 and 0016 detail three-dimensional shape estimate unit 3 of the object);
acquiring a predictive image predicted to be acquired in a case where the first subject is captured by the second imaging unit based on an estimation result of the three-dimensional shape of the first subject and an estimation result of the first subject state (figure 1 and 0016-0017, where 0017 detail estimating 3D shape from different views from first and second imaging unit); and
determining whether the first subject and the second subject are an identical subject by comparing the predictive image with the second subject image extracted (figure 5 and 0045-0053 detail extracted shapes input with object position with comparison using overlapping, previous/current time, identifiers, range of object).
Claim 19:
An image processing method (figure 5 teaches flowchart/method) comprising:
extracting, as a first extraction, a first subject image from a first image captured by a first imaging unit (figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, first image from plurality of images and first imaging unit/image capturing unit);
extracting, as a second extraction, a second subject image from a second image captured by a second imaging unit (figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, second image from plurality of images and second imaging unit/image capturing unit);
estimating, as a second estimation, a second subject state based on the second subject image extracted (figure 1 and 0016 detail three-dimensional shape estimate unit 3 of the object);
acquiring a predictive image predicted to be acquired in a case where the first subject is captured by the second imaging unit based on the first subject image extracted and an estimation result of the second subject state (figure 1 and 0016-0017, where 0017 detail estimating 3D shape from different views from first and second imaging unit); and
determining whether the first subject and the second subject are an identical subject by comparing the predictive image with the second subject image extracted in the extracting the second subject image (figure 5 and 0045-0053 detail extracted shapes input with object position with comparison using overlapping, previous/current time, identifiers, range of object).
Claim 20:
A non-transitory computer-readable storage medium storing a program for causing a computer to execute a method comprising (0076 detail non-transitory computer-readable storage medium):
extracting, as a first extraction, a first subject image from a first image captured by a first imaging unit (figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, first image from plurality of images and first imaging unit/image capturing unit);
extracting, as a second extraction, a second subject image from a second image captured by a second imaging unit (figure 1 and 0016 teaches plurality of image capturing unit 1 of the object and its position 13 this would include first subject/object, second image from plurality of images and second imaging unit/image capturing unit);
estimating, as a first estimation, a three-dimensional shape of the first subject based on the first subject image extracted (figure 1 and 0016 detail three-dimensional shape estimate unit 3 of the object);
estimating, as a second estimation, a second subject state based on the second subject image extracted (figure 1 and 0016 detail three-dimensional shape estimate unit 3 of the object);
acquiring a predictive image predicted to be acquired in a case where the first subject is captured by the second imaging unit based on an estimation result of the three-dimensional shape of the first subject and an estimation result of the second subject state (figure 1 and 0016-0017, where 0017 detail estimating 3D shape from different views from first and second imaging unit); and
determining whether the first subject and the second subject are an identical subject by comparing the predictive image with the second subject image extracted in the extracting as the second extraction (figure 5 and 0045-0053 detail extracted shapes input with object position with comparison using overlapping, previous/current time, identifiers, range of object).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kanatsu et al (US 2018/0204381) teaches IMAGE PROCESSING APPARATUS FOR GENERATING VIRTUAL VIEWPOINT IMAGE AND METHOD THEREFOR - two rendering modes, i.e., model-based rendering (MBR) and image-based rendering (IBR), are used. MBR is a method of generating a virtual viewpoint image using a three-dimensional model generated based on a plurality of captured images obtained by performing image capturing of a subject from a plurality of directions. Specifically, MBR is a technique to generate an appearance of a scene viewed from a virtual viewpoint as an image using a three-dimensional shape (model) of a target scene obtained by a three-dimensional shape reconstruction method, such as a visual volume intersection method and multi-view stereo (MVS). IBR is a technique to generate a virtual viewpoint image in which an appearance viewed from a virtual viewpoint is reconstructed by deforming and combining an input image group obtained by performing image capturing of a target scene from a plurality of viewpoints (paragraph 0099).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TSUNG-YIN TSAI whose telephone number is (571)270-1671. The examiner can normally be reached 7am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached at (571) 272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TSUNG YIN TSAI/Primary Examiner, Art Unit 2656
Prosecution Insights