DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “depth data acquisition section”, “object surface labeling section”, “image data acquisition section” and “model generation section” as initialized in claim 1, “actual depth data acquisition section” and “depth data estimation section” as initialized in claim 2, and “display control section” as initialized in claim 4.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. In this case, paragraph [0052] states that “the CPU 11 of the model generation device 1 functions as an actual depth data acquisition section 51, a depth data estimation section 52, a surface labeling section 53, an image data acquisition section 54, a three-dimensional model generation section 55, and a display control section 56.”
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 3, 6 is/are rejected under 35 U.S.C. 102(a)(1)/(a)(2) as being anticipated by Kwon et al. (US 2020/0118281 A1).
In regards to claim 1, Kwon teaches an information processing device comprising:
a depth data acquisition section (e.g. Fig.1A: processor 144) that acquires depth data at each of N (N being an integer value of two or higher) viewpoints, the depth data indicating a depth from a predetermined viewpoint to a feature point of a target object (e.g. [0045],Fig.2: in block 204, a 3D image of the object is captured by a 3D imaging sensor, such as 3D imaging sensor 116 in Fig.1A; Fig.3B is a diagram of an example of a 3D image 118 of the object 104 including a 3D point cloud 120; each point 122 in the 3D point cloud 120 includes depth information 132 associated with the corresponding location 126 or point on the object 104 and the 3D point cloud 120 includes or defines a 3D depth map 130, wherein each of the points 122 of the 3D point cloud 120 includes depth information 132 of a corresponding location 126 on the object 104; see also [0036]: the 2D imaging sensor 106 and the 3D imaging sensor 116 are associated with or mounted to a sensor platform 136; the sensor platform 136 is configured to move to different viewpoints 138 or locations to capture 2D images 108 and corresponding 3D images 118 at the different viewpoints 138 or location of the sensor platform 136; Examiner’s note: this shows that depth data for N viewpoints are acquired);
an object surface labeling section (e.g. Fig.1A: processor 144) that generates labeling data indicating a result of labeling a surface of the target object, based on the N pieces of depth data (e.g. [0047],Fig.2: a depth value is assigned from selected points 122 (of the object) of the 3D point cloud 120 or 3D depth map 130 to a matching or corresponding pixel 112 on the 2D image plane 110; Examiner’s note: as per paragraph [0055] of Instant Application’s specification, labeling refers to recording position of a surface of the object in a 3D space; as such, assigning depth points of the object (corresponding to surface points of the object via 3D depth map) to corresponding pixel on the image plane of the 2D image shows recordation of positioning of the object in the 3D space);
an image data acquisition section (e.g. Fig.1A: processor 144) that acquires M (M being an integer value of N or lower) pieces of image data obtained as a result of capturing images of the target object from viewpoints synchronized with M viewpoints, respectively, among the N viewpoints (e.g. [0044],Fig.2: in block 202, a 2D image of the object is captured by a 2D imaging sensor, such as 2D imaging sensor 106 in Fig.1A; as above, [0036]: move to different viewpoints 138 or locations to capture 2D images 108 and corresponding 3D images 118 at the different viewpoints 138; Examiner’s note: this shows that M images are captured corresponding (directly synchronized with) to the depth data for N viewpoints); and
a model generation section (e.g. Fig.1A: processor 144) that generates a three-dimensional model for the target object, based on the M pieces of image data and the labeling data (e.g. [0051],Fig.2: in block 216, the multiple upsampled 3D point clouds 148 are matched or registered and merged to generate the 3D model point cloud 102 of the object 104; Examiner’s note: upsampled 3D points clouds generated based on the aligned depth data and image data for viewpoints, and as such 3D model is based on the aligned data (labeling data)).
In regards to method claim 6, claim(s) 6 recite(s) limitations that is/are similar in scope to the limitations recited in claim 1. Therefore, claim(s) 6 is/are subject to rejections under the same rationale as applied hereinabove for claim 1.
In regards to claim 3, Kwon teaches a device, wherein the depth data acquisition section is configured to acquire the depth data calibrated in such a manner as to have the same viewpoint as each of the M pieces of image data, as the depth data synchronized with the M viewpoints, respectively, among the N viewpoints (e.g. as above, [0036]: the 2D imaging sensor 106 and the 3D imaging sensor 116 are associated with or mounted to a sensor platform 136; the sensor platform 136 is configured to move to different viewpoints 138 or locations to capture 2D images 108 and corresponding 3D images 118 at the different viewpoints 138 or location of the sensor platform 136; Examiner’s note: this shows that depth data for N viewpoints are acquired, corresponding to M viewpoints of image data).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kwon as applied to claim 1 above, and further in view of Kar et al. (US 2022/0335638 A1).
In regards to claim 2, Kwon teaches a device, wherein the depth data acquisition
section includes:
an actual depth data acquisition section that acquires M pieces of actual depth data obtained as a result of measurement from each of the M viewpoints in a real world (e.g. as above, [0045],Fig.2: in block 204, a 3D image of the object is captured by a 3D imaging sensor, such as 3D imaging sensor 116 in Fig.1A; Fig.3B is a diagram of an example of a 3D image 118 of the object 104 including a 3D point cloud 120; each point 122 in the 3D point cloud 120 includes depth information 132 associated with the corresponding location 126 or point on the object 104 and the 3D point cloud 120 includes or defines a 3D depth map 130, wherein each of the points 122 of the 3D point cloud 120 includes depth information 132 of a corresponding location 126 on the object 104; see also [0035]: the 3D imaging sensor 116 is a Light Detection and Ranging (LiDAR) imaging sensor 135 or similar imaging sensor capable of generating a 3D image or 3D point cloud);
but does not explicitly teach the device, comprising:
a depth data estimation section that estimates N pieces of depth data based on the M pieces of actual depth data, and acquires the N pieces of depth data estimated.
However, Kar teaches a device, comprising:
a depth data estimation section that estimates N pieces of depth data based on the M pieces of actual depth data, and acquires the N pieces of depth data estimated (e.g. [0042],Fig.1B: the depth estimate generator 106 may include a visual inertial motion tracker 160, a depth sensor 164, a motion stereo depth estimator 168, a sparse active depth estimator 170, and/or a pre-computed sparse map 172; the depth estimate generator 106 may select one of the sources for use in generating the depth map 138; [0046]: depth sensor 164 is configured to generate the depth estimates 108 based on the image data 104; in some examples, the depth sensor 164 includes a light detection and ranging (LiDAR) sensor; Examiner’s note: as such, the depth estimate generator may estimate N/M pieces of depth data based on the N/M pieces of actual depth data obtained).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings/combination of Kwon to estimate depth data, in the same conventional manner as taught by Kar as both deal with obtaining depth data via LIDAR. The motivation to combine the two would be that it would allow the estimation of depth based on actual depth data obtained or other sources.
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kwon as applied to claim 1 above, and further in view of Hazeghi et al. (US 2019/0108678 A1).
In regards to claim 4, Kwon teaches the device of claim 1, but does not explicitly teach the device, further comprising a display control section that performs control of displaying the three-dimensional model for the target object by drawing only a vicinity of the target object, based on the depth data.
However, Hazeghi teaches a device, further comprising a display control section that performs control of displaying the three-dimensional model for the target object by drawing only a vicinity of the target object, based on the depth data (e.g. [0161]: user interface may also show the current status of the scan, by displaying the combined point cloud that has been acquired so far during the scan (e.g. a “preview” or low fidelity version of the point cloud that is computed by the processing system 40 using decimated depth images)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings/combination of Kwon to display the 3D model, in the same conventional manner as taught by Hazeghi as both deal with generating a 3D model based on image and depth data. The motivation to combine the two would be that it would allow the user to view the current 3D model based on depth data alone.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Kwon and Hazeghi as applied to claim 4 above, and further in view of Kondo (US 2008/0186310 A1).
In regards to claim 5, the combination of Kwon and Hazeghi teaches the device of claim 4, but does not explicitly teach the device, wherein the display control section is configured to perform control of displaying an image drawn in a network representation on an as-is basis that is generated in the three-dimensional model for the target object.
However, Kondo teaches a device, wherein the display control section is configured to perform control of displaying an image drawn in a network representation on an as-is basis that is generated in the three-dimensional model for the target object (e.g. [0011]: an input unit configured to receive polygon data; a processing unit configured to generate a region surrounding an individual polygon, which is represented by the polygon data, measure a distance from a lattice point included in the region to the polygon, and draw a graphic figure on the basis of the measured distance by an implicit-function representation; and an outputting unit configured to display the graphic figure that is obtained by the implicit-function representation; Examiner’s note: paragraph [0062] of the Instant Application’s specification states that the network representation is an implicit function representation).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings/combination of Kwon and Hazeghi to display the 3D model, in the same conventional manner as taught by Kondo as both deal with display of an object. The motivation to combine the two would be that it would allow the displaying of an implicit function representation of the 3D object.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JED-JUSTIN IMPERIAL whose telephone number is (571)270-5807. The examiner can normally be reached Monday to Friday, 9am - 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JED-JUSTIN IMPERIAL/Examiner, Art Unit 2616
/DANIEL F HAJNIK/Supervisory Patent Examiner, Art Unit 2616
