Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 4-5, 7-8, 11-12, 14-15, 18-19 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shao et al. (“An interactive Approach to Semantic Modeling of Indoor Scenes with an RGBD Camera”).
Re claim 1, Shao teaches a method comprising:
obtaining a first multi-angle image of a first target scene (see Fig. 1 and 2, wherein at least a first image of a multi-angle image of a first scene is obtained)
recognizing, based on the first multi-angle image, target objects in the first target scene;
obtaining semantic features of the target objects (see Fig. 1 and 2, wherein at least a first image of a multi-angle image of a first scene is obtained and segmented into semantic regions) and (see p. 3, 3. Interactive Context-aware Image Segmentation and Labeling, wherein each pixel is assigned a semantic label).
obtaining, from a model library, first three-dimensional models that match the semantic features, wherein the first three-dimensional models comprise physical parameters of the target objects (see Fig. 2-3, wherein objects are segmented based on semantic segmentation to match the physical parameters of the captured target objects, such as 3d models of chairs for physically captured chairs. The 3d models from a database library are used to match the semantic features, as seen in Fig. 2).
generating, using the first three-dimensional models, a first three-dimensional twin model corresponding to the first target scene (see Fig. 1-3, wherein a first 3d twin model corresponds to a first target object, such as 3D chairs and 3D tables).
Re claim 4, Shao teaches claim 1. Furthermore, Shao teaches wherein recognizing the target objects comprises: performing segmentation on the first multi-angle image to obtain second images and recognizing the second images to determine the target objects.
(see Fig. 2-3, wherein segmentation of input images is performed, and the target objects in the second images is determined and recognized, such as chairs, walls, sofas, tables, etc.).
Re claim 5, Shao teaches claim 1. Furthermore, Shao teaches wherein recognizing the target objects comprises:
generating, based on the first multi-angle image, a second three-dimensional twin model; and performing segmentation on the second three-dimensional twin model to obtain the target objects (see Fig. 1-3, wherein based on a first multi-angle image, more than one target object is segmented, and recognized, so that a first and second 3d model twin is obtained, based on the model database, for example Fig. 2 image 1 is segmented and reconstructed with model matching at least first and second 3d twin models).
Re claim 7, Shao teaches claim 1. Furthermore, Shao teaches wherein the physical parameters comprise one or more of: a shape (see p. 3, 3d shape matching & Fig. 1-3, wherein 3d shape recognition is performed).
Claim 8 claims limitations in scope to claim 1 and is rejected for at least the reasons above. Furthermore, Shao teaches an apparatus comprising memory configured to store instructions and one or more processors coupled to memory (see p. 2, computer graphics, which implicitly includes a computer with processor and memory).
Claims 11-12 claim limitations in scope to claims 4-5 and are rejected for at least the reasons above.
Claims 14 claim limitations in scope to claims 7 and is rejected for at least the reasons above.
Claims 15 claim limitations in scope to claim 1 and is rejected for at least the reasons above.
Claims 18-19 claim limitations in scope to claims 4-5 and are rejected for at least the reasons above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2-3, 9-10, 16-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shao et al. (“An interactive Approach to Semantic Modeling of Indoor Scenes with an RGBD Camera”) and Mannis et al. “Vid2CAD: CAD Model Alignment Using Multi-View Constraints from Videos”).
Re claim 2, Shao teaches claim 1. Shao does not explicitly teach obtaining a second multi-angle image of the first three-dimensional twin model; and adjusting, based on a difference between the first multi-angle image and the second multi-angle image, the first three-dimensional twin model.
However, Mannis teaches obtaining a second multi-angle image of the first three-dimensional twin model; and adjusting, based on a difference between the first multi-angle image and the second multi-angle image, the first three-dimensional twin model (see Fig. 1, wherein input video comprises a plurality of input multi-angle images including at least first and second multi-angle images) and (see Fig. 4, based on differences in the images, such as input images on the left, a center and right image shows the chair object adjusted based on a difference between the two multi-angle images).
Shao and Mannis teach claim 2. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shao’s multi-angle image 3d reconstruction system to explicitly include adjusting, based on a difference between the first multi-angle image and the second multi-angle image, the first 3d twin model, as taught by Mannis, as the references are in the analogous art of multi-angle image 3d reconstruction systems for generating 3d twin models. An advantage of the modification is that it achieves the result of allowing for adjustment and modification of differences between multi-angle images to further adjust the 3d reconstruction of a scene, improving the accuracy of the model based on additional information from different angle images.
Re claim 3, Shao teaches claim 1. Shao does not explicitly teach further comprising adjusting, based on a second target scene, a model parameter of the first three-dimensional twin model to obtain a second three-dimensional twin model corresponding to the second target scene, wherein the second target scene comprises the target objects, and wherein the second target scene and the first target scene have different environments.
However, Mannis teaches further comprising adjusting, based on a second target scene, a model parameter of the first three-dimensional twin model to obtain a second three-dimensional twin model corresponding to the second target scene, wherein the second target scene comprises the target objects, and wherein the second target scene and the first target scene have different environments (see Fig. 1, wherein input video comprises a plurality of input multi-angle images including at least first and second multi-angle images, including target objects) and (see Fig. 4, based on differences in the images, such as input images on the left, a center and right image shows the chair object adjusted on scale, position, rotational parameters based on a difference between the two multi-angle images, thus obtaining a second 3d twin model corresponding to the second image, the first and second target scenes are at different angles of a scene, and is thus different environments). Shao and Mannis teach claim 3. For motivation, see claim 1.
Claims 9-10 claim limitations in scope to claims 2-3 and are rejected for at least the reasons above.
Claims 16-17 claim limitations in scope to claims 2-3 and are rejected for at least the reasons above.
Claim(s) 6, 13, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shao et al. (“An interactive Approach to Semantic Modeling of Indoor Scenes with an RGBD Camera”) and Hansen (US 20150248719).
Re claim 6, Shao teaches claim 1. Shao does not explicitly teach making a determination that the target objects do not match a second three-dimensional model in the model library;
generating, in response to the determination and based on the target objects, a third three-dimensional model; and storing the third three-dimensional model in the model library.
However, Hansen teaches making a determination that the target objects do not match a second three-dimensional model in the model library; generating, in response to the determination and based on the target objects, a third three-dimensional model; and storing the third three-dimensional model in the model library ([0040] Some embodiments of the invention relate to identifying objects from a 3D model captured from a personal electronic device. The 3D model generated from imagery captured by the portable electronic device may be compared to a database of known objects. The comparison matches features of the 3D model of the target object (i e, unknown object) with known features of known objects. In a preferred embodiment, the comparison includes comparing at least one three-dimensional feature of the target object with a three dimensional feature of a known object. The database search can include scanning a plurality of identified three-dimensional features of the target object with known features of the object data stored in the database), and ([0074] The 3D data (i.e., model data) of the reference object and/or the size information for the target object may be stored and/or added to a database for use as a reference object (i.e., data obtained for target objects may be used as a reference object data for future searches). The target object data may be stored locally on a personal electronic device or on a server computer (local or cloud storage).
Shao and Hansen teach claim 6. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shao’s multi-angle imaging system to recognize captured object for 3d modeling to explicitly include determining non-matching target objects, generating 3d models in response, and storing the 3d model, as taught by Hansen, as the references are in the analogous art of identifying objects from captured devices. An advantage of the modification is that it achieves the result of generating and storing 3d model data of non-matching/unknown objects for later use/recognition, as taught by Hansen.
Claims 13 claim limitations in scope to claim 6 and is rejected for at least the reasons above.
Claims 20 claim limitations in scope to claim 6 and is rejected for at least the reasons above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Peter Hoang whose telephone number is (571)270-1346. The examiner can normally be reached Monday-Friday 8:00 am - 5:00 pm PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hajnik F. Daniel can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER HOANG/Primary Examiner, Art Unit 2616
Prosecution Insights