Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Art Unit – Location
The Art Unit location of your application in the USPTO may have changed. To aid in correlating any papers for this application, all further correspondence regarding this application should be directed to Art Unit 2682.
Please refer to:
Marrion US 2010/0166294 A1 “Marrion”
Upendran US 2021/0183161 A1 “Upendran”
The Applicant argues: Marrion does not disclose camera poses.
The Examiner finds that Marrion teaches object poses, but not specifically camera poses.
However, Upendran teaches camera poses in FIGURE 1A and FIGURE 1B.
The Applicant argues: Marrion fails to update any geometry of a 3D model.
The Examiner responds: Marrion does not appear to update a 3D model, but Upendran teaches: to “estimate relative camera positions, followed by scaling to update those camera positions, and use the techniques described above to derive the final camera positions and then fit the model geometry to that scale. Scaling factors, then, can be determined concurrent with image capture or concurrent with constructing a 3-D representation. [0100] Upendran updates a 3-D model.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-24 are rejected under 35 U.S.C. 101 because:
The claimed invention is directed to an Abstract Idea without significantly more. The claim(s) recite(s) a mental process and mathematical manipulations. This judicial exception is not integrated into a practical application because there appears to be no significant output from the claims. The claim(s) does/do not include additional elements that amount to significantly more than the judicial exception because there is no technical improvement to the model or the processor.
Furthermore, the proposed amended claim 1 is not similar to the case of Research Corporation Technologies Inc. v. Microsoft Corp. Claim 1 does not tie the method to a particular machine.
The claim is entirely a sequence of geometric computations and information-organizing steps (identifying 2D/3D line segments, triangulating from image pairs, grouping pairs based on a parameter, and selecting “consistent” pose pairs). It does not recite any particular machine, sensor hardware, or a specific non-conventional algorithm that improves computer functionality; nor does it specify concrete thresholds, error metrics, or implementation details beyond generic “triangulating,” “grouping,” and “selecting.” Under the USPTO’s 2019 Subject Matter Eligibility Guidance and Alice/Mayo, these are mathematical concepts and data analysis performed on a computer—an abstract idea—without additional elements that integrate the idea into a practical application or provide an inventive concept.
Detailed Alice/Mayo analysis
Step 2A, Prong One (identify a judicial exception)
The claim recites mathematical concepts:
“triangulating” 2D line segments to form 3D line segments is geometric computation;
“grouping … based on a parameter” is data analysis/classification by a rule or metric;
“selecting consistent camera pose pairs from a group” is a decision rule.
Courts have treated claims focused on mathematical processing as abstract (e.g., Alice, SAP v. InvestPic, Digitech).
Step 2A, Prong Two (integrate into a practical application?)
The steps are applied to “images,” “camera poses,” and “3D models,” but the claim:
does not tie the method to a particular machine or a transformation of an article (no camera/processor details, no improved hardware operation);
does not recite a specific technical improvement in image processing or pose estimation (e.g., defined error metrics, constraints, or a novel pipeline);
uses result-oriented language (“grouping … based on a parameter,” “selecting consistent pairs”) without specifying how the computer is improved.
This is field-of-use application of mathematical concepts, not an integration into a practical application per the Guidance.
Conclusion: Directed to an abstract idea.
Step 2B (inventive concept)
Additional elements are generic data-gathering and processing:
“receiving a 3D model,” “identifying images,” “identifying a 2D line segment” are routine pre- and post-solution activity;
triangulation and grouping by a parameter are well-understood geometric/statistical operations in computer vision;
no recited unconventional data structures, algorithms, or hardware arrangement that “significantly more” transforms the claim.
Conclusion: No inventive concept; The claims are ineligible under § 101.
Therefore, claims 1-24 are rejected.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-24 are rejected under 35 U.S.C. 103 as being unpatentable over
Marrion et al. (US 2010/0166294 A1) “Marrion” in view of Upendran et al. (US 2021/0183161 A1) “Upendran”.
1. Marrion teaches: A method ("METHOD" [Title]) of grouping different camera poses ("a high-scoring pose can be a best, highest, or top-scoring pose of a group of candidate 3D poses" [0063]) , the method comprising:
receiving a 3d model comprising a plurality of 3d line segments ("pre-existing 3D model" [0008]; “ requires use of a mathematical model of the object “ [0039].) ;
identifying at least first, second, and third images that observe a first 3d line segment of the plurality of 3d line segments, wherein each of the first, second, and third images is associated with a pose (“line segments” [ABSTRACT] viewed by Cameras "120", “122”, and “170” [FIG. 1] ; [0028] each having a different pose.) ;
identifying a 2d line segment in each of the first, second, and third images that corresponds to the first 3d line segment ("the points can define a particular HLGS on the object, such as line segments" [0011]) ;
Marrion does not explicitly teach:
triangulating the 2d line segment of the first image and the second image associated with first and second camera poses to create a second 3d line segment;
triangulating the 2d line segment of the first image and the third image associated with first and third camera poses to create a third 3d line segment;
triangulating the 2d line segment of the second image and the third image associated with the second and third camera poses to create a fourth 3d line segment;
However Upendran in an analogous field of endeavor teaches:
The camera poses as shown in FIGURE 1A and FIGURE 1B;
triangulating the 2d line segment of the first image and the second image associated with first and second camera poses to create a second 3d line segment (e.g. a line segment “406” as shown in Figure 4 from vantage point “432” using triangulation.);
triangulating the 2d line segment of the first image and the third image associated with first and third camera poses to create a third 3d line segment (e.g. a line segment “406” as shown in Figure 4 from vantage point “434” using triangulation.);
triangulating the 2d line segment of the second image and the third image associated with the second and third camera poses to create a fourth 3d line segment (e.g. a line segment “406” from an arbitrary vantage point in or outside the range of vantage points 432 and 434 as shown by the multiple vantage point poses in Figures 1A and 1B which can use triangulation as shown in Figure 4 using the camera poses.).
The identification of line segments in Marrion can be modified by Upendran to identify line segments by triangulation of different camera poses as shown in FIG. 1H of Upendran showing the line point reconstruction.
The motivation for the combination is provided by Upendran to identify inlier features using “image feature triangulation” [0112].
Furthermore, the combination of Marrion and Upendran teach:
grouping pose pairs, into a plurality of groups, based on a parameter of the second 3d line segment, the third 3d line segment, and the fourth 3d line segment ("GENERATE POSES WHICH MAP FOUND LINE SEGMENT TO BE ALMOST PARALLEL TO MODEL LINE SEGMENT 722" shown in FIG. 7. “poses can be generated to map the found line pair to the model line pair, namely: Group 1, based on first mapping directed line aa' to directed line AA” [0068] of Marrion.);
selecting consistent camera pose pairs from a group comprising a largest number of camera pose pairs ("SELECT THE TOP 128 POSES WITH THE HIGHEST SCORES “1920" ”[FIG. 19]; of Marrion.); ("SELECT THE 8 POSES WITH THE HIGHEST MODEL COVERAGE SCORES 1930" [FIG. 19] of Marrion.) .
Furthermore, the combination of Marrion and Upendran teach: updating geometry of the 3d model based on the selected consistent camera pose pairs ("estimate relative camera positions, followed by scaling to update those camera positions, and use the techniques described above to derive the final camera positions and then fit the model geometry to that scale. Scaling factors, then, can be determined concurrent with image capture or concurrent with constructing a 3-D representation. " [0100] of Upendran.) .
The model of Marrion can be updated by Upendran for a better fitting 3D representation.
The motivation for the combination is provided by Upendran “ to generate accurate measurements of the building or generate realistic rendering of 3-D models of the building” [0005].
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
2. The method of claim 1, further comprising generating the 3d model based on at least the first, second, and third images ("GENERATE 3d POINT CLOUD OF OBJECT 220" [FIG. 2] of Marrion.) , wherein generating the 3d model comprises adjusting at least one of the poses associated with the first, second, or third images ("REDUCE THE NUMBER OF CANDIDATE POSES 260." [FIG. 2] of Marrion.) .
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
3. The method of claim 1, wherein the parameter is length ("the N 2D points of the line segment are sorted according to their distance along the segment…and the Nth has a distance of L, which corresponds to the total length of the segment” [0055] of Marrion.) .
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
4. The method of claim 1, wherein a bin size of each group of the plurality of groups is based on one or more expected values of the first 3d line segment ("each dimension is split into uniform bins 1810 (for example 2 mm.times.2 mm.times.2 mm can be used as the bin size for a reference object" [0104] of Marrion. the first line segment can be 60 mm [0104] of Marrion. 60 mm is an expected value.) .
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
5. The method of claim 4, wherein the one or more expected values are based on the first 3d line segment (the first line segment can be 60 mm [0104] of Marrion. 60 mm is an expected value).
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
6. The method of claim 4, wherein the one or more expected values are based on a semantic class associated with the first 3d line segment ("a measurements module 228 that calculates measurements of dimensions of a building structure (e.g., walls, dimensions of doors of the house 102) based on scaling the 3-D model generated by the 3-D model generation module 220 and the scaling factor generated by the scale calculation module 226 " [0071]. “The scale as derived from the single wall can be assigned to the entire resultant 3-D building” [0109] of Upendran. A wall or a door is a semantic class.) .
The identification of objects of Marrion can be modified by Upendran to associate a class of an object to the identified object.
The motivation for the combination is provided by Upendran “visualization tools can produce significant cost savings. Using accurate 3-D models of properties, homeowners, for instance, can estimate and plan every project.” [0004].
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
7. The method of claim 4, wherein each expected value of the one or more expected values is based on an industry standard value ("objects with predetermined or industry standard sizes" [0017] of Upendran.) .
The identification of objects of Marrion can be modified by Upendran to associate an object with an industry standard value such as size.
The motivation for the combination is provided by Upendran “visualization tools can produce significant cost savings. Using accurate 3-D models of properties, homeowners, for instance, can estimate and plan every project.” [0004].
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
8. The method of claim 4, wherein the bin size is a percentage of the one or more expected values ("2 mm.times.2 mm.times.2 mm can be used as the bin size for a reference object of 60 mm.times.60 mm.times.60 mm)." [0104] of Marrion.) . A 2 mm bin size is about 3% of the 60mm object.
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
9. The method of claim 8, wherein the percentage corresponds to an error threshold ("any noise encountered within the scoring can be addressed by applying a distance tolerance value" [0109] of Marrion.) .
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
10. The method of claim 1, further comprising: calculating a scaling factor derived from on the selected poses and the first 3d line segment; and updating the 3d model based on the scaling factor, wherein updating comprises scaling the 3d model based on the scaling factor ("calculating a scaling factor for a 3-D representation of the building structure based on correlating the reference poses with the selected candidate poses… for scaling the 3-D representations of the building structure" [ABSTRACT] of Upendran.) .
The 3D model of Marrion can be modified by Upendran to include a scaling factor.
The motivation for the combination is provided by Upendran “visualization tools can produce significant cost savings. Using accurate 3-D models of properties, homeowners, for instance, can estimate and plan every project.” [0004].
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
11. The method of claim 1, wherein updating geometry of the 3d model comprises generating a new 3d representation based on the selected poses (The 3D model can be represented based on "TOP CANDIDATE POSES SELECTED 1940" [FIG. 19] of Marrion. Selecting top candidate poses can be used to generate a new 3D representation.). Furthermore, "estimate relative camera positions, followed by scaling to update those camera positions, and use the techniques described above to derive the final camera positions and then fit the model geometry to that scale. Scaling factors, then, can be determined concurrent with image capture or concurrent with constructing a 3-D representation." [0100] of Upendran.) .
The model of Marrion can be updated by Upendran for a better fitting 3D representation.
The motivation for the combination is provided by Upendran “ to generate accurate measurements of the building or generate realistic rendering of 3-D models of the building” [0005].
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
12. The method of claim 1, further comprising generating a new pose solution based on the selected poses (The 3D model can be represented based on "TOP CANDIDATE POSES SELECTED 1940" [FIG. 19] of Marrion.) .
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
13-24. The non-transient computer-readable storage medium of claims 13 through 24 have been analyzed in view of the non-transitory computer-readable storage medium of Marrion “the systems and methods described herein can be implemented using electronic hardware, software, consisting of a computer-readable medium of program instructions” [0028, 0131]; “a non-transitory computer readable storage medium” [0022] of Upendran, and further in view of Applicants claims 1-12 respectively.
Claims 13-24 are rejected in a similar manner to claims 1-12.
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
Relevant Prior Art
Quan et al. US 8,798,965 B2
Abstract
The subject disclosure relates to generating models from images. In an aspect, multi-view semantic segmentation is provided to recognize and segment images at the pixel level into semantically meaningful areas, and which can provide labels with a specific object class. In further aspects, a partition scheme is provided that can separate objects into independent blocks using major line structures of a scene. In addition, an inverse patch-based orthographic composition and structure analysis on a block is provided that can regularize noisy and missing reconstructed 3D data to facilitate image-based modeling.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TED W BARNES whose telephone number is (571)270-1785. The examiner can normally be reached Mon-Fri. 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Tieu can be reached at 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TED W. BARNES/ Ph.D. Electrical Engineering
Primary Examiner
Art Unit 2682
/TED W BARNES/Primary Examiner, Art Unit 2682
Prosecution Insights