DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 8 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nickerson et al. (PGPUB Document No. US 2023/0359709) in view of Sawhney et al. (PGPUB Document No. US 2020/0372715) in view of Guan et al. (“A Dynamic Scene Vision SLAM Method Incorporating Object Detection and Object Characterization,” (2023), Sustainability, 15(4), 3048. https://doi.org/10.3390/su15043048).
Regarding claim 8, Nickerson teaches an electronic device, comprising:
At least one memory and at least one processor;
Wherein the at least one memory is configured to store a program code, and the at least one processor is configured to call the program code stored in the at least one memory to execute a method comprising:
Obtaining an environment image of a real environment (“augmented reality engine 304b may be running an augmented reality application” (Nickerson: 0073). “images of the physical environment were taken and objects identified” (Nickerson: 0081));
Identifying a corner point and/or an edge line of a target object in the environment image based on a vision algorithm (identifying objects in the identified location 806, comprising the steps of processing feature points of an object in an image frame (Nickerson: 0077). Further, Nickerson disclose the use of univalue segment assimilating nucleus (SUSAN) for feature detection that corresponds to the same SUSAN vision algorithm disclosed in the Applicant’s specification at para 0042);
And automatically annotating the target object in the environment image based on the identified corner point and/or edge line (“If it is determined that the object of the image frame substantially corresponds with an object identified in the augmented reality profile in decision block 810 of method 800, then method 800 proceeds to block 812, where the annotation content of the augmented reality profile is rendered relative to the object according to rendering instructions provided with the annotation content.” (Nickerson: 0083)).
However, Nickerson does not expressly teach but Sawhney teaches the annotation being a label of the target object (“Upon recognizing the real-world object, computing device 102 provides an augmentation 114 relating to the real-world object…specifically as a label that denotes a recognized identity of the real-world object.” (Sawhney: 0016, FIG.1)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the teachings of Nickerson such as to label the identified objects as taught by Sawhney, because this further aids the user in recognizing objects within the AR view.
Further, the combined teachings above do not expressly teach but Guan teaches wherein a spatial position of the target object is determined (The three-dimensional coordinates in the world coordinate system of all feature points, as well as the RGB information for these points, can then be obtained using Equation (4) (Guan: 3.4. Creation of Dense Point Cloud Map, para 2)) based on the corner point and/or edge line of the target object (“object detection on the read image data to classify the object in the scene…after the extraction of ORB feature points for the current frame is completed in the tracking module…stable static feature points are used in the subsequent tracking and map construction process” (Guan: 3.Methods, para 2)). Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to apply the SLAM teachings of Guan to the combined teachings above , because this enables an effective method of improving accuracy of constructing an XR environment.
Claim(s) 1 are corresponding method claim(s) of claim(s) 8. The limitations of claim(s) 1 are substantially similar to the limitations of claim(s) 8. Therefore, it has been analyzed and rejected substantially similar to claim(s) 1.
Claim(s) 15 are corresponding computer readable storage medium claim(s) of claim(s) 8. The limitations of claim(s) 1 are substantially similar to the limitations of claim(s) 8. Therefore, it has been analyzed and rejected substantially similar to claim(s) 1. Note, Nickerson teaches a non-transitory, machine readable medium, as presently claimed (Nickerson: 0099).
Claim(s) 2, 6, 9, 13, 16 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nickerson in view of Sawhney in view of Guan as applied to the claim(s) above, and further in view of Jovanovic et al. (PGPUB Document No. US 2019/0051054).
Regarding claim 9, the combined teachings above do not expressly teach but Jovanovic teaches the device of claim 8, wherein the method further comprises: constructing a model for the target object in an extended reality space based on a labeling result of the automatically labeling (generating an annotated 3D model of space using AR, wherein the process comprise of detecting corners of objects (Jovanovic: 0002-0003)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the modeling teaching of Jovanovic, because this enables the creation of AR space within which the user can interact within (Jovanovic: 0002).
Regarding claim 13, he combined teachings above do not expressly teach but Jovanovic teaches the device of claim 8, wherein automatically labeling the target object in the environment image based on the identified corner point and/or edge comprises:
Obtaining three-dimensional space information about the corner point and/or edge line of the target object (identifying corners within the environment to create a 3D model of the space (Jovanovic: 0044));
Rendering the corner point and/or edge line of the target object in the identified environment image based on the three-dimensional space information (the displayed corners and edges as shown in FIG.2-9 of Jovanovic);
And superposing and displaying the rendered corner point and/or edge on the environment image (refer to the superimposed corners and edges shown in FIG.2-9 of Jovanovic).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the modeling teaching of Jovanovic, because this enables the creation of AR space within which the user can interact within (Jovanovic: 0002).
Claims 2 and 16 are similar in scope to claims 9. Therefore, the rejection to claim 2 similarly applies to claims 2 and 16.
Claims 6 and 20 are similar in scope to claim 13. Therefore, the rejection to claim 2 similarly applies to claims 6 and 20.
Claim(s) 3, 10 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nickerson in view of Sawhney in view of Guan as applied to the claim(s) above, and further in view of Ye (PGPUB Document No. US 2020/0349731).
Regarding claim 10, the combined teachings teach the device of claim 8, wherein identifying the corner point and/or the edge line of the target object in the environment image based on the vision algorithm comprises:
Calculating areas of respective regions with univalue segment assimilating nucleus in the grayscale image; and determining the corner point and/or the edge line of the target object based on the areas of the regions with univalue segment assimilating nucleus (object recognition utilizing “smallest univalue segment assimilating nucleus (SUSAN), or other feature detectors” (Nickerson: 0080)).
However, the combined teachings above do not expressly teach but Ye teaches applying SUSAN on an image that is converted to a grayscale image (Ye: 0043). Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to implement SUSAN in the manner taught by Ye, because this enables an effective application of the SUSAN algorithm.
Claim 3 and 17 are similar in scope to claim 10. Therefore, the rejection to claim 10 similarly applies to claims 3 and 17.
Claim(s) 7 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nickerson in view of Sawhney in view of Guan as applied to the claim(s) above, and further in view of Chaturvedi et al. (PGPUB Document No. US 2020/0258144).
Regarding claim 14, the combined teachings above do not expressly teach but Chaturvedi teaches the device of claim 8, wherein after automatically labeling the target object in the environment image based on the identified corner point and/or edge line, the method further comprises:
In response to a confirmation operation on a labeling result obtained after automatically labeling the corner point and/or edge line of the target object superimposed and displayed on the environment image, saving the labeling result (Chaturvedi teaches a similar AR floorplan measuring process that comprises the user confirming edges of physical structures).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the user confirmation process of Chaturvedi, because this enables an improved method of surveying the physical environment.
Claim 7 is similar in scope to claim 14. Therefore, the rejection to claim 14 similarly applies to claims 7.
Allowable Subject Matter
Claims 4, 5, 11, 12, 18 and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to David H Chu whose telephone number is (571)272-8079. The examiner can normally be reached M-F: 9:30 - 1:30pm, 3:30-8:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID H CHU/Primary Examiner, Art Unit 2616
Prosecution Insights