Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments filed on 1/8/26 have been considered. Valentin teaches coordinate, wherein the at least two values are simultaneously output by the trained machine intelligence system and jointly encode the object coordinate rather than representing mutually exclusive alternative coordinate values (see equation 6 and section 3.4 last two paragraphs);
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 10, 12, 19-23 and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (“Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation”) in view of Valentin (“Exploiting Uncertainty in Regression Forests for Accurate Camera Relocalization”).
Regarding claim 1, Wang teaches a method for localizing a device with respect to a known object, comprising (abstract):
capturing, using a sensor attached to the device (section 4.2, RGB),
an image of at least a portion of the known object, wherein the image includes a set of pixels (section 5.1.1, NOCS Map head, cnn predicts class);
determining, using the image, a set of known object pixels from the set of pixels, wherein the known object occupies the set of known object pixels in the image (see section 5.1, discussion of NOCS map used first then a depth map is used with the NOCS map);
determining, using a trained machine intelligence system and the image, a set of known object coordinates for the set of known object pixels, wherein the set of known object coordinates: (i) are in a frame of reference of the known object; and (ii) encode each known object coordinate in the set of known object coordinates using an encoding with at least two values per object coordinate (see section 1, dense correspondence between pixels and coordinates and section 5.1 and figure 5, “To predict the NOCS map,….” And see section 5.1 “Loss Function” this includes encoding across multiple bin channels which reads on 2 or more values per coordinate); and
determining, at least one of a location and an orientation of the device with respect to the known object, using the set of known object coordinates from the trained machine intelligence system (see section 1, pose fitting).
Valentin teaches coordinate, wherein the at least two values are simultaneously output by the trained machine intelligence system and jointly encode the object coordinate rather than representing mutually exclusive alternative coordinate values (see equation 6 and section 3.4 last two paragraphs);
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to include in Wang the ability to represent per pixel object lcoordaes using a mixture model that takes many different variables into account as taught by Valentin. The reason is to optimize coordinate locations.
Regarding claim 2, Wang discloses the set of known object coordinates are intermediate training outputs of the trained machine intelligence system (see section 1, “We use the NOCS map together with the depth map in a pose fitting algorithm to estimate the full metric 6D pose and dimensions of objects.”)
Regarding claim 3, Wang discloses the encoding includes at least one set of channels; each channel in the at least one set of channels and the set of pixels are equal in size; and the at least two values per known object coordinate are each from different channels in the at least one set of channels (see figure 5 and section 5.1.1, regression vs classification and loss function).
Regarding claim 4, Wang discloses the object is at least one dimensional; and the set of known object coordinates includes at least one object coordinate per pixel in the set of known object pixels (section 1, “NOCS map captures the normalized shape of the visible parts of the object by predicting dense correspondences between object pixels and the NOCS. Our CNN estimates the NOCS map by formulating it either as a pixel regression or classification problem”).
Regarding claim 5, Wang discloses the image is an input to the trained machine intelligence system; and the set of known object pixels are determined using the trained machine intelligence system (section 5, “A CNN predicts class labels, masks, and NOCS maps of objects. We then use the NOCS map and the depth map to estimate the metric 6D pose and size of objects”).
Regarding claim 10, see section 5.1.1, real valued numbers.
Regarding claim 12, see figure 5 of Wang.
Regarding claim 19-23, see the rejection of claims 1-5.
Regarding claim 30, see the rejection of claims 1 and 12.
Claims 6-7, 14-19, and 24-25 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Valentin in further view of Li (“Full-Frame Scene Coordinate Regression for Image-Based Localization”).
Regarding claim 6, Li teaches the at least one of the location and the orientation of the device comprises: converting the encoding to a set of single value coordinates for the set of known object coordinates using an analytical decoder; and applying the set of single value coordinates to a perspective and point solver (see section III.D, pnp algorithm to determine final pose, see algorithm 1).
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to include in Wang and Valentin the ability to decode to single values then apply a PnP as taught by Li. The reason is to allow the use PnP solver.
Regarding claim 7, see section III.D, PnP algorithm/solver.
Regarding claim 14, see Li, section IV.B, “For the 2D transformation, we uniformly sample translation from the range [ 20%, 20%] of the image width and height for x and y respectively, sample rotation from [ 45 , 45 ]”).
Regarding claim 15, see section III.C of Li.
Regarding claims 16-19, these are intended use limitations that do not further limit the claims. The rejection of claim 1 teaches a general use.
Regarding claims 24-25, see the rejection of claims 6-7.
Claim(s) 8-9 and 26-27 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Valentin in further view of De La Cruz (20190166339).
Regarding claim 8, De La Cruz teaches each value in the encoding is binary (see par. 111).
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to include in Wang and Valentin the ability to encode as taught by De La Cruz. The reason is to allow the use different encoding.
Regarding claim 9, see par. 111 and 118 of De La Cruz.
Regarding claims 26-27, see the rejection of claims 8-9.
Claim(s) 11, 13 and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Valentin in further view of Brachmann (“Learning 6D Object Pose Estimation using 3D Object Coordinates”).
Regarding claim 11, Brachmann teaches the trained machine intelligence system outputs a probability of known object visibility map for the set of known object pixels as part of the determining of the set of known object pixels; the probability of known object visibility provides a probability that the known object is visible in a pixel of the image for each pixel in the image; and training of the trained machine intelligence system is simplified using the probability of known object visibility map (see figure 1 and the caption of figure 1).
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to include in Wang and Valentin the ability to make a probability of known object visibility map as taught by Brachmann. The reason is to allow the system to show a map.
Regarding claim 13, see the last paragraph of section 1 of Brachmann.
Regarding claim 29, see the rejection of claim 11.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HADI AKHAVANNIK whose telephone number is (571)272-8622. The examiner can normally be reached 9 AM - 5 PM Monday to Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HADI AKHAVANNIK/Primary Examiner, Art Unit 2676