DETAILED ACTION
Claims 1-15 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Objections
Claims 3 and 9 are objected to because of the following informalities: the claims recite “keypoint decriptors”, the claims should read “keypoint descriptors”. Appropriate correction is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2 and 4-8 are rejected under 35 U.S.C. 102(a)(1) as anticipated being by Sinha (US 2020/0005486) .
Regarding claim 1, Sinha teaches:
A method of estimating a location using line cloud map data, comprising:
detecting keypoints in a query image; (Sinha [0061] To estimate the pose of the HMD device, the localization program may determine a set of correspondences between the 2D query features (keypoints) and the 3D lines in the 3D line cloud.) and
estimating a location where the query image was obtained by mapping the keypoints of the query image and lines in line cloud map data, (Sinha [0061] To estimate the pose of the HMD device (i.e. location where the query image was obtained), the localization program may determine a set of correspondences between the 2D query features (keypoints) and the 3D lines in the 3D line cloud.)
wherein the line cloud map data comprise first feature information on lines determined by keypoint pairs selected from keypoints included in point cloud map data, (Sinha [0050] In some examples, the direction for each 3D line may be selected to pass through the corresponding 3D point and through an anchor location with the 3D point cloud) and second feature information on the keypoint pairs matched to the determined lines. (Sinha [0061] To estimate the pose of the HMD device, the localization program may determine a set of correspondences between the 2D query features (keypoints) and the 3D lines in the 3D line cloud.)
Regarding claim 2, Sinha teaches:
The method of claim 1, wherein the first feature information comprises direction information of the lines and location information on a reference point located on the lines. (Sinha [0050] In some examples, the direction for each 3D line may be selected to pass through the corresponding 3D point and through an anchor location with the 3D point cloud)
Regarding claim 4, Sinha teaches:
The method of claim 1, wherein the number of the lines in the line cloud map data mapped to each of the keypoints of the query image is one or two. (Sinha [0050] In some examples, the direction for each 3D line may be selected to pass through the corresponding 3D point and through an anchor location with the 3D point cloud)
Regarding claim 5, Sinha teaches:
The method of claim 1, wherein the line cloud map data is data comprising keypoints extracted from an image photographed in a target space. (Sinha [0039] In one example a person may move a camera around the room to capture images and/or video of the room that may be utilized to generate a 3D map of the room)
Regarding claim 6, Sinha teaches:
An apparatus for estimating a location using line cloud map data, comprising:
a memory; (Sinha [0091-92] processor and memory)
at least one processor electrically connected to the memory; (Sinha [0091-92] processor and memory) and
a camera configured to generate a query image, wherein the process: (Sinha [0033] RGB camera, IR camera, and/or depth camera)
detects keypoints in the query image, (Sinha [0061] To estimate the pose of the HMD device, the localization program may determine a set of correspondences between the 2D query features (keypoints) and the 3D lines in the 3D line cloud.) and
estimates a location where the query image was obtained by mapping the keypoints of the query image and lines in line cloud map data, (Sinha [0061] To estimate the pose of the HMD device (i.e. location where the query image was obtained), the localization program may determine a set of correspondences between the 2D query features (keypoints) and the 3D lines in the 3D line cloud.) and
wherein the line cloud map data comprise first feature information on lines determined by keypoint pairs selected from keypoints included in point cloud map data, (Sinha [0050] In some examples, the direction for each 3D line may be selected to pass through the corresponding 3D poin and through an anchor location with the 3D point cloud) and second feature information on the keypoint pairs matched to the determined lines. (Sinha [0061] To estimate the pose of the HMD device, the localization program may determine a set of correspondences between the 2D query features (keypoints) and the 3D lines in the 3D line cloud.)
Regarding claim 7, Sinha teaches:
A method of generating line cloud map data, comprising:
receiving point cloud map data as input; (Sinha [0061] To estimate the pose of the HMD device, the localization program may determine a set of correspondences between the 2D query features (keypoints) and the 3D lines in the 3D line cloud.)
pairing keypoints included in the point cloud map data into keypoint pairs; (Sinha [0050] In some examples, the direction for each 3D line may be selected to pass through the corresponding 3D poin and through an anchor location with the 3D point cloud) and
matching first feature information on lines determined by the keypoint pairs and second feature information of the keypoint pairs. (Sinha [0061] To estimate the pose of the HMD device, the localization program may determine a set of correspondences between the 2D query features (keypoints) and the 3D lines in the 3D line cloud.)
Regarding claim 8, Sinha teaches:
The method of claim 7, wherein the first feature information comprises direction information of the lines and location information of a reference point located on the lines. (Sinha [0050] In some examples, the direction for each 3D line may be selected to pass through the corresponding 3D point and through an anchor location with the 3D point cloud)
Allowable Subject Matter
Claims 3 and 9-15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claim 3, the closest known prior art nor any reasonable combination thereof, teaches:
the line cloud map data further comprises color information on the keypoint pairs matched to the lines.
Regarding claim 9, the closest known prior art nor any reasonable combination thereof, teaches:
The method of claim 7, wherein the matching the second feature information of the keypoint pairs matches the first and second feature information such as keypoint decriptors or scale or orientation of the keypoints, and color information on the keypoint pairs.
Regarding claim 10, the closest known prior art nor any reasonable combination thereof, teaches:
The method of claim 7, wherein the pairing the keypoints into the keypoint pairs removes an odd number of keypoints out of the keypoints included in the point cloud map data if the number of the keypoints included in the point cloud map data is an odd number.
Regarding claim 11, the closest known prior art nor any reasonable combination thereof, teaches:
The method of claim 7, wherein the pairing the keypoints into the keypoint pairs randomly selects and pairs the keypoints included in the point cloud map data into the keypoint pairs.
Regarding claim 12, the closest known prior art nor any reasonable combination thereof, teaches:
The method of claim 7, wherein the pairing the keypoints into the keypoint pairs comprises:
clustering the keypoints included in the point cloud map data into a plurality of groups; and
pairing keypoints included in the same group into the keypoint pairs.
Regarding claim 13, the closest known prior art nor any reasonable combination thereof, teaches:
The method of claim 7, wherein the pairing the keypoints into the keypoint pairs comprises:
clustering the keypoints included in the point cloud map data into a plurality of groups; and
pairing each keypoint selected from two different groups into the keypoint pairs.
Regarding claim 14, the closest known prior art nor any reasonable combination thereof, teaches:
The method of claim 7, wherein the pairing the keypoints into the keypoint pairs comprises: determining whether the keypoints included in the keypoint pairs are located on the same plane; and performing pairing again using keypoints included in keypoint pairs in which the keypoints have been determined to be located on the same plane.
Regarding claim 15, the closest known prior art nor any reasonable combination thereof, teaches:
The method of claim 14, wherein the pairing the keypoints into the keypoint pairs performs the pairing by excluding keypoints that have been determined to be located on the same plane a preset number of times.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Refer to PTO-892, Notice of References Cited for a listing of analogous art.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Molly K Wilburn whose telephone number is (571)272-3589. The examiner can normally be reached Monday-Friday 8am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Molly Wilburn/Primary Examiner, Art Unit 2666