Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings were received on 3/14/2024. These drawings are accepted.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 11 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Bagheri et al. (US20220214186, hereinafter “Bagheri”) and in view of Song et al (US20230093680, hereinafter “Song”)
Claim 1. Bagheri teaches A method, comprising:
obtaining an initial pose of a vehicle, ([0063] “map generating module 4 is configured to receive a geographical position of the vehicle from the localization system 5 of the vehicle”) a multi-modal sensor data of the vehicle, ([0043] “The perception system comprises at least one sensor type (e.g. RADAR, LIDAR, monocular camera, stereoscopic camera, infrared camera, ultrasonic sensor, etc.), and the sensor data comprises information about a surrounding environment of the vehicle.”) and a plurality of map elements for positioning the vehicle; ([0051] “a position of the second plurality of features in a global coordinate system based on the received geographical position of the vehicle.” And [0052] “For example, the first plurality of features may include one or more geometric features (e.g. lane, traffic sign, road sign, etc.) and at least one associated semantic feature (e.g. road markings, traffic sign markings, road sign markings, etc.)” is understood to be the same as the claimed map elements in light of instant specifications [0073] )
encoding the multi-modal sensor data to obtain an environmental feature; ( [0047] “LIDAR point clouds, etc.) are firstly projected on the image plane or a plane perpendicular to the direction of gravity (i.e. a bird's eye view), and projected snapshots of the environment are created.” is understood to be the same as the claimed environmental feature in light of instant specifications [0204])
encoding the plurality of map elements to obtain a map feature; ([0052] “For example, the first plurality of features may include one or more geometric features (e.g. lane, traffic sign, road sign, etc.) and at least one associated semantic feature (e.g. road markings, traffic sign markings, road sign markings, etc.). … combining, …The combination can be construed as a means for providing feature labels in the subsequently generated map.” Is understood to be the same as the claimed encoding map elements to obtain a map feature in light of instant specifications [0116])
determining, based on the environmental feature and the map feature, a target pose offset for correcting the initial pose; ([0057] “a “temporary” map of the current perceived data from the on-board sensors is compared with the generated reference local map (i.e. the generated 105 map) given the “ground truth” position given by the high precision localization system of the vehicle. The comparison results in at least one parameter (e.g. a calculated error).”)
Bagheri does not explicitly teach and superimposing the initial pose and the target pose offset to obtain a corrected pose of the vehicle.
Song teaches and superimposing ([0092] “fuse the information”) the initial pose ([0092] “obtaining pose information of the vehicle”) and the target pose offset ([0092] “offset pose”) to obtain a corrected pose of the vehicle. ([0092] “compensatory pose”)
It would have been obvious to persons of ordinary skill in the art before the effective filing date of the claimed invention to modify Bagheri to have superimposing an initial pose with a target pose offset to obtain a corrected pose as taught by Song to arrive at the claimed invention discussed above. The motivation for the proposed modification would have been to (Song et al [0049] “generate a more accurate high-precision map”)
Claim 2. Bagheri and Song teach The method according to claim 1,
Bagheri teaches wherein the multi-modal sensor data comprises a point cloud ([0068] “LIDAR”) and an image, ([0068] “monocular camera, stereoscopic camera, infrared camera”) and wherein the encoding the multi-modal sensor data to obtain the environmental feature comprises:
encoding the point cloud to obtain a point cloud feature map; ([0048] “each sensor type's characteristics to be considered separately when training each sub-model whereby more accurate “general feature maps” can be extracted….one for LIDAR”)
encoding the image to obtain an image feature map; ([0048] “each sensor type's characteristics to be considered separately when training each sub-model whereby more accurate “general feature maps” can be extracted….one for monocular cameras”)
Bagheri does not explicitly teach and fusing the point cloud feature map and the image feature map to obtain the environmental feature.
Song teaches and fusing the point cloud feature map and the image feature map ([0049] “fuse information of a pixel in an image acquired by an image sensor equipped on a vehicle and information of a point in a point cloud acquired by an equipped lidar”) to obtain the environmental feature. ([0049] “to obtain fused data, … and generate a more accurate high-precision map”
It would have been obvious to persons of ordinary skill in the art before the effective filing date of the claimed invention to modify Bagheri to have fusing point cloud map and image feature map to obtain an environmental feature as taught by Song to arrive at the claimed invention discussed above. The motivation for the proposed modification would have been to (Song et al [0049] “generate a more accurate high-precision map”)
Claim 11. Bagheri and Song teach The method according to claim 1,
Bagheri teaches wherein the determining the target pose offset for correcting the initial pose comprises:
matching the environmental feature with the map feature to determine the target pose offset. ([0057] “a “temporary” map of the current perceived data from the on-board sensors is compared with the generated reference local map (i.e. the generated 105 map) given the “ground truth” position given by the high precision localization system of the vehicle. The comparison results in at least one parameter (e.g. a calculated error).”)
Claim 19. Bagheri teaches An electronic device, comprising:
a processor; ([0019] “processors”) and
a memory communicatively connected to the processor, ([0019] “programs configured to be executed by one or more processors”) wherein
the memory stores instructions executable by the processor, and the instructions, when executed by the processor, cause the processor to perform operations ([0019] “programs comprising instructions for performing a method for automated generation according to any one of the embodiments disclosed herein.”)comprising:
obtaining an initial pose of a vehicle, ([0063] “map generating module 4 is configured to receive a geographical position of the vehicle from the localization system 5 of the vehicle”) a multi-modal sensor data of the vehicle, ([0043] “The perception system comprises at least one sensor type (e.g. RADAR, LIDAR, monocular camera, stereoscopic camera, infrared camera, ultrasonic sensor, etc.), and the sensor data comprises information about a surrounding environment of the vehicle.”)and a plurality of map elements for positioning the vehicle; ([0051] “a position of the second plurality of features in a global coordinate system based on the received geographical position of the vehicle.” And [0052] “For example, the first plurality of features may include one or more geometric features (e.g. lane, traffic sign, road sign, etc.) and at least one associated semantic feature (e.g. road markings, traffic sign markings, road sign markings, etc.)” is understood to be the same as the claimed map elements in light of instant specifications [0073] ) encoding the multi-modal sensor data to obtain an environmental feature; ( [0047] “LIDAR point clouds, etc.) are firstly projected on the image plane or a plane perpendicular to the direction of gravity (i.e. a bird's eye view), and projected snapshots of the environment are created.” is understood to be the same as the claimed environmental feature in light of instant specifications [0204]) encoding the plurality of map elements to obtain a map feature; ([0052] “For example, the first plurality of features may include one or more geometric features (e.g. lane, traffic sign, road sign, etc.) and at least one associated semantic feature (e.g. road markings, traffic sign markings, road sign markings, etc.). … combining, …The combination can be construed as a means for providing feature labels in the subsequently generated map.” Is understood to be the same as the claimed encoding map elements to obtain a map feature in light of instant specifications [0116]) determining, based on the environmental feature and the map feature, a target pose offset for correcting the initial pose; ([0057] “a “temporary” map of the current perceived data from the on-board sensors is compared with the generated reference local map (i.e. the generated 105 map) given the “ground truth” position given by the high precision localization system of the vehicle. The comparison results in at least one parameter (e.g. a calculated error).”)
Bagheri does not explicitly teach and superimposing the initial pose and the target pose offset to obtain a corrected pose of the vehicle.
Song teaches and superimposing ([0092] “fuse the information”) the initial pose ([0092] “obtaining pose information of the vehicle”) and the target pose offset ([0092] “offset pose”) to obtain a corrected pose of the vehicle. ([0092] “compensatory pose”)
It would have been obvious to persons of ordinary skill in the art before the effective filing date of the claimed invention to modify Bagheri to have superimposing an initial pose with a target pose offset to obtain a corrected pose as taught by Song to arrive at the claimed invention discussed above. The motivation for the proposed modification would have been to (Song et al [0049] “generate a more accurate high-precision map”)
Claim 20. The non-transitory computer-readable storage medium herein has been executed and performed by the device of claim 19 and is likewise rejected
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Bagheri et al. (US20220214186, hereinafter “Bagheri”) and in view of Song et al (US20230093680, hereinafter “Song”) and in view of Guo et al (US20230154203, hereinafter “Guo”)
Claim 3. Bagheri and Song teach The method according to claim 2, wherein the fusing the point cloud feature map and the image feature map to obtain the environmental feature comprises:
Bagheri and Song do not explicitly teach determining an initial environmental feature map in a target three-dimensional space based on the point cloud feature map;
fusing the initial environmental feature map and the image feature map to obtain a first environmental feature map in the target three-dimensional space; and
determining the environmental feature based on the first environmental feature map.
Guo teaches determining an initial environmental feature map in a target three-dimensional space ([0032] “The point-cloud map-retrieving device 20 detects the distance of an object on the front target road”) based on the point cloud feature map; ([0036] “point-cloud map”)
fusing the initial environmental feature map and the image feature map ([0038] “camera road image to generate a road-segmented map”) to obtain a first environmental feature map in the target three-dimensional space; ([0033] “point-cloud image-fusing module 32 is configured to receive, calibrate, and fuse the road distance point-cloud map and the first camera road image to generate a road camera point-cloud fusion map.”) and
determining the environmental feature based on the first environmental feature map. ([0033] “determine the road-line information”)
It would have been obvious to persons of ordinary skill in the art before the effective filing date of the claimed invention to modify the proposed combination of Bagheri and Song to have determine an environmental feature map based on point cloud feature map and fusing it with an image feature map to obtain an environmental feature map as taught by Guo to arrive at the claimed invention discussed above. The motivation for the proposed modification would have been because ( Guo et al [0003] “The high-precision vector map can be used to obtain accurate path points, which reduces the difficulty of path planning.”
Allowable Subject Matter
Claims 4-10 and 12-18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Brahma et al US20220261590 teaches an attention mechanism to fuse lidar data with image data and that the lidar and camera image include feature vectors but does not render obvious the claimed combination as a whole
Choi et al US20130222550 teaches synthesis of a camera image and a time-of-flight image by utilizing a cost function and probability distribution as well as estimating an error to reconstruct a depth image but does not render obvious the claimed combination as a whole
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure:
Nister et al US20230366698 teaches interpolating the vehicle pose between two frames to correct any errors between frames
Lu et al NPL “L3-Net: Towards Learning based LiDAR Localization for Autonomous Driving” teaches offset between predicted and final poses to minimize a cost between a 3D map and an online point cloud.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OWAIS MEMON whose telephone number is (571)272-2168. The examiner can normally be reached M-F (7:00am - 4:00pm) CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OWAIS I MEMON/Examiner, Art Unit 2663