DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claims 1, 10, and 19 have been amended.
No claims have been added.
No claims have been cancelled.
Claims 1-20 are currently pending.
The official correspondence below is an after non-final.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Akbarzadeh (US 11713978 B2) in view of Hirzer (US 10546387 B2).
REGARDING CLAIM 1, Akbarzadeh discloses, generating two-dimensional (2D) feature point information of landmarks (Akbarzadeh: (Col. 72, Ln. 55-65)) in a landmark-based probability map from a surrounding image (Akbarzadeh: (Col. 18, Ln. 34 - Col. 19, Ln. 3) a stream of 2D camera images ... The base conversion 506 may correspond to the landmarks … the 3D landmark locations may be converted—using base conversion 506—to a map format to generate base layer of the map 504; (Col. 11, Ln. 39-50) a raw output corresponds to a confidences for each point) acquired by a capturing device mounted on a moving object (Akbarzadeh: (Col. 4, Ln. 5-7)); obtaining landmark-based three-dimensional (3D) feature point information of the landmarks (Akbarzadeh: (Col. 18, Ln. 36-38)) from high-definition (HD) map data of a vicinity of the moving object (Akbarzadeh: (Col. 2, Ln. 62-63); (Col. 3, Ln. 15-19); (Col. 13, Ln. 59-64)); converting one of the 2D feature point information of the surrounding image to 3D data (Akbarzadeh: (Col. 8, Ln. 24-27)) or converting the 3D feature point information of the HD map data to 2D (Akbarzadeh: (Col. 28, Ln. 64-65) 2D landmarks generated from the 3D landmarks); summing, for each of the landmarks, probabilities of the landmark (Akbarzadeh: (Col. 16, Ln. 46-53) The resulting data may be aggregated, merged, edited (e.g., trajectory completion, interpolation, extrapolation, etc.), filtered (e.g., landmark filtering for creating continuous lane lines and/or road boundary lines), and/or otherwise processed to generate a mapstream 210 representing this data generated from any number of different sensors and/or sensor modalities; (Col. 18, Ln. 10-12) cost space sampling, aggregation, etc.—of pairs of segments may be executed in parallel (e.g., a first pair may be registered in parallel with another pair); (Col. 51, Ln. 57-57) a neural network that outputs a measure of confidence for each object detection. Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections. This confidence value enables the system to make further decisions regarding which detections should be considered as true positive detections rather than false positive detections. For example, the system may set a threshold value for the confidence and consider only the detections exceeding the threshold value as true positive detections; (Col. 26, Ln. 50-57) for some number of iterations (e.g., 100, 1000, 2000, etc.). Once completed for the number of iterations, the layout that had the most agreement may be used ... with respect to FIG. 6F, a cost of the error may be computed for each of the pose links 608 other than the minimum sampled—e.g., according to equation (4), below [Eq. 4: cost = sqrt(∑(u^2/s^2))]) determined based on the landmark-based probability map (Akbarzadeh: (Col. 51, Ln. 47-51) a neural network that outputs a measure of confidence for each object detection. Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections; (Col. 41, Ln. 49-53) at each frame, comparing one or more of the sensor data or the DNN outputs to map data to generate a cost space representing a probability of a vehicle being located at each of a plurality of poses; (Col. 26, Ln. 50-57) for some number of iterations (e.g., 100, 1000, 2000, etc.). Once completed for the number of iterations, the layout that had the most agreement may be used ... with respect to FIG. 6F, a cost of the error may be computed for each of the pose links 608 other than the minimum sampled—e.g., according to equation (4), below [Eq. 4: cost = sqrt(∑(u^2/s^2))]); multiplying the summed probabilities (Akbarzadeh: (Col. 26, Ln. 50-57) for some number of iterations (e.g., 100, 1000, 2000, etc.). Once completed for the number of iterations, the layout that had the most agreement may be used ... with respect to FIG. 6F, a cost of the error may be computed for each of the pose links 608 other than the minimum sampled—e.g., according to equation (4), below [Eq. 4: cost = sqrt(∑(u^2/s^2))]; (Col. 51, Ln. 47-51) a neural network that outputs a measure of confidence for each object detection. Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections; (Col. 41, Ln. 49-53) at each frame, comparing one or more of the sensor data or the DNN outputs to map data to generate a cost space representing a probability of a vehicle being located at each of a plurality of poses), each of the summed probabilities corresponding to a respective landmark one of the landmarks (Akbarzadeh: (Col. 51, Ln. 47-51) a neural network that outputs a measure of confidence for each object detection. Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections; (Col. 41, Ln. 49-53) at each frame, comparing one or more of the sensor data or the DNN outputs to map data to generate a cost space representing a probability of a vehicle being located at each of a plurality of poses; see (Col. 18, Ln. 51 - Col. 19, Ln. 9) for landmarks); determining a first similarity between the 2D feature point information and the 2D data or determining a second similarity between the 3D feature point information and the 3D data (Akbarzadeh: (Col. 28, Ln. 24-35) the 3D landmark locations from multiple maps 504 may be fused together to generate a final representation … the separate map layers may be fused to generate the aggregate map layers … where a first map layer includes data that matches up—within some threshold similarity—to data of another map layer, the matching data may be used (e.g., averaged) to generate a final representation; (Col. 22, Ln. 61-66); see at least (Col. 23, Ln. 26 - Col 24, Ln. 2; Col. 28, Ln. 23-52; Col. 29, Ln. 7 - Col. 30, Ln. 13); see at least (Col. 6, Ln. 22-55; Col. 23, Ln. 26 - Col 24) for first and second maps, points, etc., and matching and similarities; see at least (Col. 2, Ln. 64-67) for aggregating data and creating new maps) based on the multiplied summed probabilities corresponding to each landmark determined based on the landmark-based probability map (Akbarzadeh: for some number of iterations (e.g., 100, 1000, 2000, etc.). Once completed for the number of iterations, the layout that had the most agreement may be used ... with respect to FIG. 6F, a cost of the error may be computed for each of the pose links 608 other than the minimum sampled—e.g., according to equation (4), below [Eq. 4: cost = sqrt(∑(u^2/s^2))] (Col. 26, Ln. 50-57); pose graph 650 of FIG. 6G may undergo an optimization process, such as a non-linear optimization process (e.g., a bundle adjustment process) with the goal of minimizing the sum of squared costs of inliers—e.g., using the computed cost function described herein with respect to FIG. 6F (Col. 27, Ln. 13-18)); and estimating a position of the moving object (Akbarzadeh: (Col. 3, Ln. 16-20)).
The examiner respectfully submits, Akbarzadeh discloses based on the first similarity or the second similarity (Akbarzadeh: ((Col. 21, Ln. 57 - Col. 23, Ln. 4), (Col. 23, Ln. 5 - Col. 25, Ln. 24)) examiner: disclose aggregation of scores/weights (sum) and averages of landmarks in a map; (Col. 32, Ln. 26-31) the fused map to one or more vehicles for use in executing one or more operations. For example, the map data 108 representative of the final fused HD map may be transmitted to one or more vehicle 1500 for localization, path planning, control decisions, and/or other operations; Layering 2D and 3D data (Col. 6, Ln. 22-55; Col. 28, Ln. 23-52; Col. 34, Ln. 65 - Col. 35, Ln. 5; Col. 35, Ln. 39-43; Col. 72, Ln. 62 - Col. 73, Ln. 2) to determine similarities).
However, in the alternative, and in the same field of endeavor, Hirzer discloses, based on the first similarity or the second similarity (Hirzer: (Col. 16, Ln. 39-45) The pose probability block 1108 then determines a respective probability that each respective 3D rendering matches or aligns with the segmented image. The pose probability block 1108 combines the respective probabilities (i.e., the column likelihood function) over all of the plurality of regions (i.e., all of the integral columns) to provide a pose probability), for the benefit of selecting a pose from the plurality of poses, such that the 3D rendering corresponding to the selected pose aligns with the segmented image.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Akbarzadeh to include determining a confidence associated with a vehicle pose taught by Hirzer. One of ordinary skill in the art would have been motivated to make this modification, with a reasonable expectation of success, in order to select a pose from the plurality of poses, such that the 3D rendering corresponding to the selected pose aligns with the segmented image.
REGARDING CLAIM 2, Akbarzadeh, as modified, remains as applied above to claim 1, and further, Akbarzadeh also discloses, the 2D feature point information of the surrounding image (Akbarzadeh: (Col. 18, Ln. 35-38)) is obtained according to a landmark (Akbarzadeh: (Col. 12, Ln. 56-57)), based on deep neural network (DNN)- based semantic segmentation (Akbarzadeh: (Col. 10, Ln. 54-58); (Col. 11, Ln. 17-20)).
REGARDING CLAIM 3, Akbarzadeh, as modified, remains as applied above to claim 1, and further, Akbarzadeh also discloses, receiving 3D feature point information on a world domain for a landmark (Akbarzadeh: (Col. 22, Ln. 36-52)) in the vicinity of the moving object (Akbarzadeh: (Col. 22, Ln. 36-52)) from a HD map database based on position information of the moving object (Akbarzadeh: (Col. 22, Ln. 36-52)); and converting the 3D feature point information on the world domain into a local domain for the capturing device (Akbarzadeh: (Col. 18, Ln. 67 - Col. 19, Ln. 5)).
REGARDING CLAIM 4, Akbarzadeh, as modified, remains as applied above to claim 1, and further, Akbarzadeh also discloses, converting the 2D feature point information of the surrounding image to a form of a 3D probability map (Akbarzadeh: (Col. 18, Ln. 25-29); (Col. 22, Ln. 2-8)) based on inverse perspective mapping (Akbarzadeh: (Col. 10, Ln. 49-53); (Col. 19, Ln. 16-20)).
REGARDING CLAIM 5, Akbarzadeh, as modified, remains as applied above to claim 1, and further, Akbarzadeh also discloses, projecting the 3D feature point information of the HD map data onto a 2D probability map (Akbarzadeh: (Col. 22, Ln. 37-41)) obtained from the surrounding image, based on perspective mapping (Akbarzadeh: (Col. 23, Ln. 11-14)).
REGARDING CLAIM 6, Akbarzadeh, as modified, remains as applied above to claim 1, and further, Akbarzadeh also discloses, summing first probabilities of the 2D data corresponding to each landmark (Akbarzadeh: (Col. 18, Ln. Ln. 63 - Col. 19, Ln. 1-8)), in the probability map of the 2D feature point information (Akbarzadeh: (Col. 22, Ln. 28-35)); and calculating the first similarity by multiplying summed first probabilities corresponding to each landmark (Akbarzadeh: (Col. 34, Ln. 65 - Col. 35, Ln. 5); (Col. 35, Ln. 39-43); (Col. 72, Ln. 62 - Col. 73, Ln. 2)), wherein the determining of the second similarity comprises: summing the second probabilities of the 3D feature point information corresponding to each landmark, in a 3D probability map of the 3D data (Akbarzadeh: (Col. 8, Ln. 18-28); (Col. 18, Ln. 34-43)); and calculating the first similarity by multiplying the summed first probabilities corresponding to each landmark (Akbarzadeh: (Col. 51, Ln. 46-63)), wherein the determining of the second similarity comprises: summing the second probabilities of the 3D feature point information corresponding to each landmark, in a 3D probability map of the 3D data (Akbarzadeh: (Col. 22, Ln. 61-66); see at least (Col. 23, Ln. 26 - Col 24, Ln. 2; Col. 28, Ln. 23-52; Col. 29, Ln. 7 - Col. 30, Ln. 13) for comparing points, lines, and dots for matching and similarities, aggregated matching points, values of matches for 2D and 3D multi-layered maps, this includes first, second, third, and beyond similarities; see at least (Col. 6, Ln. 22-55; Col. 23, Ln. 26 - Col 24) for first and second maps, points, etc., and matching and similarities); and calculating the second similarity by multiplying summed second probabilities corresponding to each landmark (Akbarzadeh: (Col. 22, Ln. 61-66); see at least (Col. 23, Ln. 26 - Col 24, Ln. 2; Col. 28, Ln. 23-52; Col. 29, Ln. 7 - Col. 30, Ln. 13) for comparing points, lines, and dots for matching and similarities, aggregated matching points, values of matches for 2D and 3D multi-layered maps, this includes first, second, third, and beyond similarities; see at least (Col. 6, Ln. 22-55; Col. 23, Ln. 26 - Col 24) for first and second maps, points, etc., and matching and similarities).
Akbarzadeh does not explicitly recite the terminology "multiplying summed". However, Akbarzadeh discloses an accumulation/aggregation of matches and predictions. Which, the examiner respectfully submits, is parallel in service and result for a determination over an accumulation of results.
REGARDING CLAIM 7, Akbarzadeh, as modified, remains as applied above to claim 1, and further, Akbarzadeh also discloses, updating a result of estimating the position of the moving object according to a particle filter or a maximum likelihood (ML) optimization scheme, based on the first similarity of the second similarity (Akbarzadeh: (Col. 22, Ln. 61-66); see at least (Col. 23, Ln. 26 - Col 24, Ln. 2; Col. 28, Ln. 23-52; Col. 29, Ln. 7 - Col. 30, Ln. 13) for comparing points, lines, and dots for matching and similarities, aggregated matching points, values of matches for 2D and 3D multi-layered maps, this includes first, second, third, and beyond similarities; see at least (Col. 6, Ln. 22-55; Col. 23, Ln. 26 - Col 24) for first and second maps, points, etc., and matching and similarities; see at least (Col. 2, Ln. 64-67) for aggregating data and creating new maps).
REGARDING CLAIM 8, Akbarzadeh, as modified, remains as applied above to claim 1, and further, Akbarzadeh also discloses, the moving object is an autonomous vehicle or a vehicle supporting advanced driver-assistance systems (ADAS) (Akbarzadeh: (Col. 5, Ln. 4-6)).
REGARDING CLAIM 9, Akbarzadeh, as modified, remains as applied above to claim 1, and further, Akbarzadeh also discloses, the landmark comprises any one or any combination of a white lane line, a yellow lane line, a crosswalk, a speed bump, a traffic light, and a traffic sign (Akbarzadeh: (Col. 11, Ln. 20-36)).
REGARDING CLAIM 10, Akbarzadeh discloses, generating two-dimensional (2D) feature point information of landmarks (Akbarzadeh: (Col. 72, Ln. 55-65)) in a landmark-based probability map from a surrounding image (Akbarzadeh: (Col. 18, Ln. 34 - Col. 19, Ln. 3) a stream of 2D camera images ... The base conversion 506 may correspond to the landmarks … the 3D landmark locations may be converted—using base conversion 506—to a map format to generate base layer of the map 504; (Col. 11, Ln. 39-50) a raw output corresponds to a confidences for each point) acquired by a capturing device mounted on a moving object (Akbarzadeh: (Col. 4, Ln. 5-7)); obtaining landmark-based three-dimensional (3D) feature point information of the landmarks (Akbarzadeh: (Col. 18, Ln. 36-38)) from high-definition (HD) map data of a vicinity of the moving object (Akbarzadeh: (Col. 2, Ln. 62-63); (Col. 3, Ln. 15-19); (Col. 13, Ln. 59-64)); converting one of the 2D feature point information of the surrounding image to 3D data (Akbarzadeh: (Col. 8, Ln. 24-27)) or converting the 3D feature point information of the HD map data to 2D (Akbarzadeh: (Col. 28, Ln. 64-65) 2D landmarks generated from the 3D landmarks); summing, for each of the landmarks, probabilities of the landmark (Akbarzadeh: (Col. 16, Ln. 46-53) The resulting data may be aggregated, merged, edited (e.g., trajectory completion, interpolation, extrapolation, etc.), filtered (e.g., landmark filtering for creating continuous lane lines and/or road boundary lines), and/or otherwise processed to generate a mapstream 210 representing this data generated from any number of different sensors and/or sensor modalities; (Col. 18, Ln. 10-12) cost space sampling, aggregation, etc.—of pairs of segments may be executed in parallel (e.g., a first pair may be registered in parallel with another pair); (Col. 51, Ln. 57-57) a neural network that outputs a measure of confidence for each object detection. Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections. This confidence value enables the system to make further decisions regarding which detections should be considered as true positive detections rather than false positive detections. For example, the system may set a threshold value for the confidence and consider only the detections exceeding the threshold value as true positive detections; (Col. 26, Ln. 50-57) for some number of iterations (e.g., 100, 1000, 2000, etc.). Once completed for the number of iterations, the layout that had the most agreement may be used ... with respect to FIG. 6F, a cost of the error may be computed for each of the pose links 608 other than the minimum sampled—e.g., according to equation (4), below [Eq. 4: cost = sqrt(∑(u^2/s^2))]) determined based on the landmark-based probability map (Akbarzadeh: (Col. 51, Ln. 47-51) a neural network that outputs a measure of confidence for each object detection. Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections; (Col. 41, Ln. 49-53) at each frame, comparing one or more of the sensor data or the DNN outputs to map data to generate a cost space representing a probability of a vehicle being located at each of a plurality of poses; (Col. 26, Ln. 50-57) for some number of iterations (e.g., 100, 1000, 2000, etc.). Once completed for the number of iterations, the layout that had the most agreement may be used ... with respect to FIG. 6F, a cost of the error may be computed for each of the pose links 608 other than the minimum sampled—e.g., according to equation (4), below [Eq. 4: cost = sqrt(∑(u^2/s^2))]); multiplying the summed probabilities (Akbarzadeh: (Col. 26, Ln. 50-57) for some number of iterations (e.g., 100, 1000, 2000, etc.). Once completed for the number of iterations, the layout that had the most agreement may be used ... with respect to FIG. 6F, a cost of the error may be computed for each of the pose links 608 other than the minimum sampled—e.g., according to equation (4), below [Eq. 4: cost = sqrt(∑(u^2/s^2))]; (Col. 51, Ln. 47-51) a neural network that outputs a measure of confidence for each object detection. Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections; (Col. 41, Ln. 49-53) at each frame, comparing one or more of the sensor data or the DNN outputs to map data to generate a cost space representing a probability of a vehicle being located at each of a plurality of poses), each of the summed probabilities corresponding to a respective landmark one of the landmarks (Akbarzadeh: (Col. 51, Ln. 47-51) a neural network that outputs a measure of confidence for each object detection. Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections; (Col. 41, Ln. 49-53) at each frame, comparing one or more of the sensor data or the DNN outputs to map data to generate a cost space representing a probability of a vehicle being located at each of a plurality of poses; see (Col. 18, Ln. 51 - Col. 19, Ln. 9) for landmarks); determining, for each of the positions of the particles, a similarity between the projected feature point information and the 2D feature point information (Akbarzadeh: (Col. 28, Ln. 24-35); (Col. 22, Ln. 61-66); see at least (Col. 23, Ln. 26 - Col 24, Ln. 2; Col. 28, Ln. 23-52; Col. 29, Ln. 7 - Col. 30, Ln. 13) for comparing points, lines, and dots for matching and similarities, aggregated matching points, values of matches for 2D and 3D multi-layered maps, this includes first, second, third, and beyond similarities; see at least (Col. 6, Ln. 22-55; Col. 23, Ln. 26 - Col 24) for first and second maps, points, etc., and matching and similarities; see at least (Col. 2, Ln. 64-67) for aggregating data and creating new maps) based on the multiplied summed probabilities corresponding to each landmark determined based on the landmark-based probability map (Akbarzadeh: for some number of iterations (e.g., 100, 1000, 2000, etc.). Once completed for the number of iterations, the layout that had the most agreement may be used ... with respect to FIG. 6F, a cost of the error may be computed for each of the pose links 608 other than the minimum sampled—e.g., according to equation (4), below [Eq. 4: cost = sqrt(∑(u^2/s^2))] (Col. 26, Ln. 50-57); pose graph 650 of FIG. 6G may undergo an optimization process, such as a non-linear optimization process (e.g., a bundle adjustment process) with the goal of minimizing the sum of squared costs of inliers—e.g., using the computed cost function described herein with respect to FIG. 6F (Col. 27, Ln. 13-18)); and estimating a position of the moving object (Akbarzadeh: (Col. 3, Ln. 16-20)).
The examiner respectfully submits, Akbarzadeh discloses rearranging the particles based on the similarity (Akbarzadeh: ((Col. 21, Ln. 57 - Col. 23, Ln. 4), (Col. 23, Ln. 5 - Col. 25, Ln. 24)) examiner: disclose aggregation of scores/weights (sum) and averages of landmarks in a map; (Col. 32, Ln. 26-31) the fused map to one or more vehicles for use in executing one or more operations. For example, the map data 108 representative of the final fused HD map may be transmitted to one or more vehicle 1500 for localization, path planning, control decisions, and/or other operations; Layering 2D and 3D data (Col. 6, Ln. 22-55; Col. 28, Ln. 23-52; Col. 34, Ln. 65 - Col. 35, Ln. 5; Col. 35, Ln. 39-43; Col. 72, Ln. 62 - Col. 73, Ln. 2) to determine similarities; The relative pose links are then used to align the maps 504 corresponding to each of the mapstreams 210 such that landmarks and other features—e.g., points clouds, LiDAR image maps, RADAR image maps, etc.—are aligned in a final, aggregate, HD map (Col. 22, Ln. 4-8)).
However, in the alternative, and in the same field of endeavor, Hirzer discloses, based on the similarity (Hirzer: (Col. 16, Ln. 39-45) The pose probability block 1108 then determines a respective probability that each respective 3D rendering matches or aligns with the segmented image. The pose probability block 1108 combines the respective probabilities (i.e., the column likelihood function) over all of the plurality of regions (i.e., all of the integral columns) to provide a pose probability; (Col. 4, Ln. 3-13) determine a pose of an image capture device at any given time using a 3D tracker, such as visual odometry tracking or simultaneous localization and mapping (SLAM) ... The pose determination based on semantic segmentation of the captured image is considered to be ground truth, and is used to update the 3D tracker), for the benefit of selecting a pose from the plurality of poses, such that the 3D rendering corresponding to the selected pose aligns with the segmented image.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Akbarzadeh to include determining a confidence associated with a vehicle pose and updating segmentations taught by Hirzer. One of ordinary skill in the art would have been motivated to make this modification, with a reasonable expectation of success, in order to select a pose from the plurality of poses, such that the 3D rendering corresponding to the selected pose aligns with the segmented image.
REGARDING CLAIM 11, Akbarzadeh, as modified, remains as applied above to claim 10, and further, Akbarzadeh also discloses, the 2D feature point information (Akbarzadeh: (Col. 18, Ln. 35-38)) is obtained according to a landmark (Akbarzadeh: (Col. 12, Ln. 56-57)), based on deep neural network (DNN)-based semantic segmentation (Akbarzadeh: (Col. 10, Ln. 54-58); (Col. 11, Ln. 17-20)).
REGARDING CLAIM 12, Akbarzadeh, as modified, remains as applied above to claim 10, and further, Akbarzadeh also discloses, receiving 3D feature point information on a world domain for a landmark (Akbarzadeh: (Col. 22, Ln. 36-52)) in the vicinity of the moving object (Akbarzadeh: (Col. 22, Ln. 36-52)) from a HD map database based on position information of the moving object (Akbarzadeh: (Col. 22, Ln. 36-52)); and converting the 3D feature point information on the world domain into a local domain for the capturing device (Akbarzadeh: (Col. 18, Ln. 67 - Col. 19, Ln. 5)).
REGARDING CLAIM 13, Akbarzadeh, as modified, remains as applied above to claim 10, and further, Akbarzadeh also discloses, predicting the positions of the particles (Akbarzadeh: (Col. 35, Ln. 10-22)) based on position information of particles rearranged at a previous point in time and a displacement of the moving object from the previous point in time (Akbarzadeh: [ABS]).
REGARDING CLAIM 14, Akbarzadeh, as modified, remains as applied above to claim 10, and further, Akbarzadeh also discloses, the 3D feature point information is projected onto a 2D probability map (Akbarzadeh: (Col. 22, Ln. 37-41)) obtained from the surrounding image based on perspective mapping (Akbarzadeh: (Col. 23, Ln. 11-14)).
REGARDING CLAIM 15, Akbarzadeh, as modified, remains as applied above to claim 10, and further, Akbarzadeh also discloses, summing probabilities of the projected feature point information corresponding to each landmark (Akbarzadeh: (Col. 18, Ln. Ln. 63 - Col. 19, Ln. 1-8)), in the probability map (Akbarzadeh: (Col. 22, Ln. 28-35)); and multiplying summed probabilities corresponding to respective landmarks (Akbarzadeh: (Col. 34, Ln. 65 - Col. 35, Ln. 5); (Col. 35, Ln. 39-43); (Col. 72, Ln. 62 - Col. 73, Ln. 2)).
Akbarzadeh does not explicitly recite the terminology "multiplying summed". However, Akbarzadeh discloses an accumulation/aggregation of matches and predictions. Which, the examiner respectfully submits, is parallel in service and result for a determination over an accumulation of results.
REGARDING CLAIM 16, Akbarzadeh, as modified, remains as applied above to claim 1, and further, Akbarzadeh also discloses, setting weights for the respective positions of the particles according to the similarity (Akbarzadeh: (Col. 11, Ln. 39-42); (Col. 16, Ln. 25-50)); rearranging the particles according to the weights (Akbarzadeh: (Col. 16, Ln. 25-50)); see (Col. 32, Ln. 47 - Col. 33, Ln. 7) for more cost and filtering and updating (rearranging)); and estimating the position of the moving object by calculating a mean value of the rearranged particles (Akbarzadeh: ((Col. 21, Ln. 57 - Col. 23, Ln. 4), (Col. 23, Ln. 5 - Col. 25, Ln. 24)) examiner: disclose aggregation of scores/weights (sum) and averages of landmarks in a map; (Col. 32, Ln. 26-31)).
REGARDING CLAIM 17, Akbarzadeh, as modified, remains as applied above to claim 10, and further, Akbarzadeh also discloses, the moving object is an autonomous vehicle or a vehicle supporting advanced driver-assistance systems (ADAS) (Akbarzadeh: (Col. 5, Ln. 4-6)).
REGARDING CLAIM 18, Akbarzadeh, as modified, remains as applied above to claim 1, and further, Akbarzadeh also discloses, A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the operating method of claim 1 (Akbarzadeh: (Col. 5, Ln. 63-67) Various functions described herein as being performed by entities may be carried out by hardware, firmware, and/or software. For instance, various functions may be carried out by a processor executing instructions stored in memory).
REGARDING CLAIM 19, Akbarzadeh discloses, a communication module (Akbarzadeh: (Col. 5, Ln. 15-16)) configured to receive high-definition (HD) map data of a vicinity of a moving object (Akbarzadeh: (Col. 6, Ln. 27-30); (Col. 31, Ln. 19-25)) and a surrounding image acquired by a capturing device mounted on the moving object (Akbarzadeh: (Col. 53, Ln. 37-39); (Col. 63, Ln. 1-4)); and a surrounding image acquired by a capturing device mounted on the moving object (Akbarzadeh: (Col. 53, Ln. 37-39); (Col. 63, Ln. 1-4)); a memory configured to store computer-executable instructions (Akbarzadeh: (Col. 5, Ln. 63-67)), the HD map data, and the surrounding image (Akbarzadeh: (Col. 6, Ln. 26-55)); and a processor configured to execute the computer-executable instructions (Akbarzadeh: (Col. 5, Ln. 63-67)) to configure the processor to: generate two-dimensional (2D) feature point information of landmarks (Akbarzadeh: (Col. 72, Ln. 55-65)) in a landmark-based probability map from a surrounding image (Akbarzadeh: (Col. 18, Ln. 34 - Col. 19, Ln. 3); (Col. 11, Ln. 39-50)) acquired by a capturing device mounted on a moving object (Akbarzadeh: (Col. 4, Ln. 5-7)); obtaining landmark-based three-dimensional (3D) feature point information of the landmarks (Akbarzadeh: (Col. 18, Ln. 36-38)) from high-definition (HD) map data of a vicinity of the moving object (Akbarzadeh: (Col. 2, Ln. 62-63); (Col. 3, Ln. 15-19); (Col. 13, Ln. 59-64)); converting one of the 2D feature point information of the surrounding image to 3D data (Akbarzadeh: (Col. 8, Ln. 24-27)) or converting the 3D feature point information of the HD map data to 2D (Akbarzadeh: (Col. 28, Ln. 64-65) 2D landmarks generated from the 3D landmarks); summing, for each of the landmarks, probabilities of the landmark (Akbarzadeh: (Col. 16, Ln. 46-53); (Col. 18, Ln. 10-12)); (Col. 51, Ln. 57-57); (Col. 26, Ln. 50-57)) determined based on the landmark-based probability map (Akbarzadeh: (Col. 51, Ln. 47-51); (Col. 41, Ln. 49-53); (Col. 26, Ln. 50-57)); multiplying the summed probabilities (Akbarzadeh: (Col. 26, Ln. 50-57) according to equation (4), below [Eq. 4: cost = sqrt(∑(u^2/s^2))]; (Col. 51, Ln. 47-51); (Col. 41, Ln. 49-53)), each of the summed probabilities corresponding to a respective landmark one of the landmarks (Akbarzadeh: (Col. 51, Ln. 47-51); (Col. 41, Ln. 49-53); see (Col. 18, Ln. 51 - Col. 19, Ln. 9) for landmarks); determine a first similarity between the 2D feature point information and the 2D data or determine a second similarity between the 3D feature point information and the 3D data (Akbarzadeh: (Col. 28, Ln. 24-35); (Col. 22, Ln. 61-66); see at least (Col. 23, Ln. 26 - Col 24, Ln. 2; Col. 28, Ln. 23-52; Col. 29, Ln. 7 - Col. 30, Ln. 13); see at least (Col. 6, Ln. 22-55; Col. 23, Ln. 26 - Col 24); see at least (Col. 2, Ln. 64-67) for aggregating data and creating new maps) based on the multiplied summed probabilities corresponding to each landmark determined based on the landmark-based probability map (Akbarzadeh: according to equation (4), below [Eq. 4: cost = sqrt(∑(u^2/s^2))] (Col. 26, Ln. 50-57); (Col. 27, Ln. 13-18)); and estimate a position of the moving object (Akbarzadeh: (Col. 3, Ln. 16-20)).
The examiner respectfully submits, Akbarzadeh discloses based on the similarity (Akbarzadeh: ((Col. 21, Ln. 57 - Col. 23, Ln. 4), (Col. 23, Ln. 5 - Col. 25, Ln. 24)); (Col. 32, Ln. 26-31)).
However, in the alternative, and in the same field of endeavor, Hirzer discloses, based on the similarity (Hirzer: (Col. 16, Ln. 39-45)), for the benefit of selecting a pose from the plurality of poses, such that the 3D rendering corresponding to the selected pose aligns with the segmented image.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Akbarzadeh to include determining a confidence associated with a vehicle pose taught by Hirzer. One of ordinary skill in the art would have been motivated to make this modification, with a reasonable expectation of success, in order to select a pose from the plurality of poses, such that the 3D rendering corresponding to the selected pose aligns with the segmented image.
REGARDING CLAIM 20, Akbarzadeh, as modified, remains as applied above to claim 19, and further, Akbarzadeh also discloses, sum the probabilities of the 2D data corresponding to each landmark (Akbarzadeh: (Col. 18, Ln. Ln. 63 - Col. 19, Ln. 1-8)), in the probability map of the 2D feature point information (Akbarzadeh: (Col. 22, Ln. 28-35); (Col. 22, Ln. 61-66); see at least (Col. 23, Ln. 26 - Col 24, Ln. 2; Col. 28, Ln. 23-52; Col. 29, Ln. 7 - Col. 30, Ln. 13) for comparing points, lines, and dots for matching and similarities, aggregated matching points, values of matches for 2D and 3D multi-layered maps, this includes first, second, third, and beyond similarities; see at least (Col. 6, Ln. 22-55; Col. 23, Ln. 26 - Col 24) for first and second maps, points, etc., and matching and similarities; see at least (Col. 2, Ln. 64-67) for aggregating data and creating new maps), and calculate the first similarity by multiplying summed first probabilities corresponding to respective landmarks (Akbarzadeh: (Col. 34, Ln. 65 - Col. 35, Ln. 5); (Col. 35, Ln. 39-43); (Col. 72, Ln. 62 - Col. 73, Ln. 2); (Col. 22, Ln. 61-66); see at least (Col. 23, Ln. 26 - Col 24, Ln. 2; Col. 28, Ln. 23-52; Col. 29, Ln. 7 - Col. 30, Ln. 13); see at least (Col. 6, Ln. 22-55; Col. 23, Ln. 26 - Col 24) for first and second maps, points, etc., and matching and similarities; see at least (Col. 2, Ln. 64-67) for aggregating data and creating new maps), sum second probabilities of the 3D feature point information corresponding to each landmark, in the 3D probability map of the 3D data (Akbarzadeh: (Col. 22, Ln. 61-66); see at least (Col. 23, Ln. 26 - Col 24, Ln. 2; Col. 28, Ln. 23-52; Col. 29, Ln. 7 - Col. 30, Ln. 13); see at least (Col. 6, Ln. 22-55; Col. 23, Ln. 26 - Col 24) for first and second maps, points, etc., and matching and similarities; see at least (Col. 2, Ln. 64-67) for aggregating data and creating new maps); and calculate the second similarity by multiplying summed second probabilities corresponding to each landmark (Akbarzadeh: (Col. 22, Ln. 61-66); see at least (Col. 23, Ln. 26 - Col 24, Ln. 2; Col. 28, Ln. 23-52; Col. 29, Ln. 7 - Col. 30, Ln. 13); see at least (Col. 6, Ln. 22-55; Col. 23, Ln. 26 - Col 24) for first and second maps, points, etc., and matching and similarities; see at least (Col. 2, Ln. 64-67) for aggregating data and creating new maps).
Response to Arguments
Applicant's arguments, beginning on page 9, filed 01-02-2026, have been fully considered but they are not persuasive. The applicant, to the examiner’s best understanding, has contended the prior art of Akbarzadeh (US 11713978 B2) fails to disclose, “summing, for each of the landmarks, probabilities of the landmark determined based on the landmark-based probability map; multiplying the summed probabilities, each of the summed probabilities corresponding to a respective landmark one of the landmarks”. The examiner respectfully disagrees.
As cited above, Akbarzadeh (US 11713978 B2) discloses, summing, for each of the landmarks, probabilities of the landmark (Akbarzadeh: (Col. 51, Ln. 47-57) a neural network that outputs a measure of confidence for each object detection. Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections. This confidence value enables the system to make further decisions regarding which detections should be considered as true positive detections rather than false positive detections. For example, the system may set a threshold value for the confidence and consider only the detections exceeding the threshold value as true positive detections (examiner: the accumulation of measurements creates probabilities of further detections, which affects map building, thus summing probabilities)) determined based on the landmark-based probability map (Akbarzadeh: (Col. 51, Ln. 47-51) a neural network that outputs a measure of confidence for each object detection. Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections; (Col. 41, Ln. 49-53) at each frame, comparing one or more of the sensor data or the DNN outputs to map data to generate a cost space representing a probability of a vehicle being located at each of a plurality of poses (examiner: each pose gets a respective probability)); multiplying the summed probabilities (Akbarzadeh: see (Col. 51, Ln. 47-51) and (Col. 41, Ln. 49-53), supra, for summing probabilities, multiplying is adding really fast, thus interpreted as parallel in service and result), each of the summed probabilities corresponding to a respective landmark one of the landmarks (Akbarzadeh: (Col. 51, Ln. 47-51) a neural network that outputs a measure of confidence for each object detection. Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections; (Col. 41, Ln. 49-53) at each frame, comparing one or more of the sensor data or the DNN outputs to map data to generate a cost space representing a probability of a vehicle being located at each of a plurality of poses; see (Col. 18, Ln. 51 - Col. 19, Ln. 9) for landmarks).
To the examiner’s best understanding, the applicant has emphasized “landmark” and “multiplying a sum”. Regarding “landmarks”, Akbarzadeh discloses a process for map building which includes all details in a vehicle environment, including “landmarks”. Regarding “multiplying a sum”, Akbarzadeh discloses according to equation (4), below [Eq. 4: cost = sqrt(∑(u^2/s^2))], which, discloses determining as error for probabilities by finding the square-root of the sum of samples*(u^2/s^2). Which, to the examiner’s best understanding, is multiplying a sum.
Further, in considering the disclosure of a reference, it is proper to take into account not only specific teachings of the reference but also the inferences which one skilled in the art would reasonably be expected to draw therefrom (mpep 2144.01). Where the general conditions of a claim are disclosed in the prior art, it is not inventive to discover the optimum or workable ranges by routine experimentation … a change in form, proportions, or degree “will not sustain a patent” … It is a settled principle of law that a mere carrying forward of an original patented conception involving only change of form, proportions, or degree, or the substitution of equivalents doing the same thing as the original invention, by substantially the same means, is not such an invention as will sustain a patent, even though the changes of the kind may produce better results than prior inventions (mpep 2144.05.II.A). To the examiner’s best understanding, the instant application appears to carry forward the prior art, Akbarzadeh (US 11713978 B2), and claiming a change in form such as multiplying a sum for a confidence. The examiner respectfully submits Akbarzadeh (US 11713978 B2) is replete with recitations for determining a confidence based on the accumulation samples and probability determinations. Because Akbarzadeh (US 11713978 B2) discloses that which is claimed, the examiner respectfully maintains the rejection of the independent claims under 35 USC §103, obviousness.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Gausebeck (US 20190026958 A1)
Youmans (US 20200324898 A1)
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARRON SANTOS whose telephone number is (571)272-5288. The examiner can normally be reached Monday - Friday: 8:00am - 4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANGELA ORTIZ can be reached at (571) 272-1206. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.S./Examiner, Art Unit 3663
/ANGELA Y ORTIZ/Supervisory Patent Examiner, Art Unit 3663
Prosecution Insights