DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05/19/2026 has been entered.
Response to Amendments
Claims 1-30 are pending regarding this application.
The amendments to claims 1, 10, and 28 are accepted and entered.
Response to Arguments
Applicant's arguments, filed 05/19/2026, in regards to the 101 Abstract Idea rejection applied to claims 1-6, 10-15, and 28-30 have been fully considered and are persuasive. The 101 Abstract Idea rejection of claims 1-6, 10-15, and 28-30 has been withdrawn.
Applicant has acknowledged the 112(f) interpretation applied to claims 19, 20, 22, 25. None of these claims have been amended in the Claims filed 05/19/2026. As a result, the 112(f) interpretation of claims 19, 20, 22, 25 is maintained. Please note that the Claim Interpretation section has not been re-stated in this Office Action.
Applicant's arguments filed 05/19/2026 regarding the 103 Rejection applied to claims 1-4, 9-13, 18, and 28-30 have been fully considered, and are not persuasive.
Regarding the 103 rejection, applicant argues that “Millard Discloses a Sequential Prediction Pipeline, Not a Cell-by-Cell Architecture”…” Millard's data flow is: sensor data + observed occupancy maps + predictive model + predicted occupancy map. This is a sequential feed-forward pipeline. There are only two types of occupancy grids in Millard, observed and predicted, not three. Importantly, Millard does not at any point take both a predicted occupancy grid and an observed occupancy grid and combine them on a per-cell basis to produce a third, updated occupancy grid. The claimed updated occupancy grid is a third data structure that is distinct from both the predicted and observed grids and is produced by a cell-by-cell operation using corresponding cells from each.”. However, Examiner maintains that Millard teaches the updated occupancy map. Applicant has additionally amended claim 1 such that “each of a plurality of third cells of the updated occupancy grid corresponds to a respective one of the sub-regions and is determined based on a corresponding one of the plurality of first cells of the predicted occupancy grid and a corresponding one of the plurality of second cells of the observed occupancy grid”. Millard teaches that “the predictive model 222 may process preceding obstacle location predictions (e.g., predicted occupancy maps from preceding time steps) to generate obstacle location predictions for time steps further in the future” wherein “the predicted occupancy map for future time step n+1 may then be provided as input to the predictive model 222 to generate the next predicted occupancy map for future time step n+2” and “the model 222 may already be configured to output predicted occupancy map(s) in the same format as occupancy maps 216 (e.g., with only binary occupancy values assigned to each cell indicating whether or not the respective locations in the environment corresponding to cells in the map are predicted to be occupied by an obstacle at a future time step associated with the map” in col. 9, line 49 through col. 10, line 21. Here, since the “next predicted occupancy map for future time step n+2” is based on the predicted occupancy map for future time step n + 1, and “occupancy maps 216 from time steps n−10 through n” and additionally has the same cellular format as occupancy maps 216 [observed occupancy map], it is implicit that the next predicted occupancy map has a plurality of third cells which correspond to sub-regions determined based on a corresponding one of the plurality of first cells of the predicted occupancy grid (predicted occupancy map for future time step n+1) and a corresponding one of the plurality of second cells of the observed occupancy grid (occupancy maps 216).
Applicant additionally argues “The Office Action appears to rely on Millard's recurrent hidden state (col. 9, lines 36-42) as the "updated" occupancy grid. However, a hidden state in a recurrent neural network is an internal model parameter, a vector of learned representations maintained within the model's layers. It is not an occupancy grid.” …”Rather, the hidden state in Millard is updated sequentially as each new observed occupancy map is processed; it is a function of the sequence of observed maps, not a cell-by-cell combination of a predicted grid and an observed grid.”. Examiner has updated the citations below to include additional information regarding the mapping of the “updated occupancy grid”. Please see col. 9, line 49 through col. 10, line 21 which provides additional insight regarding the generation of the “next predicted occupancy map for future time step n+2” which is being interpreted as equivalent to the updated occupancy grid in the 103 rejection below.
Applicant argues that “Komorkiewicz does not teach or suggest determining an updated occupancy grid on a per-cell basis from corresponding cells of a predicted occupancy grid and an observed occupancy grid”. However, Millard teaches the updated occupancy grid on a per-cell basis, it is not necessary for Komorkiewicz to additionally teach an updated occupancy grid in the same manner. The Komorkiewicz reference is solely used to teach the process of using machine learning to determine an observed occupancy grid. Please see the 103 rejection and motivation of claims 1, 10, and 28 below regarding this matter.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 10-13, 18, and 28-30 are rejected under 35 U.S.C. 103 as being unpatentable over Millard (U.S. Patent No. 11016491 B1) in view of Komorkiewicz et al. (U.S. Publication No. 2021/0165093 A1), hereinafter Komorkiewicz.
Regarding claim 1, Millard teaches an apparatus comprising:
a memory; and a processor communicatively coupled to the memory (Millard teaches and a “processor 610 [] capable of processing instructions stored in the memory 620” in col. 16, lines 23-24), and configured to:
determine a predicted occupancy grid based on a previous occupancy grid (Millard teaches that “the occupancy prediction can be, for example, in the form of a predicted occupancy map that includes a grid of occupancy cells and that indicates, for each occupancy cell, a likelihood (e.g., a probability) that the location of the environment corresponding to the occupancy cell will be occupied by an obstacle at the future time step” in col. 14, lines 12-18, wherein “for time steps after a first future time step, predicted occupancy map from a preceding time step can be fed back to the predictive model as an input to generate a new predicted occupancy map for the next future time step” in col. 14, lines 19-22), the predicted occupancy grid comprising a plurality of first cells corresponding to sub-regions of a region, each of the plurality of first cells including a plurality of predicted indications of probability each indicative of a predicted probability of a respective possible type of occupier of the respective first cell (Millard teaches “a grid of occupancy cells and that indicates, for each occupancy cell, a likelihood (e.g., a probability) that the location of the environment corresponding to the occupancy cell will be occupied by an obstacle at the future time step” in col. 14, lines 12-18; here, the probability is based on a prediction of whether the obstacle is dynamic or static, and the prediction of whether the obstacle is dynamic or static is interpreted as the possible type of occupier which is inherently tied to the probability value indicated by the prediction model. Since Millard recites “for each cell”, it is inherent that this process occurs for a first cell, wherein the cells each represent sub-regions of a region (see col. 7, lines 23-32));
determine, (Millard teaches “the occupancy maps 216 that are provided as output from mapping engine 214 indicate current or recent locations of obstacles that were observed by the robot 202” in col. 7, line 66 through col. 8, line 2, wherein “the map 300 includes a two-dimensional grid of cells arranged in a collection of rows and columns” as shown in col. 7, lines 33-36; see also FIG. 3, wherein the cells of the grid are interpreted as the second cells corresponding to the sub-regions of the region; Millard additionally teaches “the mapping engine 214 may receive the sensor data 212 that includes measurements from one or more sensors on the robot 202” in col. 6, lines 33-35. Here, the one or more sensors on the robot are interpreted as equivalent to the claimed first sensor of a vehicle);
determine an updated occupancy grid based on the observed occupancy grid and the predicted occupancy grid (Millard teaches that “the predictive model 222 may process preceding obstacle location predictions (e.g., predicted occupancy maps from preceding time steps) to generate obstacle location predictions for time steps further in the future” wherein “the predicted occupancy map for future time step n+1 may then be provided as input to the predictive model 222 to generate the next predicted occupancy map for future time step n+2” and “the model 222 may already be configured to output predicted occupancy map(s) in the same format as occupancy maps 216 (e.g., with only binary occupancy values assigned to each cell indicating whether or not the respective locations in the environment corresponding to cells in the map are predicted to be occupied by an obstacle at a future time step associated with the map” in col. 9, line 49 through col. 10, line 21. Here, since the “next predicted occupancy map for future time step n+2” is based on the predicted occupancy map for future time step n + 1, and “occupancy maps 216 from time steps n−10 through n” and additionally has the same cellular format as occupancy maps 216 [observed occupancy map]), wherein each of a plurality of third cells of the updated occupancy grid corresponds to a respective one of the sub-regions and is determined based on a corresponding one of the plurality of first cells of the predicted occupancy grid and a corresponding one of the plurality of second cells of the observed occupancy grid (Millard teaches that “the predictive model 222 may process preceding obstacle location predictions (e.g., predicted occupancy maps from preceding time steps) to generate obstacle location predictions for time steps further in the future” wherein “the model 222 may already be configured to output predicted occupancy map(s) in the same format as occupancy maps 216 (e.g., with only binary occupancy values assigned to each cell indicating whether or not the respective locations in the environment corresponding to cells in the map are predicted to be occupied by an obstacle at a future time step associated with the map” in col. 9, line 49 through col. 10, line 21); and
provide the updated occupancy grid to a vehicle control system of the vehicle to control operation of the vehicle based on the updated occupancy grid (Millard teaches that “the local planner 232 receives the planned route from global planner 230, along with obstacle prediction data 226 from the obstacle location prediction engine 218, and generates a planned path from the current location of the robot 202 to a farther point along the route. The local planner 232 can constantly update the path as the robot travels” in col. 12, lines 23-28. Here, the robot is equivalent to the claimed vehicle, and the prediction data 226 includes the updated occupancy grid as shown in col. 12, lines 44-61. See also col. 11, line 65 through col. 13, line 27).
Millard fails to teach using machine learning to determine an observed occupancy grid.
However, Komorkiewicz teaches using machine learning to determine an observed occupancy grid (Kormorkiewicz teaches “camera-radar fusion for determination of an occupancy map using a neural network” in para. [0047] and FIG. 1; see para. [0046] wherein the occupancy map is a occupancy grid).
Millard and Komorkiewicz are both considered to be analogous to the claimed invention because they are in the same field of developing occupancy grids for a moving vehicle. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Millard to incorporate the teachings of Komorkiewicz and include “using machine learning to determine an observed occupancy grid”. The motivation for doing so would have been to “improve the determination of the (occupancy grid) patch because it may contain the high level understanding of a scene (since each object class is marked as different color) as well as improve the sharpness of the edges in a given occupancy grid map”, as suggested by Komorkiewicz in para. [0066]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Millard with Komorkiewicz to obtain the invention specified in claim 1.
Regarding claim 2, Millard and Komorkiewicz teach the apparatus of claim 1, further comprising:
a first sensor configured to obtain the first sensor measurements (Komorkiewicz teaches a “second sensor [which] may include or may be a camera” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075]; see also FIG. 4; the second sensor/measurements is interpreted as equivalent to the claimed first sensor/measurements); and
a second sensor configured to obtain second sensor measurements (Komorkiewicz teaches a “first sensor [which] may include or may be a radar sensor” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075]; see also FIG. 4; the first sensor/measurements is interpreted as equivalent to the claimed second sensor/measurements);
wherein the processor is communicatively coupled to the first sensor and the second sensor (Komorkiewicz teaches a computer which implements the process of acquiring the measurements taken by the first and second sensor in para. [0035]) (Millard also teaches the processor(s) being coupled to the sensor(s) in FIG. 2 and FIG. 6, wherein “each of the components 610, 620, 630, and 640 are interconnected using a system bus 650” as shown in col. 16, lines 12-14), and wherein to determine the observed occupancy grid the processor is configured to use, for each of the plurality of second cells,
(path 1) a respective first portion of first information corresponding to the first sensor measurements (Komorkiewicz teaches “determining a segmented image based on the second sensor data; and determining a segmented image portion of the segmented image. The entry of the occupancy map may be determined further based on the segmented image portion” in para. [0076]; here the occupancy grid is determined based on the second sensor data, wherein the occupancy map is interpreted as equivalent to the observed occupancy grid. The entry here is interpreted as the second information (see definition of entry as described in para. [0009] which corresponds to a cell; see also computer implemented method wherein the processor carries out the above process in para. [0035]; the second sensor data is interpreted as equivalent to the claimed first sensor measurements),
(path 2) a respective second portion of second information corresponding to the second sensor measurements, or
(path 3) a combination thereof (Komorkiewicz, see FIG. 2B). Similar motivations as applied to claim 1 can be applied here.
Note: Only one limitation need be found in the prior art due to the “or” language in the claim.
Regarding claim 3, Millard and Komorkiewicz teach the apparatus of claim 2,
wherein the first information comprises the first sensor measurements (Komorkiewicz teaches a “second sensor [which] may include or may be a camera” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075], wherein the plurality of measurements from the second sensor is interpreted as the first information; see also FIG. 4) and
the second information comprises the second sensor measurements (Komorkiewicz teaches a “first sensor [which] may include or may be a radar sensor” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075]; see also FIG. 4 wherein a portion of the first sensor data is determined to correspond to a potential object. This determination is interpreted as the second information), and
wherein to determine the observed occupancy grid the processor is configured to use, for each of the plurality of second cells,
(path 1) at least one of the first sensor measurements (Komorkiewicz teaches “determining a segmented image based on the second sensor data; and determining a segmented image portion of the segmented image. The entry of the occupancy map may be determined further based on the segmented image portion” in para. [0076]; here the occupancy grid is determined based on the second sensor data (which is equivalent to the first sensor measurements as noted above), wherein the occupancy map is interpreted as equivalent to the observed occupancy grid. The entry here is interpreted as a cell (see definition of entry as described in para. [0009]; see also computer implemented method wherein the processor carries out the above process in para. [0035]; see FIG. 4),
(path 2) at least one of the second sensor measurements, or
(path 3) a combination thereof (Komorkiewicz, see FIG. 2B). Similar motivations as applied to claim 1 can be applied here.
Note: Only one limitation need be found in the prior art due to the “or” language in the claim.
Regarding claim 4, Millard and Komorkiewicz teach the apparatus of claim 2,
wherein the first information is derived from the first sensor measurements (Komorkiewicz teaches a “second sensor [which] may include or may be a camera” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075], wherein the plurality of measurements from the second sensor is interpreted as the first information; see also FIG. 4) and
the second information is derived from the second sensor measurements (Komorkiewicz teaches a “first sensor [which] may include or may be a radar sensor” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075]; see also FIG. 4 wherein a portion of the first sensor data is determined to correspond to a potential object. This determination is interpreted as the second information). Similar motivations as applied to claim 1 can be applied here.
Regarding claim 9, Millard and Komorkiewicz teach the apparatus of claim 1,
wherein the plurality of predicted indications of probability are each indicative of a plausibility of the respective possible type of occupier of the respective first cell actually occupying the respective first cell (Millard teaches “the obstacle location predictive model 222 may be trained to implicitly predict motion/travel characteristics of different types of obstacles in an environment” in col. 11, lines 49-51; here, the probability is based on a prediction of whether the obstacle is dynamic or static; the prediction of whether the obstacle is dynamic or static is interpreted as the possible type of occupier).
Regarding claim 10, Millard teaches an occupancy grid determination method comprising:
determining a predicted occupancy grid based on a previous occupancy grid (Millard teaches that “the occupancy prediction can be, for example, in the form of a predicted occupancy map that includes a grid of occupancy cells and that indicates, for each occupancy cell, a likelihood (e.g., a probability) that the location of the environment corresponding to the occupancy cell will be occupied by an obstacle at the future time step” in col. 14, lines 12-18, wherein “for time steps after a first future time step, predicted occupancy map from a preceding time step can be fed back to the predictive model as an input to generate a new predicted occupancy map for the next future time step” in col. 14, lines 19-22), the predicted occupancy grid comprising a plurality of first cells corresponding to sub-regions of a region, each of the plurality of first cells including a plurality of predicted indications of probability each indicative of a predicted probability of a respective possible type of occupier of the respective first cell (Millard teaches “a grid of occupancy cells and that indicates, for each occupancy cell, a likelihood (e.g., a probability) that the location of the environment corresponding to the occupancy cell will be occupied by an obstacle at the future time step” in col. 14, lines 12-18; here, the probability is based on a prediction of whether the obstacle is dynamic or static, and the prediction of whether the obstacle is dynamic or static is interpreted as the possible type of occupier which is inherently tied to the probability value indicated by the prediction model. Since Millard recites “for each cell”, it is inherent that this process occurs for a first cell, wherein the cells each represent sub-regions of a region (see col. 7, lines 23-32));
determining, (Millard teaches “the occupancy maps 216 that are provided as output from mapping engine 214 indicate current or recent locations of obstacles that were observed by the robot 202” in col. 7, line 66 through col. 8, line 2, wherein “the map 300 includes a two-dimensional grid of cells arranged in a collection of rows and columns” as shown in col. 7, lines 33-36; see also FIG. 3, wherein the cells of the grid are interpreted as the second cells corresponding to the sub-regions of the region; Millard additionally teaches “the mapping engine 214 may receive the sensor data 212 that includes measurements from one or more sensors on the robot 202” in col. 6, lines 33-35. Here, the one or more sensors on the robot are interpreted as equivalent to the claimed first sensor of a vehicle);
determining an updated occupancy grid based on the observed occupancy grid and the predicted occupancy grid (Millard teaches that “the predictive model 222 may process preceding obstacle location predictions (e.g., predicted occupancy maps from preceding time steps) to generate obstacle location predictions for time steps further in the future” wherein “the predicted occupancy map for future time step n+1 may then be provided as input to the predictive model 222 to generate the next predicted occupancy map for future time step n+2” and “the model 222 may already be configured to output predicted occupancy map(s) in the same format as occupancy maps 216 (e.g., with only binary occupancy values assigned to each cell indicating whether or not the respective locations in the environment corresponding to cells in the map are predicted to be occupied by an obstacle at a future time step associated with the map” in col. 9, line 49 through col. 10, line 21. Here, since the “next predicted occupancy map for future time step n+2” is based on the predicted occupancy map for future time step n + 1, and “occupancy maps 216 from time steps n−10 through n” and additionally has the same cellular format as occupancy maps 216 [observed occupancy map]), wherein each of a plurality of third cells of the updated occupancy grid corresponds to a respective one of the sub-regions and is determined based on a corresponding one of the plurality of first cells of the predicted occupancy grid and a corresponding one of the plurality of second cells of the observed occupancy grid (Millard teaches that “the predictive model 222 may process preceding obstacle location predictions (e.g., predicted occupancy maps from preceding time steps) to generate obstacle location predictions for time steps further in the future” wherein “the model 222 may already be configured to output predicted occupancy map(s) in the same format as occupancy maps 216 (e.g., with only binary occupancy values assigned to each cell indicating whether or not the respective locations in the environment corresponding to cells in the map are predicted to be occupied by an obstacle at a future time step associated with the map” in col. 9, line 49 through col. 10, line 21); and
providing the updated occupancy grid to a vehicle control system of the vehicle to control operation of the vehicle based on the updated occupancy grid (Millard teaches that “the local planner 232 receives the planned route from global planner 230, along with obstacle prediction data 226 from the obstacle location prediction engine 218, and generates a planned path from the current location of the robot 202 to a farther point along the route. The local planner 232 can constantly update the path as the robot travels” in col. 12, lines 23-28. Here, the robot is equivalent to the claimed vehicle, and the prediction data 226 includes the updated occupancy grid as shown in col. 12, lines 44-61. See also col. 11, line 65 through col. 13, line 27).
Millard fails to teach using machine learning to determine an observed occupancy grid.
However, Komorkiewicz teaches using machine learning to determine an observed occupancy grid (Kormorkiewicz teaches “camera-radar fusion for determination of an occupancy map using a neural network” in para. [0047] and FIG. 1; see para. [0046] wherein the occupancy map is a occupancy grid).
Millard and Komorkiewicz are both considered to be analogous to the claimed invention because they are in the same field of developing occupancy grids for a moving vehicle. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Millard to incorporate the teachings of Komorkiewicz and include “using machine learning to determine an observed occupancy grid”. The motivation for doing so would have been to “improve the determination of the (occupancy grid) patch because it may contain the high level understanding of a scene (since each object class is marked as different color) as well as improve the sharpness of the edges in a given occupancy grid map”, as suggested by Komorkiewicz in para. [0066]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Millard with Komorkiewicz to obtain the invention specified in claim 10.
Regarding claim 11, Millard and Komorkiewicz teach the occupancy grid determination method of claim 10, further comprising:
obtaining the first sensor measurements from a first sensor (Komorkiewicz teaches a “second sensor [which] may include or may be a camera” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075]; see also FIG. 4; the second sensor is interpreted as equivalent to the first sensor); and
obtaining second sensor measurements from a second sensor (Komorkiewicz teaches a “first sensor [which] may include or may be a radar sensor” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075]; see also FIG. 4; the first sensor is interpreted as equivalent to the second sensor);
wherein determining the observed occupancy grid the processor using, for each of the plurality of second cells (see below citation),
(path 1) a respective first portion of first information corresponding to the first sensor measurements (Komorkiewicz teaches “determining a segmented image based on the second sensor data; and determining a segmented image portion of the segmented image. The entry of the occupancy map may be determined further based on the segmented image portion” in para. [0076]; here the occupancy grid is determined based on the second sensor data, wherein the occupancy map is interpreted as equivalent to the observed occupancy grid. The entry here is interpreted as the second information (see definition of entry as described in para. [0009] which corresponds to a cell; see also computer implemented method wherein the processor carries out the above process in para. [0035]),
(path 2) a respective second portion of second information corresponding to the second sensor measurements, or
(path 3) a combination thereof (Komorkiewicz, see FIG. 2B). Similar motivations as applied to claim 10 can be applied here.
Note: Only one limitation need be found in the prior art due to the “or” language in the claim.
Regarding claim 12, Millard and Komorkiewicz teach the occupancy grid determination method of claim 11,
wherein the first information comprises the first sensor measurements (Komorkiewicz teaches a “second sensor [which] may include or may be a camera” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075], wherein the plurality of measurements from the second sensor is interpreted as the first information; see also FIG. 4) and
the second information comprises the second sensor measurements (Komorkiewicz teaches a “first sensor [which] may include or may be a radar sensor” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075]; see also FIG. 4 wherein a portion of the first sensor data is determined to correspond to a potential object. This determination is interpreted as the second information), and
wherein to determine the observed occupancy grid the processor is configured to use, for each of the plurality of second cells,
(path 1) at least a first one of the first sensor measurements (Komorkiewicz teaches “determining a segmented image based on the second sensor data; and determining a segmented image portion of the segmented image. The entry of the occupancy map may be determined further based on the segmented image portion” in para. [0076]; here the occupancy grid is determined based on the second sensor data (equivalent to the claimed first sensor measurements), wherein the occupancy map is interpreted as equivalent to the observed occupancy grid. The entry here is interpreted as a cell (see definition of entry as described in para. [0009]; see also computer implemented method wherein the processor carries out the above process in para. [0035]; see FIG. 4),
(path 2) at least a second one of the second sensor measurements, or
(path 3) a combination thereof (Komorkiewicz, see FIG. 2B). Similar motivations as applied to claim 10 can be applied here.
Note: Only one limitation need be found in the prior art due to the “or” language in the claim.
Regarding claim 13, Millard and Komorkiewicz teach the occupancy grid determination method of claim 11,
further comprising deriving the first information from the first sensor measurements (Komorkiewicz teaches a “second sensor [which] may include or may be a camera” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075], wherein the plurality of measurements from the second sensor is interpreted as the first information; see also FIG. 4) and
deriving the second information is derived from the second sensor measurements (Komorkiewicz teaches a “first sensor [which] may include or may be a radar sensor” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075]; see also FIG. 4 wherein a portion of the first sensor data is determined to correspond to a potential object. This determination is interpreted as the second information). Similar motivations as applied to claim 10 can be applied here.
Regarding claim 18, Millard and Komorkiewicz teach the occupancy grid determination method of claim 10,
wherein the plurality of predicted indications of probability are each indicative of a plausibility of the respective possible type of occupier of the respective first cell actually occupying the respective first cell (Millard teaches “the obstacle location predictive model 222 may be trained to implicitly predict motion/travel characteristics of different types of obstacles in an environment” in col. 11, lines 49-51; here, the probability is based on a prediction of whether the obstacle is dynamic or static; the prediction of whether the obstacle is dynamic or static is interpreted as the possible type of occupier).
Regarding claim 28, Millard teaches a non-transitory, processor-readable storage medium comprising processor-readable instructions (Millard teaches that “computer-readable media may be non-transitory, and can be part of a computing system that includes the one or more processors in one or more computers” in col. 3, lines 8-11) to cause a processor to:
determine a predicted occupancy grid based on a previous occupancy grid (Millard teaches that “the occupancy prediction can be, for example, in the form of a predicted occupancy map that includes a grid of occupancy cells and that indicates, for each occupancy cell, a likelihood (e.g., a probability) that the location of the environment corresponding to the occupancy cell will be occupied by an obstacle at the future time step” in col. 14, lines 12-18, wherein “for time steps after a first future time step, predicted occupancy map from a preceding time step can be fed back to the predictive model as an input to generate a new predicted occupancy map for the next future time step” in col. 14, lines 19-22), the predicted occupancy grid comprising a plurality of first cells corresponding to sub-regions of a region, each of the plurality of first cells including a plurality of predicted indications of probability each indicative of a predicted probability of a respective possible type of occupier of the respective first cell (Millard teaches “a grid of occupancy cells and that indicates, for each occupancy cell, a likelihood (e.g., a probability) that the location of the environment corresponding to the occupancy cell will be occupied by an obstacle at the future time step” in col. 14, lines 12-18; here, the probability is based on a prediction of whether the obstacle is dynamic or static, and the prediction of whether the obstacle is dynamic or static is interpreted as the possible type of occupier which is inherently tied to the probability value indicated by the prediction model. Since Millard recites “for each cell”, it is inherent that this process occurs for a first cell, wherein the cells each represent sub-regions of a region (see col. 7, lines 23-32));
determine, to the sub-regions of the region (Millard teaches “the occupancy maps 216 that are provided as output from mapping engine 214 indicate current or recent locations of obstacles that were observed by the robot 202” in col. 7, line 66 through col. 8, line 2, wherein “the map 300 includes a two-dimensional grid of cells arranged in a collection of rows and columns” as shown in col. 7, lines 33-36; see also FIG. 3, wherein the cells of the grid are interpreted as the second cells corresponding to the sub-regions of the region; Millard additionally teaches “the mapping engine 214 may receive the sensor data 212 that includes measurements from one or more sensors on the robot 202” in col. 6, lines 33-35. Here, the one or more sensors on the robot are interpreted as equivalent to the claimed first sensor of a vehicle);
determine an updated occupancy grid based on the observed occupancy grid and the predicted occupancy grid (Millard teaches that “the predictive model 222 may process preceding obstacle location predictions (e.g., predicted occupancy maps from preceding time steps) to generate obstacle location predictions for time steps further in the future” wherein “the predicted occupancy map for future time step n+1 may then be provided as input to the predictive model 222 to generate the next predicted occupancy map for future time step n+2” and “the model 222 may already be configured to output predicted occupancy map(s) in the same format as occupancy maps 216 (e.g., with only binary occupancy values assigned to each cell indicating whether or not the respective locations in the environment corresponding to cells in the map are predicted to be occupied by an obstacle at a future time step associated with the map” in col. 9, line 49 through col. 10, line 21. Here, since the “next predicted occupancy map for future time step n+2” is based on the predicted occupancy map for future time step n + 1, and “occupancy maps 216 from time steps n−10 through n” and additionally has the same cellular format as occupancy maps 216 [observed occupancy map]), wherein each of a plurality of third cells of the updated occupancy grid corresponds to a respective one of the sub-regions and is determined based on a corresponding one of the plurality of first cells of the predicted occupancy grid and a corresponding one of the plurality of second cells of the observed occupancy grid (Millard teaches that “the predictive model 222 may process preceding obstacle location predictions (e.g., predicted occupancy maps from preceding time steps) to generate obstacle location predictions for time steps further in the future” wherein “the model 222 may already be configured to output predicted occupancy map(s) in the same format as occupancy maps 216 (e.g., with only binary occupancy values assigned to each cell indicating whether or not the respective locations in the environment corresponding to cells in the map are predicted to be occupied by an obstacle at a future time step associated with the map” in col. 9, line 49 through col. 10, line 21); and
provide the updated occupancy grid to a vehicle control system of the vehicle to control operation of the vehicle based on the updated occupancy grid (Millard teaches that “the local planner 232 receives the planned route from global planner 230, along with obstacle prediction data 226 from the obstacle location prediction engine 218, and generates a planned path from the current location of the robot 202 to a farther point along the route. The local planner 232 can constantly update the path as the robot travels” in col. 12, lines 23-28. Here, the robot is equivalent to the claimed vehicle, and the prediction data 226 includes the updated occupancy grid as shown in col. 12, lines 44-61. See also col. 11, line 65 through col. 13, line 27).
Millard fails to teach using machine learning to determine an observed occupancy grid.
However, Komorkiewicz teaches using machine learning to determine an observed occupancy grid (Kormorkiewicz teaches “camera-radar fusion for determination of an occupancy map using a neural network” in para. [0047] and FIG. 1; see para. [0046] wherein the occupancy map is a occupancy grid).
Millard and Komorkiewicz are both considered to be analogous to the claimed invention because they are in the same field of developing occupancy grids for a moving vehicle. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Millard to incorporate the teachings of Komorkiewicz and include “using machine learning to determine an observed occupancy grid”. The motivation for doing so would have been to “improve the determination of the (occupancy grid) patch because it may contain the high level understanding of a scene (since each object class is marked as different color) as well as improve the sharpness of the edges in a given occupancy grid map”, as suggested by Komorkiewicz in para. [0066]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Millard with Komorkiewicz to obtain the invention specified in claim 28.
Regarding claim 29, Millard and Komorkiewicz teach the non-transitory, processor-readable storage medium of claim 28, further comprising processor-readable instructions to cause the processor to:
obtain the first sensor measurements from a first sensor (Komorkiewicz teaches a “second sensor [which] may include or may be a camera” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075]; see also FIG. 4; the second sensor is interpreted as equivalent to the claimed first sensor); and
obtain second sensor measurements from a second sensor (Komorkiewicz teaches a “first sensor [which] may include or may be a radar sensor” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075]; see also FIG. 4; the first sensor is interpreted as equivalent to the claimed second sensor);
wherein the processor-readable instructions to cause the processor to determine the observed occupancy grid comprise processor-readable instructions to cause the processor to use, for each of the plurality of second cells (see below citation),
(path 1) a respective first portion of first information corresponding to the first sensor measurements (Komorkiewicz teaches “determining a segmented image based on the second sensor data; and determining a segmented image portion of the segmented image. The entry of the occupancy map may be determined further based on the segmented image portion” in para. [0076]; here the occupancy grid is determined based on the second sensor data, wherein the occupancy map is interpreted as equivalent to the observed occupancy grid. The entry here is interpreted as the second information (see definition of entry as described in para. [0009] which corresponds to a cell; see also computer implemented method wherein the processor carries out the above process in para. [0035]),
(path 2) a respective second portion of second information corresponding to the second sensor measurements, or
(path 3) a combination thereof (Komorkiewicz, see FIG. 2B). Similar motivations as applied to claim 28 can be applied here.
Note: Only one limitation need be found in the prior art due to the “or” language in the claim.
Regarding claim 30, Millard and Komorkiewicz teach the non-transitory, processor-readable storage medium of claim 29,
wherein the first information comprises the first sensor measurements (Komorkiewicz teaches a “second sensor [which] may include or may be a camera” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075], wherein the plurality of measurements from the second sensor is interpreted as the first information; see also FIG. 4) and
the second information comprises the second sensor measurements (Komorkiewicz teaches a “first sensor [which] may include or may be a radar sensor” in para. [0074] configured to acquire a plurality of measurements as shown in para. [0075]; see also FIG. 4 wherein a portion of the first sensor data is determined to correspond to a potential object. This determination is interpreted as the second information), and
wherein the processor-readable instructions to cause the processor to determine the observed occupancy grid comprise processor-readable instructions to cause the processor to use (see below citation), for each of the plurality of second cells,
(path 1) at least one of the first sensor measurements (Komorkiewicz teaches “determining a segmented image based on the second sensor data; and determining a segmented image portion of the segmented image. The entry of the occupancy map may be determined further based on the segmented image portion” in para. [0076]; here the occupancy grid is determined based on the second sensor data, wherein the occupancy map is interpreted as equivalent to the observed occupancy grid. The entry here is interpreted as a cell (see definition of entry as described in para. [0009]; see also computer implemented method wherein the processor carries out the above process in para. [0035]; see FIG. 4),
(path 2) at least one of the second sensor measurements, or
(path 3) a combination thereof (Komorkiewicz, see FIG. 2B). Similar motivations as applied to claim 28 can be applied here.
Note: Only one limitation need be found in the prior art due to the “or” language in the claim.
Claims 5 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Millard (U.S. Patent No. 11016491 B1) in view of Komorkiewicz et al. (U.S. Publication No. 2021/0165093 A1), hereinafter Komorkiewicz in view of Trojahner (U.S. Publication No. 2023/0056589 A1).
Regarding claim 5, Millard and Komorkiewicz teach the apparatus of claim 4.
Millard and Komorkiewicz fail to teach wherein the first information comprises a bird’s-eye view of the region.
However, Trojahner teaches wherein the first information comprises a bird’s-eye view of the region (Trojahner teaches a system and method for generating occupancy occlusion grids wherein “the one or more sensors can be positioned on the vehicle to provide a bird's eye view of the one or more objects” in para. [0040]).
Millard, Komorkiewicz, and Trojahner are all considered to be analogous to the claimed invention because they are in the same field of generating occupancy grid maps to represent dynamic and static objects. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Millard (as modified by Komorkiewicz) to incorporate the teachings of Trojahner and include “wherein the first information comprises a bird’s-eye view of the region”. The motivation for doing so would have been to “facilitate the capture of information associated with one or more objects in the environment surrounding a vehicle”, as suggested by Trojahner in para. [0040]. Evaluation. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Millard and Komorkiewicz with Trojahner to obtain the invention specified in claim 5.
Regarding claim 14, Millard and Komorkiewicz teach the occupancy grid determination method of claim 13.
Millard and Komorkiewicz fail to teach wherein the first information comprises a bird’s-eye view of the region.
However, Trojahner teaches wherein the first information comprises a bird’s-eye view of the region (Trojahner teaches a system and method for generating occupancy occlusion grids wherein “the one or more sensors can be positioned on the vehicle to provide a bird's eye view of the one or more objects” in para. [0040]).
Millard, Komorkiewicz, and Trojahner are all considered to be analogous to the claimed invention because they are in the same field of generating occupancy grid maps to represent dynamic and static objects. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Millard (as modified by Komorkiewicz) to incorporate the teachings of Trojahner and include “wherein the first information comprises a bird’s-eye view of the region”. The motivation for doing so would have been to “facilitate the capture of information associated with one or more objects in the environment surrounding a vehicle”, as suggested by Trojahner in para. [0040]. Evaluation. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Millard and Komorkiewicz with Trojahner to obtain the invention specified in claim 14.
Claims 6-8 and 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Millard (U.S. Patent No. 11016491 B1) in view of Komorkiewicz et al. (U.S. Publication No. 2021/0165093 A1), hereinafter Komorkiewicz in view of Hoermann et al. (“Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling”), hereinafter Hoermann.
Regarding claim 6, Millard and Komorkiewicz teach the apparatus of claim 4.
Millard and Komorkiewicz fail to teach wherein the first information comprises a plurality of first indications of probability each indicative of a first present probability of a first respective possible type of occupier of a first respective one of the sub-regions, and the second information comprises a plurality of second indications of probability each indicative of a second present probability of a second respective possible type of occupier of a second respective one of the sub-regions.
However, Hoermann teaches wherein the first information comprises a plurality of first indications of probability each indicative of a first present probability of a first respective possible type of occupier of a first respective one of the sub-regions (Hoermann teaches “a dynamic occupancy grid map” section II. Filtered Dynamic Input, wherein the map represents probabilities of objects being dynamic; here, the probability of objects being dynamic is interpreted as the first information and the dynamic represents the respective possible type of occupier of the sub-region, wherein there exists “a sequence (PO,d(k)) describing the occupancy probability originated solely from dynamic elements” in Section III. Automatic Output Label Generation), and
the second information comprises a plurality of second indications of probability each indicative of a second present probability of a second respective possible type of occupier of a second respective one of the sub-regions (Hoermann teaches a segmentation method that outputs “a constant occupancy probability PO,s of the static environment” in Section III. Automatic Output Label Generation, wherein the second respective possible type of occupier is static, and the second present probability of a region being static is the second information).
Millard, Komorkiewicz, and Hoermann are all considered to be analogous to the claimed invention because they are in the same field of generating occupancy grid maps to represent dynamic and static objects. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Millard (as modified by Komorkiewicz) to incorporate the teachings of Hoermann and include “wherein the first information comprises a plurality of first indications of probability each indicative of a first present probability of a first respective possible type of occupier of a first respective one of the sub-regions, and the second information comprises a plurality of second indications of probability each indicative of a second present probability of a second respective possible type of occupier of a second respective one of the sub-regions”. The motivation for doing so would have been to allow for a “better performance for classifying static regions in the grid map”, as suggested by Hoermann in Section VII. Evaluation. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Millard and Komorkiewicz with Hoermann to obtain the invention specified in claim 6.
Regarding claim 7, Millard and Komorkiewicz teach the apparatus of claim 2, wherein the processor is further configured to:
determine the first information by applying the image-to-occupancy-grid transformation to third information corresponding to an image corresponding to the first sensor measurements, the first sensor comprising a camera (Millard teaches an image-to-occupancy grid transformation of third information in col. 7, line 66 through col. 8, line 2 wherein Millard teaches “the occupancy maps 216 that are provided as output from mapping engine 214 indicate current or recent locations of obstacles that were observed by the robot 202”, wherein the third information is the updated current location of the obstacle which can either be the observed or predicted location. Millard additionally teaches “the mapping engine 214 may receive the sensor data 212 that includes measurements from one or more sensors on the robot 202” in col. 6, lines 33-35, wherein the sensor may be a camera as shown in col. 6, lines 11-12) (See also Komorkiewicz wherein the second sensor (which is mapped as equivalent to the claimed first sensor in claim 2) is a camera in para. [0013]).
Millard and Komorkiewicz fail to teach determine, through machine learning, an occupancy-grid-to-image transformation; determine an image-to-occupancy-grid transformation based on the occupancy-grid-to-image transformation.
However, Hoermann (in view of Millard) teaches determine, through machine learning, an occupancy-grid-to-image transformation (Hoermann teaches inputting a grid map in order to output “predicted occupancy from 0.5 s to 3.0 s is illustrated in RGB images in the bottom row” in FIG. 5, wherein the transformation is a “learning based prediction” as shown in Section VII. Evaluation);
determine an image-to-occupancy-grid transformation based on the occupancy-grid-to-image transformation (Hoermann teaches inputting a grid map in order to output “predicted occupancy from 0.5 s to 3.0 s is illustrated in RGB images in the bottom row” in FIG. 5. Wherein this process is interpreted as the occupancy-grid-to-image transformation. Additionally, Millard teaches constantly performing an image-to-occupancy-grid transformation based on newly added sensor data (see claim 10). By combining Millard’s teaching of constantly updating an image-to-occupancy-grid transformation based on image data, with Hoermann’s teaching of occupancy-grid-to-image transformation, it is obvious that one of ordinary skill in the art could use Hoermann’s occupancy-grid-to-image transformation to update Millard’s teaching of an image-to-occupancy-grid transformation based on image data with the image data produced by Hoerrman in FIGS. 5 and 6).
Millard, Komorkiewicz, and Hoermann are all considered to be analogous to the claimed invention because they are in the same field of generating occupancy grid maps to represent dynamic and static objects. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Millard (as modified by Komorkiewicz) to incorporate the teachings of Hoermann and “determine, through machine learning, an occupancy-grid-to-image transformation; determine an image-to-occupancy-grid transformation based on the occupancy-grid-to-image transformation”. The motivation for doing so would have been to allow for a “better performance for classifying static regions in the grid map”, as suggested by Hoermann in Section VII. Evaluation. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Millard and Komorkiewicz with Hoermann to obtain the invention specified in claim 7.
Regarding claim 8, Millard, Komorkiewicz, and Hoermann teach the apparatus of claim 7,
wherein the occupancy-grid-to-image transformation maps between an occupancy grid, comprising a plurality of occupancy grid cells, and the third information, comprising a plurality of third-information regions (Hoermann an occupancy-grid-to-image transformation in FIG. 6 in which the input occupancy grid is transformed into a multitude of RGB images (see also FIG. 5); Hoermann further teaches that this transformation occurs between an input occupancy grid and third information regions wherein the third information regions are the static regions which remain the same as the original input image captured by the car sensors), and
the image-to-occupancy-grid transformation maps between the third information and the occupancy grid (see claim 7), and wherein:
(path 1) the occupancy-grid-to-image transformation maps at least two of the plurality of occupancy grid cells to a single pixel of the plurality of third-information regions; or
(path 2) the occupancy-grid-to-image transformation maps a single occupancy grid cell of the plurality of occupancy grid cells to at least two of the plurality of third-information regions; or
(path 3) the image-to-occupancy-grid transformation maps at least two of the plurality of third-information regions to a single one of the plurality of occupancy grid cells; or
(path 4) the image-to-occupancy-grid transformation maps a single one of the plurality of third-information regions to at least two of the plurality of occupancy grid cells (Komorkiewicz teaches “determining the entry of the occupancy map may include determining more than only one entry of the occupancy map (i.e. the entries of more than one cell of the occupancy map)” in para. [0012], wherein the third information region is interpreted as one of the second sensor data portion which is captured by a camera and used to update the entries of the occupancy map as shown in para. [0073-0075] and FIG. 4); or
(path 5) a combination of two or more thereof;
whereby there is a non-uniform mapping between the occupancy grid and the third information (Komorkiewicz teaches that the entry may include determining more than only one entry of the occupancy map in para. [0012]; since the entry is determined based on a subjective size of a portion of the first and second sensor data, it is inherent that the mapping is non-uniform). Similar motivations as applied to claims 1 and 7 can be applied here to claim 8.
Note: Only one limitation need be found in the prior art due to the “or” language in the claim.
Regarding claim 15, Millard and Komorkiewicz teach the occupancy grid determination method of claim 13.
Millard and Komorkiewicz fail to teach wherein the first information comprises a plurality of first indications of probability each indicative of a first present probability of a first respective possible type of occupier of a first respective one of the sub-regions, and the second information comprises a plurality of second indications of probability each indicative of a second present probability of a second respective possible type of occupier of a second respective one of the sub-regions.
However, Hoermann teaches wherein the first information comprises a plurality of first indications of probability each indicative of a first present probability of a first respective possible type of occupier of a first respective one of the sub-regions (Hoermann teaches “a dynamic occupancy grid map” section II. Filtered Dynamic Input, wherein the map represents probabilities of objects being dynamic; here, the probability of objects being dynamic is interpreted as the first information and the dynamic represents the respective possible type of occupier of the sub-region, wherein there exists “a sequence (PO,d(k)) describing the occupancy probability originated solely from dynamic elements” in Section III. Automatic Output Label Generation), and
the second information comprises a plurality of second indications of probability each indicative of a second present probability of a second respective possible type of occupier of a second respective one of the sub-regions (Hoermann teaches a segmentation method that outputs “a constant occupancy probability PO,s of the static environment” in Section III. Automatic Output Label Generation, wherein the second respective possible type of occupier is static, and the second present probability of a region being static is the second information).
Millard, Komorkiewicz, and Hoermann are all considered to be analogous to the claimed invention because they are in the same field of generating occupancy grid maps to represent dynamic and static objects. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Millard (as modified by Komorkiewicz) to incorporate the teachings of Hoermann and include “wherein the first information comprises a plurality of first indications of probability each indicative of a first present probability of a first respective possible type of occupier of a first respective one of the sub-regions, and the second information comprises a plurality of second indications of probability each indicative of a second present probability of a second respective possible type of occupier of a second respective one of the sub-regions”. The motivation for doing so would have been to allow for a “better performance for classifying static regions in the grid map”, as suggested by Hoermann in Section VII. Evaluation. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Millard and Komorkiewicz with Hoermann to obtain the invention specified in claim 15.
Regarding claim 16, Millard and Komorkiewicz teach the occupancy grid determination method of claim 11, further comprising:
determining the first information by applying the image-to-occupancy-grid transformation to third information corresponding to an image corresponding to the first sensor measurements, the first sensor comprising a camera (Millard teaches an image-to-occupancy grid transformation of third information in col. 7, line 66 through col. 8, line 2 wherein Millard teaches “the occupancy maps 216 that are provided as output from mapping engine 214 indicate current or recent locations of obstacles that were observed by the robot 202”, wherein the third information is the updated current location of the obstacle which can either be the observed or predicted location. Millard additionally teaches “the mapping engine 214 may receive the sensor data 212 that includes measurements from one or more sensors on the robot 202” in col. 6, lines 33-35, wherein the sensor may be a camera as shown in col. 6, lines 11-12) (See also Komorkiewicz wherein the second sensor (which is mapped as equivalent to the claimed first sensor in claim 11) is a camera in para. [0013]).
Millard and Komorkiewicz fail to teach determining, through machine learning, an occupancy-grid-to-image transformation; determining an image-to-occupancy-grid transformation based on the occupancy-grid-to-image transformation.
However, Hoermann (in view of Millard) teaches determining, through machine learning, an occupancy-grid-to-image transformation (Hoermann teaches a training process wherein actual data is used to determine inputting a grid map in order to output “predicted occupancy from 0.5 s to 3.0 s is illustrated in RGB images in the bottom row” in FIG. 5, wherein the transformation is a “learning based prediction” as shown in Section VII. Evaluation);
determining an image-to-occupancy-grid transformation based on the occupancy-grid-to-image transformation (Hoermann teaches inputting a grid map in order to output “predicted occupancy from 0.5 s to 3.0 s is illustrated in RGB images in the bottom row” in FIG. 5. Wherein this process is interpreted as the occupancy-grid-to-image transformation. Additionally, Millard teaches constantly performing an image-to-occupancy-grid transformation based on newly added sensor data (see claim 10). By combining Millard’s teaching of constantly updating an image-to-occupancy-grid transformation based on image data, with Hoermann’s teaching of occupancy-grid-to-image transformation, it is obvious that one of ordinary skill in the art could use Hoermann’s occupancy-grid-to-image transformation to update Millard’s teaching of an image-to-occupancy-grid transformation based on image data with the image data produced by Hoerrman in FIGS. 5 and 6).
Millard, Komorkiewicz, and Hoermann are all considered to be analogous to the claimed invention because they are in the same field of generating occupancy grid maps to represent dynamic and static objects. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Millard (as modified by Komorkiewicz) to incorporate the teachings of Hoermann and include “determining, through machine learning, an occupancy-grid-to-image transformation; determining an image-to-occupancy-grid transformation based on the occupancy-grid-to-image transformation”. The motivation for doing so would have been to allow for a “better performance for classifying static regions in the grid map”, as suggested by Hoermann in Section VII. Evaluation. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Millard and Komorkiewicz with Hoermann to obtain the invention specified in claim 16.
Regarding claim 17, Millard, Komorkiewicz, and Hoermann teach the occupancy grid determination method of claim 16,
wherein the occupancy-grid-to-image transformation maps between an occupancy grid, comprising a plurality of occupancy grid cells, and the third information, comprising a plurality of third-information regions (Hoermann an occupancy-grid-to-image transformation in FIG. 6 in which the input occupancy grid is transformed into a multitude of RGB images (see also FIG. 5); Hoermann further teaches that this transformation occurs between an input occupancy grid and third information regions wherein the third information regions are the static regions which remain the same as the original input image captured by the car sensors), and
the image-to-occupancy-grid transformation maps between the third information and the occupancy grid (see claim 16), and wherein:
(path 1) the occupancy-grid-to-image transformation maps at least two of the plurality of occupancy grid cells to a single pixel of the plurality of third-information regions; or
(path 2) the occupancy-grid-to-image transformation maps a single occupancy grid cell of the plurality of occupancy grid cells to at least two of the plurality of third-information regions; or
(path 3) the image-to-occupancy-grid transformation maps at least two of the plurality of third-information regions to a single one of the plurality of occupancy grid cells; or
(path 4) the image-to-occupancy-grid transformation maps a single one of the plurality of third-information regions to at least two of the plurality of occupancy grid cells (Komorkiewicz teaches “determining the entry of the occupancy map may include determining more than only one entry of the occupancy map (i.e. the entries of more than one cell of the occupancy map)” in para. [0012], wherein the third information region is interpreted as one of the second sensor data portion which is captured by a camera and used to update the entries of the occupancy map as shown in para. [0073-0075] and FIG. 4); or
(path 5) a combination of two or more thereof;
whereby there is a non-uniform mapping between the occupancy grid and the third information (Komorkiewicz teaches that the entry may include determining more than only one entry of the occupancy map in para. [0012]; since the entry is determined based on a subjective size of a portion of the first and second sensor data, it is inherent that the mapping is non-uniform). Similar motivations as applied to claims 10 and 16 can be applied here to claim 17.
Note: Only one limitation need be found in the prior art due to the “or” language in the claim.
Allowable Subject Matter
Claims 19-27 are allowed. The following is a statement of reasons for the indication of allowable subject matter.
The best prior art of record is Millard, Komorkiewicz, and Hoermann. Prior art applied alone or in combination with fails to anticipate or render obvious claims 19-27.
Claim 19
Regarding claim 19, Millard teaches an apparatus comprising:
means for determining a predicted occupancy grid based on a previous occupancy grid (Millard teaches that “the occupancy prediction can be, for example, in the form of a predicted occupancy map that includes a grid of occupancy cells and that indicates, for each occupancy cell, a likelihood (e.g., a probability) that the location of the environment corresponding to the occupancy cell will be occupied by an obstacle at the future time step” in col. 14, lines 12-18, wherein “for time steps after a first future time step, predicted occupancy map from a preceding time step can be fed back to the predictive model as an input to generate a new predicted occupancy map for the next future time step” in col. 14, lines 19-22), the predicted occupancy grid comprising a plurality of first cells corresponding to sub-regions of a region, each of the plurality of first cells including a plurality of predicted indications of probability each indicative of a predicted probability of a respective possible type of occupier of the respective first cell (Millard teaches “a grid of occupancy cells and that indicates, for each occupancy cell, a likelihood (e.g., a probability) that the location of the environment corresponding to the occupancy cell will be occupied by an obstacle at the future time step” in col. 14, lines 12-18; here, the probability is based on a prediction of whether the obstacle is dynamic or static, and the prediction of whether the obstacle is dynamic or static is interpreted as the possible type of occupier which is inherently tied to the probability value indicated by the prediction model. Since Millard recites “for each cell”, it is inherent that this process occurs for a first cell, wherein the cells each represent sub-regions of a region (see col. 7, lines 23-32));
means for determining, (Millard teaches “the occupancy maps 216 that are provided as output from mapping engine 214 indicate current or recent locations of obstacles that were observed by the robot 202” in col. 7, line 66 through col. 8, line 2, wherein “the map 300 includes a two-dimensional grid of cells arranged in a collection of rows and columns” as shown in col. 7, lines 33-36; see also FIG. 3, wherein the cells of the grid are interpreted as the second cells corresponding to the sub-regions of the region; Millard additionally teaches “the mapping engine 214 may receive the sensor data 212 that includes measurements from one or more sensors on the robot 202” in col. 6, lines 33-35); and
means for determining an updated occupancy grid based on the observed occupancy grid and the predicted occupancy grid (Millard teaches using an observed occupancy grid and a predicted occupancy grid to determine an updated occupancy grid in col. 14, lines 6-18, wherein “an obstacle location prediction engine provides a sequence of occupancy maps for a series of time steps to a predictive model. The predictive model processes the sequence of occupancy maps one at a time and then outputs an occupancy prediction for a future time step”. Millard additionally teaches “the predictive model 222 includes recurrent layers that maintain a hidden state that is updated each time a new occupancy map 216 is processed. As such, the obstacle location prediction for a given future time step can be based on not just the single most recent occupancy map 216, but is also partly a function of occupancy maps 216 from one or more preceding time steps” in col. 9, lines 36-42).
Komorkiewicz further teaches using machine learning to determine an observed occupancy grid (Kormorkiewicz teaches “camera-radar fusion for determination of an occupancy map using a neural network” in para. [0047] and FIG. 1; see para. [0046] wherein the occupancy map is a occupancy grid).
However, neither Millard, nor Kormorkiewicz, nor Hoermann, nor the combination, teaches means for determining a predicted occupancy grid based on a previous occupancy grid, the predicted occupancy grid comprising a plurality of first cells corresponding to sub-regions of a region, each of the plurality of first cells including a plurality of predicted indications of probability each indicative of a predicted probability of a respective possible type of occupier of the respective first cell; means for determining, using machine learning and based on first sensor measurements, an observed occupancy grid comprising a plurality of second cells corresponding to the sub-regions of the region; and means for determining an updated occupancy grid based on the observed occupancy grid and the predicted occupancy grid and their corresponding algorithm which can be found in the specification in para. [0058]-[0077]. See also the 112(f) section above which notes the specific algorithms associated with each respective “means for” limitation.
Claims 20-27 contain allowable subject matter by virtue of being dependent upon claim 19.
***Please note that the “means for determining a predicted occupancy grid”, “means for determining, using machine learning and based on first sensor measurements, an observed occupancy grid”, and “means for determining an updated occupancy grid” in claim 19 are being interpreted under 112(f) as a computer-implemented means-plus-function limitation, wherein the corresponding algorithm of the means for executing the following functions are being read into these limitations: “means for determining a predicted occupancy grid based on a previous occupancy grid, the predicted occupancy grid comprising a plurality of first cells corresponding to sub-regions of a region, each of the plurality of first cells including a plurality of predicted indications of probability each indicative of a predicted probability of a respective possible type of occupier of the respective first cell; means for determining, using machine learning and based on first sensor measurements, an observed occupancy grid comprising a plurality of second cells corresponding to the sub-regions of the region; and means for determining an updated occupancy grid based on the observed occupancy grid and the predicted occupancy grid”. As shown in the 112(f) section in the Final Rejection mailed on 02/20/2026, the corresponding structure for the above limitations is a processor and the corresponding algorithm in the applicant’s specification. Claiming a means for performing a specific computer-implemented function and disclosing only a general-purpose computer as its structure amounts to pure functional claiming. Aristocrat, 521 F.3d 1328 at 1333, 86 USPQ2d at 1239. In this instance, the structure corresponding to a 35 U.S.C. 112(f) claim limitation for a computer-implemented function must include the algorithm needed to transform the general purpose computer or microprocessor disclosed in the specification. See MPEP 2181(II)(B). The specific information in the specification regarding the algorithm associated with the above means-plus function limitations that makes this limitation allowable when analyzed in conjunction with the rest of the claim elements includes, but is not limited to, para. [0058] through para. [0078].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Natroshvili (U.S. Publication No. 2019/0188862 A1) teaches determining an updated occupancy grid based on a predicted occupancy grid and an observed occupancy grid.
Kang (U.S. Publication No. 2023/0161030 A1) teaches determining an updated predicted occupancy probability grid map based on a predicted occupancy map and vehicle driving information.
Yguel et al. (U.S. Publication No. 2008/0252433 A1) teaches determining an updated occupancy grid on a per-cell basis from corresponding cells of a predicted occupancy grid and an observed occupancy grid.
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to KYLA G ALLEN whose telephone number is (703)756-5315. The examiner can
normally be reached M-F 7:30am - 4:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a
USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use
the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
John Villecco can be reached on (571) 272-7319. The fax phone number for the organization where this
application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users. To
file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and
https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/Kyla Guan-Ping Tiao Allen/
Examiner, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661
Prosecution Insights