DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Responsive to communications on 03/20/2023
Claims 1-20 pending
Claims 1-20 Rejected
Priority
Responsive to application data sheet received on 03/20/2023. Application data sheet does not make claim to foreign or domestic priority. Application Data sheet accepted by the examiner.
Drawings
Responsive to the drawings received on 03/20/2023. Drawings are accepted by the examiner.
Specification
Responsive to abstract received on 03/20/2023. Examiner notes that abstract is preferably 50-150 according to MPEP 1826. Abstract received on 03/20/2023 is 151 words, but contains abbreviations that contribute to the wordcount (i.e.: “autonomous vehicle (AV)” is counted as three words). Because of this the examiner finds the abstract acceptable. Abstract is accepted by the examiner.
Specification received on 03/20/2023 is accepted by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1 , 14, and 20 recite the limitation receive one or more object detections in the scene based on the sensor data captured by a second sensor of the AV; . There is insufficient antecedent basis for this limitation in the claim. This appears to be separate sensor data then the “sensor data captured by a first sensor”
Claims 6 and 19 recites the limitation in response to a determination that the area of the group of the one or more cells that are indicative of the missing object exceeds the area threshold. There is insufficient antecedent basis for this limitation in the claim. The claim is discussing the “non-existent” objects not the missing object.
Claims 2-5 and 7-13, and 15-18 are rejected based on their dependence on the above claims.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 9-12 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. This is because claim 1 states to “identify a missing object or a non-existent object in the scene.” Therefore claim 1 requires either a missing object or a non-existent object, but does not require both. Later on, dependent claims 9-12 reference “the missing object” and “the non-existent object.” Because neither of these terms are actually required in the claim, claims 9-12 are all technically optional limitations.
Applicant may cancel the claims, amend the claims to place the claims in proper dependent form, rewrite the claims in independent form, or present a sufficient showing that the dependent claims complies with the statutory requirements.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention recites a judicial exception, an abstract idea, which has not been integrated into practical application and the claims further do not recite significantly more than the judicial exception.
Claim 1
Step 1: Is the claimed invention one of the four statutory categories? :
YES. The claim recites “a system” which is a machine.
Step 2A Prong 1, inquiry "Is the claim directed to a law of nature, a natural phenomenon or an abstract idea?":
YES. compare, for each cell in the grid map, the one or more object detections in the scene against the occupancy-space probability and the free-space probability in the scene;
This claim encompasses an individual looking at different pieces of data (object detections in the scene, occupancy space probability, and free space probability) and seeing if they align (a comparison against each other). This is an observation of different pieces of data and then an evaluation of their differences. The MPEP 2106.04(a)(2)(III) states “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions. “ Therefore this claim recites an abstract idea.
identify a missing object or a non-existent object in the scene based on the comparison of the one or more object detections against the occupancy-space probability and the free-space probability;
This claim encompasses an individual determining that there is an incongruency between the two different pieces of data. For example, if sensor 1 states that there is a high probability of occupancy, and sensor 2 says there is no object detected, the user could determine that there is a missing object in the scene. This is a user making a judgement based on pieces of data. The MPEP 2106.04(a)(2)(III) states “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions. “ Therefore this claim recites an abstract idea.
Step 2A Prong 2, Does the claim recite additional elements that integrate the judicial exception into a practical application?
A system comprising: a memory; and one or more processors coupled to the memory, the one or more processors being configured to:
This claim limitation simply states the computing components used in the above claim. The MPEP 2106.05(f)(2) states “Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more.” These are general purpose computer components and therefore do not integrate a judicial exception into a practical application or provide significantly more
receive an occupancy-space probability and a free-space probability for each cell in a grid map representing a scene, wherein the occupancy-space probability and the free- space probability are based on sensor data captured by a first sensor of an autonomous vehicle (AV) in the scene;
Receiving occupancy and free space probabilities for each cell in a grid map representing a scene is receiving data which is then used in the judicial exception. The probabilities being based on sensor data captured by an AV describes how the data is received. The MPEP 2106.05(g) gives examples of mere data gathering, with an example being “Determining the level of a biomarker in blood … (assessing or measuring data derived from an ultrasound scan, to be used in a diagnosis).” This limitation can be understood as “Determining the probabilities of occupancy of a street … ( assessing or measuring data derived from sensors, to be used in a comparison).” Because this claim limitation is Mere data gathering it does not integrate a judicial exception into a practical application.
receive one or more object detections in the scene based on the sensor data captured by a second sensor of the AV;
Receiving object detections in a scene is receiving data which is then used in the judicial exception. The data being from sensor data captured by an AV describes how the data is received. The MPEP 2106.05(g) gives examples of mere data gathering, with an example being “Determining the level of a biomarker in blood … (assessing or measuring data derived from an ultrasound scan, to be used in a diagnosis).” This limitation can be understood as “Determining object detections of a street … ( assessing or measuring data derived from sensors, to be used in a comparison).” Because this claim limitation is Mere data gathering it does not integrate a judicial exception into a practical application.
and initiate one or more remedial actions with respect to the missing object or the non- existent object in the scene.
This claim states to “initiate one or more remedial actions” after making a determination. This claim does not define the steps of the action, nor does it tie those steps to the determination
(the missing or non-existent object). The MPEP 2106.05(f)(1) states “The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words ‘apply it’” Therefore this claim limitation does not integrate a judicial exception into a practical application or provide significantly more.
Step 2B, does the claim recites additional elements that amount to significantly more than the judicial exception.
NO. As stated in Step 2A Prong 2, wherein the occupancy-space probability and the free- space probability are based on sensor data captured by a first sensor of an autonomous vehicle (AV) in the scene;
The examiner has determined the above limitations to be data gathering. The examiner will also outline that the way the data is gathered is well understood and conventional in the art. Malik_2022 states in “How Do Autonomous Vehicles Decide?” page 2 par 2: “(i) Perception: The perception component of AV collects the information from different on-board sensors (LiDAR, RADAR, and camera) and external sources (high-definition maps), extracts the relevant knowledge, and develop an understanding of the environment focusing on; Which type of objects are in the vicinity of the vehicle? How far is the next obstacle? How effective is the detection of traffic signs, road marking, curves, neighboring vehicles, pedestrians, cyclists, other objects, and so on? The perception has a direct impact on the planning and decision-making of AVs allowing them to react to the events of the environment accordingly. Therefore, the wider and more accurate environment understanding is, the better AVs make the decision” As understood in the art, it is widely understood that AV’s use sensor data to capture information towards perception of what is in the road. Therefore, this form of data gathering is also well understood and does not amount to significantly more than the judicial exception.
based on the sensor data captured by a second sensor of the AV;
The examiner has determined the above limitations to be data gathering. The examiner will also outline that the way the data is gathered is well understood and conventional in the art. Malik_2022 states in “How Do Autonomous Vehicles Decide?” page 2 par 2: “(i) Perception: The perception component of AV collects the information from different on-board sensors (LiDAR, RADAR, and camera) and external sources (high-definition maps), extracts the relevant knowledge, and develop an understanding of the environment focusing on; Which type of objects are in the vicinity of the vehicle? How far is the next obstacle? How effective is the detection of traffic signs, road marking, curves, neighboring vehicles, pedestrians, cyclists, other objects, and so on? The perception has a direct impact on the planning and decision-making of AVs allowing them to react to the events of the environment accordingly. Therefore, the wider and more accurate environment understanding is, the better AVs make the decision” As understood in the art, it is widely understood that AV’s use sensor data to capture information towards perception of what is in the road. Therefore, this form of data gathering is also well understood and does not amount to significantly more than the judicial exception.
Based on the above facts, the office concludes that claim 1 is not eligible under 35 USC 101.
Claim 2:
The system of claim 1, wherein to compare the one or more object detections in the scene against the occupancy-space probability and the free-space probability in the scene, the one or more processors are configured to: overlay the one or more object detections onto the grid map of the occupancy-space probability and the free-space probability.
Claim 2 pertains to the comparison made against the object detections in the scene and the occupancy-space probability/ the free-space probability. The limitation to “overlay” the one or more object detections onto the grid map is understood as placing the objects found onto a 2D paper or conceptual grid to determine if there are any inconsistencies between them. This overlay could also encompass some combination of mathematic representations. Both of these representations can reasonably be performed by an individual in the mind with a pen and paper. The MPEP 2106.04(a)(2)(III)(B) states “If a claim recites a limitation that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper, the limitation falls within the mental processes grouping, and the claim recites an abstract idea.” Therefore this claim limitation is a further recitation of an abstract idea.
Claim 3:
The system of claim 1, wherein to compare the occupancy-space probability and the free-space probability with the one or more object detections, the one or more processors are configured to:
to determine that the cell indicates the missing object in response to a determination that the occupancy-space probability exceeds a threshold in a cell that has no object detection in the scene.
This claim limitation pertains to the identification of the missing object in the scene based on the object detections and probability threshold comparisons. The claim states that if the cell has a high occupancy probability, but no objects detected (exceeding a threshold in a cell that has no object detection), to define the cell as indicating a missing object. This is an observation of two pieces of data and then evaluating whether the occupancy is at a high enough threshold (ie: if occupancy is greater than or equal to 90% but no objects are detected then there is a missing object). The MPEP 2106.04(a)(2)(III) states “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions. “ Therefore this claim limitation is a further recitation of the abstract idea.
Claim 4:
The system of claim 3, wherein the one or more processors are configured to:
group one or more cells that indicate the missing object; compare an area of the group of the one or more cells that indicate the missing object to an area threshold; and in response to a determination that the area of the group of the one or more cells that are indicative of the missing object exceeds the area threshold,
This claim is performed after the abstract idea of determining which cells indicate a missing object. This claim is evaluating a grid to determine the area of the cells which indicate a missing object, and then evaluating if the area is large enough to pass some threshold (i.e.: if the area of the missing objects encompasses >10% of the total scene). The MPEP 2106.04(a)(2)(III) states “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions. “ and further the MPEP 2106.04(a)(2)(III)(B) states “If a claim recites a limitation that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper, the limitation falls within the mental processes grouping, and the claim recites an abstract idea.” Therefore because this limitation can be performed in the mind and with a pen and paper this limitation further recites an abstract idea.
initiate the one or more remedial actions with respect to the missing object.
As already stated, initiating a remedial action is determined to be a mere recitation of Apply it. This claim does not define the steps of the action, nor does it tie those steps to the determination. The MPEP 2106.05(f)(1) states “The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words ‘apply it’” Therefore this claim limitation does not integrate a judicial exception into a practical application or provide significantly more.
Claim 5:
The system of claim 1, wherein comparing the occupancy-space probability and a free- space probability with the one or more object detections comprises:
in response to a determination that the free-space probability exceeds a threshold in a cell that includes at least one object detection in the scene, determining that the cell indicates the non-existent object.
This claim limitation pertains to the identification of the non-existent object in the scene based on the object detections and probability threshold comparisons. This claim states that if the cell has a high free-space probability, but an objects is detected, to define the cell as indicating a non-existent object. This is an observation of two pieces of data and then evaluating whether the free-space is at a high enough threshold (ie: if free-space is greater than or equal to 90% but objects are detected we declare a non-existent object). The MPEP 2106.04(a)(2)(III) states “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions. “ Therefore this claim limitation is a further recitation of the abstract idea.
Claim 6:The system of claim 5, wherein the one or more processors are configured to:
group one or more cells that indicate the non-existent object; compare an area of the group of the one or more cells that indicate the non-existent object to an area threshold; and in response to a determination that the area of the group of the one or more cells that are indicative of the missing [non-existent] object exceeds the area threshold,
This claim is performed after the abstract idea of determining which cells indicate a missing object/nonexistent object. This claim evaluates a grid to determine the area of the cells which indicate a non-existent object, and then evaluating if the area is large enough to pass some threshold (i.e.: if the area of the non-existent objects encompasses >10% of the total scene). The MPEP 2106.04(a)(2)(III) states “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions. “ and further the MPEP 2106.04(a)(2)(III)(B) states “If a claim recites a limitation that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper, the limitation falls within the mental processes grouping, and the claim recites an abstract idea.” Therefore because this limitation can be performed in the mind and with a pen and paper this limitation further recites an abstract idea.
initiate the one or more remedial actions with respect to the non-existent object.
As already stated, initiating a remedial action is determined to be a mere recitation of Apply it. This claim does not define the steps of the action, nor does it tie those steps to the determination. The MPEP 2106.05(f)(1) states “The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words ‘apply it’” Therefore this claim limitation does not integrate a judicial exception into a practical application or provide significantly more.
Claim 7:
The system of claim 1, wherein the one or more remedial actions comprise: regenerating the sensor data captured by the first sensor or the second sensor of the AV in the scene.
As already stated, initiating a remedial action is determined to be a mere recitation of Apply it. This claim does not define the steps of the action, nor does it tie those steps to the determination. For example, claim 1 simply states to initiate a remedial action with respect to the missing object or non-existent object. Claim 7 does not tie in the missing object or non-existent object into the actions of the claim. Claim 7 also does not state how the determinations and comparisons made in claim 1 are integrated into the choice of regenerating sensor data. Claim 7 simply amounts to “after doing claim 1, regenerate the data.” The MPEP 2106.05(f)(1) states “The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words ‘apply it’” Therefore this claim limitation does not integrate a judicial exception into a practical application or provide significantly more.
Claims 8-12:Claim 8-12 contain limitations which could provide a practical application for the judicial exceptions. The examiner recommends to not simply state that the claim limitations pertain to “a planning,” as a planning could be interpreted as an abstract idea, but to instead ensure that the claims effectively apply the judicial exception into the real world. I.e.: the abstract idea is used to improve some actual algorithm used in the car, is used to actually move the car, or is used to actually improve the sensors of the car in some meaningful way which could apply the judicial exception into a practical application. As stated above under claim interpretation, claims 8-12 do not meaningfully limit claim 1 due to the presence of optional limitations, and therefore are directed to an abstract idea which does not contain additional elements that integrate the judicial exception into a practical application or provide significantly more.
Claim 13:The system of claim 1, wherein the one or more object detections include an object defined by a geometry of the object including at least one of a width, a depth, a height, and a footprint.
This claim limitation pertains to the mere data gathering step of claim 1. This claim states that the object detected includes some types of parameters. This claim limtiat6ion does not provide more to the data gathering of step 1, and does note integrate the judicial exception into a practical application or provide significantly more.
Claims 14-19:Claims 14-19 are effective duplicates of claims 1-6. The difference being that claims 14-19 are directed towards a process rather than a machine. Therefore claims 14-19 are rejected under the same rationales as claims 1-6 above.
Claim 20:
Claim 20 is an effective duplicate of claim 1. Claim 20 states additional limitations of
A non-transitory computer-readable medium having stored thereon instructions which, when executed by one or more processors, cause the one or more processors to:
The MPEP 2106.05(f)(2) states “Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more.”
Therefore claim 20 is rejected under the same rational as claim 1 as well as in relation to the additional limitation above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 10-11, and 14-20 are rejected under 35 U.S.C. 103 as being unpatentable over Buerkle_2020 (US 2020/0326721 A1) and further in view of Porebski_2022 (“Occupancy grid environmental modeling for automotive applications”)
Claim 1:
Buerkle_2020 makes obvious A system comprising: (par 47: “FIG. 1 shows a vehicle 100 including a safety system 200 (see also FIG. 2) in accordance with various aspects of the present disclosure.”)
a memory; and (par 31: “As used herein, “memory” is understood as a computer-readable medium in which data or information can be stored for retrieval.”)
one or more processors coupled to the memory, the one or more processors being configured to: (par 51: “Any of the processors 214, 216, 218 disclosed herein may be configured to perform certain functions in accordance with program instructions which may be stored in a memory of the one or more memories “)
receive an occupancy-space probability par 83: “ As described herein, the Safety Driving Module Input Validator 408 may compare this received object information with cell information of a dynamic occupancy grid, including, but not limited to, occupancy probability information of one or more cells, and optionally velocity information”) … par 99: “ The occupancy grid may provide the spatial occupancy probability for a position ζ, which is denoted P(ζ)”)
wherein the occupancy-space probability and par 74: “One or more processors may estimate the dynamic occupancy grid based on information from one or more sensors (e.g., LIDAR information, Radar information, camera information, ultrasound information, etc.) by using particle filters. Emerging sensor technologies such as HD Radar or LIDAR may provide both location and velocity, thereby allowing the processors to derive the occupancy grid information directly from the sensor information.”)
receive one or more object detections in the scene based on the sensor data captured by a second sensor of the AV; (par 79: “ Object identification systems may utilize information from the dynamic occupancy system and optionally from one or more other data sources (e.g., vehicle sensors such as LIDAR, radar, ultrasound, video, or any combination thereof) to identify one or more objects in a vicinity of the vehicle. “ … par 84: “ FIG. 5 discloses a safety driving module according to an aspect of the disclosure. The Safety Driving Module may include an input module 502, which may receive sensor information, which may then processed by the complex function (object extraction, tracking and fusion, i.e., perception) 504 and the monitor 506. The sensor information may be any sensor information whatsoever for detecting objects for an AV, including, but not limited to, image sensor data, LIDAR data, Radar data, ultrasound data, etc. “)
compare, for each cell in the grid map, the one or more object detections in the scene against the occupancy-space probability par 83: “The Safety Driving Module Input Validator 408 may receive information about detected objects, or information about an absence of a detected object, optionally along with object velocity information. As described herein, the Safety Driving Module Input Validator 408 may compare this received object information with cell information of a dynamic occupancy grid, including, but not limited to, occupancy probability information of one or more cells, and optionally velocity information. On the basis of these comparisons as described in greater detail herein, the Safety Driving Module Input Validator 408 may accept the detected object information, determine a false positive object detection to have occurred, determine a false negative object detection to have occurred, determine an incorrect velocity to have been detected, correct any of the previously listed detections, or any combination thereof.”
identify a missing object or a non-existent object in the scene based on the comparison of the one or more object detections against the occupancy-space probability and
(par 93-94: “IGS. 8A and 8B show a false-positive, according to an aspect of the disclosure. In FIG. 8A, and object 802 is detected on the basis of one or more cells 804 in a dynamic occupancy grid 8A. It can be seen that the location of the detected object in FIG. 8A does not correspond closely to the locations of the occupied cells 804 in FIG. 8B. The occupied cells 804 in FIG. 8B may correspond to a sensor error, sensor noise, or otherwise. In this example, a significant difference between the location of the detected object in FIG. 8A and the locations of the occupied cells in FIG. 8B is present. The one or more processors may analyze this difference and, if the one or more processors determine that the difference is outside of a predetermined range, the one or more processors may deem the detected object of FIG. 8A to represent a false positive. (Examiner note: Where a false positive is understood to be a non-existent object)
FIGS. 9A and 9B show a false-negative, according to an aspect of the disclosure. In these examples, the one or more processors (including but not limited to an IA) analyze dynamic occupancy grid data and detect that object is present within a given area. This negative detection is depicted in FIG. 9A, in which no obstacle is shown as being present. In contrast, however, FIG. 9B, on which the detection of no obstacle is based, shows a plurality of occupied cells 902. The one or more processors may analyze the lack of a detected object for an area corresponding to the plurality of occupied cells 902, and if a difference between the lack of the detected object and the plurality of occupied cells is outside of a predetermined range, the one or more processors may determine the lack of a detected obstacle to be a false negative. (Examiner note: Where a false negative is understood to be a missing object)
and initiate one or more remedial actions with respect to the missing object or the non- existent object in the scene. (par 96: “If errors in the objects are identified (e.g., false positive object detection, false negative object detection, or false velocity), the following possibilities for recovery may be available. First, in the event of a false positive, the one or more processors may remove the object representing the false positive from the list of detected objects or otherwise deactivate the object. “)
Buerkle_2020 does not expressly recite [both a] occupancy-space probability and a free-space probability
Porebeski_2022 however makes obvious [both a] occupancy-space probability and a free-space probability (Porebeski_2022 page 48 par 2: “As a solution to that problem, Foroughi et al. in [Foroughi et al., 2015] introduced two independent grid maps: one for occupancy and one for free space accumulation. This approach requires separation of cell states, but can be used to solve sensor conflicts [Foroughi et al., 2015] or to estimate additional environment parameters [Valente et al., 2018].”)
Buerkle_2020 and Porebeski_2022 are analogous art to the claimed invention because they are from the same field of endeavor called autonomous vehicle perception. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Buerkle_2020 and Porebeski_2022.
The rationale for doing so would have been to follow a teaching and motivation proposed in the prior art. Buerkle_2020 teaches the use of occupancy grids, where each cell gives the occupancy probability for each cell in the grid. As understood by one ordinarily skilled in the art, the occupancy grid probability mathematically includes the free space probability when it is understood that the probability of a free space is the complementary event to the probability of an occupied space. (i.e.: a 80% occupancy probability inherently implies a 20% free space probability). Buerkle_2020 does not teach the usage of a separate free and occupied probability. Porebeski_2022 states that the use of two grid maps with occupancy and free space separately is used to solve sensor conflicts or to estimate additional environment parameters. In order to help solve sensor conflicts (which is what the invention of Buerkle_2020 intends), one normally skilled in the art would integrate the separate free and occupancy probabilities of Porebeski_2022. Therefore, it would have been obvious to combine the sensor validation workflow with occupancy maps of Buerkle_2020 with the addition of a separate map for free spaces of Porebeski_2022 for the benefit of allowing a higher degree of solving sensor conflicts from between the different sensors of Buerkle_2020 to obtain the invention as specified in the claims.
Claim 2:
The system of claim 1, wherein to compare the one or more object detections in the scene against the occupancy-space probability and the free-space probability in the scene, the one or more processors are configured to: (see claim 1)
Buerkle_2020 makes obvious overlay the one or more object detections onto the grid map of the occupancy-space probability and the free-space probability. (par 97-98: “ In the following, the comparison between the presence or absence of an object with one or more cells of the dynamic occupancy grid will be described. Various AV safety models may utilize inputs including the states of all detected objects in an environment O. To be able to compare the object information with the occupancy grid information, the one or more processors may convert an object track into a spatial occupancy. Therefore, the one or more processors may first determine the region that is covered by the object at a given point in time (e.g., the current time). The one or more processors may determine region Ao by the state o and its covariance matrix Σ(o). The coverage function may be defined as c(ζ; o) that will determine the coverage for a position ζ and a given object o.”
Claim 3:
The system of claim 1, wherein to compare the occupancy-space probability and the free-space probability with the one or more object detections, the one or more processors are configured to: (see claim 1)
Buerkle_2020 makes obvious to determine that the cell indicates the missing object in response to a determination that the occupancy-space probability exceeds a threshold in a cell that has no object detection in the scene. (par 105: “If a false negative occurs, there is a region of high conflict K. In order to recover from these false negatives, the one or more processors may create a new object for each cell with K>tfn (Examiner note: where the conflict is greater than a threshold), where -tfn may be a user-defined parameter (for example, a parameter that permits tuning the sensitivity to False Negatives).”) Examiner note: Where conflict K is a mathematical formula based on the conflict between the occupational probability and object position for each cell in a grid. Where “for each cell” implies that a cell by cell determination was previously made that marked the cell as a false negative (missing object) cell which may be corrected.
Claim 4:
The system of claim 3, wherein the one or more processors are configured to:
Buerkle_2020 makes obvious group one or more cells that indicate the missing object; (par 94: “In contrast, however, FIG. 9B, on which the detection of no obstacle is based, shows a plurality of occupied cells 902.”)
compare an area of the group of the one or more cells that indicate the missing object to an area threshold; ( “FIGS. 9A and 9B show a false negative, according to an aspect of the disclosure. In these examples, the one or more processors (including but not limited to an IA) analyze dynamic occupancy grid data and detect that object is present within a given area. This negative detection is depicted in FIG. 9A, in which no obstacle is shown as being present. In contrast, however, FIG. 9B, on which the detection of no obstacle is based, shows a plurality of occupied cells 902. The one or more processors may analyze the lack of a detected object for an area corresponding to the plurality of occupied cells 902, and if a difference between the lack of the detected object and the plurality of occupied cells is outside of a predetermined range, the one or more processors may determine the lack of a detected obstacle to be a false negative.”
and in response to a determination that the area of the group of the one or more cells that are indicative of the missing object exceeds the area threshold, initiate the one or more remedial actions with respect to the missing object. (par 96: “If errors in the objects are identified (e.g., false positive object detection, false negative object detection, or false velocity), the following possibilities for recovery may be available … Second, in the event of a false negative, the one or more processors may recover the object position and velocity information from the dynamic occupancy cell information.”)
Claim 5:
The system of claim 1, wherein comparing the occupancy-space probability and a free- space probability with the one or more object detections comprises: (see claim 1)
Buerkle_2020 makes obvious in response to a determination that the free-space probability exceeds a threshold in a cell that includes at least one object detection in the scene, determining that the cell indicates the non-existent object.
PNG
media_image1.png
598
642
media_image1.png
Greyscale
Examiner note: Where n is the consistency (agreement) between the position information and occupancy information based on a calculation occurring in each cell. Where t is a threshold. Where if the agreement is below a threshold, the cell is considered to be a false positive (non-existent object) . Where this calculation utilizes./implies a low occupancy probability as a free space probability. See claim 1 for why the examiner views the low occupancy to make obvious high free space probability.
Claim 6:
The system of claim 5, wherein the one or more processors are configured to:
Buerkle_2020 makes obvious group one or more cells that indicate the non-existent object; (par 93: “FIGS. 8A and 8B show a false-positive, according to an aspect of the disclosure. In FIG. 8A, and object 802 is detected on the basis of one or more cells 804 in a dynamic occupancy grid 8A. It can be seen that the location of the detected object in FIG. 8A does not correspond closely to the locations of the occupied cells 804 in FIG. 8B.” )
compare an area of the group of the one or more cells that indicate the non-existent object to an par 93: “The occupied cells 804 in FIG. 8B may correspond to a sensor error, sensor noise, or otherwise. In this example, a significant difference between the location of the detected object in FIG. 8A and the locations of the occupied cells in FIG. 8B is present. The one or more processors may analyze this difference and, if the one or more processors determine that the difference is outside of a predetermined range, the one or more processors may deem the detected object of FIG. 8A to represent a false positive.” )
and in response to a determination that the area of the group of the one or more cells that are indicative of the missing [non-existent] object exceeds the par 96: “If errors in the objects are identified (e.g., false positive object detection, false negative object detection, or false velocity), the following possibilities for recovery may be available. First, in the event of a false positive, the one or more processors may remove the object representing the false positive from the list of detected objects or otherwise deactivate the object. “)
Buerkle_2020 does not expressly recite area
Buerkle_2020 in another embodiment however makes obvious area ( “FIGS. 9A and 9B show a false negative, according to an aspect of the disclosure. In these examples, the one or more processors (including but not limited to an IA) analyze dynamic occupancy grid data and detect that object is present within a given area. This negative detection is depicted in FIG. 9A, in which no obstacle is shown as being present. In contrast, however, FIG. 9B, on which the detection of no obstacle is based, shows a plurality of occupied cells 902. The one or more processors may analyze the lack of a detected object for an area corresponding to the plurality of occupied cells 902, and if a difference between the lack of the detected object and the plurality of occupied cells is outside of a predetermined range, the one or more processors may determine the lack of a detected obstacle to be a false negative.”
Buerkle_2020 is analogous art to the claimed invention, because it is in the same field of endeavor of AV perception. Before the effective filing date, it would have been obvious to combine the different embodiments of Buerkle_2020. The motivation would be to use a known technique to improve the same device in the same way. The prior art of Buerkle_2020 flags objects as false positives or false negatives. The prior art of Buerkle_2020 described how for false negatives, it compares the area corresponding to the cells if its outside a predetermined range (threshold). While the prior art of Buerkle_2020 is silent that this is done for the false positives as well, one ordinarily skilled in the art would understand that this methodology would apply directly to the base device, and could also be used to outline false positives like it is used for false negatives. Therefore, it would have been obvious to use the area comparisons using predetermined ranges of the false negatives and also apply it to the false positives to improve the devices in the same way.
Claim 10:
The system of claim 1, wherein the one or more remedial actions comprise:
Buerkle_2020 makes obvious adding at least a portion of the missing object in the scene for a planning of controlling the AV. (par 96: “ If errors in the objects are identified (e.g., false positive object detection, false negative object detection, or false velocity), the following possibilities for recovery may be available. … Second, in the event of a false negative, the one or more processors may recover the object position and velocity information from the dynamic occupancy cell information. “) … par 148: “In example 27, the occupancy verification device of any one of examples 19 to 26, wherein the trust data include an instruction to consider an object of the one or more objects at a location associated with the object vacancy data in a driving decision if a result of the comparison is outside of a predetermined range”)
Claim 11:
The system of claim 1, wherein the one or more remedial actions comprise:
Buerkle_2020 makes obvious removing at least a portion of the non-existent object in the scene for a planning of controlling the AV. (par 120: “If the results of the comparison indicate that a detected object is a false positive, the one or more processors may include a tag as an instruction to deactivate the object. In this manner, the object may be removed from one or more registers, arrays, matrices, vectors, or otherwise. The object may be deleted or ignored, such that it is not incorporated in a driving decision made by the Safety Driving Module.”)
Claims 14-19:Claims 14-19 are substantially similar to claims 1-6, except that claim 14 is “A method comprising” (Buerkle_2020 par 117: “FIG. 17 shows a method of occupancy verification according to an aspect of the disclosure”) Therefore claims 14-19 are rejected under the same rational as claims 1-6.
Claim 20:
Claim 20 is substantially similar to claim 1, except that claim 20 is “A non-transitory computer-readable medium having stored thereon instructions which, when executed by one or more processors, cause the one or more processors to” (Buerkle_2020 par 180: “In example 59, a non-transitory computer readable medium, including instructions which, if executed, cause one or more processors to perform any of the methods of examples 30 to 58.”) . Therefore claim 20 is rejected under the same rational as claim 1.
Claims 7, 9, and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Buerkle_2020, Porebski_2022, and Frazzoli_2021 (US 2021/0163021 A1).
Claim 7:
The system of claim 1, wherein the one or more remedial actions comprise:
Buerkle_2020 does not expressly recite regenerating the sensor data captured by the first sensor or the second sensor of the AV in the scene.
Frazzoli_2021 however makes obvious regenerating the sensor data captured by the first sensor or the second sensor of the AV in the scene. Frazzoli_2021 par 341: “the processor 3250 communicates with a diagnostic module to resolve the abnormal condition by performing tests or resets of the sensors 3210a-b.”
Buerkle_2020 and Frazzoli_2021 are analogous art to the claimed invention because they are from the same field of endeavor called vehicle perception. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Buerkle_2020 and Frazzoli_2021. The rationale for doing so would have been to follow a teaching and motivation proposed in the art. Both Buerkle_2020 and Frazzoli_2021 compare between two sensors to compare and outline the differences between them. When there is an issue, Frazzoli_2021 proposes to reset the sensors in order to resolve the abnormal condition. One ordinarily skilled in the art would recognize resetting the sensors would also apply to the prior art of Buerkle_2020 and would provide a reasonable expectation for success for resolving the abnormal conditions/conflicts. Therefore, it would have been obvious to combine the vehicle perception workflow of Buerkel_2020 with the regeneration of sensors of Frazzoli_2021 for the benefit of resolving abnormal conditions/conflicts to obtain the invention as specified in the claims.
Claim 9:
The system of claim 1, wherein the one or more remedial actions comprise:
Buerkle_2020 does not expressly recite simulating the scene with the missing object for a planning of controlling the AV.
Frazzoli_2021 however makes obvious simulating the scene with the missing object for a planning of controlling the AV. (Frazzoli_2021 par 337: “In an embodiment, the anomaly detector 3240 is configured to detect an abnormal condition based on a difference between the sensor data streams being produced by respective sensors 3210a-b. In some implementations, an abnormal condition is detected based on one or more samples values that are indicative of a sensor malfunction or a sensor blockage such as one caused by dirt or another substance covering a sensor 3210a-b. In some implementations, an abnormal condition is detectable based on one or more missing samples. For example, the first sensor 3210a may have produced a sample for a particular time index, but the second sensor 3210b did not produce a sample for the same time index (Examiners note: missing). … Frazzoli_2021 par 948: “Item 202. A method performed by an autonomous vehicle (AV), the method comprising: performing, by a first simulator, a first simulation of a first AV process/system using data output by a second AV process/system; performing, by a second simulator, a second simulation of the second AV process/system using data output by the first AV process/system; comparing, by one or more processors, the data output by the first and second process/system with data output by the first and second simulators; and in accordance with a result of the comparing, causing the AV to perform a safe mode maneuver or other action.) Examiner note: Where the above makes obvious simulating two different processes for controlling an AV, where one simulation would be with the missing object based on the sensors used.
As stated previously, Buerkle_2020 and Frazzoli_2021 are analogous art to the claimed invention because they are from the same field of endeavor called vehicle perception. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Buerkle_2020 and Frazzoli_2021. The rationale for doing so would have been to follow a teaching and motivation proposed in the art. Both Buerkle_2020 and Frazzoli_2021 compare between two sensors to compare and outline the differences between them. Frazzoli_2021 simulates a comparison between two outputs of the systems in order to determine safe maneuvers or actions in the scenario. In order to determine the safe course of action, the user of Buerkle_2020 would be motivated to simulate the multiple different sensor perceptions. Therefore, it would have been obvious to combine the vehicle perception workflow of Buerkel_2020 with the simulation of worldviews of Frazzoli_2021 for the benefit of choosing a safe trajectory for the AV to obtain the invention as specified in the claims.
Claim 12:
The system of claim 1, wherein the one or more remedial actions comprise:
Buerkle_2020 does not expressly recite marking the non-existent object in the scene as uncertain for a planning of controlling the AV.
Frazzoli_2021 however makes obvious marking the non-existent object in the scene as uncertain for a planning of controlling the AV. (par 540: “The processor 6810 compares and fuses the independent outputs from the perception components 6802 and 6803 to produce a unionized model of the operating environment 6814. In one example, each perception output from a perception component is associated with a confidence score indicating the probability that the output is accurate. The perception component generates a confidence score based on factors that can affect the accuracy of the associated data, e.g., data generated during a rainstorm may have a lower confidence score than data generated during clear weather. The degree of unionization is based on the confidence scores and the desired level of caution for the unionization. For example, if false positives are much preferred to false negatives, a detected object with a low confidence score will still be added to a detected free space with a high confidence score.”) Examiner note: Where a lower confidence score for a detected object is understood to be “uncertain”
As stated previously, Buerkle_2020 and Frazzoli_2021 are analogous art to the claimed invention because they are from the same field of endeavor called vehicle perception. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Buerkle_2020 and Frazzoli_2021. The rationale for doing so would have been to follow a teaching and motivation proposed in the art. Both Buerkle_2020 and Frazzoli_2021 compare between two sensors to compare and outline the differences between them. Frazzoli_2021 incorporates confidence scores, which gives flexibility in different situations, for example, data in a rainstorm has lower confidence than clear weather. Therefore, it would have been obvious to combine the vehicle perception workflow of Buerkel_2020 with the confidence values /uncertainty of Frazzoli_2021 for the benefit of being able to modify the degree of false positives/negatives as outlined by Buerkle_2020 to improve safety in the vehicles navigation.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Buerkle_2020, Porebski_2022, and Wang_2020 (US 11,673,581 B2)
Claim 8:
The system of claim 1, wherein the one or more remedial actions comprise:
Buerkle_2020 makes obvious marking an area that consists of one or more cells that are indicative of the missing object (par 94: “The one or more processors may analyze the lack of a detected object for an area corresponding to the plurality of occupied cells 902, and if a difference between the lack of the detected object and the plurality of occupied cells is outside of a predetermined range, the one or more processors may determine the lack of a detected obstacle to be a false negative.”)
Buerkle_2020 does not expressly recite to avoid during a planning of controlling the AV.
Wang_2020 however makes obvious to avoid during a planning of controlling the AV. (Wang_2020 par 70: “or example, when a plurality of cells that are clustered together have a probability that meets the aforementioned predetermined threshold, the computing devices 110 may publish information such as “a puddle of size X was detected at location Y with probability Z”. The occupancy grid and/or the aforementioned published information may be used as input to vehicle 100's motion control systems (deceleration system 160, acceleration system 162, steering system 164), planning system 168, and perception system 174. This may enable the vehicle 100 to avoid puddles and/or reduce the effects of the puddles on the control of the vehicle. “)
Buerkle_2020 and Wang_2020 are analogous art to the claimed invention because they are from the same field of endeavor called vehicle perception. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Buerkle_2020 and Wang_2020. The rationale for doing so would be a motivation proposed in the prior art. Both Buerkle_2020 and Wang_2020 utilize occupancy grid maps to detect objects (or more specifically puddles in Wang_2020). Wang_2020 teaches to be able to avoid the detected objects. While the prior art of Buerkle_2020 does not expressly recite avoiding the detected objects, one ordinarily skilled in the art would recognize that the prior art of Buerkle_2020 could be used in the exact same way, where if a missing object is detected, that the user of Buerkle_2020 would necessarily want to avoid that object/potential crash. See Buerkle_2020 par 72: “Safety Driving Modules can evaluate potential conflicts among these cells and by that means avoid dangerous situations” and par 61: “If you can avoid an accident without causing another one, you must do it.” Therefore, it would have been obvious to combine the vehicle perception workflow of Buerkle_2020 with the object avoidance of Wang_2020 for the benefit of avoiding obstacles to improve safety and obtain the invention as specified in the claims.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Buerkle_2020, Porebski_2022, and Erkent_2021 (“GridTrack: Detection and Tracking of Multiple Objects in Dynamic Occupancy Grids”)
Claim 13:
The system of claim 1, wherein the one or more object detections include
Buerkle_2020 does not expressly recite an object defined by a geometry of the object including at least one of a width, a depth, a height, and a footprint.
Erkent_2021 however makes obvious an object defined by a geometry of the object including at least one of a width, a depth, a height, and a footprint. (Erkent_2021 page 6 par 3: “ We use two different off-the-shelf 3D object detectors for estimating the 3D poses of the objects in the grid. Both of them use point clouds as input ([13], [26]). We require the following outputs from the detector: the class type of the detected object, its center location on the grid, dimension (width and length) and orientation. It should be reminded that the detector can use any sensor type as long as it provides the required outputs. Furthermore, we use it as an independent module in our framework so that it can be interchanged if necessary, such as to detect another object class, or interchange it with another sensor modality in another data domain.”
Buerkle_2020 and Erkent_2021 are analogous art to the claimed invention because they are from the same field of endeavor called vehicle perception. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Buerkle_2020 and Erkent_2021. The rationale for doing so would have been a combination of prior art elements to yield predictable results. Buerkle_2020 states par 84: “The Safety Driving Module may include an input module 502, which may receive sensor information, which may then be processed by the complex function (object extraction, tracking and fusion, i.e., perception) 504 and the monitor 506. The sensor information may be any sensor information whatsoever for detecting objects for an AV, including, but not limited to, image sensor data, LIDAR data, Radar data, ultrasound data, etc. “). Erkent_2021 teaches a system which combines a dynamic occupancy map with an object detector. Erkent_2021 in their experiment page 9 states they their system “has RGB camera, LIDAR and corresponding calibration parameters for the sensors and GPS data for the vehicle localization.” Therefore, the sensors of Erkent_2021 and Buerkle_2020 encompass the same type of sensors. One reasonably skilled in the art would then understand that the sensors of Beurkle_2020, although not explicitly stated, is capable of being used to receive at least a width of the detected objects. One ordinarily skilled in the art would recognize that they could combine the sensors of Buerkle_2020 with the receiving of at least width information of the detected objects. Therefore, it would have been obvious to combine the workflow and camera sensors of Burkle_2020 with width detection of Erkent_2021 for the recognized predictable combination of using the known object detectors to detect at least a width to obtain the invention as specified in the claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHMAD HUSSAM SHALABY whose telephone number is (571)272-7414. The examiner can normally be reached Mon-Fri 7:30am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emerson Puente can be reached at 5712723652. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.H.S./Examiner, Art Unit 2187
/BRIAN S COOK/Primary Examiner, Art Unit 2187
Prosecution Insights