DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the amendment filed on Apr. 9th, 2026. The amendments are linked to the original application filed on Mar. 30th, 2023.
Priority
Acknowledgment is made of applicant's claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. EP22166079.8, filed on Mar. 31st, 2022.
Response to Amendment
The Examiner thanks the Applicant for the remarks, edits and arguments.
Regarding Specification Objection – Title Objection
The Applicant has made amendments to the title of the invention and believes that the title is now more descriptive and reflects the core elements of the invention. The Applicant requests the withdrawal of the title objection. The Examiner has evaluated the title and also believes it is clear and more descriptive of the invention itself. Therefore, the Examiner has withdrawn the title objection.
Regarding Specification Objection – Drawing Objection
The Applicant has made amendments to the specification and drawings to ensure correct figure numbers and corresponding information in the specification of the figures. The Applicant requests the withdrawal of the drawing objection. The Examiner has reviewed the amendments of the drawings and specification and believes the drawings are correctly labeled corresponding information from the specification. Therefore, the Examiner has withdrawn the drawing objection.
Regarding Claim Rejections – 35 U.S.C. 101
The Applicant has amended the claims to further recite a technical improvement of the invention to overcome a 101 rejection. The Applicant believes the current claims current claims do not recite abstract ideas or mental process because the limitations are implemented on a computing system which is able to evaluate the environment around itself using different sensors to predict agents’ movements in that environment concurrently. Further the Applicant believes, even if the claims recite abstract concepts or mental process, the claims use the additional limitations to integrate the subject matter into a practical application. Finally, the Applicant believes, even if the claims recite an abstract idea or mental process, the current amended recite significantly more that than abstract ideas. After each amendment the Examiner must reapply the Alice/Mayo test using the broadest reasonable interpretation of the claims in light of the specification. After reviewing the claims, the Examiner has noted several claim limitation that can be considered abstract ideas or mental process. However, in Step 2A, Prong 2, of the Alice/Mayo test the Examiners believes that the claims sufficiently recite the technical improvements stated in the specification. The Applicant has added the new subject matter and subject matter from claims 5 and 6 to the independent claims which, the Examiner believes, recites sufficient technical improvements which are stated in the specification. The Examiner would like to note that the claims recite the technical improvement stated in the specification, however this does not relate to the invention being novel or not. Therefore, the Examiner believes the current claims comply with 35 USC 101 and this rejection has been withdrawn.
Regarding Claim Rejections – 35 U.S.C. 103
The Applicant argues that the claims have been amended and the proposed arts fail to teach or disclose the amended elements of the independent claims. In particular the Applicant argues that the Usman and Eggert fail to disclose, “calculating an importance score for each agent of the at least one other agent based on the acquired sensor data and the acquired information on position and motion, wherein the importance score is a quantitative physical value describing a relevance of the particular agent for the operated agent based on the acquired sensor data and the acquired information on position and motion of the agent" and "calculating the importance score comprises applying a risk shadowing process to the data on the at least one other agent and on the physical structures in the environment, and the risk shadowing process comprises performing a reachability analysis based on the sensor data and the position and motion information for determining occupied areas for the agent and the at least one other agent in the environment over time, determining overlapping areas from the occupied areas of the agent and the at least one other agent, and determining a relevance of each of the at least one other agent based on the determined overlapping areas" (Emphasis added). These limitations originated from claims 5 and 6 and the Applicant believes Eggert fails to teach or disclose these elements. After further review of the amendments to the claims, Remarks, and the specification the Examiner has found the applicants argument on Eggert persuasive. The added subject matter, the Examiner believes, overcomes Eggert and therefore the Examiner no longer relies on Eggert to disclose the elements of the current amended claims. After each amendment the Examiner is required to perform a complete and thorough search of the arts to ensure the current amended claims comply with 35 U.S.C. 102/103. While performing this search the Examiner has discovered new art. The new art, Ding et al, the Examiner believes, teaches the calculation and utilization of importance scores for each agent in an environment. Further, the Examiner has discovered art that is able to teach or disclose partitioning of importance scores, risk shadowing, and detecting overlapping paths or trajectories of other agents in a roadway environment containing other agents from gathered sensor information. Because of this, the Examiner believes that the combination of previously proposed art, Usman, and the newly discovered art teaches or discloses the amened claims. Therefore, the Examiner has upheld the rejection under 35 U.S.C. 103, see 103 rejection below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, and 7-15 are rejected under 35 U.S.C. 103 as being unpatentable over Usman et al., (Usman et al., "SYSTEMS AND METHODS FOR PREDICTING AGENT TRAJECTORY", US 2021/0304018 A1, Filed 2020, hereinafter "Usman") in view of Ding et al, (Ding et al, “AGENT PRIORITIZATION FOR AUTONOMOUS VEHICLES”, US 2020/0159215 A1, Filed 2018, hereinafter “Ding”).
Regarding claim 1, Usman discloses, “A computer-implemented method for assisting operation of an agent operating in a dynamic environment involving at least one other agent in the environment of the agent, wherein the method comprises:” (Detailed Description, pp. 9, [0062]; "FIG. 6B illustrates an example method 620, according to an embodiment of the present technology. At block 622, a first observed position of an agent at a first time can be determined from sensor data captured. At block 624, a plurality of trajectory prediction algorithms can be applied to the agent at the first observed position to predict respective positions of the agent at a second time as predicted by the plurality of trajectory prediction algorithms. At block 626, a second observed position of the agent at the second time can be determined from the sensor data captured. At block 628, the second observed position and the respective positions can be compared to generate a plurality of performance metrics." This application discloses a method for operation of an autonomous or semi- autonomous vehicle. This system uses sensors and vehicle data to evaluate the environment and roadway. This system provided assistance to the driver in avoiding obstacles and general operation of the vehicle.)
“acquiring sensor data from at least one sensor physically sensing the environment, including information on the at least one other agent and on physical structures in the environment;” (Detailed Description, pp. 3, [0027]; “In this example, the vehicle 154 can detect the presence of an incoming vehicle 156, an outgoing vehicle 164, and a bicyclist 160. The vehicle 154 can then determine various attributes and values of the attributes for the incoming vehicle 156 based on sensor data captured by sensors of the vehicle 154 and, optionally, based on semantic map data available to the vehicle 154 and localization data of the vehicle 154.” This system will acquire data from sensors on the vehicle to evaluate the environment around the vehicle. As see in figure 1B, the vehicle is able to acquire information about other vehicles, bicyclists and other physical objects in the real world.)
“acquiring, from at least one ego-sensor, data including information on position and motion of the agent, wherein the information on position and motion includes information on a position, a heading, a velocity and an acceleration or deceleration of the agent;” (Detailed Description, pp. 4, [0031]; “In general, a vehicle can process sensor data to determine various information about itself and its surroundings. For example, the vehicle may determine its position, or "pose", based on the sensor data. In another example, the vehicle may determine its velocity and acceleration, a direction of one or more wheels of the vehicle, a lane in which the vehicle is located, among other types of information. Additionally, the vehicle may determine various agents present in its surrounding environment and their respective motion information (e.g., position, direction, velocity, acceleration, etc.) based on the sensor data.” This system is able to evaluate the environment around the user’s vehicle as well as the current vehicles motion data. It can evaluate other agents or vehicles in the roadway and determine their velocity and heading as well.)
“generating a prediction result by predicting a behavior of the at least one agent included in the at least one partition based on the sensor data and the position and motion information using a selected behavior prediction model, wherein the selected behavior prediction model is selected from a plurality of behavior prediction models based on the importance score of the at least one agent included in the at least one partition; and” (Detailed Description, pp. 5, [0035]; "The algorithm selection module 210 can be configured to select a trajectory prediction algorithm, from a plurality of trajectory prediction algorithms, to predict a trajectory for an agent. The selection of a trajectory prediction algorithm can be based on a context associated with the agent as indicated by the agent's attribute values." This model will use data from the environment and the current system to predict behaviors of other agents. This system uses multiple different trajectory prediction algorithms and one must be selected. This system will rank the agents on a list as stated earlier and this list is used to determine a priority score, which is also considered when using the trajectory prediction algorithm.)
“outputting the prediction result to a planning and control system for operating the agent or for generating and outputting an information signal or warning signal via a human machine interface.” (Detailed Description, pp. 3, [0026]; “In this example, the vehicle may select the first trajectory prediction to predict the object's trajectory when time is a factor and the second trajectory prediction to predict the object's trajectory when time is not a factor. As a result, the improved approach permits the vehicle to more efficiently and accurately predict agent trajectories and navigate environments more effectively and safely.” This system is able to generate and select appropriate trajectory predictions for a vehicle to avoid collisions. This will use the selected trajectory prediction model to output a prediction result to the system which allows the agent to navigate the environment.) and (Detailed Description, pp. 4, [0027]; “By permitting the vehicle 154 to intelligently select and apply different types of trajectory prediction algorithms based on detected agents and their associated attribute values, the improved approach helps produce more accurate trajectory predictions for the agents. Further, by permitting the vehicle 154 to also select between different trajectory prediction algorithms based on their own algorithm-specific properties (e.g., computation time, prediction reliability, prediction accuracy, etc.), the improved approach provides an improvement in the functioning of computing systems that navigate the vehicle. More details discussing the present technology are provided below.” This system uses the predictions to help the control system, or the user, to navigate the environment around the agent appropriately.)
Usman fails to explicitly disclose the remaining elements of this claim. However, Ding discloses, “calculating an importance score for each agent of the at least one other agent based on the acquired sensor data and the acquired information on position and motion, wherein the importance score is a quantitative physical value describing a relevance of the particular agent for the operated agent based on the acquired sensor data and the acquired information on position and motion of the agent, and” (Detailed Description, pp. 5, [0058]; “The importance scoring system 114 is configured to process sensor data 118 (generated by the sensor system 104 of the vehicle 102) which characterizes the environment 202 in the vicinity of the vehicle 102 to generate a respective importance score 116 for each agent in the vicinity of the vehicle 102.” This system uses and calculates importance scores based on gathered sensor data from the environment. This system will generate an importance score for each vehicle in the area of the user vehicle in the given environment.) and (Detailed Description, pp. 5, [0062]; “The on-board system 100 determines which of the agents in the vicinity of the vehicle 102 to designate as high-priority agents using an importance scoring system 114. The importance scoring system 114 is configured to generate a respective importance score 116 for each agent in the vicinity of the vehicle 102 using an importance scoring model. The importance scoring model can be implemented, for example, as a neural network model, a random forest model, a support vector machine (SVM) model, or as any other type of trainable machine learning model.” The on-board system will calculate a value for the importance of a vehicle in proximity of the user’s vehicle. The process of generating an importance score includes using processes such as machine learning models to produce a quantitative result with real numerical values.)
“wherein calculating the importance score comprises applying a risk shadowing process to the data on the at least one other agent and on the physical structures in the environment, and” (Detailed Description, pp. 6, [0062]; “A feature representation 208 of an agent refers to numerical data represented in any appropriate numerical format (e.g., as a vector or a matrix) which characterizes the agent. The feature representation of an agent can include one or more of: a velocity of the agent (e.g., measured in miles per hour-mph), an acceleration of the agent ( e.g., measured in feet per second squared), a type of the agent (e.g., pedestrian, vehicle, bicyclist, and the like), a distance from the agent to the vehicle 102 (e.g. measured in feet), data indicating whether (and when) a predicted future trajectory of the agent will overlap with a future trajectory of the vehicle 102, a position of the agent relative to the vehicle 102, and data characterizing the velocity, acceleration, and heading of the agent over a previous window of time (e.g., 5 seconds).” This system will generate an importance score based on the attributes of the other vehicle and the objects in the environment. This will generate a feature representation of the agents in the environment. The feature representation as disclosed will use information acquired from sensors to evaluate other agents in the area of the user’s vehicle as well as overlapping trajectories.) and (Detailed Description, pp. 5-6, [0063]; The importance scoring system 114 processes each of the agent feature representations 208 using an importance scoring model 210 to generate a respective importance score 116 for each of the agents in the vicinity of the vehicle 102. Each of the importance scores 116 is represented as a respective numerical value.” The system produces the importance score using information from the feature representation, which is interpreted to be the “risk shadowing process”.)
“the risk shadowing process comprises performing a reachability analysis based on the sensor data and the position and motion information for determining occupied areas for the agent and the at least one other agent in the environment over time, determining overlapping areas from the occupied areas of the agent and the at least one other agent, and determining a relevance of each of the at least one other agent based on the determined overlapping areas;” (Detailed Description, pp. 6, [0072]; “The system generates a respective feature representation of each of the agents in the vicinity of the vehicle (304). The feature representation of an agent can include one or more of: a velocity of the agent (e.g., measured in miles per hour-mph), an acceleration of the agent (e.g., measured in feet per second squared), a type of the agent (e.g., pedestrian, vehicle, bicyclist, and the like), a distance from the agent to the vehicle (e.g. measured in feet), and data indicating whether a predicted future trajectory of the agent will overlap with a future trajectory of the vehicle.” This system is able to generate a feature representation of other agents in the environment. The system will evaluate the other agent’s attributes such as velocity and heading to determine if the other agent’s trajectory overlaps the user’s vehicle or path.) and (Detailed Description, pp. 7, [0074]; “The system generates a respective importance score for each agent in the vicinity of the vehicle by processing the agent feature representations using an importance scoring model (306).) and (Detailed Description, pp. 5, [0058]; “The importance scoring system 114 is configured to process sensor data 118 (generated by the sensor system 104 of the vehicle 102) which characterizes the environment 202 in the vicinity of the vehicle 102 to generate a respective importance score 116 for each agent in the vicinity of the vehicle 102.” These two citations teach that the importance score is generated using the information from the generated feature representation of the other agents in the environment. This is able to determine overlapping regions or trajectories of the user’s vehicle and future position of agents in the environment.)
“generating a list comprising the at least one other agent based on the calculated importance scores of the at least one other agent;” (Detailed Description, pp. 6, [0065]; “The prediction system 106 processes the importance scores 116 using a ranking engine 212 to identify which (if any) of the agents in the vicinity of the vehicle 102 are high-priority agents. For example, the ranking engine 212 may identify each agent with an importance score that satisfies a predetermined threshold as a high-priority agent. In a particular example, the ranking engine may identify each agent with an importance score that exceeds the predetermined threshold of 0.8 as a high-priority agent.” This system will generate an importance score of all agents in an environment. The system will then rank the agents based on the calculated importance scores. Ranking is interpreted to be ordering a set of importance scores into a ranked list or set.)
“partitioning the generated list in at least one partition based on the importance score of the at least one other agent, wherein each partition includes at least one other agent;” (Detailed Description, pp. 6, [0065]; “The prediction system 106 processes the importance scores 116 using a ranking engine 212 to identify which (if any) of the agents in the vicinity of the vehicle 102 are high-priority agents. For example, the ranking engine 212 may identify each agent with an importance score that satisfies a predetermined threshold as a high-priority agent. In a particular example, the ranking engine may identify each agent with an importance score that exceeds the predetermined threshold of 0.8 as a high-priority agent. As another example, the ranking engine 212 may identify a predetermined number of the agents with the highest importance scores as high-priority agents.” After the importance scores are generated, this system will rank the list of importance scores. The system will determine, or partition, the list into high and low priority agents based on the generated importance scores.) and (Detailed Description, pp. 6, [0066]; “After identifying the high-priority agents 214, the prediction system 106 uses a high-priority prediction model 216 to generate respective prediction data 108 for each of the high-priority agents 214” The high priority agents are partitioned into another category and are handled differently than other importance scores from the ranked list.) and (Detailed Description, pp. 6, [0067]; “The prediction system 106 can use low-priority prediction models 218 to generate respective prediction data 108 for each of the remainder of the agents which the ranking engine 212 does not identify as high-priority agents 214 (i.e., the low-priority agents 220).” The low priority agents are partitioned into another category and are handled differently than other importance scores from the ranked list.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Usman and Ding. Usman teaches a system which is able to assist or control an autonomous vehicle within the systems environment. Ding teaches a system that is able to evaluate an environment for potential collisions with other agents and evaluate their trajectories. One of ordinary skill would have motivation to combine teachings of two different systems that are able to monitor and detect other agents in their environments and determine how to avoid collisions with those agents. A person of ordinary skill in the art would have motivation be combine the teaching of these arts to solve the similar problem of predicting trajectories of objects and agents in their environments , “In order for a planning system of a vehicle to generate planning decisions which cause the vehicle to travel along a safe and comfortable trajectory, the planning system must be provided with timely and accurate prediction data (e.g., behavior prediction data) for the agents in the vicinity of the vehicle. However, generating prediction data for an agent in the vicinity of the vehicle may require substantial computational resources (e.g., memory, computing power, or both). In certain situations, a large number of other agents in the environment may be in the vicinity of the vehicle (e.g., when the vehicle is traveling along a busy highway). In these situations, the limited computational resources available on-board the vehicle may be insufficient to generate timely prediction data for all the agents in the vicinity of the vehicle. The on-board system described in this specification can concentrate the limited computational resources available on-board the vehicle (e.g., memory, computing power, or both) on generating precise prediction data for only those agents which are likely to have the biggest impact on the planning decisions generated by the planning system.” (Ding, Summary, pp. 2, [0025]-[0026]).
Regarding claim 2, Usman discloses, “the plurality of behavior prediction models comprises individual behavior prediction models which differ with respect to at least one of prediction accuracy and computational complexity.” (Detailed Description, pp. 5, [0035]; "The algorithm selection module 210 can be configured to select a trajectory prediction algorithm, from a plurality of trajectory prediction algorithms, to predict a trajectory for an agent. The selection of a trajectory prediction algorithm can be based on a context associated with the agent as indicated by the agent's attribute values. As mentioned, a one-size-fits-all algorithm may not provide accurate trajectory predictions for all agents in various contexts. For example, a trajectory prediction algorithm may be better suited for quickly predicting linear trajectories. However, some agents may require a more sophisticated trajectory prediction algorithm to predict their complex non-linear trajectories." This model uses multiple different trajectory models to predict the trajectory of another agent in an environment. Based on a given ranking by the system different trajectory algorithms can be applied to different agents. As stated above the system can evaluate an agent to determine the priority of that agent and apply different algorithms to this agent to predict its trajectory. Agents that meet certain criteria will have use a more complex trajectory algorithm when compared to lower priority agents.)
Regarding claim 3, Usman discloses, “wherein the at least one partition comprise at least a first partition including the other agents having a first range of the importance score and a second partition including the other agents having a second range of the importance score,” (Detailed Description, pp. 5, [0032]; "The agent ranking module 206 can be configured to rank (or filter) agents (e.g., agents identified from the sensor data module 204) based on various ranking criteria to generate a ranked list of agents. The ranked list can be used to prioritize trajectory determination for agents ranked higher in the ranked list." The system will rank the agents around the current vehicle. This will generate a list of agents and rank them based on different factors. This system can also filter the agent based on certain threshold criteria. This article mentions that some agents can have a higher rank and a lower rank.)
“the first range includes importance scores, which are smaller than the importance scores included in the second range, and” (Detailed Description, pp. 5, [0032]; "Ranking criteria can be based on relevance, such as importance or criticality, of an identified agent. For example, if an agent is within a threshold distance of a vehicle, that agent may be determined to be of higher relevance to the vehicle than an agent that is not within the threshold distance of the vehicle. In some embodiments, agents can be ranked based on various attributes associated with the agents. For example, an agent that has threshold likelihood of interacting with the vehicle can be ranked higher than an agent that does not satisfy the threshold likelihood of interacting with the vehicle. In another example, an agent that is traveling in an opposite direction of the vehicle can be ranked lower than agents traveling in the same direction as the vehicle. By ranking agents for prioritizing trajectory determination, the agent ranking module 206 permits efficient allocation of computational resources to agents that are more likely to impact operation of the vehicle while accounting for any time-based constraints for completing agent trajectory predictions." This system will rank the agents according to different factors. The agents that have the higher score have a higher chance of collision or proximity to the systems vehicle, therefore that vehicle will have a higher score when compared to a vehicle that is not travelling in the same direction and wont intercept the systems vehicle.)
“the method comprises selecting the behavior prediction model for the first partition with a lower computational complexity than the behavior prediction model for the second partition.” (Detailed Description, pp. 3, [0026]; "In some embodiments, these algorithm properties can also be considered by the vehicle when selecting a trajectory prediction Page 45 algorithm to predict an agent's trajectory. For example, a first trajectory prediction algorithm may require a lesser amount of time to compute a trajectory for a given object than a second trajectory prediction algorithm. However, the second trajectory prediction algorithm may more accurately predict a trajectory for the object than the first trajectory prediction algorithm. In this example, the vehicle may select the first trajectory prediction to predict the object's trajectory when time is a factor and the second trajectory prediction to predict the object's trajectory when time is not a factor." This system uses an algorithm to determine which trajectory algorithm to use. Some of the trajectory algorithms are more complex than others and require more resources. Therefore, this system will only use the computationally complex algorithms on high priority, or higher ranked vehicles, and lower-level complexity algorithms on lower priority, or ranked, vehicles.)
Regarding claim 4, Usman discloses, “the method comprises discarding other agents with a calculated importance score below a threshold when generating the list.” (Detailed Description, pp. 7, [0051]; "At block 454, optionally, the vehicle can filter the identified agents based on various criteria. For example, if an agent is a threshold distance away from the vehicle (e.g.,> or =100 m, etc.), then the agent may be considered not sufficiently relevant and thus a trajectory need not be determined for the agent." When analyzing the agents around a vehicle the system can disregard certain objects or actions that do not meet a threshold. This is designed so that the system does not need waster time and energy on non-essential or critical agents.)
Regarding claim 7, Usman discloses, determining the relevance of each of the at least one other agent includes disregarding the at least one other agent in case the occupied areas of the agent and the at least one other agent do not exist, or” (Detailed Description, pp. 7, [0051]; "At block 454, optionally, the vehicle can filter the identified agents based on various criteria. For example, if an agent is a threshold distance away from the vehicle (e.g.,> or =100 m, etc.), then the agent may be considered not sufficiently relevant and thus a trajectory need not be determined for the agent." The system proposed will evaluate different agents and filter out agents that do not meet a threshold. This is used to save on computations. The system will disregard agents based on different attributes such as distance.)
“computing a continuous relevance score for the at least one other agent by calculating a size of the overlapping area or a distance of the occupied areas.” (Detailed Description, pp. 9, [0062]; "FIG. 6B illustrates an example method 620, according to an embodiment of the present technology. At block 622, a first observed position of an agent at a first time can be determined from sensor data captured. At block 624, a plurality of trajectory prediction algorithms can be applied to the agent at the first observed position to predict respective positions of the agent at a second time as predicted by the plurality of trajectory prediction algorithms. At block 626, a second observed position of the agent at the second time can be determined from the sensor data captured. At block 628, the second observed position and the respective positions can be compared to generate a plurality of performance metrics." This figure discloses two different sets of data on a single agent over a period of time. This teaches that this system will regularly repeat evaluation of agents over time and will alter the metrics based on the updated sensor data after a period of time.)
Regarding claim 8, Usman discloses, “calculating the importance score comprises decreasing an influence of a first agent included in the at least one other agent based on an influence of at least one second agent of the at least one other agent.” (Detailed Description, pp. 5, [0033]; "The agent attributes module 208 can be configured to determine attributes and values of the attributes for agents detected in an environment. Each of the identified agents can be associated with various attributes and values for the attributes. The agent attributes module 208 can determine the values of the attributes based on various information acquired from various sources. For example, the agent attributes module 208 can determine attribute values by processing sensor data obtained by the sensor data module 204 and semantic map data for determining semantic locations of agents in an environment.") and (Detailed Description, pp. 5, [0037]; "FIG. 3 illustrates example attributes 300 that can be associated with an agent, according to an embodiment of the present technology. Each attribute may include an attribute description 302, an associated value 304, one or more sources 306 from which the attribute value is determined, and/or a priority associated with the attribute 308. As shown, one example attribute 310 can indicate whether an agent should yield. Another example attribute 312 can indicate an agent's direction relative to "ego" (or a vehicle that is predicting a trajectory of the agent). The "Direction relative to Ego" attribute 312 may be associated with a value, which can be one of the values in an enumeration of "Same", "Opposite", or "Crossing." Further, the "Direction relative to Ego" attribute 312 value may be determined from sources including perception data ("pep") describing surroundings of the ego vehicle, position data ("pos") describing positions of the agent and the ego vehicle, and map data ("map") describing geographic and semantic locations of the agent and ego vehicle." This model will consider the attributes of another agent in the sensor area. These attributes influence the ranking of the agent and can increase/decrease the risk of that agent depending on the agents predicted trajectory and other attributes and agent characteristics.)
Regarding claim 9, Usman discloses, “calculating the importance score includes applying a risk model that regards an influence of the at least one other agent on the agent for computing the importance score of the at least one other agent.” (Detailed Description, pp. 5, [0034]; "The agent attributes module 208 can be configured to determine attributes and values of the attributes for agents detected in an environment. Each of the identified agents can be associated with various attributes and values for the attributes. The agent attributes module 208 can determine the values of the attributes based on various information acquired from various sources. For example, the agent attributes module 208 can determine attribute values by processing sensor data obtained by the sensor data module 204 and semantic map data for determining semantic locations of agents in an environment.") and (Detailed Description, pp. 5, [0032]; "The agent ranking module 206 can be configured to rank (or filter) agents (e.g., agents identified from the sensor data module 204) based on various ranking criteria to generate a ranked list of agents. The ranked list can be used to prioritize trajectory determination for agents ranked higher in the ranked list. Ranking criteria can be based on relevance, such as importance or criticality, of an identified agent." This system uses a ranking module to evaluate the different agents based on ranking criteria. This ranking can use the different attributes assigned to agents by the attribute module. This system will also use sensor, motion and positional data to evaluate the risk of another agent.)
Regarding claim 10, Usman discloses, “the method comprises calculating the importance score as a first importance score for each of a plurality of other agents by applying a first method, discarding a subset of the plurality other agent based on a first filter criterion for the calculated first importance score for generating a first filtered list of the other agents,” (Detailed Description, pp. 8, [0058]-[0059]; "In some embodiments, clusters can be determined for similar contexts based on a clustering technique. Any suitable conventional clustering technique can be used. Once clusters are determined, various trajectory prediction algorithms can be applied to the clusters to determine confidence scores associated with the trajectory prediction algorithms with respect to the clusters. Each cluster can be associated with a trajectory prediction algorithm that best predicts trajectories for contexts represented by the cluster. [0059] In the example diagram 500, three clusters are associated with their own trajectory prediction algorithms. A cluster 504 of vehicles with a velocity that falls within a first pre-defined range is associated with algorithm C because algorithm C predicts trajectories for similar contexts in the cluster 504 better than algorithm A and algorithm B (e.g., performance metrics of the algorithm Care superior to the corresponding performance metrics of algorithms A and B)." The system proposed in Usman is able to group and cluster agents accordingly. These different clusters have different risks associated and are ranked by ranking module. The system is able to determine, based on many factors, which trajectory algorithm is to be used. Some of the algorithms are computationally complex and take a disclosed amount of time to complete. Further, this system is able to identify agents that pose little to no risk and are not evaluated by the trajectory algorithms.)
“calculating the importance score as a second importance score by applying a second method for the plurality of other agents, discarding a second subset of the plurality of other agents included in the generated list based on a second filter criterion for the calculated second importance score for generating a second filtered list of the plurality of other agents.” (Detailed Description, pp. 8, [0059]; "A cluster 506 of vehicles with a velocity that falls within a second pre-defined range is associated with the algorithm A, because algorithm A predicts trajectories for contexts represented by the cluster 506 better than algorithm B or algorithm C. A cluster 508 of pedestrians with a velocity that falls within a third pre-defined range is associated with algorithm B, because algorithm B predicts trajectories for the contexts represented by the cluster 508 better than algorithm A or algorithm C. In some cases, it can be determined that none of the trajectory prediction algorithms accurately predict trajectories for some contexts 510, such as a context 502d. The outlier contexts 510 can indicate a need to tune an existing trajectory prediction algorithm or, in some cases, introduce a new trajectory prediction algorithm to more accurately predict trajectories for the outlier contexts 510." This system uses different trajectory algorithms to predict agents' movements. This system can cluster agents and a determined trajectory algorithm might be applied to a cluster based on algorithm selection module. Depending on many factors, a more complex algorithm might be applied to one agent in a cluster. Finally, agents may move from cluster to cluster based on updated information, causing the agents to be evaluated by a different less complex algorithm.)
Regarding claim 11, Usman discloses, “the first method for calculating the first importance score has a lower computational complexity than the second method for calculating the second importance score.” (Detailed Description, pp. 5, [0035]; "The algorithm selection module 210 can be configured to select a trajectory prediction algorithm, from a plurality of trajectory prediction algorithms, to predict a trajectory for an agent. The selection of a trajectory prediction algorithm can be based on a context associated with the agent as indicated by the agent's attribute values. As mentioned, a one-size-fits-all algorithm may not provide accurate trajectory predictions for all agents in various contexts. For example, a trajectory prediction algorithm may be better suited for quickly predicting linear trajectories. However, some agents may require a more sophisticated trajectory prediction algorithm to predict their complex non-linear trajectories. For example, a vehicle blinking a right turn signal and decelerating while at a rightmost lane is more likely to make a right turn at an intersection than another vehicle blinking a right turn signal at a leftmost lane and maintaining velocity. Furthermore, some trajectory prediction algorithms may better predict trajectories for certain types of agents with a certain set of attribute-values indicating a certain context." This model uses multiple trajectory algorithms to predict agents' movements. Some of the algorithms are more complex and computationally expense when compared to others. As stated above, a one size-fits-all model does not work in this environment, therefore they developed multiple algorithms of different complexities to perform predictions.)
Regarding claim 12, Ding discloses, “calculating the importance score uses a path distance model, a trajectory distance model, a Gaussian model, or a survival analysis model.” (Detailed Description, pp. 7, [0074]; “The importance scoring model can be implemented as any appropriate prediction model, for example, as a neural network, a random forest, or support vector machine. When the importance scoring model is implemented as a neural network, the importance scoring neural network can have any appropriate neural network architecture. For example, the importance scoring neural network may be configured to separately process each of the agent feature representations to generate respective importance scores for each of the agents. In this example, the architecture of the importance scoring neural network may be defined by a sequence of fully-connected layers followed by an output layer consisting of a single neuron. The importance score output of the importance scoring neural network may be defined as the activation of this neuron.” This system will generate an importance score using machine learning. As disclosed above, part of calculating the importance score involves processing feature representations of the other agents in the environment based on their trajectories. This also teaches that the importance score is generated using path and distance metrices as well as trajectory models to produce trajectories of other agents.)
Regarding claim 13, Usman discloses, “the agent is a mobile robotic device, and the at least one other agent includes other mobile devices, in particular other mobile robotic devices, or” (Detailed description, pp. 4, [0029]" In various embodiments, the context specific prediction module 202 can be implemented by a vehicle being driven semi autonomously or autonomously. While the vehicle navigates an environment, the context specific prediction module 202 can detect various agents present in the environment based on sensor data captured by sensors of the vehicle. For example, the context-specific prediction module 202 can detect agents such as other vehicles, pedestrians, bicyclists, animals, among other objects, to name some examples. The context-specific prediction module 202 can also determine related attribute values for the detected agents." This system, as stated above, can work on fully autonomous vehicles. These systems disclosed are considered mobile autonomous devices which are able to evaluate other mobile autonomous or manual devices in an environment.)
Ding discloses, “the agent is an ego-vehicle, and the at least one other agent is at least one other traffic participant in a road traffic scenario, in particular at least one of other vehicles, cyclists and pedestrians, or” (Detailed Description, pp. 3, [0038]; “As another example, for each of the multiple agents in the vicinity of the vehicle 102, the prediction data 108 may be classification prediction data which defines respective probabilities that the agent is each of a predetermined number of possible agent types (e.g., animal, pedestrian, bicyclist, car, or truck).” The system in this article relates to a vehicle operating a real physical traffic environment. This model will evaluate a roadway using sensors. The roadway can contain other vehicles, bicyclists and pedestrians.) and (Detailed Description, pp. 3, [0039]; “The on-board system 100 can provide the prediction data 108 generated by the prediction system 106 to a planning system 110, a user interface system 112, or both. [0040] When the planning system 110 receives the prediction data 108, the planning system 110 can use the prediction data 108 to generate planning decisions 144 which plan the future trajectory of the vehicle 102. The planning decisions 144 generated by the planning system 110 can include, for example: yielding (e.g., to pedestrians), stopping (e.g., at a "Stop" sign), passing other vehicles, adjusting vehicle lane position to accommodate a bicyclist, slowing down in a school or construction zone, merging (e.g., onto a highway), and parking.” The on-board system is used to evaluate the senor data from the real environment and provide prediction information to the control system to avoid collisions with other agents in the environment.)
Both Usman and Ding fail to explicitly disclose, “the operated agent and the at least one other agent are vessels in a maritime traffic scenario, or”, “the operated agent and the at least one other agent are air vehicles in an air traffic scenario, or”, and “the operated agent and the at least one other agent are space vehicles in a space environment.” However, the way the claim has been composed, it designates each of these limitations as potential possibilities of operating environment while using the keyword "or". Under the broadest reasonable interpretation and use of the keyword "or", these limitations are optional and therefore have not been mapped to prior art.
Regarding claim 14, Usman discloses, “A computer program product comprising a non-transitory computer readable medium having a computer readable program embodied therein, wherein the computer readable program, when executed on a computing device, causes the computing device to perform a method for assisting operation of an agent operating in a dynamic environment involving at least one other agent in the environment of the agent, wherein the method comprises:” (Detailed Description, pp. 13, [0083]; "In particular embodiments, memory 804 includes main memory for storing instructions for processor 802 to execute or data for processor 802 to operate on. As an example, and not by way of limitation, computer system 800 may load instructions from storage 806 or another source (such as another computer system 800) to memory 804. Processor 802 may then load the instructions from memory 804 to an internal register or internal cache. To execute the instructions, processor 802 may retrieve the instructions from the internal register or internal cache and decode them. During or after execution of the instructions, processor 802 may write one or more results (which may be intermediate or final results) to the internal register or internal cache." The proposed system uses a computing system similar to a generic computer. This includes using processors which are linked to memory which stores machine code for the processing system. This system is designed to operate in a dynamic environment using sensors to perform different vehicle operations.)
“acquiring sensor data from at least one sensor physically sensing the environment, including information on the at least one other agent and on physical structures in the environment;” (Detailed Description, pp. 3, [0027]; “In this example, the vehicle 154 can detect the presence of an incoming vehicle 156, an outgoing vehicle 164, and a bicyclist 160. The vehicle 154 can then determine various attributes and values of the attributes for the incoming vehicle 156 based on sensor data captured by sensors of the vehicle 154 and, optionally, based on semantic map data available to the vehicle 154 and localization data of the vehicle 154.” This system will acquire data from sensors on the vehicle to evaluate the environment around the vehicle. As see in figure 1B, the vehicle is able to acquire information about other vehicles, bicyclists and other physical objects in the real world.)
“acquiring, from at least one ego-sensor, data including information on position and motion of the agent, wherein the information on position and motion includes information on a position, a heading, a velocity and an acceleration or deceleration of the agent;” (Detailed Description, pp. 4, [0031]; “In general, a vehicle can process sensor data to determine various information about itself and its surroundings. For example, the vehicle may determine its position, or "pose", based on the sensor data. In another example, the vehicle may determine its velocity and acceleration, a direction of one or more wheels of the vehicle, a lane in which the vehicle is located, among other types of information. Additionally, the vehicle may determine various agents present in its surrounding environment and their respective motion information (e.g., position, direction, velocity, acceleration, etc.) based on the sensor data.” This system is able to evaluate the environment around the user’s vehicle as well as the current vehicles motion data. It can evaluate other agents or vehicles in the roadway and determine their velocity and heading as well.)
“generating a prediction result by predicting a behavior of the at least one agent included in the at least one partition based on the sensor data and the position and motion information using a selected behavior prediction model, wherein the selected behavior prediction model is selected from a plurality of behavior prediction models based on the importance score of the at least one agent included in the at least one partition; and” (Detailed Description, pp. 5, [0035]; "The algorithm selection module 210 can be configured to select a trajectory prediction algorithm, from a plurality of trajectory prediction algorithms, to predict a trajectory for an agent. The selection of a trajectory prediction algorithm can be based on a context associated with the agent as indicated by the agent's attribute values." This model will use data from the environment and the current system to predict behaviors of other agents. This system uses multiple different trajectory prediction algorithms and one must be selected. This system will rank the agents on a list as stated earlier and this list is used to determine a priority score, which is also considered when using the trajectory prediction algorithm.)
“outputting the prediction result to a planning and control system for operating the agent or for generating and outputting an information signal or warning signal via a human machine interface.” (Detailed Description, pp. 3, [0026]; “In this example, the vehicle may select the first trajectory prediction to predict the object's trajectory when time is a factor and the second trajectory prediction to predict the object's trajectory when time is not a factor. As a result, the improved approach permits the vehicle to more efficiently and accurately predict agent trajectories and navigate environments more effectively and safely.” This system is able to generate and select appropriate trajectory predictions for a vehicle to avoid collisions. This will use the selected trajectory prediction model to output a prediction result to the system which allows the agent to navigate the environment.) and (Detailed Description, pp. 4, [0027]; “By permitting the vehicle 154 to intelligently select and apply different types of trajectory prediction algorithms based on detected agents and their associated attribute values, the improved approach helps produce more accurate trajectory predictions for the agents. Further, by permitting the vehicle 154 to also select between different trajectory prediction algorithms based on their own algorithm-specific properties (e.g., computation time, prediction reliability, prediction accuracy, etc.), the improved approach provides an improvement in the functioning of computing systems that navigate the vehicle. More details discussing the present technology are provided below.” This system uses the predictions to help the control system, or the user, to navigate the environment around the agent appropriately.)
Usman fails to explicitly disclose the remaining elements of this claim. However, Ding discloses, “calculating an importance score for each agent of the at least one other agent based on the acquired sensor data and the acquired information on position and motion, wherein the importance score is a quantitative physical value describing a relevance of the particular agent for the operated agent based on the acquired sensor data and the acquired information on position and motion of the agent, and” (Detailed Description, pp. 5, [0058]; “The importance scoring system 114 is configured to process sensor data 118 (generated by the sensor system 104 of the vehicle 102) which characterizes the environment 202 in the vicinity of the vehicle 102 to generate a respective importance score 116 for each agent in the vicinity of the vehicle 102.” This system uses and calculates importance scores based on gathered sensor data from the environment. This system will generate an importance score for each vehicle in the area of the user vehicle in the given environment.) and (Detailed Description, pp. 5, [0062]; “The on-board system 100 determines which of the agents in the vicinity of the vehicle 102 to designate as high-priority agents using an importance scoring system 114. The importance scoring system 114 is configured to generate a respective importance score 116 for each agent in the vicinity of the vehicle 102 using an importance scoring model. The importance scoring model can be implemented, for example, as a neural network model, a random forest model, a support vector machine (SVM) model, or as any other type of trainable machine learning model.” The on-board system will calculate a value for the importance of a vehicle in proximity of the user’s vehicle. The process of generating an importance score includes using processes such as machine learning models to produce a quantitative result with real numerical values.)
“wherein calculating the importance score comprises applying a risk shadowing process to the data on the at least one other agent and on the physical structures in the environment, and” (Detailed Description, pp. 6, [0062]; “A feature representation 208 of an agent refers to numerical data represented in any appropriate numerical format (e.g., as a vector or a matrix) which characterizes the agent. The feature representation of an agent can include one or more of: a velocity of the agent (e.g., measured in miles per hour-mph), an acceleration of the agent ( e.g., measured in feet per second squared), a type of the agent (e.g., pedestrian, vehicle, bicyclist, and the like), a distance from the agent to the vehicle 102 (e.g. measured in feet), data indicating whether (and when) a predicted future trajectory of the agent will overlap with a future trajectory of the vehicle 102, a position of the agent relative to the vehicle 102, and data characterizing the velocity, acceleration, and heading of the agent over a previous window of time (e.g., 5 seconds).” This system will generate an importance score based on the attributes of the other vehicle and the objects in the environment. This will generate a feature representation of the agents in the environment. The feature representation as disclosed will use information acquired from sensors to evaluate other agents in the area of the user’s vehicle as well as overlapping trajectories.) and (Detailed Description, pp. 5-6, [0063]; The importance scoring system 114 processes each of the agent feature representations 208 using an importance scoring model 210 to generate a respective importance score 116 for each of the agents in the vicinity of the vehicle 102. Each of the importance scores 116 is represented as a respective numerical value.” The system produces the importance score using information from the feature representation, which is interpreted to be the “risk shadowing process”.)
“the risk shadowing process comprises performing a reachability analysis based on the sensor data and the position and motion information for determining occupied areas for the agent and the at least one other agent in the environment over time determining overlapping areas from the occupied areas of the agent and the at least one other agent, and determining a relevance of each of the at least one other agent based on the determined overlapping areas;” (Detailed Description, pp. 6, [0072]; “The system generates a respective feature representation of each of the agents in the vicinity of the vehicle (304). The feature representation of an agent can include one or more of: a velocity of the agent (e.g., measured in miles per hour-mph), an acceleration of the agent (e.g., measured in feet per second squared), a type of the agent (e.g., pedestrian, vehicle, bicyclist, and the like), a distance from the agent to the vehicle (e.g. measured in feet), and data indicating whether a predicted future trajectory of the agent will overlap with a future trajectory of the vehicle.” This system is able to generate a feature representation of other agents in the environment. The system will evaluate the other agent’s attributes such as velocity and heading to determine if the other agent’s trajectory overlaps the user’s vehicle or path.) and (Detailed Description, pp. 7, [0074]; “The system generates a respective importance score for each agent in the vicinity of the vehicle by processing the agent feature representations using an importance scoring model (306).) and (Detailed Description, pp. 5, [0058]; “The importance scoring system 114 is configured to process sensor data 118 (generated by the sensor system 104 of the vehicle 102) which characterizes the environment 202 in the vicinity of the vehicle 102 to generate a respective importance score 116 for each agent in the vicinity of the vehicle 102.” These two citations teach that the importance score is generated using the information from the generated feature representation of the other agents in the environment. This is able to determine overlapping regions or trajectories of the user’s vehicle and future position of agents in the environment.)
“generating a list comprising the at least one other agent based on the calculated importance scores of the at least one other agent;” (Detailed Description, pp. 6, [0065]; “The prediction system 106 processes the importance scores 116 using a ranking engine 212 to identify which (if any) of the agents in the vicinity of the vehicle 102 are high-priority agents. For example, the ranking engine 212 may identify each agent with an importance score that satisfies a predetermined threshold as a high-priority agent. In a particular example, the ranking engine may identify each agent with an importance score that exceeds the predetermined threshold of 0.8 as a high-priority agent.” This system will generate an importance score of all agents in an environment. The system will then rank the agents based on the calculated importance scores. Ranking is interpreted to be ordering a set of importance scores into a ranked list or set.)
“partitioning the generated list in at least one partition based on the importance score of the at least one other agent, wherein each partition includes at least one other agents;” (Detailed Description, pp. 6, [0065]; “The prediction system 106 processes the importance scores 116 using a ranking engine 212 to identify which (if any) of the agents in the vicinity of the vehicle 102 are high-priority agents. For example, the ranking engine 212 may identify each agent with an importance score that satisfies a predetermined threshold as a high-priority agent. In a particular example, the ranking engine may identify each agent with an importance score that exceeds the predetermined threshold of 0.8 as a high-priority agent. As another example, the ranking engine 212 may identify a predetermined number of the agents with the highest importance scores as high-priority agents.” After the importance scores are generated, this system will rank the list of importance scores. The system will determine, or partition, the list into high and low priority agents based on the generated importance scores.) and (Detailed Description, pp. 6, [0066]; “After identifying the high-priority agents 214, the prediction system 106 uses a high-priority prediction model 216 to generate respective prediction data 108 for each of the high-priority agents 214” The high priority agents are partitioned into another category and are handled differently than other importance scores from the ranked list.) and (Detailed Description, pp. 6, [0067]; “The prediction system 106 can use low-priority prediction models 218 to generate respective prediction data 108 for each of the remainder of the agents which the ranking engine 212 does not identify as high-priority agents 214 (i.e., the low-priority agents 220).” The low priority agents are partitioned into another category and are handled differently than other importance scores from the ranked list.)
Regarding claim 15, Usman discloses, “A system for assisting operation of an agent operating in a dynamic scenario involving at least one other agent in an environment of the agent, the system comprises:” (Detailed Description, pp. 13, [0081]; "In particular embodiments, computer system 800 includes a processor 802, memory 804, storage 806, an input/output (1/0) interface 808, a communication interface 810, and a bus 812. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement." The proposed system uses a computing system similar to a generic computer. This includes using processors which are linked to memory which stores machine code for the processing system. This system is designed to operate in a dynamic environment using sensors to perform different vehicle operations.)
“at least one sensor configured to acquire sensor data from the environment including information on the at least one other agent and on physical structures in the environment;” (Detailed Description, pp. 3, [0027]; “In this example, the vehicle 154 can detect the presence of an incoming vehicle 156, an outgoing vehicle 164, and a bicyclist 160. The vehicle 154 can then determine various attributes and values of the attributes for the incoming vehicle 156 based on sensor data captured by sensors of the vehicle 154 and, optionally, based on semantic map data available to the vehicle 154 and localization data of the vehicle 154.” This system will acquire data from sensors on the vehicle to evaluate the environment around the vehicle. As see in figure 1B, the vehicle is able to acquire information about other vehicles, bicyclists and other physical objects in the real world.)
“at least one ego-sensor configured to acquire data including information on position and motion on the agent, wherein the information on position and motion includes information on a position, a heading, a velocity and an acceleration or deceleration of the agent; and” (Detailed Description, pp. 4, [0031]; “In general, a vehicle can process sensor data to determine various information about itself and its surroundings. For example, the vehicle may determine its position, or "pose", based on the sensor data. In another example, the vehicle may determine its velocity and acceleration, a direction of one or more wheels of the vehicle, a lane in which the vehicle is located, among other types of information. Additionally, the vehicle may determine various agents present in its surrounding environment and their respective motion information (e.g., position, direction, velocity, acceleration, etc.) based on the sensor data.” This system is able to evaluate the environment around the user’s vehicle as well as the current vehicles motion data. It can evaluate other agents or vehicles in the roadway and determine their velocity and heading as well.)
“generate a prediction result by predicting a behavior of the at least one other agent included in the at least one partition based on the sensor data and the position and motion data using a selected behavior prediction model, wherein the selected behavior prediction model is selected from a plurality of behavior prediction models based on the importance score of the at least one other agent included in the at least one partition, and” (Detailed Description, pp. 5, [0035]; "The algorithm selection module 210 can be configured to select a trajectory prediction algorithm, from a plurality of trajectory prediction algorithms, to predict a trajectory for an agent. The selection of a trajectory prediction algorithm can be based on a context associated with the agent as indicated by the agent's attribute values." This model will use data from the environment and the current system to predict behaviors of other agents. This system uses multiple different trajectory prediction algorithms and one must be selected. This system will rank the agents on a list as stated earlier and this list is used to determine a priority score, which is also considered when using the trajectory prediction algorithm.)
“to output the prediction result to a planning and control system for operating the agent or to a human machine interface for generating and outputting an information signal or warning signal.” (Detailed Description, pp. 3, [0026]; “In this example, the vehicle may select the first trajectory prediction to predict the object's trajectory when time is a factor and the second trajectory prediction to predict the object's trajectory when time is not a factor. As a result, the improved approach permits the vehicle to more efficiently and accurately predict agent trajectories and navigate environments more effectively and safely.” This system is able to generate and select appropriate trajectory predictions for a vehicle to avoid collisions. This will use the selected trajectory prediction model to output a prediction result to the system which allows the agent to navigate the environment.) and (Detailed Description, pp. 4, [0027]; “By permitting the vehicle 154 to intelligently select and apply different types of trajectory prediction algorithms based on detected agents and their associated attribute values, the improved approach helps produce more accurate trajectory predictions for the agents. Further, by permitting the vehicle 154 to also select between different trajectory prediction algorithms based on their own algorithm-specific properties (e.g., computation time, prediction reliability, prediction accuracy, etc.), the improved approach provides an improvement in the functioning of computing systems that navigate the vehicle. More details discussing the present technology are provided below.” This system uses the predictions to help the control system, or the user, to navigate the environment around the agent appropriately.)
Usman fails to explicitly disclose the remaining elements of this claim. However, Ding discloses, “at least one processor configured to: calculate an importance score for each agent of the at least one other agent based on the acquired sensor data and the acquired position and motion data, wherein the importance score is a quantitative physical value describing a relevance of the particular agent for the operated agent based on the acquired sensor data and the acquired information on position and motion of the agent, and” (Detailed Description, pp. 5, [0058]; “The importance scoring system 114 is configured to process sensor data 118 (generated by the sensor system 104 of the vehicle 102) which characterizes the environment 202 in the vicinity of the vehicle 102 to generate a respective importance score 116 for each agent in the vicinity of the vehicle 102.” This system uses and calculates importance scores based on gathered sensor data from the environment. This system will generate an importance score for each vehicle in the area of the user vehicle in the given environment.) and (Detailed Description, pp. 5, [0062]; “The on-board system 100 determines which of the agents in the vicinity of the vehicle 102 to designate as high-priority agents using an importance scoring system 114. The importance scoring system 114 is configured to generate a respective importance score 116 for each agent in the vicinity of the vehicle 102 using an importance scoring model. The importance scoring model can be implemented, for example, as a neural network model, a random forest model, a support vector machine (SVM) model, or as any other type of trainable machine learning model.” The on-board system will calculate a value for the importance of a vehicle in proximity of the user’s vehicle. The process of generating an importance score includes using processes such as machine learning models to produce a quantitative result with real numerical values.)
“wherein the processor is configured to calculate the importance score comprising applying a risk shadowing process to the data on the at least one other agent and on the physical structures in the environment, and” (Detailed Description, pp. 6, [0062]; “A feature representation 208 of an agent refers to numerical data represented in any appropriate numerical format (e.g., as a vector or a matrix) which characterizes the agent. The feature representation of an agent can include one or more of: a velocity of the agent (e.g., measured in miles per hour-mph), an acceleration of the agent ( e.g., measured in feet per second squared), a type of the agent (e.g., pedestrian, vehicle, bicyclist, and the like), a distance from the agent to the vehicle 102 (e.g. measured in feet), data indicating whether (and when) a predicted future trajectory of the agent will overlap with a future trajectory of the vehicle 102, a position of the agent relative to the vehicle 102, and data characterizing the velocity, acceleration, and heading of the agent over a previous window of time (e.g., 5 seconds).” This system will generate an importance score based on the attributes of the other vehicle and the objects in the environment. This will generate a feature representation of the agents in the environment. The feature representation as disclosed will use information acquired from sensors to evaluate other agents in the area of the user’s vehicle as well as overlapping trajectories.) and (Detailed Description, pp. 5-6, [0063]; The importance scoring system 114 processes each of the agent feature representations 208 using an importance scoring model 210 to generate a respective importance score 116 for each of the agents in the vicinity of the vehicle 102. Each of the importance scores 116 is represented as a respective numerical value.” The system produces the importance score using information from the feature representation, which is interpreted to be the “risk shadowing process”.)
“the risk shadowing process comprises performing a reachability analysis based on the sensor data and the position and motion information for determining occupied areas for the agent and the at least one other agent in the environment over time, determining overlapping areas from the occupied areas of the agent and the at least one other agent, and determining a relevance of each of the at least one other agent based on the determined overlapping areas,” (Detailed Description, pp. 6, [0072]; “The system generates a respective feature representation of each of the agents in the vicinity of the vehicle (304). The feature representation of an agent can include one or more of: a velocity of the agent (e.g., measured in miles per hour-mph), an acceleration of the agent (e.g., measured in feet per second squared), a type of the agent (e.g., pedestrian, vehicle, bicyclist, and the like), a distance from the agent to the vehicle (e.g. measured in feet), and data indicating whether a predicted future trajectory of the agent will overlap with a future trajectory of the vehicle.” This system is able to generate a feature representation of other agents in the environment. The system will evaluate the other agent’s attributes such as velocity and heading to determine if the other agent’s trajectory overlaps the user’s vehicle or path.) and (Detailed Description, pp. 7, [0074]; “The system generates a respective importance score for each agent in the vicinity of the vehicle by processing the agent feature representations using an importance scoring model (306).) and (Detailed Description, pp. 5, [0058]; “The importance scoring system 114 is configured to process sensor data 118 (generated by the sensor system 104 of the vehicle 102) which characterizes the environment 202 in the vicinity of the vehicle 102 to generate a respective importance score 116 for each agent in the vicinity of the vehicle 102.” These two citations teach that the importance score is generated using the information from the generated feature representation of the other agents in the environment. This is able to determine overlapping regions or trajectories of the user’s vehicle and future position of agents in the environment.)
“generate a list comprising the at least one other agent based on the calculated importance scores of the at least one other agent,” (Detailed Description, pp. 6, [0065]; “The prediction system 106 processes the importance scores 116 using a ranking engine 212 to identify which (if any) of the agents in the vicinity of the vehicle 102 are high-priority agents. For example, the ranking engine 212 may identify each agent with an importance score that satisfies a predetermined threshold as a high-priority agent. In a particular example, the ranking engine may identify each agent with an importance score that exceeds the predetermined threshold of 0.8 as a high-priority agent.” This system will generate an importance score of all agents in an environment. The system will then rank the agents based on the calculated importance scores. Ranking is interpreted to be ordering a set of importance scores into a ranked list or set.)
“partition the generated list in at least one partition based on the importance score of the at least one other agent, wherein each partition includes at least one other agent,” (Detailed Description, pp. 6, [0065]; “The prediction system 106 processes the importance scores 116 using a ranking engine 212 to identify which (if any) of the agents in the vicinity of the vehicle 102 are high-priority agents. For example, the ranking engine 212 may identify each agent with an importance score that satisfies a predetermined threshold as a high-priority agent. In a particular example, the ranking engine may identify each agent with an importance score that exceeds the predetermined threshold of 0.8 as a high-priority agent. As another example, the ranking engine 212 may identify a predetermined number of the agents with the highest importance scores as high-priority agents.” After the importance scores are generated, this system will rank the list of importance scores. The system will determine, or partition, the list into high and low priority agents based on the generated importance scores.) and (Detailed Description, pp. 6, [0066]; “After identifying the high-priority agents 214, the prediction system 106 uses a high-priority prediction model 216 to generate respective prediction data 108 for each of the high-priority agents 214” The high priority agents are partitioned into another category and are handled differently than other importance scores from the ranked list.) and (Detailed Description, pp. 6, [0067]; “The prediction system 106 can use low-priority prediction models 218 to generate respective prediction data 108 for each of the remainder of the agents which the ranking engine 212 does not identify as high-priority agents 214 (i.e., the low-priority agents 220).” The low priority agents are partitioned into another category and are handled differently than other importance scores from the ranked list.)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL MICHAEL GALVIN-SIEBENALER whose telephone number is (571)272-1257. The examiner can normally be reached Monday - Friday 8AM to 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL M GALVIN-SIEBENALER/Examiner, Art Unit 2147
/ERIC NILSSON/Primary Examiner, Art Unit 2151
Prosecution Insights