DETAILED ACTION
Examiner Remarks
In response to Applicant’s Remarks submitted on 12/11/2025, Examiner has withdrawn the objections to the Specification and Claims and has also withdrawn the 112(b) rejections. Claims 1-20 are currently pending.
Response to Arguments
Applicant argues that the prior art of Wray does not teach the claim one limitation of a probability associated with experienced operational scenarios that are determined to be similar to one another. See pg., 9 of Applicant’s Remarks submitted on 12/11/2025. Applicant supports this assertion by arguing that the probability in Wray is a variable adjustment within an existing model, not a parameter generated based on multiple, similar experienced operational scenarios. Id.
Respectfully, Examiner disagrees. As a preliminary matter, in response to Applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies upon (i.e., based on multiple) are not recited in the rejected claims. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Rather the rejected limitation as claimed recites the following: generating a parameter based on the group of state-action history entries, wherein the parameter represents a probability associated with experienced operational scenarios that are determined to be similar to one another. No where does the above claim limitation recite that the parameter is generated based on multiple, similar experienced operational scenarios.
Furthermore, as col. 43, lines 9-41 state: “the operational scenarios are updated using stored information from a history database. For example, if the autonomous vehicle observes that other vehicles cut off the autonomous vehicle consistently, the autonomous vehicle then updates the probability that other vehicles will cut it off. Accordingly, the next time the autonomous vehicle observes the same operational scenario in a new location, it can use its prior experience to improve its behavior.” Accordingly, Wray does teach the claim limitation of a probability associated with experienced operational scenarios[the probability that other vehicles will cut it off] that are determined to be similar to one another[the autonomous vehicle observes the same operational scenario in a new location, it can use its prior experience to improve its behavior].
Applicant then argues that the prior art of Wiest does not teach the amended claim limitation of determined when collecting the state-action entry and representing a condition associated with the experienced operational scenario. See pg., 10 of Applicant’s Remarks submitted on 12/11/2025. Applicant supports this assertion by arguing that the variables
X
0
:
t
,
A
t
, and
C
t
are inputs to a mathematical formulation and are not features of a state-action history entry. Id.
Respectfully, Examiner disagrees. As paragraph [0024] of Applicant’s Specification states “[a]n individual state-action history entry may include state-action samples that are collected when experiencing an operational scenario (e.g., when traversing an intersection, changing lanes, or encountering a crosswalk).”(Emphasis Added). Accordingly, with respect to the Broadest Reasonable Interpretation (BRI) in light of the Specification Wiest teaches the newly amended claim limitation of determined when collecting the state-action entry and representing a condition associated with the experienced operational scenario. As Wiest teaches in col. 16, lines 34-61 “FIG. 6 illustrates an overview of example stages of developing, deploying and using state prediction models for autonomous vehicles... as part of a continuous or ongoing data collection procedure 601, data about driving environments may be collected from a variety of vehicle-based sensors in numerous geographical regions.” (Emphasis added). Accordingly, the state-action-context entries of the state prediction model i.e.,
X
0
:
t
,
A
t
, and
C
t
are collected while the vehicle is driving i.e., amounting to experienced operational scenario. See the Current Office Action for the detailed teaching.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wray et al., US 11,027,751 B2 (“Wray”) in view of Wiest et al. US 11,537,134 Bl (“Wiest”).
Regarding claim 1, Wray teaches a method for use in traversing a vehicle transportation network, the method comprising:
determining a group of state-action history entries by filtering state-action history entries stored in a database, wherein a state-action history entry represents an experienced operational scenario, [wherein the state-action history entry is associated with a feature determined when collecting the state-action history entry and representing a condition associated with the experienced operational scenario, and wherein the group of state- action history entries is determined based on a similarity of the feature](Wray, col. 42, lines 19-45, see also figs. 5 and 6, “The autonomous vehicle associates the observed new state and the corresponding candidate vehicle action taken to generate a state-action history entry 6110. The autonomous vehicle may store the state-action history entry in a history database[determining a group of state-action history entries from state-action history entries stored in a database,], for example a scenario-specific operation control database[wherein a state-action history entry represents an experienced operational scenario]. A state-action history entry may include an action performed, a vehicle and/or operational environment state resulting from the action performed, and the time the action was performed.”);1
generating a parameter based on the group of state-action history entries, wherein the parameter represents a probability associated with experienced operational scenarios that are determined to be similar to one another; and generating a model based on the parameter, wherein the model is configured for use in an operational scenario that is similar to the experienced operational scenarios when traversing a portion of the vehicle transportation network(Wray, col. 43, lines 9-41, see also fig. 7, “Detecting an operational scenario 7040 may include determining a SSOCEM based on the operational scenario. The SSOCEM may include one or more models that determine a candidate vehicle control action and may be based on an operational environment of the autonomous vehicle[generating a model]. At 7045, the one or more models are optionally re-solved and the operational scenarios are updated using stored information from a history database[based on the group of state-action history entries,]. For example, if the autonomous vehicle observes that other vehicles cut off the autonomous vehicle consistently, the autonomous vehicle then updates the probability that other vehicles will cut it off. Accordingly, the next time the autonomous vehicle observes the same operational scenario in a new location, it can use its prior experience to improve its behavior[generating a parameter wherein the parameter represents a probability associated with experienced operational scenarios that are determined to be similar to one another; based on the parameter, wherein the model is configured for use in an operational scenario that is similar to the experienced operational scenarios when traversing a portion of the vehicle transportation network].”)
While Wray does teach the state-action history entry, Wray does not teach: wherein the state-action history entry is associated with a feature determined when collecting the state-action history entry and representing a condition associated with the experienced operational scenario, and wherein the group of state- action history entries is determined based on a similarity of the feature
However, Wiest teaches:
wherein the state-action history entry is associated with a feature determined when collecting the state-action history entry and representing a condition associated with the experienced operational scenario, and wherein the group of state- action history entries is determined based on a similarity of the feature (Wiest, cols. 13-14, lines 31-67 & lines 1-57 and cols. 16-17, lines 34-67 & lines 1-30, see also figs. 4 and 6, “FIG. 4 illustrates an example mathematical formulation of a general state prediction problem...[i]n the formulation 401 of the depicted embodiment, the term
X
0
:
t
denotes information about the current state (at time t) and previous states (starting from some initial state represented by time zero)[state]… [t]he term
A
t
denotes the actions taken by various agents at time t[action]… while the term
C
t
represents several external aspects the context of the autonomous vehicle[wherein the state-action history entry is associated with a feature]… [t]he policy term 407 may represent the probabilities of higher-level actions taken by various entities or agents given a context and a current world state[and wherein the group of state- action history entries is determined based on a similarity of the feature]... as part of a continuous or ongoing data collection procedure 601, data about driving environments may be collected from a variety of vehicle-based sensors in numerous geographical regions... [a]t least a subset of the decisions made at the vehicle... may be transmitted back to the data centers as part of the ongoing data collection approach, and uses to improve and update the state prediction models[determined when collecting the state-action history entry and representing a condition associated with the experienced operational scenario]....”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Wray with the teachings of Wiest the motivation to do so would be to implement predictions regarding vehicle trajectories for better predictability with regards to driving behaviors(Wiest, col. 1, lines 18-59, “[T]he task of making
timely and reasonable decisions (which are based neither on excessively pessimistic assumptions, nor on excessively optimistic assumptions) regarding an autonomous vehicle's
trajectory in the context of unpredictable behaviors of other entities ( such as other drivers or other autonomous vehicles) and incomplete or noisy data about the vehicle's environment
in real-world traffic remains a significant challenge. [Accordingly], [v]arious embodiments of methods and apparatus for generating joint state predictions to be used for decision making regarding movements or trajectories of an autonomous vehicle are described.”).
Regarding claim 2, Wray in view of Wiest teaches the method of claim 1, wherein the experienced operational scenarios include at least one of: traversing an intersection; changing lanes; or encountering a crosswalk(Wray, col. 6, lines 1-14, “The embodiments disclosed herein describe methods and vehicles configured to gain experience in the form of state-action
and/or action-observation histories for an operational scenario as the vehicle traverses a vehicle transportation network. These histories may be incorporated into a model in the form of learning to improve the model over time. These histories may be used to customize solutions for
specific intersections, locations of merges, and pedestrian crosswalks, etc[traversing an intersection, changing lanes; or encountering a crosswalk].”).2
Regarding claim 3, Wray in view of Wiest teaches the method of claim 1, wherein the feature includes at least one of: a coordinate location; a time of day; a density of traffic; or a driver aggressiveness(Wiest, col. 14, lines 1-27, see also fig. 4, “The context
C
t
may include time-constant object specific properties (such as the type of an object, color, brand, age, gender, size, etc.)… as well as time-varying higher level, non-observable variables such as desired goals (reflected in
G
t
), agents' awareness of other agents, agent driving styles (reflected in
D
t
)[ a driver aggressiveness] etc”).3
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Wray with the above teachings of Wiest for the same rationale stated at Claim 1.
Regarding claim 4, Wray in view of Wiest teaches the method of claim 1, further comprising: transferring the model to multiple vehicles in a fleet, wherein the model is transferred to the multiple vehicles based on the multiple vehicles having a same vehicle characteristic (Wiest, col. 17, lines 1-30, see also figs. 5 and 6, “Trained models 650, which may for example comprise implementations of the layers indicated in FIG. 5, may be transmitted to autonomous vehicles 672 (e.g., AV 672A-672C) of fleets 670… [t]he trained models may be executed using local computing resources at the autonomous vehicles and data collected by local sensors of the autonomous vehicles, e.g., to predict joint world states at desired time horizons as discussed earlier[transferring the model to multiple vehicles in a fleet, wherein the model is transferred to the multiple vehicles based on the multiple vehicles having a same vehicle characteristic].”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Wray with the above teachings of Wiest for the same rationale stated at Claim 1.
Regarding claim 5, Wray in view of Wiest teaches the method of claim 1, further comprising: blocking a first state-action history entry from the group of state-action history entries based on a difference in the feature associated with the first state-action history entry(Wiest, col. 18, lines 10-57, see also fig. 7, “[A]dding dynamic object representations representing, for example…control decisions[action]…respective state vectors[state] may be generated for each dynamic object or entity…[i]n some embodiments, a padded vector technique may be used, in which a maximum number or upper bound of dynamic objects that can be represented may be chosen…[i]f the total number of dynamic or moving entities detectable in the environment exceeds the maximum, some may be discarded from the representation, e.g., based on relevance criteria or factors such as proximity, size, speed, and the like[blocking a first state-action history entry from the group of state-action history entries based on a difference in the feature associated with the first state-action history entry].”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Wray with the above teachings of Wiest for the same rationale stated at Claim 1.
Regarding claim 6, Wray in view of Wiest teaches the method of claim 1, further comprising: generating the parameter, based on the group of state-action history entries, by averaging state-action history entries in the group of state-action history entries(Wray, col. 23, lines 1-46, “Identifying the vehicle control action from the candidate vehicle control actions may include generating or calculating a weighted average for each type of vehicle control action based on the defined vehicle control action identification metrics, the instantiated scenarios, weights associated with the instantiated scenarios, the candidate vehicle control actions, weights associated with the candidate vehicle control actions or a combination thereof[generating the parameter, based on the group of state-action history entries, by averaging state-action history entries in the group of state-action history entries]. ”).
Regarding claim 7, Wray in view of Wiest teaches the method of claim 1, further comprising: generating the model, based on the parameter, by implementing a Partially Observable Markov Decision Process (POMDP) or a Markov Decision Process (MDP)( Wray, col. 23, lines 1-46, “[I]dentifying the vehicle control action from the candidate vehicle control actions may include implementing an MDP or a POMDP, which may describe how respective candidate vehicle control actions affect subsequent candidate vehicle control actions affect, and may include a reward function that outputs a positive or negative reward for respective vehicle control actions[generating the model, based on the parameter, by implementing a Partially Observable Markov Decision Process (POMDP) or a Markov Decision Process (MDP)]”).
Regarding claim 8, Wray in view of Wiest teaches the method of claim 1, wherein the state-action history entry includes state-action samples, and wherein a first state-action history entry of the group of state-action history entries includes more state-action samples than a second state-action history entry of the group of state- action history entries(Wiest, col. 18, lines 10-57, see also fig. 7, “The combination of the occupancy grid map 702 and the
static infrastructure graph 704 may be further enhanced by adding dynamic object representations representing, for example, the autonomous vehicle for which motion control
decisions are to be generated…[and] respective state vectors may be generated for each dynamic object or entity[wherein the state-action history entry includes state-action samples]… [i]n another approach towards handling varying numbers of dynamic objects, a mapping or embedding from a first representation space to a second representation space may
be used, in which the dimensionality of the two representation spaces may differ. For example, if there are 20 moving entities to be represented, the states of the 20 entities may be mapped to a space with a smaller dimensionality…while if there are no more than 10 objects to be represented, the mapping or embedding transformation may not be required[and wherein a first state-action history entry of the group of state-action history entries includes more state-action samples than a second state-action history entry of the group of state- action history entries].”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Wray with the above teachings of Wiest for the same rationale stated at Claim 1.
Regarding claim 9, Wray in view of Wiest teaches the method of claim 1, further comprising: collecting state-action samples when experiencing an operational scenario(Wray, col. 27, lines 22-59, “An MDP model may model a distinct vehicle operational
scenario using a set of states, a set of actions, a set of state transition probabilities, a reward function, or a combination thereof… [a]lthough any number or cardinality of states may be used,
the number or cardinality of states included in a model may be limited to a defined maximum number of states, such as 300 states… [t]he set of actions may include vehicle control actions
available to the MDP model at each state in the set of states[collecting state-action samples when experiencing an operational scenario].”);
determining the feature when collecting the state-action samples; storing the state-action samples as the state-action history entry in the database; and storing the feature as metadata, associated with the state-action history entry, in the database(Wray, col. 42, lines 19-45, “The autonomous vehicle may store the state-action history entry in a history database[storing the state-action samples as a state-action history entry in the database], for example a scenario-specific operation control database… [a] state-action history entry may include an action performed, a vehicle and/or operational environment state resulting from the action performed, and the time the action was performed[determining the feature when collecting the state-action samples and storing the feature as metadata, associated with the state-action history entry, in the database].”).
Regarding claim 10, Wray in view of Wiest teaches the method of claim 1, wherein the parameter represents a probability associated with movement of a vehicle(Wray, col. 24, lines 23-35, “A probability of a vehicle control action, may indicate a probability or likelihood that the autonomous vehicle may traverse a portion of, or spatial location within, the vehicle transportation network safely, such as unimpeded by an external object, such as a remote vehicle or a pedestrian[wherein the parameter represents a probability associated with movement of a vehicle].”).
Regarding claim 11, Wray in view of Wiest teaches the method of claim 1, further comprising: determining the similarity of the feature based on a threshold distance of features associated with state-action history entries stored in the database(Wray, col. 34, lines 48-60, see also fig. 4, “The candidate vehicle control actions output by the instances of the SSOCEMs 4400 may include, or be associated with, operational environment information, such as state information, temporal information, or both[of features associated with state-action history entries stored in the database]… [t]he autonomous vehicle operational management controller 4100 may identify stale candidate vehicle control actions representing past temporal locations, states having a probability of occurrence below a minimum threshold[determining the similarity of the feature based on a threshold distance] and may delete, omit, or ignore the stale candidate vehicle control actions.”).
Regarding claim 12, Wray in view of Wiest teaches the method of claim 1, wherein the model is a deep reinforcement learning (DRL) model, and wherein the parameter is a parameter for the DRL model(Wray, col. 42, lines 1-18, “State-action-reward-state-action (SARSA), or Deep Leaming to update the SSOCEMI 6100… SARSA is an algorithm for learning an MDP policy used in RL where a SARSA agent interacts with the environment and updates the policy based on actions taken, known as an on-policy learning algorithm[wherein the model is a deep reinforcement learning (DRL) model, and wherein the parameter is a parameter for the DRL model].”).
Regarding claim 13, Wray teaches an apparatus for use in traversing a vehicle transportation network, the apparatus comprising: a non-transitory computer readable medium; and a processor configured to execute instructions stored on the non-transitory computer readable medium to(Wray, col. 2, lines 23-32, “The autonomous vehicle may include a memory and a processor configured to execute instructions stored on a non-transitory computer readable medium.”) and for all other claim limitations they are rejected on the same basis as independent claim 1 since they are analogous claims.
Referring to dependent claims 14-20 they are rejected on the same basis as dependent claims 4-7, 9, and 11-12 since they are analogous claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ADAM C STANDKE whose telephone number is (571)270-1806. The examiner can normally be reached Gen. M-F 9-9PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Adam C Standke/
Primary Examiner
Art Unit 2129
1 Examiner Notes: The claim limitations that are not in bold and contained with square brackets i.e., [ ] are claim limitations that are not taught by Wray
2 Examiner Notes: According to the broadest reasonable interpretation (BRI), the use of alternative language amounts to the claim requiring one or more elements but not all.
3 Examiner Notes: According to the broadest reasonable interpretation (BRI), the use of alternative language amounts to the claim requiring one or more elements but not all.
Prosecution Insights