Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see arguments , filed06/08/2026, with respect to the rejection(s) of claim(s) 1-20 (In particular the limitation from claim 4 which is now integrated into the independent claims) under Konrardy in view of Liu have been fully considered and are persuasive. Therefore, the rejection has been withdrawn.
However, upon further consideration, a new ground(s) of rejection is made in view of Konrardy in view of Liu (same NPL as previously cited however relying on a different section (related works section of NPL) and thus is a new grounds of rejection); the applicant is correct in that the previously cited section does not teach the feedback based on the results/output of the model, instead it focus on internal feature selection, however upon further review of Liu is discloses for single (and multi) actor re-enforcement learning, providing feedback (that is the results/performance) of the output of a given iteration of the models task and using it to train the model is WURC for re-enforcement learning; thus in render obvious in the context of ML model of Konrardy (ML which predicts the proper level of autonomous driving) having the results of the switching being feedback into the ML model to update the model weights to improve its performance in subsequent predictions. (Liu, 2 Related Work, Multi-Agent Reinforcement Learning “Our work is related to multi-agent reinforcement learning, where multiple agents share a complex environment and interact with each other [24]. In the single-agent formulation, the reinforcement learning agent takes an action to change the environment, and get a reward as feedback to evaluate its action, so as to improve its next decision on action [25]. In the multi-agent formulation, agents not only need to interact with the environment, but also need to interact with each other.)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Konrardy and further in view of NPL, “Automated Feature Selection: A Reinforcement Learning Perspective”, Liu et al.
Regarding Claim 1, Konrardy et al teaches “A computer-implemented method, comprising: receiving contextual data from an autonomous vehicle, wherein the contextual data includes data obtained from the autonomous vehicle and data obtained from other vehicles;”( [0148] “FIG. 7 illustrates a flow diagram of an exemplary automatic usage optimization method 700 for monitoring and adjusting autonomous operation feature usage levels of a vehicle 108 having a plurality of autonomous operation features. The method 700 may begin by receiving autonomous operation suitability data for a plurality of road segments (block 702), which may form a route the vehicle 108 will traverse. While vehicle operation continues, vehicle operation may be monitored for each road segment along which the vehicle 108 travels (block 704), which may include obtaining vehicle operating data relating to location and environmental conditions. Based upon the monitored vehicle operating data and the suitability data, appropriate usage levels for one or more autonomous operation features may be determined (block 706), which may include only optimal settings or may include other allowable settings.” + [0039] which teaches that external sources (such as other vehicles and/or infrastructures) can also provide supplemental data);” ; in response to receiving the contextual data from the autonomous vehicle, determining a level of driving automation, from a plurality of levels of driving automation, , that is predicted to provide a highest level of efficiency”([0147] “As a vehicle traverses various road segments and as conditions change, the optimal usage levels of autonomous operation features (i.e., the settings and configurations that are most efficient, safest, etc.) may likewise change.”Here teaches that determining the usage level of autonomous operation features is/includes determining which is “most efficient”);” of the autonomous vehicle based on the contextual data and using a machine learning model;”([0232]-[0233] here teaches that the Fig. 7 [0148] cited above, can be implemented via ML models specifically in [0233] teaches using ML models to predict/optimize the recommend optimal autonomous feature usage);” and transmitting, to the autonomous vehicle, a recommendation including the determined level of driving automation.”([0148] “… When the current usage levels are not appropriate for the road segment and conditions, the usage levels of the one or more autonomous operation features may be automatically adjusted to appropriate levels (block 710). Such adjustments may include enabling or disabling features, as well as adjusting settings associated with the features. Once the usage levels are at appropriate levels, the method may check whether the vehicle 108 has reached the end of its route or whether operation is ongoing (block 712). The method 700 may then continue to monitor vehicle operation and adjust usage levels as needed (blocks 704-710) until the vehicle operation is determined to be discontinued (block 712), at which point the method may terminate.” The recommended autonomous operation features levels are implemented in block 710 (inherently requiring they are transmitting within/to the autonomous vehicle controller))
While Konrardy does teach the use of machine learning to perform the prediction of the proper autonomous driving levels ([0232]-[0241]) it does not specifically teach “; and receiving feedback from the autonomous vehicle based on the autonomous vehicle switching from a current level of driving automation to the recommended level of driving automation.” i.e. re-enforcement learning based on the results/performance (feedback) of the machine learning model from past/previous iterations.
Liu teaches a system for reinforcement learning, which teaches that as part of said learning the system assigns a reward for a given action (output) of its given task and that reward is used to train the model to improve its performance ((Liu, 2 Related Work, Multi-Agent Reinforcement Learning “Our work is related to multi-agent reinforcement learning, where multiple agents share a complex environment and interact with each other [24]. In the single-agent formulation, the reinforcement learning agent takes an action to change the environment, and get a reward as feedback to evaluate its action, so as to improve its next decision on action [25]. In the multi-agent formulation, agents not only need to interact with the environment, but also need to interact with each other.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the application to modify Konrardy to include the rewards based on the action as feedback as taught by Liu. One would be motivated to implement this to allow for the ML model of Konrardy to improve its predictive accuracy/performance ( in predicting the proper level of autonomous driving) thereby improving operation of the system of Konrardy. ((Liu, 2 Related Work, Multi-Agent Reinforcement Learning “Our work is related to multi-agent reinforcement learning, where multiple agents share a complex environment and interact with each other [24]. In the single-agent formulation, the reinforcement learning agent takes an action to change the environment, and get a reward as feedback to evaluate its action, so as to improve its next decision on action [25]. In the multi-agent formulation, agents not only need to interact with the environment, but also need to interact with each other.)
Regarding Claim 2, while Konrardy et al teaches the use of ML models to predict the optimal level of autonomous driving features based on driving and environmental data. (Claim 1’s rejection);It does not explicitly recite determining teach determining “key” features and the subsequent inputting of the Key features as a query into the model
Liu et al teaches for machine learning models the concept of determining extracting key features for generalized input data and subsequently inputting of those key features as the query into the machine learning model (Introduction: “feature selection aims to select the optimal subset of relevant features for a downstream predictive task [1], [2]. Effective feature selection can help to reduce dimensionality, shorten training time, enhance generalization, avoid overfitting, improve predictive accuracy, and provide better
interpretation and explanation.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the application to modify Konrardy et al to implement a feature selection/filtering algorithm as generally taught by Liu before query/feeding the resulting key features into the ML model(s) taught in Konrardy to predict the optimal level of autonomous driving features/functions. One would be motivated to implement the (key) feature selection/extraction in order to reduce the dimensionality of the determination, shortening training time and computational cost of running the model, and also improving the generalized accuracy/functioning of the model. Liu teaches these improvements resulting from feature selection as taught in the introduction section cited above.
Regarding Claim 3, modified Konrardy teaches “The computer-implemented method of claim 2, further comprising applying the query as an input to the machine learning model.”( Liu, introduction: “feature selection aims to select the optimal subset of relevant features for a downstream predictive task “ here the downstream prediction task teaches inputing of the relevant features into the ML model (i.e. applying the query))
Regarding Claim 4, modified Konrardy teaches “re-training the machine learning model using the feedback.”( (Liu, 2 Related Work, Multi-Agent Reinforcement Learning “Our work is related to multi-agent reinforcement learning, where multiple agents share a complex environment and interact with each other [24]. In the single-agent formulation, the reinforcement learning agent takes an action to change the environment, and get a reward as feedback to evaluate its action, so as to improve its next decision on action [25]. In the multi-agent formulation, agents not only need to interact with the environment, but also need to interact with each other.) here teaches that the feedback (reward) is used to evaluate to improve (i.e. retrain) the model for its next decision on an action (next prediction/iteration))
Regarding Claim 5, Konrardy et al teaches “. The computer-implemented method of claim 1, further comprising: creating a knowledge base including historic data from plural vehicles; determining( [0234] A processor or a processing element may be trained using supervised or unsupervised machine learning, and the machine learning program may employ a neural network, which may be a convolutional neural network, a deep learning neural network, or a combined learning module or program that learns in two or more fields or areas of interest. Machine learning may involve identifying and recognizing patterns in existing data (such as autonomous vehicle system, feature, or sensor data, autonomous vehicle system control signal data, vehicle-mounted sensor data, mobile device sensor data, and/or telematics, image, or radar data) in order to facilitate making predictions for subsequent data (again, such as autonomous vehicle system, feature, or sensor data, autonomous vehicle system control signal data, vehicle-mounted sensor data, mobile device sensor data, and/or telematics, image, or radar data).” Here Konrardy teaches that the ML models learn to identify patterns in “existing’ (historical) data and through training on the set of data (knowledge base) they learn to predict outcomes from a given subsequent/currently inputted data.)
Konrardy et al however does not disclose determining “Key” features,
Liu et al teaches for machine learning models the concept of determining extracting key features for generalized input data and subsequently inputting of those key features as the query into the machine learning model (Introduction: “feature selection aims to select the optimal subset of relevant features for a downstream predictive task [1], [2]. Effective feature selection can help to reduce dimensional ity, shorten training time, enhance generalization, avoid overfitting, improve predictive accuracy, and provide better
interpretation and explanation.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the application to modify Konrardy et al to implement a feature selection/filtering algorithm as generally taught by Liu before query/feeding the resulting key features into the ML model(s) taught in Konrardy to predict the optimal level of autonomous driving features/functions. One would be motivated to implement the (key) feature selection/extraction in order to reduce the dimensionality of the determination, shortening training time and computational cost of running the model, and also improving the generalized accuracy/functioning of the model. Liu teaches these improvements resulting from feature selection as taught in the introduction section cited above.
Regarding Claim 6, Konrardy et al teaches “The computer-implemented method of claim 1, wherein the contextual data includes vehicular data,”( [0148] FIG. 7 illustrates a flow diagram of an exemplary automatic usage optimization method 700 for monitoring and adjusting autonomous operation feature usage levels of a vehicle 108 having a plurality of autonomous operation features. The method 700 may begin by receiving autonomous operation suitability data for a plurality of road segments (block 702), which may form a route the vehicle 108 will traverse. While vehicle operation continues, vehicle operation may be monitored for each road segment along which the vehicle 108 travels (block 704), which may include obtaining vehicle operating data relating to location and environmental conditions. Based upon the monitored vehicle operating data and the suitability data, appropriate usage levels for one or more autonomous operation features may be determined (block 706), which may include only optimal settings or may include other allowable settings” Here teaches obtaining operating data (vehicular data) + environmental data );” road condition data, and environmental data.”( [0009] In some embodiments, the environmental information may include one or more of the following environmental conditions: time of day, type of roadway, traffic conditions, weather conditions, and/or construction conditions. In such embodiments, obtaining the environmental information may include monitoring at least one environmental condition based upon sensor data generated by one or more sensors” Here teaches that the environmental conditions (in [0148]) includes road conditions and weather (environmental))
Regarding Claim 7, Konrardy et al teaches “The computer-implemented method of claim 1, wherein the machine learning model is trained to predict an optimal level of driving automation for a vehicle that will achieve a highest level of efficiency for the vehicle based on a set of contextual data associated with the vehicle.”([0148] teaches determining optimal level of autonomous features (level of driving autonomation) + [0147] teaches the optimal level of autonomous features is/includes most efficient level + [0232]-[0233] teaches that the determinations can be implemented via ML models trained to predict the corresponding optimal levels)
Regarding Claim 8 it is a computer program product equivalent to claim 1’s method. It has the same overall grounds of rejection.
Regarding Claims 12-13 they are computer program product equivalents to method claims 6-7 above, they have the same grounds of rejection.
Regarding Claim 14, Konrardy et al teaches “. The computer program product of claim 8, wherein the contextual data further includes data obtained from one or more Internet-of-Things sensors along a roadway on which the autonomous vehicle is driving.”([0033] “The front end components 102 may further include a communication component 122 to transmit information to and receive information from external sources, including other vehicles, infrastructure, or the back-end components 104.” Here teaches external sources include infrastructure (i.e. sensors along the road/the internet of things) + [0041] teaches transmission over internet))Which is read in light of [0147]-[0148]/claim 1’s rejection)
Regarding Claim 15, it is a computer system equivalent to the claims 8 + claim 13 + claim 14. It has the same grounds of rejection as those claims.
Claims 9-11 and 16-18 are computer program products and computer system equivalents to the method claims 2-5 above, they have the same overall grounds of rejection, combination, and motivation combination as their respective equivalents above.
Regarding Claim 19, it is equivalent to claim 6.
Regarding Claim 20, it is equivalent to claim 7.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNETH MICHAEL DUNNE whose telephone number is (571)270-7392. The examiner can normally be reached Mon-Thurs 8:30-6:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Navid Z Mehdizadeh can be reached at (571) 272-7691. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KENNETH M DUNNE/Primary Examiner, Art Unit 3669
Prosecution Insights