DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-3, 5-10, 12-16, and 18-20 are pending and have been examined.
Amendments
This Office Action is in response to the amendment filed on July 1, 2025.
Claims 1-3, 5-10, 12-16, and 18-20 have been amended.
Claims 4, 11, and 17 have been cancelled.
No new claims have been added.
The objections and rejections from the prior correspondence that are not restated herein are withdrawn.
Response to Arguments
Applicant's arguments filed on July 1, 2025 have been fully considered.
Applicant’s arguments regarding the 35 U.S.C. 112(b) rejections of the previous office action have been fully considered, and are persuasive. The rejections have been withdrawn due to claim amendments.
Applicant’s arguments regarding the 35 U.S.C. 101 rejections of the previous office action have been fully considered, and are persuasive. The rejections have been withdrawn due to claim amendments.
Applicant’s arguments regarding the 35 U.S.C. 103 rejections of the previous office action have been fully considered, but are moot because the arguments allege that only the newly added limitations are not taught by the prior art of record. It should be noted that new prior art references to Sarferaz in view of Murray and Zhang teaches the newly added limitations as shown in the rejections below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-8, 12-13, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Sarferaz (US 20210073627 A1) in view of Murray (US 20210089903 A1), Athreya (US 20230048206 A1), and Zhang (US 20220297726 A1), hereafter Sarferaz, Murray, Athreya, and Zhang respectively.
Regarding Claim 1, Sarferaz teaches:
A method of training a machine learning (ML) model, the method comprising:
training the pretrained ML model with a first training dataset for a first number of iterations based on a first configuration to yield a first ML model; (Sarferaz [0043] teaches: "At 302, the controller 110 may train, based at least on a first training dataset, a machine learning model. For example, the controller 110 may train, based at least on a first training data, the machine learning model 150 prior to deploying the machine learning model 150 to the enterprise resource planning system. The machine learning model 150 may be trained to perform text classification in order to assign, based at least on the text associated with the ticket 160 generated by the issue tracking system 165, a priority corresponding to the severity of the error associated with the ticket 160." Additionally, Sarferaz [0008] teaches: "The machine learning model may be trained and retrained by at least minimizing the error in the output of the machine learning model. [...] until a gradient of an error function associated with the machine learning model converges to a threshold value." Examiner's note: Sarferaz explicitly teaches training a machine learning model with a first training dataset to perform specific functions (i.e., based on a first configuration). Additionally, Fig. 4A and 4B of Sarferaz shows the iteration in training the model and generating metrics. Under BRI, "for a first number of iterations" can be interpreted as the training and retraining until the error converges. Furthermore, "to yield a first ML model" can be reasonably interpreted as the trained model that reached convergence.)
analyzing the first ML model based on a convergence of the first ML model and a previous iteration of training to identify at least one constraint; (Sarferaz [008] teaches: "The error in the output of the machine learning model may be minimized by at least adjusting one or more weights applied by the machine learning model until a gradient of an error function associated with the machine learning model converges to a threshold value." Sarferaz [0045] teaches: "At 304, the controller 110 may detect a degradation of the machine learning model." Sarferaz [0051] teaches: "The degradation component of the controller 400 may interpret the accuracy key performance indicators and generate recommendations for the monitoring component of the controller 400 that include, for example, retrain the machine learning model, adjust the definition of the machine learning model, and/or the like." Examiner's note: under BRI, "to identify at least one constraint" can be interpreted as detecting (i.e., analyzing) a degradation of the machine learning model based on the interpretation of accuracy key performance indicators of the machine learning model.)
generating a report based on the analysis of the first ML model, […] (Sarferaz [0051] teaches: "[…] generate recommendations for the monitoring component of the controller 400 that include, for example, retrain the machine learning model, adjust the definition of the machine learning model, and/or the like." Examiner's note: under BRI, "a report based on the analysis of the first ML model" can be interpreted as the generated recommendations for the monitoring component.)
based on the at least one constraint, further training the first ML model for a second number of iterations based on a second configuration to yield a second ML model, the second configuration configured to cause an […] modification to the second ML model relative to the first training model; (Sarferaz [0048] teaches: "At 306, the controller 110 may respond to detecting the degradation of the machine learning model by at least retraining, based at least on a second training dataset that includes at least one training sample not included in the first training dataset, the machine learning model." Examiner’s note: under BRI, retraining a machine learning model causes a modification to the model’s parameters or weights.)
Sarferaz is not relied upon for teaching:
pretraining an uninitialized ML model to yield a pretrained ML model;
[…] the report comprising the at least one identified constraint, the at least one constraint comprising one of a model capacity, a scenario imbalance, a target category imbalance, or a data diversity issue;
[…] to cause an architectural modification to the second ML model relative to the first training model;
deploying the second ML model to a local computing device of an autonomous vehicle having a plurality of mechanical systems; and
controlling, by the deployed second ML model executing on the local computing device, at least one mechanical system of the plurality of mechanical systems to navigate the autonomous vehicle within a physical environment, the at least one mechanical system comprising at least one of a steering system, a braking system, or a propulsion system.
However, Murray teaches: pretraining an uninitialized ML model to yield a pretrained ML model; (Murray [0098] teaches: "the model is initialized using a model pre-trained on a large dataset that does not have semantic annotations.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Sarferaz and Murray before them, to include Murray's pretraining and initialization of a model into Sarferaz' model degradation detection method. One would have been motivated to make such a combination in order to significantly improve the performance metrics of the model (Murray [0124]).
Sarferaz in view of Murray is not relied upon for teaching:
[…] the report comprising the at least one identified constraint, the at least one constraint comprising one of a model capacity, a scenario imbalance, a target category imbalance, or a data diversity issue;
[…] to cause an architectural modification to the second ML model relative to the first training model;
deploying the second ML model to a local computing device of an autonomous vehicle having a plurality of mechanical systems; and
controlling, by the deployed second ML model executing on the local computing device, at least one mechanical system of the plurality of mechanical systems to navigate the autonomous vehicle within a physical environment, the at least one mechanical system comprising at least one of a steering system, a braking system, or a propulsion system.
However, Athreya teaches: […] the report comprising the at least one identified constraint, the at least one constraint comprising one of a model capacity, a scenario imbalance, a target category imbalance, or a data diversity issue; (Athreya [0061] teaches: "For example, the processor 304 may modify the machine learning model structure to reduce the complexity, processing load, and/or power consumption of the machine learning model structure while maintaining (e.g., satisfying) a target accuracy". Athreya [0026] teaches: "A processing load is an amount of processing (e.g., processor cycles, processing complexity, proportion of processing bandwidth, memory usage, and/or memory bandwidth, etc.)." Examiner’s note: Sarferaz’ method generates recommendations for retraining the model based on performance indicators. A person having ordinary skill in the art would have recognized that monitoring the model’s complexity, processing load, and/or power consumption as Athreya describes would have been beneficial in yielding a more robust model.)
to cause an architectural modification to the second ML model relative to the first training model; (Athreya [0061] teaches: "For example, the processor 304 may modify the machine learning model structure to reduce the complexity, processing load, and/or power consumption of the machine learning model structure while maintaining (e.g., satisfying) a target accuracy". Athreya [0026] teaches: "A processing load is an amount of processing (e.g., processor cycles, processing complexity, proportion of processing bandwidth, memory usage, and/or memory bandwidth, etc.).")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Sarferaz, Murray, and Athreya before them, to include Athreya's model structure modification in Sarferaz and Murray’s model degradation detection method. One would have been motivated to make such a combination in order to improve inferencing performance, improve inferencing efficiency, and/or reduce power consumption during inferencing (Athreya [0011]) by including such information in Sarferaz’ recommendation (i.e., report).
The combination Sarferaz/Murray/Athreya is not relied upon for teaching:
deploying the second ML model to a local computing device of an autonomous vehicle having a plurality of mechanical systems; and
controlling, by the deployed second ML model executing on the local computing device, at least one mechanical system of the plurality of mechanical systems to navigate the autonomous vehicle within a physical environment, the at least one mechanical system comprising at least one of a steering system, a braking system, or a propulsion system.
However, Zhang teaches: deploying the second ML model to a local computing device of an autonomous vehicle having a plurality of mechanical systems; and controlling, by the deployed second ML model executing on the local computing device, at least one mechanical system of the plurality of mechanical systems to navigate the autonomous vehicle within a physical environment, the at least one mechanical system comprising at least one of a steering system, a braking system, or a propulsion system. (Zhang [0023] teaches: "Embedded algorithms and/or logic that determine driving decisions, such as slowing down or stopping, undergo rigorous testing through simulations to ensure safety of the embedded algorithms and/or logic before deploying to autonomous vehicles in road testing." Zhang [0024] teaches: "The trained machine learning model may be part of the embedded algorithms and/or logic, or may provide input to the embedded algorithms and/or logic." Zhang [0029] teaches: "In general, the autonomous vehicle 100 can effectuate any control to itself that a human driver can on a conventional vehicle. For example, the autonomous vehicle 100 can accelerate, brake, turn left or right, or drive in a reverse direction just as a human driver can on the conventional vehicle." Examiner's note: under BRI, "within a physical environment" can be reasonably interpreted as deploying to autonomous vehicles in road testing. A person having ordinary skill in the art would recognize that training models under simulated real-world driving conditions and then deploying the model for road testing involves testing the autonomous vehicle in physical environments. Additionally, the “second ML model” can be interpreted as Sarferaz’ model after retraining based on the recommendations and/or the constraints as outlined above.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Sarferaz, Murray, Athreya, and Zhang before them, to include Zhang’s model deployment to autonomous vehicles in Sarferaz/Murray/Athreya’s model degradation detection method. One would have been motivated to make such a combination in order to have a machine learning model that better identifies dangerous situations causing the vehicle to respond accordingly (Zhang [0038]).
Regarding Claim 2, the combination of Sarferaz/Murray/Athreya/Zhang teaches the elements of claim 1 as outlined above. The combination of Sarferaz/Murray/Athreya/Zhang also teaches:
The method of claim 1, wherein the further training of the first ML model is performed with a second training dataset. (Sarferaz [0010] teaches: "[…] the machine learning model retrained based at least on a second training dataset that includes at least one training sample not included in the first training dataset." Examiner’s note: under BRI, “further training of the first ML model” can be interpreted as retraining the machine learning model.)
Regarding Claim 3, the combination of Sarferaz/Murray/Athreya/Zhang teaches the elements of claim 1 as outlined above. The combination of Sarferaz/Murray/Athreya/Zhang also teaches:
The method of claim 1,wherein analyzing the first ML model based on the convergence of the first ML model comprises:
determining an impact of the first training dataset based on the convergence; and (Murray [0012] teaches: "However, the chosen threshold is arbitrary and can introduce noise when training. In fact, it has been found that, when training aesthetic classification models using thresholded scores as labels, removing training images with scores close to the threshold resulted in faster model convergence and similar test-time performance. Because generative adversarial network training is sensitive to noisy annotations, it is preferable to avoid thresholding." Examiner's note: under BRI, “determining an impact” can be interpreted as determining that removing images with scores close to the threshold results in faster convergence, thus less computation time. Additionally, the “first training dataset” can be reasonably interpreted as Sarferaz’ training dataset.)
identifying discrete portions of the first training dataset having the impact on the convergence, (Examiner’s note: a training image, or a certain number of training images are considered discrete portions of a dataset.)
wherein the report identifies the discrete portions of the first training dataset. (Examiner’s note: similar to Sarferaz’ recommendation, the test-time performance indicates the images that are causing the model to converge slowly during training. A person having ordinary skill in the art would be able to recognize the importance of having models converge faster by identifying training images that are causing the model to converge slowly, thus saving computational time and resources.)
Regarding Claim 5, the combination of Sarferaz/Murray/Athreya/Zhang teaches the elements of claim 3 as outlined above. The combination of Sarferaz/Murray/Athreya/Zhang also teaches:
The method of claim 3, wherein the discrete portions of the first training dataset cause the impact on the convergence based on a volume of data used in the training and an extra compute time, wherein the extra compute time is a compute time greater than a compute time of the previous iteration of training. (Examiner’s note: Murray discloses that images may cause the model to converge slower during training if these images (i.e. discrete portions of the dataset) are not removed. Therefore, these portions of the dataset result in more computational time due to slower convergence. A person having ordinary skill in the art would recognize that the presence of images, or samples, with values close to the threshold cause slow convergence, and the larger the amount of such images in the dataset, the slower the convergence and more compute time would be required during training.)
Regarding Claim 6, the combination of Sarferaz/Murray/Athreya/Zhang teaches the elements of claim 3 as outlined above. The combination of Sarferaz/Murray/Athreya/Zhang also teaches:
The method of claim 3, wherein noise in annotations in the first training dataset causes the impact on the convergence. (Murray [0012] teaches: "However, the chosen threshold is arbitrary and can introduce noise when training. In fact, it has been found that, when training aesthetic classification models using thresholded scores as labels, removing training images with scores close to the threshold resulted in faster model convergence and similar test-time performance. Because generative adversarial network training is sensitive to noisy annotations, it is preferable to avoid thresholding.")
Regarding Claim 7, the combination of Sarferaz/Murray/Athreya/Zhang teaches the elements of claim 1 as outlined above. The combination of Sarferaz/Murray/Athreya/Zhang also teaches:
The method of claim 1, further comprising:
generating second metrics based on the training of the second ML model; and (Sarferaz [0015] teaches: "in response to detecting the degradation of the machine learning model, retraining the machine learning model, the machine learning model retrained based at least on a second training dataset that includes at least one training sample not included in the first training dataset." Additionally, Sarferaz [0048] teaches: "For instance, the controller 110 may retrain the machine learning model to at least increase the predictive power metric and/or the predictive confidence metric of the machine learning model 150 such that the retrained machine learning model 150 is able to assign a correct label to input samples in input datasets having a same characteristic as the second training dataset used to retrain the machine learning model 150." Examiner's note: under BRI, the second ML model may be interpreted as a retrained machine learning model from Sarferaz. The “second metrics” can be interpreted as the metrics generated during retraining using the second dataset.)
comparing the second metrics to first metrics generated during the training of the first ML model to determine that a first scenario in the first ML model is unresolved in the second ML model, (Sarferaz [0011] teaches: “The one or more accuracy key performance indicators may further include a prediction confidence metric measuring an ability of the machine learning model to achieve a same performance for different input datasets having one or more same characteristics as the first training dataset.” Examiner’s note: under BRI “comparing second metrics to first metrics” can be interpreted as measuring the ability of the machine learning model to achieve same performance for different datasets with similar characteristics as the first training dataset. Under BRI, an “unresolved” scenario can be interpreted as the ML model outputting an incorrect label when using the second dataset, thus decreasing the accuracy of the model.)
wherein a scenario is unresolved when the second ML model fails to process the scenario that was successfully processed by the first ML model, (Sarferaz [0037] teaches: "The controller 110 may monitor the performance of the machine learning model 150 including by monitoring the one or more accuracy key performance metrics (KPIs). For example, the controller 110 may monitor the predictive power metric and/or the predictive confidence metric associated with the machine learning model 150. A decrease in the predictive power metric and/or the predictive confidence metric associated with the machine learning model 150 may indicate a change in the relationship between the input samples being received at the machine learning model 150 and the correct labels associated with these input samples. Accordingly, in response to detecting the decrease in the predictive power metric and/or the predictive confidence metric of the machine learning model 150, the controller 110 may retrain the machine learning model 150 based on a training dataset that includes at least one training sample that is not present in the training dataset previously used to train the machine learning model 150." Additionally, Sarferaz [0045] teaches: "At 304, the controller 110 may detect a degradation of the machine learning model. In some example embodiments, the controller 110 may detect the degradation of the machine learning model 150 based at least on one or more accuracy key performance indicators (KPIs) including, for example, a prediction power metric, a prediction confidence metric, and/or the like. The prediction power metric of the machine learning model 150 may measure an ability of the machine learning model 150 to generate, for each input value, a correct output value. Meanwhile, the prediction confidence metric of the machine learning model 150 may an ability of the machine learning model 150 to achieve a same performance across different input datasets having the same characteristics as the first training dataset. A decrease in the predictive power metric and/or the predictive confidence metric associated with the machine learning model 150 may indicate a change in the relationship between the input samples being received at the machine learning model 150 and the correct labels associated with these input samples." Examiner's note: under BRI, “a second ML model” can be interpreted as the result of retraining a first ML model. Moreover, Sarferaz teaches monitoring key performance metrics to detect a decrease in accuracy. Under BRI, a decrease in model accuracy indicates a model's output incorrectly labeling a previously correctly labeled scenario (e.g., resolved and unresolved). Therefore, “a scenario is unresolved when the second ML model fails to process the scenario that was successfully processed by the first ML model" can be interpreted as the ML model outputting an incorrect label (i.e., fails to process) when using the second dataset, thus decreasing the accuracy of the model.)
Regarding Claim 8, the combination of Sarferaz/Murray/Athreya/Zhang teaches the elements of claim 7 as outlined above. The combination of Sarferaz/Murray/Athreya/Zhang also teaches:
The method of claim 7, wherein the first scenario was resolved during the training of the first ML model. (Sarferaz [0010] teaches: “The method may include: training, based at least on a first training dataset, a machine learning model; detecting, based at least on one or more accuracy key performance indicators associated with the machine learning model, a degradation of the machine learning model, the one or more accuracy key performance indicators including a prediction power metric measuring an ability of the machine learning model to generate, for each input value, a correct output value; and in response to detecting the degradation of the machine learning model, retraining the machine learning model, the machine learning model retrained based at least on a second training dataset that includes at least one training sample not included in the first training dataset.” Examiner’s note: under BRI, “the first scenario was resolved during the training of the first ML model” can be interpreted as outputting correct values when training the model with the first training dataset.)
Regarding Claim 12, the combination of Sarferaz/Murray/Athreya/Zhang teaches the elements of claim 1 as outlined above. The combination of Sarferaz/Murray/Athreya/Zhang also teaches:
The method of claim 1, wherein further training the first ML model based on the second configuration comprises:
generating a second training dataset and a third training dataset based on a curriculum for the first ML model to learn; (Zhang [0035] teaches: "The training engine 136 can be configured to generate and/or organize training datasets". Examiner's note: under broadest reasonable interpretation, a "curriculum" is structured training for a machine learning model to learn how to perform a task; Zhang explicitly teaches training a model to detect unsafe/dangerous situations based on training datasets.)
training the first ML model with the second training dataset; and (Zhang [0035] teaches: "The training engine 136 can be configured to generate and/or organize training datasets, and feed such training datasets, to train one or more machine learning models to detect unsafe or dangerous situations. In some embodiments, the one or more machine learning models may further be trained to generate scenarios, such as from dangerous situations, used to test algorithms and/or logic of the vehicle 101." Examiner's note: under BRI, a "curriculum" is structured training for a machine learning model to learn how to perform a task; Zhang explicitly teaches training a model to detect unsafe/dangerous situations based on training datasets.)
after training the first ML model with the second training dataset, (taught above in claim 1)
training the first ML model with the third training dataset, wherein the second training dataset and the third training dataset are generated based on the curriculum for the first ML model to learn. (Zhang [0035] teaches: "The training engine 136 can be configured to generate and/or organize training datasets, and feed such training datasets, to train one or more machine learning models to detect unsafe or dangerous situations. In some embodiments, the one or more machine learning models may further be trained to generate scenarios, such as from dangerous situations, used to test algorithms and/or logic of the vehicle 101." Examiner's note: under BRI, a "curriculum" is structured training for a machine learning model to learn how to perform a task; Zhang explicitly teaches training a model to detect unsafe/dangerous situations based on training datasets (i.e., more than one). Examiner’s note: under BRI, the “first ML model” can be interpreted as the one or more machine learning models that are further trained, which can reasonably be Sarferaz’ machine retrained machine learning model.)
Regarding Claim 13, the combination of Sarferaz/Murray/Athreya/Zhang teaches the elements of claim 12 as outlined above. The combination of Sarferaz/Murray/Athreya/Zhang also teaches:
The method of claim 12, wherein the second training dataset comprises a first scenario to learn first and the third training dataset comprises at least one scenario of the first scenario. (Zhang [0024] teaches: "[…] At least some scenarios used to create the simulations may also be generated from the sensor data frames, so that at least some of the scenarios may be similar or same as portions of the sensor data frames and reflect realistic scenarios." (Zhang [0035] teaches: "The training engine 136 can be configured to generate and/or organize training datasets". Examiner’s note: under BRI, the “third training dataset” can be interpreted as the generated dataset from the Zhang reference.)
Regarding Claim 18, the combination of Sarferaz/Murray/Athreya/Zhang teaches the elements of claim 1 as outlined above. The combination of Sarferaz/Murray/Athreya/Zhang also teaches:
The method of claim 1,wherein the at least one constraint further comprises at least one of a […]
model information, (Athreya [0061] teaches: "For example, the processor 304 may modify the machine learning model structure to reduce the complexity, processing load, and/or power consumption of the machine learning model structure while maintaining (e.g., satisfying) a target accuracy".)
evaluation information, (Sarferaz [0011] teaches: “The one or more accuracy key performance indicators may further include a prediction confidence metric measuring an ability of the machine learning model to achieve a same performance for different input datasets having one or more same characteristics as the first training dataset.”)
or optimization information. (Sarferaz [0008] teaches: “The machine learning model may be trained and retrained by at least minimizing the error in the output of the machine learning model. The error in the output of the machine learning model may be minimized by at least adjusting one or more weights applied by the machine learning model until a gradient of an error function associated with the machine learning model converges to a threshold value.” Examiner’s note: under BRI, “optimization information” can be interpreted as the error in the output to determine if the error of the machine learning model has been minimized.)
Regarding Claim 19, the claim recites similar limitations as corresponding claim 1 and is rejected for similar reasons as claim 1 using similar teachings and rationale. Additionally, the combination of Sarferaz/Murray/Athreya/Zhang also teaches:
A system comprising:
one or more processors; and (Sarferaz [0015] teaches: "[…] at least one data processor.")
at least one non-transitory computer-readable medium having stored thereon instructions that, when executed by the one or more processors, cause the one or more processors to: (Sarferaz [0015] teaches: "In another aspect, there is provided a computer program product that includes a non-transitory computer readable storage medium. The non-transitory computer-readable storage medium may include program code that causes operations when executed by at least one data processor.")
Regarding Claim 20, the combination of Sarferaz/Murray/Athreya/Zhang teaches the limitations of claim 19 as outlined above. Additionally, the claim recites similar limitations as corresponding claims 2 and 3 and is rejected for similar reasons as claims 2 and 3 using similar teachings and rationale.
Claims 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Sarferaz in view of Murray, Athreya, and Zhang as applied above in claim 1, and further in the view of Ji (US 20220366263 A1), hereafter Ji.
Regarding Claim 9, the combination of Sarferaz/Murray/Athreya/Zhang teaches the elements of claim 1 as outlined above. However, the combination of Sarferaz/Murray/Athreya/Zhang does not teach, but Ji teaches:
The method of claim 1, further comprising:
training a third ML model using the second ML model as a teacher model in a teacher-student distillation process, the training based on a compute budget associated with the autonomous vehicle with a second training dataset, wherein the compute budget comprises computational constraints of the local computing device of the autonomous vehicle. (Ji [0012] teaches: "A distilled student machine learning model that is easier to deploy than a cumbersome teacher machine learning model, i.e., because it requires less computation, memory, or both, to generate outputs at run time than the cumbersome teacher machine learning model, can be trained using the cumbersome teacher model that has already been trained" Ji [0085] teaches: "Thus, the student model, once trained, is feasible for deployment on a device with limited computational power or resources, e.g., the on-board system of a vehicle, to compute inferences at run time." Examiner's note: under BRI, the "third ML model" can be interpreted as the student model, and the "second ML model as a teacher model" can be interpreted as Sarferaz' retrained model.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Sarferaz/Murray/Athreya/Zhang/Ji before them, to include Ji's distillation training in Sarferaz/Murray/Athreya/Zhang's model degradation detection method. One would have been motivated to make such a combination in order to train models suitable for deployment on a device with limited computational power or resources (Ji [0012]).
Regarding Claim 10, the combination of Sarferaz/Murray/Athreya/Zhang/Ji teaches the elements of claim 9 as outlined above. The combination of Sarferaz/Murray/Athreya/Zhang/Ji also teaches:
The method of claim 9, wherein the training of the third ML model comprises at least one:
training the third ML model with a second testing dataset different from the first testing dataset; and (Examiner's note: under BRI, the "third ML model" can be interpreted as the Ji's student model. Ji [Abstract] also discusses using second training data (i.e., different from the first training dataset) for the training the student model. A person having ordinary skill in the art would recognize that both Sarferaz and Ji use different datasets, Sarferaz during retraining and Ji during training the student model via distillation. Therefore, “training the third model” can be interpreted as training the student model using a second training dataset, such as the one from the Sarferaz reference.)
training the third ML model with a different architecture than the second ML model (Ji [0024] teaches: "Generally, the student machine learning model is a model that has a different architecture from the teacher machine learning model that makes it easier to deploy than the teacher machine learning model, e.g., because the student machine learning model requires less computation, memory, or both, to generate outputs at run time than the teacher machine learning model". Examiner’s note: under BRI, the teacher machine learning model can be interpreted as Sarferaz’ retrained model using the second training dataset.)
Claims 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Sarferaz in view of Murray, Athreya, and Zhang as applied above in claim 1, and further in the view of Wang ("Risky Driver Recognition with Class Imbalance Data and Automated Machine Learning Framework"), hereafter Wang.
Regarding Claim 14, the combination of Sarferaz/Murray/Athreya/Zhang teaches the elements of claim 1 as outlined above. The combination of Sarferaz/Murray/Athreya/Zhang also teaches:
The method of claim 1,wherein further training the first ML model based on the second configuration comprises:
identifying at least one annotation in the first training dataset set to emphasize wherein to emphasize the at least one annotation comprises (Zhang [0034] teaches: "The data synchronization engine 134 can be configured to associate the disengagement data 120 and sensor data such as point cloud data frames generated by the data fusion engine 132 to indicate whether the vehicle 100 is disengaged from the autonomous mode at a time of capture of each sensor data frame." Additionally, Zhang [0035] teaches: "[…] One example of the juxtaposition of a first training dataset that includes data corresponding to a safe situation and a second training dataset that includes data corresponding to an unsafe situation occurs when one training dataset includes a frame or a series of frames from a particular location determined or labeled as safe and a second training dataset includes a frame or a series of frames from that same particular location determined or labeled as unsafe. […] By juxtaposing the training datasets, in which at least one factor, such as a time range or a location, is kept constant or similar among the pair of training datasets, such as a time range or a location, the machine learning model may determine or infer whether, and to what extent, any of the differences between the juxtaposed data frames cause a situation to be unsafe. For example, if a difference between a situation determined to be safe and another situation determined to be unsafe at a same location, and/or having similar traffic density, is that the situation determined to be safe was obtained during daytime and the other situation determined to be unsafe was at nighttime, the machine learning model may infer that a lack of lighting causes a situation to be unsafe. By using multiple juxtaposed training datasets, the machine learning model may further isolate, refine and/or confirm parameters and/or factors used to determine a level of safety of a situation." Examiner's note: under BRI, “annotations to emphasize” can be interpreted as data that was labeled safe in training using the first training dataset, but then labeled unsafe in retraining using the second training dataset for data frames from the same location and/or similar traffic conditions. Additionally, “to emphasize” can be interpreted as isolating, refining and/or confirming parameters and/or factors used to determine the level of safety of a situation.)
training the first ML model with a second training dataset based on the identification of annotations to emphasize. (Zhang [0035] teaches: "The training engine 136 can be configured to generate and/or organize training datasets, and feed such training datasets, to train one or more machine learning models to detect unsafe or dangerous situations. In some embodiments, the one or more machine learning models may further be trained to generate scenarios, such as from dangerous situations, used to test algorithms and/or logic of the vehicle 101. Zhang [0047] teaches: “in a neural network, weights and/or biases of sigmoid functions in each of the perceptrons may be adjusted based on an accuracy, precision, and/or recall.”)
However, the combination of Sarferaz/Murray/Athreya/Zhang is not relied upon for teaching, but Wang teaches: causing a model trainer to increase weights associated with the at least one annotation relative to other annotations; and (Wang [pg. 7, section 3.3. Sampling Methods] teaches: "The sampling method partially balances data before classifier training, and cost-sensitive XGBoost is trained with minority examples’ weight increased. With hyperparameter optimization, we can optimize the sampling rate and example weights and create a more flexible way to handle imbalanced data (i.e., one annotation relative to other annotations).” Examiner’s note: under BRI, “a model trainer” can be interpreted as the hyperparameter optimization which optimizes (i.e., increases) the weights in order to be more flexible when handling imbalanced data. Additionally, “at least one annotation relative to other annotations” can be interpreted as the imbalanced data, which is caused by the having a minority class relative to the majority class in the data.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Sarferaz, Murray, Athreya, Zhang and Wang before them, to include Wang's weight increasing in Sarferaz, Murray, Athreya, and Zhang’s model degradation detection method. One would have been motivated to make such a combination in order to create a more flexible way to handle imbalanced data when training machine learning models (Wang [pg. 7, section 3.3. Sampling Methods]).
Regarding Claim 15, the combination of Sarferaz/Murray/Athreya/Zhang/Wang teaches the elements of claim 14 as outlined above. The combination of Sarferaz/Murray/Athreya/Zhang/Wang also teaches:
The method of claim 14, further comprising:
generating a second training dataset from the first training dataset based on the identification of the annotations to emphasize. (Zhang [0024] teaches: "[…] At least some scenarios used to create the simulations may also be generated from the sensor data frames, so that at least some of the scenarios may be similar or same as portions of the sensor data frames and reflect realistic scenarios." Zhang [0035] teaches: " The training engine 136 can be configured to generate and/or organize training datasets". […] By juxtaposing the training datasets, in which at least one factor, such as a time range or a location, is kept constant or similar among the pair of training datasets, such as a time range or a location, the machine learning model may determine or infer whether, and to what extent, any of the differences between the juxtaposed data frames cause a situation to be unsafe.” Examiner’s note: under BRI, the “second training dataset” can be interpreted as the generated dataset from the Zhang reference, and the “first training dataset” can be interpreted as the first training dataset from Sarferaz. Furthermore, the Zhang reference teaches that juxtaposing datasets when a situation has been determined safe helps the machine learning model determine the differences in frames that are causing the situation to be unsafe.)
Regarding Claim 16, the combination of Sarferaz/Murray/Athreya/Zhang/Wang teaches the elements of claim 14 as outlined above. The combination of Sarferaz/Murray/Athreya/Zhang/Wang also teaches:
The method of claim 14, wherein a ML model trainer that performs each iteration of the further training receives the identification of the annotations to emphasize. (Zhang [0021] teaches: "FIG. 13 illustrates a computing component that includes one or more hardware processors and machine-readable storage media storing a set of machine-readable/machine-executable instructions that, when executed, cause the hardware processor(s) to perform an illustrative method for training a machine learning model, according to an embodiment of the present disclosure." Examiner’s note: under BRI, “a model trainer” can be interpreted as the computing component that executes the instructions for training the machine learning model. Additionally, Zhang uses the juxtaposing of training datasets during machine learning model training to determine what is causing a scenario to be unsafe when it was previously determined to be safe.)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
ZAREMBA (US 20210356968 A1) relates to training machine learning models with various traffic scenarios.
CHOI (US 20180268292 A1) relates to knowledge distillation.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Alvaro S Laham Bauzo whose telephone number is (571)272-5650. The examiner can normally be reached Mon-Fri 7:30 AM - 11:00 AM | 1:00 PM - 5:30 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew J. Jung can be reached on (571) 270-3779. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.S.L./Examiner, Art Unit 2146
/ANDREW J JUNG/Supervisory Patent Examiner, Art Unit 2146