DETAILED ACTION
This action is responsive to the application filed on 01/12/2026. Claims 1-5, 9-24 are pending and have been examined. This action is Non-Final.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C.
120, 121, 365(c), or 386(c) is acknowledged.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 01/12/2026 has been entered.
Response to Arguments
Argument 1: The applicant argues (see pages 9-11) that the claims are not directed to an abstract idea but instead recite a specific technological improvement to machine learning systems, particularly through the use of uncertainty values during training to selectively truncate processing and redirect certain inputs. They contend that these operations improve model accuracy and reduce computational resource consumption by preventing low-certainty inputs from being processed unnecessarily and instead routing them to an external entity. The applicant emphasizes that this is not a mental process or mere mathematical concept, but rather a practical application that improves how the machine learning model operates, including efficiency and development time, and therefore integrates any alleged abstract idea into a practical application and amounts to significantly more than routine or conventional activity.
Examiner Response to Argument 1: The applicant’s argument is not persuasive because the claims, as presently drafted in light of amendments, remain directed to abstract concepts involving evaluation and conditional decision-making based on data, rather than a specific technological implementation that integrates the exception into a practical application. As set forth in the rejection, the claim recites “determining…uncertainty values,” “segregating…based on the uncertainty values,” and “truncating processing…in response to determining… uncertainty values that are greater than a sufficiency threshold,” which amount to evaluating information and making threshold-based decisions, i.e., mental processes. The additional limitations do not change that conclusion. The recitation of “at least one processor” and “at least one memory” merely applies the abstract idea on generic computer components in a routine manner, and the recitation of “resulting in improved accuracy…and reduced consumption of computing resources” states a desired outcome rather than a particular technological mechanism for achieving that outcome. Likewise, “redirecting the processing to an entity not associated with the machine learning system” merely limits the environment in which the abstract decision-making is used and does not integrate the exception into a practical application. Accordingly, the claims do not recite significantly more than the abstract idea and remain non-patent eligible for the reasons already set forth in the rejection, and thus the rejection is maintained.
Argument 2: The applicant argues (see pages 11-14) that the cited references fail to teach or suggest the claimed combination of truncating processing during training based on uncertainty values and redirecting processing to an external entity. Specifically, Kung’s BranchyNet is said to allow early exiting only when the model is confident (i.e., low uncertainty), which is the opposite of the claimed approach that truncates processing for high-uncertainty inputs, and Kung does not disclose redirecting processing after exiting. Chen similarly fails to remedy these deficiencies. Geifman and Wegkamp are argued to relate to reject-option frameworks where predictions may be withheld, but they do not disclose performing additional actions such as redirecting processing to another entity after rejection. The applicant therefore asserts that the references, alone or in combination, do not teach the claimed training-time control and redirection mechanism, and dependent claims are patentable for at least the same reasons.
Examiner Response to Argument 2: The applicant’s argument is not persuasive. The rejection does not rely on Kung alone to teach the full claimed workflow. Rather, Kung is relied upon for threshold-based uncertainty routing and truncation, including that “the sample exits the network with the prediction result at this exit point, and is not processed by the higher network layers” and that “If the entropy value is above the threshold, then the classifier at this exit point is deemed not confident,” which correspond to the claimed segregation and truncation based on uncertainty values and a threshold. Chen is relied upon for updating the machine learning model and penalty-related loss values using batch-derived training information, including “we update our loss weights wi(t) to move gradient norms towards this target for each task” and “tuning loss weights in a multi-task learning setting.” Kang is expressly relied upon for the claimed redirection to an external entity, teaching that “NSmobile sends the output of that layer from the mobile device to NSserver residing on the server side. NSserver then executes the remaining DNN layers,” which is not merely exiting or abstaining, but an explicit transfer of ongoing processing to a separate entity where continued execution occurs. Kang also teaches the claimed efficiency benefits, including that the approach “reduces mobile energy consumption” and “improves datacenter throughput.” Thus, the cited references, when considered in the manner set forth in the rejection, do teach or at least render obvious the claimed combination of uncertainty-based truncation during training, redirection to an external entity, and associated performance/resource improvements, and the rejections are maintained.
Argument 3: The applicant argues (see pages 15-18) that Ziyin and Cortes disclose abstention or reject-option techniques but do not teach or suggest redirecting processing to an external entity after abstaining, nor do they disclose the claimed training-time truncation based on uncertainty thresholds. For additional references such as Lin, the applicant acknowledges iterative optimization techniques but argues that Lin does not teach the specific process of setting an adjustable penalty value based on a mean loss and repeatedly updating a loss vector across batches until stabilization according to a defined criterion. Accordingly, the applicant maintains that the cited combinations fail to disclose the structured training procedure and specific integration of uncertainty-based truncation, redirection, and iterative loss updating as claimed, and therefore do not render the claims obvious.
Examiner Response to Argument 3: The applicant’s argument is not persuasive because it again addresses references in isolation rather than the combined teachings applied in the rejection. Ziyin is relied upon for determining uncertainty values corresponding to model outputs and for adapting the model based on those uncertainty values, including that “We call g(x) the uncertainty score of x” and that the selection function corresponds to an output of the model. Cortes is relied upon for updating a penalty value based on batch-level loss information, including selecting α based on a batch-averaged objective. Kung is relied upon for threshold-based truncation of processing, including that selected samples “are not processed by the higher network layers,” and Kang is relied upon for redirecting processing externally, including that “NSserver then executes the remaining DNN layers.” For claim 24 and the stabilization aspect, Han is relied upon for the iterative update continuing “until convergence,” which corresponds to the claimed repetition until the adjustable penalty value is stabilized according to a defined stability criterion. Accordingly, the rejection does not depend on any single reference teaching all claimed features, but instead on the combined teachings of references that each address complementary aspects of the claimed training procedure. The rejection also sets forth articulated reasons why a person of ordinary skill in the art would have combined these teachings, namely to improve training effectiveness, convergence behavior, computational efficiency, and resource utilization. Therefore, the cited combinations do teach or render obvious the claimed structured integration of uncertainty-based truncation, redirection, and iterative loss updating, and the rejection maintains.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition
of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the
conditions and requirements of this title.
Claims 1-5, 9-24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1,Step 1: The claim is directed to a device, which is in the category of a machine. The claim satisfies step 1.
Step 2A Prong 1:(a) “determining, based on first inputs to a machine learning system, uncertainty values corresponding to outputs of the machine learning system;” – The limitation is directed to determining a value that corresponds to an output based on inputs to a computer system. Under a broadest reasonable interpretation, determining and assigning such values can be performed using observation, evaluation, and judgment by the human mind with aid of pen and paper, and thus it is directed to a mental process.
(b) “segregating, during training of the machine learning system, the outputs of the machine learning system into a first portion of the outputs and a second portion of the outputs based on the uncertainty values, the segregating of the outputs comprising truncating processing, using the machine learning system,” -- The limitation is directed to separating or classifying outputs by a threshold and deciding how the outputs are grouped. This can be performed in the human mind using evaluation, observation, and judgment, with aid of pen and paper, and hence is directed to a mental process.
(c) “truncating processing, using the machine learning system, of respective ones of the first inputs corresponding to the second portion of the outputs…in response to determining that the respective ones of the first inputs correspond to respective ones of the uncertainty values that are greater than a sufficiency threshold.”-- The limitation is directed to deciding whether to continue or stop processing based on a threshold, particularly greater than. Such conditional decision-making can be performed in the human mind using evaluation and judgment, and therefore is directed to a mental process.
Step 2A Prong 2 and Step 2B:
“A device, comprising: at least one processor; and at least one memory that stores executable instructions that, when executed by the at least one processor, facilitate performance of operations, comprising” -- The limitation recites mere instructions to apply operations on a generic computer in a routine manner. It does not integrate the judicial exception into a practical application and does not provide significantly more than the judicial exception (see MPEP 2106.05(f)).
“updating a machine learning model employed by the machine learning system based on the uncertainty values, and a loss function, the loss function being a function of at least a conventional loss function and an adjustable penalty value, wherein updating the machine learning model comprises updating the adjustable penalty value based on a mean value of the conventional loss function as applied to a batch of second inputs processed before the first inputs;” -- The limitation recites updating a model based on calculated values and adjusting a penalty term using a mean of prior loss results. Merely updating data or parameters and performing mathematical calculations are insignificant extra-solution activities that do not integrate the judicial exception into a practical application. Under step 2B, computing means, setting thresholds, and tuning parameters in model training are well-understood, routine, and conventional activities that do not amount to significantly more than the abstract idea.
“resulting in improved accuracy of the machine learning model and reduced consumption of computing resources by the machine learning system compared to operation of the machine learning system absent the truncating and the redirecting” -- The limitation recites a desired result of the claimed operations, namely improving accuracy and reducing computational resource usage, without reciting a specific technological mechanism for achieving such improvements. This constitutes a statement of intended results and functional language describing an outcome of the abstract idea rather than a particular implementation, and does not integrate the judicial exception into a practical application, (see MPEP 2106.05(g); see also MPEP 2111.04). Under Step 2B, the limitation does not provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
“and redirecting the processing to an entity not associated with the machine learning system” -- The limitation recites redirecting the processing to an associated that is not associated with the ML system. The limitation amounts to no more than mere further limiting to a field of use/environment, and it does integrate to a practical application, nor provide significantly more than the judicial exception (see MPEP 2106.05(h)).
Therefore, claim 1 is non-patent eligible. Claims 11 and 16 are analogous to claim 1, aside from claim type, and thus the same rejection can be applied.
Regarding claim 2,Step 1: The claim is directed to a device, which is in the category of a machine. The claim satisfies step 1.
Step 2A Prong 1:(a) “wherein the loss function facilitates determining a loss vector based on results of the conventional loss function and the uncertainty value.” – The limitation is directed to forming a vector using results of a loss function and an uncertainty value. Determining such a vector is a mathematical calculation that can be performed using pen and paper by a human and thus is directed to a mathematical concept and a mental process.There are no elements to be evaluated under Step 2A and Prong 2B.
Therefore, claim 2 is non-patent eligible.
Regarding claim 4,Step 1: The claim is directed to a device, which is in the category of a machine. The claim satisfies step 1.Step 2A Prong 1:(a) “The device of claim 2, wherein the loss function is Lnew(xj)=(1-puncert)Lold(xj)+puncertCpenalty, wherein puncert is the uncertainty value, wherein Lold(xj) are the results of the conventional loss function, and wherein Cpenalty is the adjustable penalty value.” – The limitation is directed to the loss function and describes the different elements of the loss function and how it pertains to different values and results. The limitation is directed to the use of a mathematical operation, and therefore the limitation is directed to math.There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.Therefore, claim 4 is non-patent eligible.
Regarding claim 5,Step 1: The claim is directed to a device, which is in the category of a machine. The claim satisfies step 1.Step 2A Prong 1:(a) “The device of claim 2, wherein the adjustable penalty value, in a first iteration of evaluating the loss function, is initially set to at least a defined value, resulting in the adjustable penalty value initially dominating the loss function.” – The limitation is directed to setting values to a defined value that will result in the adjustable penalty function to overcome the loss function, for which can be performed in a mental process.There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.Therefore, claim 5 is non-patent eligible.
Regarding claim 9,Step 1: The claim is directed to a device, which is in the category of a machine. The claim satisfies step 1.
Step 2A Prong 1:(a) “The device of claim 1, wherein the updating of the adjustable penalty value comprises restricting the adjustable penalty value from traversing a floor penalty value.” – The limitation recites updating the penalty value which comprises of restricting the adjustable penalty value from traversing the floor penalty value. The limitation is directed to mathematical concept/relationship, and thus the claim is directed to math.
There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.
Therefore, claim 9 is non-patent eligible.
Regarding claim 10,Step 1: The claim is directed to a device, which is in the category of a machine. The claim satisfies step 1.Step 2A Prong 1:(a) “wherein the indicator value is based on a mean of the uncertainty values comprising a respective uncertainty value in each iteration of the evaluating the loss function” – The limitation is directed to computing a mean of uncertainty values for the indicator value that will comprise a respective uncertainty value for each iteration. The limitation is directed to a mathematical concept, and thus it is directed to math.(b) “and wherein the hyperparameter is less than 1.” – The limitation is directed to comparing a value to a pre-set value to 1. Determining if a hyperparameter value is less than 1 is a process that can be done using evaluation, observation, and judgement of the human mind, and thus the limitation is directed to a mental process.Step 2A Prong 2 and Step 2B:(a) “The device of claim 1, wherein the adjustable penalty value is iteratively updated based on a hyperparameter to cause an indicator value to converge to a target value across iterations of evaluating the loss function” – The limitation is directed to iterative updating a value based on a hyperparameter to get an indicator value to converge to a target value. The limitation is directed to merely updating gathered data, for which is an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, the act of updating data on a repeated basis is considered both a repetitive calculation and electronic recordkeeping, both for which are well-understood, routine, and conventional activity (WURC) that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).(b) “wherein the target value is selectable based on user input received via a user interface, “ – This limitation is directed to having a selectable target value based on user input that is received by the user interface. The limitation is directed to an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under step 2B, the act of transmitting (receiving/sending) data over a network is a well-understood, routine, and conventional activity (WURC) that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).Therefore, claim 10 is non-patent eligible.
Regarding claim 12,Step 1: The claim is directed to a method, which is in the category of a process. The claim satisfies step 1.Step 2A Prong 1:(a) “wherein the loss function is Lnew(xj)=(1-puncert)Lold(xj)+puncertCpenalty, wherein puncert is an uncertainty value of the uncertainty values, wherein Lold(xj) are second results of the conventional loss function, and wherein Cpenalty is the penalty value.” – The limitation is directed to further describing the loss function that will be used to increase efficacy is calculated using a mathematical operation, and thus the limitation is directed to math.Step 2A Prong 2 and Step 2B:(a) “The method of claim 11, wherein the updating of the penalty value and the adapting of the machine learning model increase an efficacy of the loss function according to a defined performance criterion” – The limitation recites updating the penalty value and adapting (also considered updating) the model to increase loss function efficacy. Updating values and a ML model is directed to an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 21106.05(g)). Furthermore, under Step 2B, the act of updating values is directed to electronic recordkeeping, and it is a well-understood, routine, and conventional activity that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).Therefore, claim 12 is non-patent eligible.
Regarding claim 13,Step 1: The claim is directed to a method, which is in the category of a process. The claim satisfies step 1.Step 2A Prong 1:(a) “The method of claim 11, wherein the updating of the penalty value comprises evaluating the formula Cpenalty=mean(Lold(xbatch_j)), and wherein Lold(xbatch_j) are the first results of the conventional loss function.” – The limitation is directed to the updating of the penalty value will involve evaluating a mathematical operation, and that part of that operation will be considered the first results of the conventional loss function. The limitation is directed to math.There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.Therefore, claim 13 is non-patent eligible.
Regarding claim 14,Step 1: The claim is directed to a method, which is in the category of a process. The claim satisfies step 1.Step 2A Prong 1:(a) “The method of claim 13, wherein the updating of the penalty value comprises preventing the penalty value from dropping below a minimum value according to the formula Cpenalty=max(Cpenalty, Cmin) wherein Cmin is the minimum value.” – The limitation directed to the updating of the penalty value that will further comprise preventing the penalty value from dropping below a thresholded minimum value according to the formula provided in the claim. The limitation is directed to the use of a mathematical calculation/relationship, and thus it is directed to math.There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.Therefore, claim 14 is non-patent eligible.
Regarding claim 15,Step 1: The claim is directed to a method, which is in the category of a process. The claim satisfies step 1.Step 2A Prong 1:(a) “The method of claim 11, further comprising tuning…the penalty value based on a mean of previous uncertainty values” – The limitation is directed to tuning penalty values based on the mean of previous uncertainty values, which is considered to be directed to math.(b) “wherein the tuning comprises incrementing the penalty value by a first amount that is less than 1 in response to the mean of the previous uncertainty values being less than a target value, and wherein the tuning comprises decrementing the penalty value by a second amount that is less than 1 in response to the mean of the previous uncertainty values being greater than the target value.” – The limitation is directed to the tuning process includes adjusting the penalty value incrementally or decrementally based on a comparison to a target value. This limitation can be performed using evaluation, observation, and judgement, using pen and paper, to complete. Thus, the limitation is directed to a mental process.Step 2A Prong 2 and Step 2B:(a) “by the system” – The limitation merely states that the tasks are going to be performed (applied onto) these computers, and thus do not integrate to a practical application, nor provide significantly more than the judicial exception (see MPEP 2106.05(f)).Therefore, claim 15 is non-patent eligible.
Regarding claim 17,Step 1: The claim is directed to a non-transitory computer-readable medium, which is in the category of a manufacture. The claim satisfies step 1.There are no elements to be evaluated under Step 2A Prong 1.Step 2A Prong 2 and Step 2B:(b) “The non-transitory machine-readable storage medium of claim 16, wherein the updating of the penalty value improves an optimization of the loss function according to a defined performance criterion.” – The limitation recites updating the penalty value to improve the optimization of the loss function. This limitation is directed to an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, the act of updating a value to improve optimization is a well-understood, routine, and conventional activity (WURC) that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).Therefore, claim 17 is non-patent eligible.
Regarding claim 18,Step 1: The claim is directed to a non-transitory computer-readable medium, which is in the category of a manufacture. The claim satisfies step 1.Step 2A Prong 1:(a) “The non-transitory machine-readable storage medium of claim 17, wherein the loss function is Lnew(xj)=(1-puncert)Lold(xj)+puncertCpenalty, wherein puncert is an uncertainty value of the uncertainty values, wherein Lold(xj) are current results of the conventional loss function, and wherein Cpenalty is the penalty value.” – The limitation is similar to one that was recited in claim 12. The limitation can still be analyzed and rejected in the same manner as claim 12. The limitation is directed to further describing the loss function that will be used to increase efficacy is calculated using a mathematical operation, and thus the limitation is directed to math.There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.Therefore, claim 18 is non-patent eligible.
Regarding claim 19,Step 1: The claim is directed to a non-transitory computer-readable medium, which is in the category of a manufacture. The claim satisfies step 1.Step 2A Prong 1:(a) “The non-transitory machine-readable medium of claim 16, wherein the updating of the penalty value comprises evaluating the formula Cpenalty=max(mean(Lold(xbatch_j)), Cmin), wherein Cpenalty is the penalty value, wherein Lold(xbatch_j) are the previous results of the conventional loss function, and wherein Cmin is a minimum allowable penalty value.” – The limitation is directed to the process of updating the penalty for which would involve the mathematical operation formula set forth above, which is directed to the use of math.There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.Therefore, claim 19 is non-patent eligible.
Regarding claim 20,Step 1: The claim is directed to a non-transitory computer-readable medium, which is in the category of a manufacture. The claim satisfies step 1.Step 2A Prong 1:(a) “The non-transitory machine-readable medium of claim 16, wherein the operations further comprise adjusting the penalty value based on a mean of previous uncertainty values, wherein the adjusting comprises incrementing the penalty value by a first amount that is less than 1 in response to the mean of the previous uncertainty values being less than a target value, and wherein the adjusting comprises decrementing the penalty value by a second amount that is less than 1 in response to the mean of the previous uncertainty values being greater than the target value.” – The limitation is directed to the operations will further comprise adjusting the penalty values based on a mean of previous uncertainty values, and incrementing/decrementing to an amount less than 1 depending on the mean of the previous uncertainty value to be less than/greater than the target value. Adjusting penalty values based on math of other values (uncertainty values) through increments then comparing the result is directed to a mathematical calculation/relationship, and thus the limitation is directed to math.There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.Therefore, claim 20 is non-patent eligible.
Regarding claim 21,Step 1: The claim is directed to a device, which is in the category of a machine. The claim satisfies step 1.Step 2A Prong 1:(a) “The device of claim 1, wherein the first inputs correspond to incoming calls to a call center, and wherein the operations further comprise: routing an incoming call, of the incoming calls and corresponding to a first input of the first inputs, to a device associated with an operator of the call center based on determining that the first input corresponds to one of the first portion of the outputs; and routing the incoming call to another device associated with a human expert and other than the device, instead of to any device of any operator of the call center, for further action based on determining that the first input corresponds to one of the second portion of the outputs.” – The limitation is directed to inputs that correspond to a call center, where the operations will comprise of routing calls to a device associated with an operator, and routing calls to a human expert and to any operator of the call center based in determination on which input corresponds to a second portion of outputs. The limitation is directed to a process that can be performed in the human mind using evaluation, observation, and judgement (with aid of pen and paper), and thus the limitation is directed to a mental process.There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.Therefore, claim 21 is non-patent eligible.
Regarding claim 22,Step 1: The claim is directed to a device, which is in the category of a machine. The claim satisfies step 1.There are no elements to be evaluated under Step 2A Prong 1.Step 2A Prong 2 and Step 2B:(a) “The device of claim 2, wherein the operations further comprise: repeating the updating of the machine learning model until the loss vector and the adjustable penalty value are stabilized according to a defined stability criterion.” – The limitation recites that the operations will comprise repetitive updating of the model until the loss vector and penalty value is stabilized. The limitation recites an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, the act of repetitively updating a model is a well-understood, routine, and conventional activity (WURC) that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
Therefore, claim 22 is non-patent eligible.
Regarding claim 23,Step 1: The claim is directed to a device, which is in the category of a machine. The claim satisfies step 1.
Step 2A Prong 1:(a) “The device of claim 22, wherein the operations further comprise: in response to the adjustable penalty value being determined to be stabilized according to the defined stability criterion” – The limitation is directed to determining stabilization of a value with respect to a criterion is an evaluative/observe/judgement of a mathematical assessment (convergence testing), and thus is directed to a mathematical concept and a mental process.(b) “modifying the adjustable penalty value by an incremental value based on a difference between loss values associated with the loss vector and a defined preferred loss value.” – The limitation is directed to modifying a value by computing a difference between loss values and incrementally adjusting/changing the value of a penalty term, which is directed to mathematical calculation/relationship, and thus the limitation is directed to math.
There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.
Therefore, claim 23 is non-patent-eligible.
Regarding claim 24,Step 1: The claim is directed to a device, which is in the category of a machine. The claim satisfies step 1.
Step 2A Prong 1:
“setting the adjustable penalty value to a mean value of the conventional loss function” -- The limitation recites setting the adjustable penalty value to a mean value of the loss function. The limitation is directed to the use of mathematical operations/relationships/calculation, and thus the limitation is directed to math.
Step 2A Prong 2 and Step 2B:
“as applied to a first batch of the second inputs processed before the first inputs” -- The limitation recites applying a first batch of inputs that were processed (by the model) before the first inputs. The limitation amounts to no more than mere instructions to apply onto a computer, and does not integrate to practical application, nor provide significantly more than the judicial exception (see MPEP 2106.05(f)).
“updating the loss vector based on the adjustable penalty value; and repeating the setting of the adjustable penalty value and the updating of the loss vector for other batches of the second inputs, other than the first batch, until the adjustable penalty value is stabilized according to a defined stability criterion.” --The limitation is directed to updating the loss vector based on an adjustable value, and repeatedly setting the value and the updating the loss vector until the penalty value is stabilized. The limitation is directed to an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, the act of updating/repetitive updating until a criteria is met is a well-understood, routine, and conventional activity (WURC) that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
Therefore, claim 24 is non-patent-eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this
Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not
identically disclosed as set forth in section 102, if the differences between the claimed invention and the
prior art are such that the claimed invention as a whole would have been obvious before the effective filing
date of the claimed invention to a person having ordinary skill in the art to which the claimed invention
pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are
summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over NPL reference “BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks”, by Kung et. al. (referred herein as Kung) in view of NPL reference “GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks” by Chen et. al. (referred herein as Chen) further in view of NPL reference “Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge”, by Kang et. al. (referred herein as Kang).
Regarding claim 1, Kung teaches:
A device, comprising: at least one processor; and at least one memory that stores executable instructions that, when executed by the at least one processor, facilitate performance of operations, comprising: determining, based on first inputs to a machine learning system, uncertainty values corresponding to outputs of the machine learning system; ([Kung, page 3] “Let y be a one-hot ground-truth label vector, x be an input sample and C be the set of all possible labels. The objective function can be written as…where fexitn is the output of the n-th exit branch and θ represents the parameters of the layers from an entry point to the exit point”, wherein the examiner interprets “input sample and C be the set of all possible labels. The objective function can be written as…where fexitn is the output of the n-th exit branch and θ represents the parameters of the layers” to be the same as determining values based on inputs within a ML system.)
segregating, during training of the machine learning system, the outputs of the machine learning system into a first portion of the outputs and a second portion of the outputs based on the uncertainty values, ([Kung, page 2] “If the entropy of a test sample is below a learned threshold value, meaning that the classifier is confident in the prediction, the sample exits the network with the prediction result at this exit point, and is not processed by the higher network layers. If the entropy value is above the threshold, then the classifier at this exit point is deemed not confident, and the sample continues to the next exit point in the network. If the sample reaches the last exit point, which is the last layer of the baseline neural network, it always performs classification.”, wherein the examiner interprets “If the entropy of a test sample is below a learned threshold value” and “If the entropy value is above the threshold” to be the same as segregating, during training of the machine learning system, the outputs of the machine learning system into a first portion of the outputs and a second portion of the outputs based on the uncertainty values because they are both directed to separating outputs into different groups according to whether a thresholded uncertainty/confidence condition is satisfied.)
the segregating of the outputs comprising truncating processing, using the machine learning system, of respective ones of the first inputs corresponding to the second portion of the outputs ([Kung, page 2] “the sample exits the network with the prediction result at this exit point, and is not processed by the higher network layers.”, wherein the examiner interprets “the sample exits the network with the prediction result at this exit point, and is not processed by the higher network layers” to be the same as the segregating of the outputs comprising truncating processing, using the machine learning system, of respective ones of the first inputs corresponding to the second portion of the outputs because they are both directed to halting further machine-learning processing for a subset of inputs once the threshold-based routing decision has been made.)
in response to determining that the respective ones of the first inputs correspond to respective ones of the uncertainty values that are greater than a sufficiency threshold, ([Kung, page 2] “If the entropy value is above the threshold, then the classifier at this exit point is deemed not confident, and the sample continues to the next exit point in the network.”, wherein the examiner interprets “entropy value is above the threshold” to be the same as uncertainty values that are greater than a sufficiency threshold because they are both directed to comparing an uncertainty/confidence measure against a threshold in order to determine how processing should proceed for a given input.)
resulting in improved accuracy of the machine learning model … ([Kung, page 2, section 3] “In training the classifiers at these exit branches, we also consider network regularization and mitigation of vanishing gradients in backprogation. For the former, branches will provide regularization on the main branch (baseline network), and vice versa. For the latter, a relatively shallower branch at a lower layer will provide more immediate gradient signal in backpropagation, resulting in discriminative features in lower layers of the main branch, thus improving its accuracy.”, wherein the examiner interprets “resulting in discriminative features in lower layers of the main branch, thus improving its accuracy” to be the same as resulting in improved accuracy of the machine learning model because they are both directed to enhancing the predictive performance of the machine learning model through improved feature learning and training.)
Kung does not teach updating a machine learning model employed by the machine learning system based on the uncertainty values and a loss function, the loss function being a function of at least a conventional loss function and an adjustable penalty value, wherein updating the machine learning model comprises updating the adjustable penalty value based on a mean value of the conventional loss function as applied to a batch of second inputs processed before the first inputs; and … redirecting the processing to an entity not associated with the machine learning system … and reduced consumption of computing resources by the machine learning system compared to operation of the machine learning system absent the truncating and the redirecting.
Chen teaches:
updating a machine learning model employed by the machine learning system based on the uncertainty values and a loss function, the loss function being a function of at least a conventional loss function and an adjustable penalty value, ([Chen, page 3] “we update our loss weights wi(t) to move gradient norms towards this target for each task. GradNorm is then implemented as an L1 loss function Lgrad between the actual and target gradient norms…The computed gradients ∇wiLgrad are then applied via standard update rules to update each wi”, wherein the examiner interprets “update our loss weights wi(t) to move gradient norms towards this target for each task. GradNorm is then implemented as an L1 loss function Lgrad between the actual and target gradient norms” to be the same as updating an ML model based on values that are adjustable, because they are both directed to ML updating and using a loss function with values that are updated and changed).
wherein updating the machine learning model comprises updating the adjustable penalty value based on a mean value of the conventional loss function as applied to a batch of second inputs processed before the first inputs; ([Chen, page 8] “We introduced GradNorm, an efficient algorithm for tuning loss weights in a multi-task learning setting based on balancing the training rates of different tasks.”, wherein the examiner interprets dynamically tuning a loss-side weight from batch-level training signals (including averages computed each iteration) to be the same as updating the adjustable penalty value based on a mean value of the conventional loss function as applied to a batch of second inputs processed before the first inputs because they are both directed to adjusting a coefficient in the objective using statistics derived from a previous mini-batch’s conventional loss).
Kung and Chen do not teach and redirecting the processing to an entity not associated with the machine learning system … and reduced consumption of computing resources by the machine learning system compared to operation of the machine learning system absent the truncating and the redirecting.
Kang teaches and redirecting the processing to an entity not associated with the machine learning system ([Kang, page 623, sec 5.3] “NSmobile sends the output of that layer from the mobile device to NSserver residing on the server side. NSserver then executes the remaining DNN layers. Upon the completion of the DNN execution, the final result is sent back to NSmobile on the mobile device from NSserver. Note that there is exactly one partition point within the DNN for which information is sent from the mobile device to the cloud.”, wherein the examiner interprets “sends the output of that layer from the mobile device to NSserver residing on the server side” to be the same as redirecting the processing because they are both directed to transferring ongoing computation from one execution location to another rather than continuing all processing at the original location).
and reduced consumption of computing resources by the machine learning system compared to operation of the machine learning system absent the truncating and the redirecting. ([Kang, page 615] “We evaluate Neurosurgeon on a state-of-the-art mobile development platform and show that it improves end-to-end latency by 3.1× on average and up to 40.7×, reduces mobile energy consumption by 59.5% on average and up to 94.7%, and improves datacenter throughput by 1.5× on average and up to 6.7×”, wherein the examiner interprets “reduces mobile energy consumption by 59.5% on average and up to 94.7%” and “improves datacenter throughput by 1.5× on average and up to 6.7×” to be the same as reduced consumption of computing resources by the machine learning system and improved efficiency of processing workloads because they are both directed to decreasing the amount of computational resources required during execution of the machine learning system and increasing processing efficiency such that fewer resources are required per task.)
Kung, Chen, Kang, and the instant application are analogous art because they are all directed to machine learning systems that improve model performance and computational efficiency.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the objective function and thresholding technique disclosed by Kung to include the gradient norm modification technique disclosed by Chen. One would be motivated to do so to efficiently improve training stability and performance of the machine learning model through adaptive loss balancing, as suggested by Chen ([Chen, page 3] “update our loss weights wi(t) to move gradient norms towards this target for each task.”).
It would have also been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the objective function and thresholding technique disclosed by Kung to include the technique for shifting processing of NN to another remote device disclosed by Kang disclosed by Kang. One would be motivated to do so to effectively distribute computational workloads across multiple processing entities and reduce local resource consumption, as suggested by Kang (Kang, [p. 623, sec 5.3] “NSmobile sends the output of that layer from the mobile device to NSserver residing on the server side.”).
Regarding claim 16, Kung teaches:
A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by at least one processor, facilitate performance of operations, comprising: determining, based on first inputs to a machine learning system, uncertainty values that respectively correspond to outputs of the machine learning system; ([Kung, page 3] “Let y be a one-hot ground-truth label vector, x be an input sample and C be the set of all possible labels. The objective function can be written as…where f_exit^n is the output of the n-th exit branch and θ represents the parameters of the layers from an entry point to the exit point”, wherein the examiner interprets that the predictive distribution over C (the model’s output vector) is used to compute an uncertainty measure such as entropy per sample x; because the uncertainty is computed from (and thus corresponds to) the model’s outputs for the same first inputs within the ML system.)
correlating the output of the machine learning system with the uncertainty values to enable segregation of outputs of the machine learning system into at least first outputs and second outputs; and ([Kung, page 2] “If the entropy of a test sample is below a learned threshold value…the sample exits the network…and is not processed by the higher network layers.”, wherein the examiner interprets “entropy of a test sample” to be the same as the uncertainty values and the comparison of that entropy to “a learned threshold value” to be the same as correlating the output of the machine learning system with the uncertainty values because they are both directed to using an uncertainty measure derived from the model output in a threshold-based decision process. The examiner further interprets, “the sample exits the network…and is not processed by the higher network layers” to be the same as enable segregation of outputs of the machine learning system into at least first outputs and second outputs because they are both directed to separating outputs into one set that continues to be processed and another set that does not continue to be processed based on the thresholded uncertainty measure.)
truncating, during training of the machine learning system, processing by the machine learning system of respective ones of the first inputs corresponding to the second outputs … in response to determining that the respective ones of the first inputs correspond to respective ones of the uncertainty values that are greater than a sufficiency threshold, ([Kung, page 2] “If the entropy of a test sample is below a learned threshold value, meaning that the classifier is confident in the prediction, the sample exits the network with the prediction result at this exit point, and is not processed by the higher network layers. If the entropy value is above the threshold, then the classifier at this exit point is deemed not confident, and the sample continues to the next exit point in the network. If the sample reaches the last exit point, which is the last layer of the baseline neural network, it always performs classification.”, wherein the examiner interprets “the sample exits the network…and is not processed by the higher network layers” to be the same as truncating, during training of the machine learning system, processing by the machine learning system of respective ones of the first inputs corresponding to the second outputs because they are both directed to halting further machine-learning processing for selected inputs once a threshold-based condition is met. The examiner further interprets “entropy of a test sample” to be the same as the uncertainty values and “below a learned threshold value” to be the same as greater than a sufficiency threshold because they are both directed to applying a threshold to a confidence/uncertainty measure in order to decide whether to continue or halt further processing for a given input.)
resulting in improved accuracy of the machine learning model (Kung, page 2, section 3] “In training the classifiers at these exit branches, we also consider network regularization and mitigation of vanishing gradients in backprogation. For the former, branches will provide regularization on the main branch (baseline network), and vice versa. For the latter, a relatively shallower branch at a lower layer will provide more immediate gradient signal in backpropagation, resulting in discriminative features in lower layers of the main branch, thus improving its accuracy.”, wherein the examiner interprets “resulting in discriminative features in lower layers of the main branch, thus improving its accuracy” to be the same as resulting in improved accuracy of the machine learning model because they are both directed to enhancing the predictive performance of the machine learning model through improved feature learning and training.)
Kung does not teach iteratively updating a penalty value of a loss function based on a mean value of previous results of a conventional loss function as applied to a batch of second inputs processed before the first inputs, iteratively adapting a machine learning model employed by the machine learning system based on the uncertainty values, … and redirecting the processing to an entity that is not part of the machine learning system … and reduced consumption of computing resources by the machine learning system compared to operation of the machine learning system absent the truncating and the redirecting.
Chen teaches iteratively updating a penalty value of a loss function based on a mean value of previous results of a conventional loss function as applied to a batch of second inputs processed before the first inputs, ([Chen, page 3] “we update our loss weights w_i(t) to move gradient norms towards this target for each task. GradNorm is then implemented as an L1 loss function L_grad between the actual and target gradient norms…The computed gradients ∇_w_i L_grad are then applied via standard update rules to update each w_i”, and [Chen, page 8] “an efficient algorithm for tuning loss weights in a multi-task learning setting based on balancing the training rates of different tasks.”, wherein the examiner interprets the per-iteration weight updates w_i(t) driven by statistics aggregated over the current/previous mini-batch (i.e., batch-level averages of conventional losses/gradients computed on the “second inputs” processed earlier in the loop) to be the same as “iteratively updating a penalty value … based on a mean value of previous results of a conventional loss function as applied to a batch of second inputs processed before the first inputs,” because both are directed to adjusting a penalty/weight term each iteration using batch-mean conventional loss signals derived from the immediately preceding batch.)
iteratively adapting a machine learning model employed by the machine learning system based on the uncertainty values, ([Chen, page 3] “The computed gradients…are then applied via standard update rules…”, wherein the examiner interprets that, because the loss function being optimized mixes a conventional loss with adjustable weights/penalties that are updated using batch statistics (which, in Kung, arise from uncertainty-driven routing/entropy), the model parameters are iteratively adapted in view of signals that include (or are conditioned by) the uncertainty values; this is the same as “iteratively adapting … based on the uncertainty values,” since the uncertainty-conditioned routing contributes to the loss landscape and thus to the iterative parameter updates.)
Chen does not teach and redirecting the processing to an entity that is not part of the machine learning system … and reduced consumption of computing resources by the machine learning system compared to operation of the machine learning system absent the truncating and the redirecting.
Kang teaches redirecting the processing to an entity that is not part of the machine learning system ([Kang, p. 623, sec 5.3] “NSmobile sends the output of that layer from the mobile device to NSserver residing on the server side. NSserver then executes the remaining DNN layers. Upon the completion of the DNN execution, the final result is sent back to NSmobile on the mobile device from NSserver. Note that there is exactly one partition point within the DNN for which information is sent from the mobile device to the cloud.”, wherein the examiner interprets “NSserver then executes the remaining DNN layers” to be the same as to an entity that is not part of the machine learning system because they are both directed to a separate server-side entity external to the originating machine-learning execution environment performing the continued processing.)
and reduced consumption of computing resources by the machine learning system compared to operation of the machine learning system absent the truncating and the redirecting ([Kang, page 615] “We evaluate Neurosurgeon on a state-of-the-art mobile development platform and show that it improves end-to-end latency by 3.1× on average and up to 40.7×, reduces mobile energy consumption by 59.5% on average and up to 94.7%, and improves datacenter throughput by 1.5× on average and up to 6.7×”, wherein the examiner interprets “reduces mobile energy consumption by 59.5% on average and up to 94.7%” and “improves datacenter throughput by 1.5× on average and up to 6.7×” to be the same as reduced consumption of computing resources by the machine learning system and improved efficiency of processing workloads because they are both directed to decreasing the amount of computational resources required during execution of the machine learning system and increasing processing efficiency such that fewer resources are required per task.)
Kung, Chen, Kang, and the instant application are analogous art because they are all directed to machine learning systems that utilize uncertainty-based processing to improve model performance and efficiency.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the objective function disclosed by Kung to include the NN weight updating technique disclosed by Chen. One would be motivated to do so to efficiently improve training performance by adaptively balancing loss contributions across training iterations, as suggested by Chen ([Chen, page 3] “we update our loss weights w_i(t) to move gradient norms towards this target for each task.”). It would have also been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the objective function disclosed by Kung to include the technique to transfer processing between devices disclosed by Kang. One would be motivated to do so to effectively reduce local computational resource usage by offloading processing to an external system, thereby improving execution efficiency, as suggested by Kang ([Kang, p. 623, sec 5.3] “NSmobile sends the output of that layer from the mobile device to NSserver residing on the server side.”).
Claims 2,5, 10, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Kung in view of Chen in view of Kang further in view of NPL reference “SelectiveNet: A Deep Neural Network with an Integrated Reject Option”, by Geifman et. al. (referred herein as Geifman).
Regarding claim 2, Kung, Chen, and Kang teaches The device of claim 1 (see rejection of claim 1).
Kung, Chen, and Kang do not teach wherein the loss function facilitates determining a loss vector based on results of the conventional loss function and the uncertainty value.
Geifman teaches wherein the loss function facilitates determining a loss vector based on results of the conventional loss function and the uncertainty value. ([Geifman, page 3, sec 4.2] “This results in the following unconstrained objective, which is averaged over the samples in Sm, Eq, 2] where c is the target coverage, λ is a hyperparameter controlling the relative importance of the constraint, and Ψ is a quadratic penalty function.”, wherein the examiner interprets the objective that combines r̂ℓ (a conventional prediction loss) with λΨ(·) (a tunable penalty driven by the selection head g) to be the same as a loss function being a function of at least a conventional loss function and an adjustable penalty value because they are both directed to updating the model using a base task loss plus an adjustable penalty that is set from uncertainty-related values.)
Kung, Chen, Kang, Geifman, and the instant application are analogous art because they are all directed to employing loss functions that combine a conventional prediction loss with a uncertainty-driven term.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the device employing a machine-learning model of claim 1 disclosed by Kung, Chen, and Kang to include the confidence term, g(x), in the loss function as disclosed by Geifman [Eq. (2)]. One would be motivated to do so to efficiently enforce a target acceptance/coverage level while training the model, as suggested by Geifman ([Geifman, page 3] “a deep neural architecture allowing end-to-end optimization of selective models … to train it for any desired target coverage...L(f,g) = r̂(f,g|S_m) + λΨ(c - φ̂(g|S_m))” and “a neural network with an integrated reject option, designed to be optimal for the required coverage slice.”).
Regarding claim 5, Kung, Chen, Kang, and Geifman teaches The device of claim 2, (see rejection of claim 2).
Geifman further teaches wherein the adjustable penalty value, in a first iteration of evaluating the loss function, is initially set to at least a defined value, resulting in the adjustable penalty value term initially dominating the loss function. ([Geifman, page 3, section 4.2] “To enforce the coverage constraint, we utilize a variant of the well-known Interior Point Method (IPM). This results in the following unconstrained objective, which is averaged over the samples in Sm
PNG
media_image1.png
57
287
media_image1.png
Greyscale
where c is the target coverage, λ is a hyperparameter controlling the relative importance of the constraint, and Ψ is a quadratic penalty function…… λ was set to 32 (this value was found to be large enough to preserve the constraint through the training process).”, wherein the examiner interprets “λ was set to 32 (this value was found to be large enough to preserve the constraint through the training process)” to be the same as “the adjustable… value [loss, penalty, any value]… initially set to at least a defined value, resulting in the adjustable penalty value term initially dominating the loss function” as both are used to initialize to a defined value where the penalty component (i.e. ψ) dominates the loss function over the empirical risk term (i.e. r) during the initial phases of the training process and then later updates adjust the balance.)
Kung, Chen, Kang, Geifman, and the instant application are analogous art because they are all directed to tuning or adjusting loss components during the training of a machine learning system based on model uncertainty.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the device of claim 2 disclosed by Kung, Chen, Kang, and Geifman to include the value adjustment approach disclosed by Geifman. One would be motivated to do so to effectively improve the model’s ability to account for prediction uncertainty during training, as suggested by Geifman ([Geifman, page 3, section 4.2] “To enforce the coverage constraint, we utilize a variant of the well-known Interior Point Method (IPM).
Regarding claim 10, Kung, Chen, Kang teaches The device of claim 1, (see rejection of claim 1).
Kung, Chen, and Kang do not teach wherein the adjustable penalty value is iteratively updated based on a hyperparameter to cause an indicator value to converge to a target value across iterations of evaluating the loss function, wherein the indicator value is based on a mean of the uncertainty values comprising a respective uncertainty value in each iteration of the evaluating the loss function, wherein the target value is selectable based on user input received via a user interface, and wherein the hyperparameter is less than 1.
Geifman teaches:
wherein the adjustable penalty value is iteratively updated based on a hyperparameter to cause an indicator value to converge to a target value across iterations of evaluating the loss function, ([Geifman, page 1, Introduction] “A selective loss function that optimizes a specified coverage slice using a variant of the interior point optimization method” (IPM) and [Geifman, page 3, section 4.2] “To enforce the coverage constraint, we utilize a variant of the well-known Interior Point Method (IPM)”, wherein the examiner interprets utilizing “a variant of the well-known IPM” where the IPM is a common method which iteratively reweights a penalty-augmented loss function to enforce convergence of empirical coverage (a.k.a. “indicator value”) toward a user-specified target to be the same as iteratively updating the adjustable loss value based on a hyperparameter to cause an indicator value to “converge to a target value across iterations of evaluating the loss function.” because both accomplish the same goal.)
wherein the indicator value is based on a mean of the uncertainty values comprising a respective uncertainty value in each iteration of the evaluating the loss function, ([Geifman, page 2, section 2, “Selective Prediction Problem Formulation”] “the empirical coverage is”
PNG
media_image2.png
53
159
media_image2.png
Greyscale
Furthermore,
PNG
media_image3.png
46
249
media_image3.png
Greyscale
”
wherein the examiner interprets the “empirical coverage” indicator to be the same as “mean of uncertainty values comprising a respective uncertainty value” because both are the mean of per-sample uncertainty/selection outputs of a selection/abstention function, g(x) = {0, 1} where g(x) =1 means the model is confident enough to output f(x), or if g(x)=0 means the model doesn’t know if it is the correct choice; it is an iterative process.)
wherein the target value is selectable based on user input received via a user interface, and wherein the hyperparameter is less than 1. ([Geifman, page 1, Introduction] “Within an abstention-constrained framework, whereby the user specifies the desired target coverage, we optimize, end-to-end, a neural network with an integrated reject option, designed to be optimal for the required coverage slice. … In all our experiment we used α = 0.5 without any hyperparameter optimization.”, wherein the examiner interprets the weighting hyperparameter α that blends the adjustable loss with an auxiliary loss and was set to α=0.5 (50/50 mix) to be adjustable but less than 1 and is interpreted to be the same as “target value is selectable based on user input received via a user interface, and wherein the hyperparameter is less than 1” because they both are aiming to have a selectable target value via a hyperparameter.)
Kung, Chen, Kang, Geifman, and the instant application are analogous art because they are all directed to updating or modifying a machine learning model based on confidence or uncertainty measurements to improve prediction reliability.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the device of claim 1 disclosed by Kung, Chen, and Kang to include the process to include the empirical coverage disclosed by Geifman. One would be motivated to do so to effectively integrate uncertainty/average of uncertainty-based rejection into the model’s training process, as suggested by Geifman (see Geifman quote above.)
Regarding claim 17, Kung, Chen, and Kang teaches The non-transitory machine-readable storage medium of claim 16, (see rejection of claim 16).
Kung, Chen, and Kang do not teach wherein the updating of the penalty value improves an optimization of the loss function according to a defined performance criterion.
Geifman teaches wherein the updating of the penalty value improves an optimization of the loss function according to a defined performance criterion. ([Geifman, Abstract] “SelectiveNet is trained to optimize both classification (or regression) and rejection simultaneously, end-to-end. The result is a deep neural network that is optimized over the covered domain.” wherein the examiner interprets adjusting the rejection mechanism (e.g. a penalty or cost function), to “optimize both classification (or regression) and rejection simultaneously” to generate a “neural network that is optimized over the covered domain” is the same as “updating the penalty value improves an optimization of the loss function according to a defined performance criterion” because both are optimizing a loss function according to a performance criterion.)
Kung, Chen, Kang, Geifman, and the instant application are analogous art because they are all directed to adaptive adjustment of a penalty term in response to uncertainty in order to optimize a learning objective.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the CRM claim 16 disclosed by Kung, Chen, and Kang to include the “optimize both classification (or regression) and rejection simultaneously, end-to-end” disclosed by Geifman. One would be motivated to do so to effectively optimize/update that can be applied to a loss function, as suggested by Geifman ([Geifman, Abstract] “SelectiveNet is trained to optimize both classification (or regression) and rejection simultaneously, end-to-end. The result is a deep neural network that is optimized over the covered domain.”)
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Kung in view of Chen in view of Kang in view of Geifman further in view of NPL reference “Classification with a Reject Option using a Hinge Loss.”, by Wegkemp et. al. (referred herein as Wegkemp).
Regarding claim 18, Kung, Chen, Kang, and Geifman teaches The non-transitory machine-readable storage medium of claim 17, (see rejection of claim 17).
Kung, Chen, Kang, and Geifman does not teach wherein the loss function is Lnew(xj)=(1-puncert)Lold(xj)+puncertCpenalty, wherein puncert is an uncertainty value of the uncertainty values, wherein Lold(xj) are current results of the conventional loss function, and wherein Cpenalty is the penalty value.
Wegkamp teaches wherein the loss function is Lnew(xj)=(1-puncert)Lold(xj)+puncertCpenalty, wherein puncert is an uncertainty value of the uncertainty values, wherein Lold(xj) are current results of the conventional loss function, and wherein Cpenalty is the penalty value. ([Wegkamp, page 2, Introduction] “We propose to incorporate the reject option into our classification scheme by using a threshold value 0 ≤ δ < 1 as follows. Given a discriminant function f : X → R, we report sgn(f(x))) ∈ {-1,1} if | f(x)| > δ, but we withhold decision if | f(x)| ≤ δ and report r. In this note, we assume that the cost of making a wrong decision is 1 and the cost of using the reject option is d > 0. The appropriate risk function is then
PNG
media_image4.png
115
486
media_image4.png
Greyscale
, wherein the examiner interprets P{|Y f(X)| ≤ δ} to be the same as puncert (probability / uncertainty of the ambiguous region) because both have to do with uncertainty. Further clarification is seen from the specification of the instant application “[0023] As an example, Uncertainty-type Loss Function (ULF) component 260 can employ the formula Lnew(xj)=(1-puncert)Lold(xj)+puncertCpenalty where L(x,) is conventional loss function data, e.g., CLF data from CLF component 250, where puncert information embodied in UPS 240, where Cpenalty is a penalty value discussed in further detail elsewhere herein, and where L, (x,) is uncertainty based loss data derived from CLF data.”. The examiner further interprets P{Y f(X) < -δ} to be the same as Lold(xj) because they are both related to the ordinary classification error of a neural network / ML model. Furthermore, the constant d in the above formula is interpreted to be the same as Cpenalty because they both have to do with cost of abstaining. The examiner finally interprets that together with previous interpretations, Ld,δ (f) is the same as Lnew(xi) because they are both associated with the total loss.”)
Kung, Chen, Kang, Geifman, Wegkamp, and the instant application are analogous art because they are all directed to uncertainty-aware loss formulations for selective prediction systems.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the CRM of claim 17 disclosed by Kung, Chen, Kang, and Geifman to include the threshold-based abstention loss calculation disclosed by Wegkamp. One would be motivated to do so to efficiently use the risk function to determine loss as a loss function as suggested by Wegkamp ([Wegkamp, page 2, Introduction] “In this note, we assume that the cost of making a wrong decision is 1 and the cost of using the reject option is d > 0. The appropriate risk function is then [Eq].”)
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Kung in view of Chen in view of Kang further in view of Cortes and Ziyin.
Regarding claim 20, Kung, Chen, and Kang teaches The non-transitory machine-readable storage medium of claim 16, (see rejection of claim 16).
Kung, Chen, and Kang do not teach wherein the operations further comprise adjusting the penalty value based on a mean of previous uncertainty values, wherein the adjusting comprises incrementing the penalty value by a first amount that is less than 1 in response to the mean of the previous uncertainty values being less than a target value, and wherein the adjusting comprises decrementing the penalty value by a second amount that is less than 1 in response to the mean of the previous uncertainty values being greater than the target value.
Cortes and Ziyin teaches wherein the operations further comprise adjusting the penalty value based on a mean of previous uncertainty values, wherein the adjusting comprises incrementing the penalty value by a first amount that is less than 1 in response to the mean of the previous uncertainty values being less than a target value, and wherein the adjusting comprises decrementing the penalty value by a second amount that is less than 1 in response to the mean of the previous uncertainty values being greater than the target value. ([Cortes, page 8, section 6] “We introduced a general framework for classification with abstention where the predictor and abstention functions are learned simultaneously.” and [Ziyin, Abstract] “aims to balance between betting on an outcome (making a prediction) when confident and reserving one’s winnings (abstaining) when not confident. This loss function allows us to train neural networks and characterize the disconfidence of prediction in an end-to-end fashion”, wherein the examiner interprets adjusting prediction behavior based on confidence levels to control abstention, including the use of loss functions that emphasize either prediction or abstention depending on confidence, to be the same as “adjusting the penalty value based on a mean of previous uncertainty values” such that the tuning increments or decrements the penalty value depending on whether the mean is less than or greater than a target value. The examiner further interprets that this iterative process of adjusting prediction behavior would take steps of 1 for adjusting the penalty or prediction behavior.)
Kung, Chen, Kang, Cortes and Ziyin, and the instant application are analogous art because they are all directed to training-time optimization and adaptive tuning of machine learning systems using uncertainty or confidence-based feedback.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the CRM of claim 16 disclosed by Kung, Chen, and Kang to include the uncertainty-based adjustment framework disclosed by Cortes and Ziyin. One would be motivated to do so to effectively stabilize and improve convergence of training by incorporating uncertainty-aware penalty tuning that adaptively controls abstention and prediction behavior, as suggested by Cortes and Ziyin ([Cortes, page 8, section 6] “a general framework for classification with abstention where the predictor and abstention functions are learned simultaneously AND [Ziyin, page 1] “aims to balance between betting on an outcome (making a prediction) when confident and reserving one’s winnings (abstaining) when not confident”)
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Kung in view of Chen in view of Kang in view of Geifman further in view of Wegkamp.
Regarding claim 4, Kung, Chen, Kang, and Geifman teaches The device of claim 2, (see rejection of claim 2).
Kung, Chen, Kang, and Geifman do not teach wherein the loss function is Lnew(xj)=(1-puncert)*Lold(xj)+puncert*Cpenalty, wherein puncert is the uncertainty value, wherein Lold(xj) are the results of the conventional loss function, and wherein Cpenalty is the adjustable penalty value.
Wegkamp teaches wherein the loss function is Lnew(xj)=(1-puncert)*Lold(xj)+puncert*Cpenalty, wherein puncert is the uncertainty value, wherein Lold(xj) are the results of the conventional loss function, and wherein Cpenalty is the adjustable penalty value. ([Wegkamp, page 2] Introduction] “We propose to incorporate the reject option into our classification scheme by using a threshold value 0 ≤ δ < 1 as follows. Given a discriminant function f : X → R, we report sgn(f(x))) ∈ {-1,1} if | f(x)| > δ, but we withhold decision if | f(x)| ≤ δ and report r. In this note, we assume that the cost of making a wrong decision is 1 and the cost of using the reject option is d > 0. The appropriate risk function is then
PNG
media_image5.png
175
648
media_image5.png
Greyscale
.” wherein the examiner interprets P{|Y f(X)| ≤ δ} to be the same as puncert (probability / uncertainty of the ambiguous region) because both have to do with uncertainty. Further clarification is seen from the specification of the instant application “[0023] As an example, Uncertainty-type Loss Function (ULF) component 260 can employ the formula Lnew(xj)=(1-puncert)*Lold(xj)+puncert*Cpenalty where L(x,) is conventional loss function data, e.g., CLF data from CLF component 250, where puncert information embodied in UPS 240, where Cpenalty is a penalty value discussed in further detail elsewhere herein, and where L, (x,) is uncertainty based loss data derived from CLF data.”. The examiner further interprets P{Y f(X) < -δ} to be the same as Lold(xj) because they are both related to the ordinary classification error of a neural network / ML model. Furthermore, the constant d in the above formula is interpreted to be the same as Cpenalty because they both have to do with cost of abstaining. The examiner finally interprets that together with previous interpretations, Ld,δ (f) is the same as Lnew(xi) because they are both associated with the total loss.”).
Kung, Chen, Kang, Geifman, Wegkamp, and the instant application are analogous art because they are all directed to loss functions that integrate uncertainty and cost-sensitive abstention mechanisms in machine learning systems.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the device of claim 2 disclosed by Kung, Chen, Kang, and Geifman to include the “cost of using the reject option is d > 0” disclosed by Wegkamp. One would be motivated to do so to effectively account for the cost tradeoff associated with abstaining from low-confidence predictions, as suggested by Wegkamp ([Wegkamp, page 2] “In this note, we assume that the cost of making a wrong decision is 1 and the cost of using the reject option is d > 0.”).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Kung in view of Chen in view of Kang further in view of Wegkamp.
Regarding claim 9, Kung, Chen, and Kang teaches The device of claim 1 (see rejection of claim 1).
Kung, Chen, and Kang do not teach wherein the updating of the adjustable penalty value comprises restricting the adjustable penalty value from traversing a floor penalty value.
Wegkamp teaches wherein the updating of the adjustable penalty value comprises restricting the adjustable penalty value from traversing a floor penalty value. ([Wegkamp, page 2, Introduction] “we assume that the cost of making a wrong decision is 1 and the cost of using the reject option is d > 0, i.e., a minimum penalty that cannot be crossed] ... [Eq. 1] Ld,δ(f)=Pr{Yf(X)<-δ}+dPr{∣Yf(X)∣≤δ}”, wherein the examiner interprets this fixed positive reject cost as a floor on the penalty term (a lower bound that the penalty cannot go below), which restricts the penalty from traversing that floor during training/optimization of the objective, thereby meeting the claim language that the updating “comprises restricting … from traversing a floor penalty value.”)
Kung, Chen, Kang, Wegkamp, and the instant application are analogous art because each addresses loss-function penalties that shape model training or behavior, including enforcing a floor on the penalty.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the device of claim 1 disclosed by Kung, Chen, and Kang to include Wegkamp’s fixed positive reject cost d>0. One would be motivated to do so to effectively prevent collapse of the penalty to zero and preserve a meaningful abstain trade-off, i.e., to efficiently enforce a floor on the penalty during updates, as suggested by Wegkamp ([Wegkamp, page 4]) “we assume that the cost of making a wrong decision is 1 and the cost of using the reject option is d > 0.”)
Claims 11, and 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over NPL reference “Deep gamblers: Learning to abstain with portfolio theory.”, by Ziyin et. al. (referred herein as Ziyin) in view of NPL reference “Boosting with abstention.” by Cortes et. al. (referred herein as Cortes) in view of Kung further in view of Kang.
Regarding claim 11, Ziyin teaches:
A method, comprising: determining, by a system comprising at least one processor, uncertainty values, based on first inputs to a machine learning system, wherein the uncertainty values correspond to outputs of the machine learning system, ([Ziyin, page 2, section 2] “the model abstains from making a prediction when the selection function g(x) falls below a predetermined threshold h. We call g(x) the uncertainty score of x;” and [Ziyin, page 4, section 4.3] “for an m-class classification task … we add an m+1-th class, which stands for reservation … the selection function g(·) is then f_w(·){m+1},” wherein the examiner interprets defining a numerical uncertainty score g(x) for each input and equating it with the network’s (m+1)-th output probability f_w(x){m+1} (the reservation/abstention output) to be the same as determining “uncertainty values … correspond[[s]] to [[an]] outputs of the machine learning system,” because both are directed to producing an uncertainty signal that is realized as (and thus corresponds to) a component of the model’s output vector.)
adapting, by the system, a machine learning model employed by the machine learning system based on the uncertainty values, ([Ziyin, page 3, section 4] “We then describe how to adapt portfolio theory for classification problems in machine learning and derive our adapted loss function that trains a model to predict a rejection score” AND ([Ziyin, page 1] “This loss function allows us to train neural networks and characterize the disconfidence of prediction in an end-to-end fashion.”, wherein the examiner interprets the inclusion of g(x), the uncertainty value, in the loss function in which the “adapted loss function that trains a model to predict a rejection score” to be the same as “adapting… a machine learning model … based on the uncertainty value”, as both are directed to adapting a ML model based on an uncertainty value from an ML system.)
correlating, by the system, the outputs of the machine learning system with the uncertainty values, and ([Ziyin, page 4, section 4.3] “For a m-class classification task [output probability] we transform it to a horse race with reservation by adding a m+1-th class, which stands for reservation. The objective function for a mini-batch of size B, and for constant o over all categories … The selection function g(.) is then fw(.) m+1 (cf. Equation 1)” and [Ziyin, page 2, section. 2] “ We call g(x) the uncertainty score of x”, wherein the examiner interprets “g(x) the uncertainty score” as the uncertainty value. The examiner further interprets f w(x) m+1; the network’s output probability for the added reservation class; to be numerically equal to g(x) since it is produced as a direct component of the network’s output vector and is the same as “correlating … the output of the machine learning system with the uncertainty value”. Furthermore, in the abstract, the statement that the (m+1)th class “represents disconfidence” confirms that the same output probability encapsulates uncertainty.).
Ziyin does not teach updating, by the system, a penalty value of a loss function based on a mean value of first results of a conventional loss function as applied to a batch of second inputs processed before the first inputs; truncating, by the system and during training of the machine learning system, processing, using the machine learning system, of respective ones of the first inputs and redirecting the processing to an entity distinct from the machine learning system in response to determining that the respective ones of the first inputs correspond to respective ones of the uncertainty values that are greater than a sufficiency threshold, resulting in improved accuracy of the machine learning model and reduced consumption of computing resources by the machine learning system compared to operation of the machine learning system absent the truncating and the redirecting.
Cortes teaches updating, by the system, a penalty value of a loss function based on a mean value of first results of a conventional loss function as applied to a batch ([Cortes, page 5] “For other surrogate losses, the step size η_t is found via a line search or other numerical methods by solving argmin_η F(αt-1+ηek) ... this suggests to select α as the solution of min_α>=0[(1/m) ∑ L_sb(h,r,xi,yi)]”, wherein the examiner interprets by solving argmin_η F(αt-1+ηek) to be the same as “updating, by the system, a penalty value of a loss function”, because they are both directed to changing the loss-function weights α that control the rejection/penalty term’s contribution to the objective between iterations. The examiner further interprets “select α as the solution of min_α>=0[(1/m) ∑ L_sb(h,r,xi,yi)]” to be the same as minimizing a 1/m average over the training batch (xi, yi) and this bath average to be the same as “based on a mean value of first results of a conventional loss function as applied to a batch” because they are both directed to computing the objective as a 1/m average over the training batch and using that batch-mean loss to drive updates.)
of second inputs processed before the first inputs; ([Cortes, page 5, Fig. 3] “Pseudocode of the boosting-stylealgorithm (BA) algorithm ... Dt+1(i,1) ←ϕ0[rt(xi) - yiht(xi)]/Zt + 1; Dt + 1(i,2) ← ϕ0[⋯rt(xi)⋯]/Zt+1”, wherein the examiner interprets, line 15] to be the same as “…as applied to a batch of second inputs processed before the first inputs,” because they are both directed to forming the current round’s batch expectations/means using statistics derived from the previous round’s model outputs, i.e., means based on inputs processed earlier (the “second inputs”) before processing the current (“first”) inputs. Note D is a set of weights over your training examples used to take weighted averages. D(t+1) is the updated weights on the next round computed from the model’s outputs at round t.)
Ziyin and Cortes does not teach truncating, by the system and during training of the machine learning system, processing, using the machine learning system, of respective ones of the first inputs and redirecting the processing to an entity distinct from the machine learning system in response to determining that the respective ones of the first inputs correspond to respective ones of the uncertainty values that are greater than a sufficiency threshold, resulting in improved accuracy of the machine learning model and reduced consumption of computing resources by the machine learning system compared to operation of the machine learning system absent the truncating and the redirecting.
Kung teaches truncating, by the system and during training of the machine learning system, processing, using the machine learning system, of respective ones of the first inputs ([Kung, page 2] “If the entropy of a test sample is below a learned threshold value…the sample exits the network…and is not processed by the higher network layers.”, wherein the examiner interprets “the sample exits the network…and is not processed by the higher network layers” to be the same as truncating, by the system and during training of the machine learning system, processing, using the machine learning system, of respective ones of the first inputs because they are both directed to halting further machine-learning processing for selected inputs once a threshold-based condition is met).
in response to determining that the respective ones of the first inputs correspond to respective ones of the uncertainty values that are greater than a sufficiency threshold, ([Kung, page 2] “If the entropy of a test sample is below a learned threshold value”, wherein the examiner interprets “entropy of a test sample” to be the same as uncertainty values and “below a learned threshold value” to be the same as greater than a sufficiency threshold because they are both directed to applying a threshold to a confidence/uncertainty measure in order to decide whether to continue or halt further processing for a given input).
Kung teaches resulting in improved accuracy of the machine learning model ([Kung, page 2, section 3] “In training the classifiers at these exit branches, we also consider network regularization and mitigation of vanishing gradients in backprogation. For the former, branches will provide regularization on the main branch (baseline network), and vice versa. For the latter, a relatively shallower branch at a lower layer will provide more immediate gradient signal in backpropagation, resulting in discriminative features in lower layers of the main branch, thus improving its accuracy.”, wherein the examiner interprets “resulting in discriminative features in lower layers of the main branch, thus improving its accuracy” to be the same as resulting in improved accuracy of the machine learning model because they are both directed to enhancing the predictive performance of the machine learning model through improved feature learning and training.)
Ziyin, Cortes and Kung do not teach redirecting the processing to an entity distinct from the machine learning system … and reduced consumption of computing resources by the machine learning system compared to operation of the machine learning system absent the truncating and the redirecting.
Kang teaches redirecting the processing to an entity distinct from the machine learning system ([Kang, p. 623, sec 5.3] “NSmobile sends the output of that layer from the mobile device to NSserver residing on the server side. NSserver then executes the remaining DNN layers. Upon the completion of the DNN execution, the final result is sent back to NSmobile on the mobile device from NSserver. Note that there is exactly one partition point within the DNN for which information is sent from the mobile device to the cloud.”, wherein the examiner interprets “sends the output of that layer from the mobile device to NSserver residing on the server side” to be the same as redirecting the processing because they are both directed to transferring ongoing computation from one execution location to another rather than continuing all processing at the original location.)
redirecting the processing to an entity distinct from the machine learning system ([Kang, p. 623, sec 5.3] “NSmobile sends the output of that layer from the mobile device to NSserver residing on the server side. NSserver then executes the remaining DNN layers. Upon the completion of the DNN execution, the final result is sent back to NSmobile on the mobile device from NSserver. Note that there is exactly one partition point within the DNN for which information is sent from the mobile device to the cloud.”, wherein the examiner interprets “NSserver then executes the remaining DNN layers” to be the same as to an entity distinct from the machine learning system because they are both directed to a separate server-side entity external to the originating machine-learning execution environment performing the continued processing.)
and reduced consumption of computing resources by the machine learning system compared to operation of the machine learning system absent the truncating and the redirecting ([Kang, page 615] “We evaluate Neurosurgeon on a state-of-the-art mobile development platform and show that it improves end-to-end latency by 3.1× on average and up to 40.7×, reduces mobile energy consumption by 59.5% on average and up to 94.7%, and improves datacenter throughput by 1.5× on average and up to 6.7×”, wherein the examiner interprets “reduces mobile energy consumption by 59.5% on average and up to 94.7%” and “improves datacenter throughput by 1.5× on average and up to 6.7×” to be the same as reduced consumption of computing resources by the machine learning system and improved efficiency of processing workloads because they are both directed to decreasing the amount of computational resources required during execution of the machine learning system and increasing processing efficiency such that fewer resources are required per task.)
Ziyin, Cortes, Kung, Kang, and the instant application are analogous art because they are all directed to machine learning systems that utilize uncertainty-based decision-making to improve model performance and efficiency.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the loss function disclosed by Ziyin to include the solution selection technique disclosed by Cortes. One would be motivated to do so to efficiently update a penalty value based on batch-level loss information to improve training effectiveness and convergence behavior, as suggested by Cortes ([Cortes, page 5] “select α as the solution of min_α>=0[(1/m) ∑ L_sb(h,r,xi,yi)]”). It would have also been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the loss function disclosed by Ziyin to include the sample exiting strategy disclosed by Kung. One would be motivated to do so to effectively reduce unnecessary computation by conditionally truncating processing based on uncertainty thresholds, as suggested by Kung ([Kung, page 2] “the sample exits the network…and is not processed by the higher network layers.”). It would have also been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the loss function disclosed by Ziyin to include the technique for shifting of processing of NN to another remote device as disclosed by Kang. One would be motivated to do so to effectively distribute computational processing across external systems to reduce local computational resource usage and improve system efficiency, as suggested by Kang ([Kang, p. 623, sec 5.3] “NSmobile sends the output of that layer from the mobile device to NSserver residing on the server side.”).
Regarding claim 13, Ziyin, Cortes, Kung, and Kang teaches The method of claim 11, (see rejection of claim 11).
Cortes further teaches wherein the updating of the penalty value comprises evaluating the formula Cpenalty = mean(Lold(xbatch_j)), and wherein Lold(xbatch_j) are the first results of the conventional loss function. ([Cortes, page 2, section 2.1] “The abstention cost c(x) is assumed known to the learner. In the following, we assume that c is a constant function…
PNG
media_image6.png
55
550
media_image6.png
Greyscale
.”, wherein the examiner interprets defining a constant abstention cost c(x) that is applied within the per-sample conventional loss function to be the same as evaluating the formula for Cpenalty as a mean of the first results of the conventional loss function, where the “abstention cost” is the same as “penalty value” as they both are a “cost” in governing the contribution of terms.)
Ziyin, Cortes, Kung, Kang, and the instant application are analogous art because they are all directed to modifying or adjusting a penalty value in a loss function to manage model uncertainty or abstention behavior during learning.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method of claim 11 disclosed by Ziyin, Cortes, Kung, and Kang to include the definition of a constant abstention cost used in the per-sample loss function disclosed by Cortes. One would be motivated to do so to effectively make a determination of when to abstain, as suggested by Ziyin (Ziyin, page 2] section 2) “the model abstains from making a prediction when the selection function g(x) falls below a predetermined threshold h.”)
Regarding claim 14, Ziyin, Cortes, Kung, and Kang teaches The method of claim 13, (see rejection of claim 13).
Cortes further teaches wherein the updating of the penalty value comprises preventing the penalty value from dropping below a minimum value according to the formula Cpenalty=max(Cpenalty, Cmin) wherein Cmin is the minimum value. ([Cortes, page 3, section 3.1] “the abstention loss L(h, r, x, y) can be equivalently expressed as L(h, r, x, y) =
PNG
media_image7.png
12
196
media_image7.png
Greyscale
The abstention cost c(x) is assumed known to the learner. In the following, we assume that c is a constant function,” wherein the examiner interprets the loss (penalty), L, is the maximum of the ordinary error term and the constant c. Thus, whenever the ordinary term would fall below c, the function “clips” it at c, which is the same as like Cpenalty = max(Cpenalty, Cmin).
Ziyin, Cortes, Kung, Kang, and the instant application are analogous art because they are all directed to ensuring a lower bound on a penalty value within a loss function to stabilize model behavior.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method of claim 11 disclosed by Ziyin, Cortes, Kung, and Kang to include the penalty clipping technique disclosed by Cortes. One would be motivated to do so to effectively make a determination of when to abstain, as suggested by Ziyin ([Ziyin, page 2, section 2] “the model abstains from making a prediction when the selection function g(x) falls below a predetermined threshold h.”)
Regarding claim 15, Ziyin, Cortes, Kung, and Kang teaches The method of claim 11, (see rejection of claim 11).
Ziyin further teaches further comprising tuning, by the system, the penalty value based on a mean of previous uncertainty values, wherein the tuning comprises incrementing the penalty value by a first amount that is less than 1 in response to the mean of the previous uncertainty values being less than a target value, and wherein the tuning comprises decrementing the penalty value by a second amount that is less than 1 in response to the mean of the previous uncertainty values being greater than the target value. ([Ziyin, page 2] “A prediction model augmented with a rejection option is a pair of functions (f, g) such that gₕ : X → R is a selection function … i.e., the model abstains from making a prediction when the selection function g(x) falls below a predetermined threshold h. We call g(x) the uncertainty score of x … The covered dataset is defined to be {x : gₕ(x) ≥ h}, and the coverage is the ratio … One may trade-off coverage for lower risk … We gradually decrease the threshold h … This shows how we might calibrate threshold h to control coverage.”, wherein the examiner interprets that the gradual adjustment of the threshold h based on aggregated uncertainty scores corresponds to tuning the penalty value based on a mean of previous uncertainty values, because both describe feedback-driven calibration of a control parameter to maintain target coverage or confidence. The examiner further interprets the phrase “gradually decrease the threshold h” as corresponding to incrementing or decrementing by small fractions (less than 1) where fine-grained, progressive threshold updates to achieve smooth adjustment toward a target coverage are achieved.)
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Kung in view of Chen and Kang further in view of Cortes.
Regarding Claim 19, Kung, Chen, and Kang teaches The non-transitory machine-readable storage medium of claim 16, (see rejection of claim 16).
Kung, Chen, and Kang do not teach wherein the updating the penalty value comprises evaluating the formula Cpenalty=max(mean(Lold(xbatch_j)), Cmin), wherein Cpenalty is the penalty value, wherein Lold(xbatch_j) are the previous results of the conventional loss function, and wherein Cmin is a minimum allowable penalty value.
Cortes teaches wherein the updating the penalty value comprises evaluating the formula Cpenalty=max(mean(Lold(xbatch_j)), Cmin), wherein Cpenalty is the penalty value, wherein Lold(xbatch_j) are the previous results of the conventional loss function, and wherein Cmin is a minimum allowable penalty value. [Cortes, page 2, section 2.1] “The abstention cost c(x) is assumed known to the learner. In the following, we assume that c is a constant function…
PNG
media_image6.png
55
550
media_image6.png
Greyscale
and [Cortes, page 8, section 5] “For each cost c, the hyperparameter configuration was chosen to be the set of parameters that attained the smallest average rejection loss on the validation set. For that set of parameters we report the results on the test set.”, wherein the examiner interprets the constant abstention cost c selected to minimize average rejection loss and applied in the per-example abstention loss function to be the same as Cpenalty, the penalty value that is updated based on a mean of previous conventional loss results and constrained by a minimum allowable value, as both terms are directed to a cost that modulates the loss function during optimization.)
Kung, Chen, Kang, Cortes, and the instant application are analogous art because they are all directed to machine learning systems that incorporate selective prediction with adjustable or learnable abstention penalties within a loss function.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the machine-readable storage medium of claim 16 disclosed by Kung, Chen, and Kang to include the abstention-aware loss function using a fixed cost disclosed by Cortes. One would be motivated to do so to effectively use the cost function to calculate loss as suggested by Cortes ([Cortes, page 2, section 2.1] “The abstention cost c(x) is assumed known to the learner. In the following, we assume that c is a constant function…[Cortes, page 8, section 5] “For each cost c, the hyperparameter configuration was chosen to be the set of parameters that attained the smallest average rejection loss on the validation set.”)
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Ziyin in view of Cortes in view of Kung in view of Kang in view of Geifman further in view of Wegkamp.
Regarding claim 12, Ziyin, Cortes, Kung, and Kang teaches The method of claim 11, (see rejection of claim 11).
Ziyin, Cortes, Kung, and Kang do not teach wherein the updating the penalty value and the adapting the machine learning model increase an efficacy of the loss function according to a defined performance criterion, wherein the loss function is Lnew(xj)=(1-puncert)*Lold(xj)+puncert*Cpenalty, wherein puncert is the uncertainty value, wherein Lold(xj) are second results of the conventional loss function, and wherein Cpenalty is the penalty value.
Geifman teaches wherein the updating the penalty value and the adapting the machine learning model increase an efficacy of the loss function according to a defined performance criterion, ([Geifman, page 1, Introduction] “A selective loss function that optimizes a specified coverage slice using a variant of the interior point optimization method”, wherein the examiner interprets the “selective loss function that optimizes a specified coverage slice” to be the same as the “adapting of the machine learning model increase an efficacy of the loss function according to a defined performance criterion”, as they both optimize the selective loss, matching the notion that adapting the model increases an “efficacy” of the loss function.
Ziyin, Cortes, Kung, Kang, Geifman, and the instant application are analogous art, because they are all directed to updating a clue and machine learning adapting for optimal performance.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method of claim 11 disclosed by Ziyin and Cortes to include the “selective loss function that optimizes a specified coverage slice using a variant of the interior point optimization method” disclosed by Geifman. One would be motivated to do so to efficiently use a loss function to optimize loss in hopes to improve performance/efficacy as suggested by Geifman ([Geifman, page 1, Introduction] “A selective loss function that optimizes a specified coverage slice using a variant of the interior point optimization method”.)
Ziyin, Cortes, Kung, Kang, Geifman do not teach wherein the loss function is Lnew(xj)=(1-puncert)*Lold(xj)+puncert*Cpenalty, wherein puncert is the uncertainty value, wherein Lold(xj) are second results of the conventional loss function, and wherein Cpenalty is the penalty value.
Wegkamp teaches wherein the loss function is Lnew(xj)=(1-puncert)*Lold(xj)+puncert*Cpenalty, wherein puncert is the uncertainty value, wherein Lold(xj) are second results of the conventional loss function, and wherein Cpenalty is the penalty value. ([Wegkamp, page 2, Introduction] “We propose to incorporate the reject option into our classification scheme by using a threshold value 0 ≤ δ < 1 as follows. Given a discriminant function f : X → R, we report sgn(f(x))) ∈ {-1,1} if | f(x)| > δ, but we withhold decision if | f(x)| ≤ δ and report r. In this note, we assume that the cost of making a wrong decision is 1 and the cost of using the reject option is d > 0. The appropriate risk function is then
PNG
media_image5.png
175
648
media_image5.png
Greyscale
.”, wherein the examiner interprets P{|Y f(X)| ≤ δ} to be the same as puncert (probability / uncertainty of the ambiguous region) because both have to do with uncertainty. Further clarification is seen from the specification of the instant application “[0023] As an example, Uncertainty-type Loss Function (ULF) component 260 can employ the formula Lnew(xj)=(1-puncert)*Lold(xj)+puncert*Cpenalty where L(x,) is conventional loss function data, e.g., CLF data from CLF component 250, where puncert information embodied in UPS 240, where Cpenalty is a penalty value discussed in further detail elsewhere herein, and where L, (x,) is uncertainty based loss data derived from CLF data.”. The examiner further interprets P{Y f(X) < -δ} to be the same as Lold(xj) because they are both related to the ordinary classification error of a neural network / ML model. Furthermore, the constant d in the above formula is interpreted to be the same as Cpenalty because they both have to do with cost of abstaining. The examiner finally interprets that together with previous interpretations, Ld,δ (f) is the same as Lnew(xi) because they are both associated with the total loss.”).
Ziyin, Cortes, Kung, Kang, Geifman, Wegkamp, and the instant application are analogous art because they are all directed to loss functions that integrate uncertainty and cost-sensitive abstention mechanisms in machine learning systems.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method of claim 11 disclosed by Ziyin, Cortes, Kung, and Kang, and the coverage constraint optimization disclosed by Geifman and to include the “cost of using the reject option is d > 0” disclosed by Wegkamp. One would be motivated to do so to effectively account for the cost tradeoff associated with abstaining from low-confidence predictions, as suggested by Wegkamp (Wegkamp, page 2] “In this note, we assume that the cost of making a wrong decision is 1 and the cost of using the reject option is d > 0.”)
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Kung in view of Chen further in view of NPL reference “Vector-Based Natural Language Call Routing” by Carpenter et. al. (referred herein as Carpenter).
Regarding claim 21, Kung, Chen, and Kang teaches The device of claim 1 (see rejection of claim 1).
Kung, Chen, and Kang do not teach wherein the first inputs correspond to incoming calls to a call center, and wherein the operations further comprise: routing an incoming call, of the incoming calls and corresponding to a first input of the first inputs, to a device associated with an operator of the call center based on determining that the first input corresponds to one of the first portion of the outputs; and routing the incoming call to another device associated with a human expert and other than the device, instead of to any device of any operator of the call center, for further action based on determining that the first input corresponds to one of the second portion of the outputs.Carpenter teaches:
wherein the first inputs correspond to incoming calls to a call center; ([Carpenter, Abstract] “This paper describes a domain independent, automatically trained natural language call router for directing incoming calls in a call center.”, wherein the examiner interprets directing incoming calls in a call center to be the same as first inputs correspond to incoming calls to a call center because they are both directed to treating each call as an input for routing within a call-center system).
and wherein the operations further comprise: routing an incoming call, of the incoming calls and corresponding to a first input of the first inputs, to a device associated with an operator of the call center based on determining that the first input corresponds to one of the first portion of the outputs; routing the incoming call to another device associated with a human expert and other than the device, instead of to any
device of any operator of the call center, for further action based on determining that the first input corresponds to one of the second portion of the outputs; ([Carpenter, page 20] “the first caller utterance was given to the router as input. The outcome of the router was classified as follows:... (c) Reject, if the router could not form a sensible query or was unable to gather sufficient information from the user after its queries and routed the call to a human operator.”, wherein the examiner interprets “given to the router as input.... if the router could not form a sensible query or was unable to gather sufficient information from the user after its queries and routed the call to a human operator.” to be the same as inputs correspond to routing of a calls to a human expert/operators device or to another operators device if needed.)
Kung, Chen, Kang, Carpenter, and the instant application are analogous art, because they are all directed to call-center workflows that use automated classification.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the device of claim 1 disclosed by Kung, Chen, and Kang to include the call direction determination process disclosed by Carpenter. One would be motivated to do so to efficiently route routine calls to operator devices while escalating uncertain or difficult calls to expert devices to improve handling quality and first-contact resolution, as suggested by Carpenter ([Carpenter, Abstract] “directing incoming calls in a call center.”).
Claim 22 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Kung in view of Chen in view of Kang in view of Geifman further in view of NPL reference “Pareto Multi-Task Learning” by Lin et. al. (referred herein as Lin).
Regarding claim 22, Kung, Chen, Kang, and Geifman teaches The device of claim 2 (see rejection of claim 2).
Kung, Chen, and Geifman do not teach wherein the operations further comprise: repeating the updating of the machine learning model until the loss vector and the adjustable penalty value are stabilized according to a defined stability criterion.
Lin teaches wherein the operations further comprise: repeating the updating of the machine learning model until the loss vector the adjustable penalty value are stabilized according to a defined stability criterion. ([Lin, page 3] “A MTL problem involves a set of m correlated tasks with a loss vector...where Li(θ) is the loss of the i-th task. A MTL algorithm is to optimize all tasks simultaneously by exploiting the shared structure and information among them…“combine the losses of all tasks into a single surrogate loss: [Eq.2] where wi is the weight for the i-th task” AND [Lin, page 4-5] “The update rule of the algorithm is...where η is the step size and the search direction dt is obtained as follows:...and will be stopped once a feasible solution is found or a predefined number of iterations is met.”, wherein the examiner interprets the loss vector, and the update rule that will be stopped once a “pre-defined number of iterations”/stability is met to be the same as “defined stability criterion”. The examiner further interprets the task weighting vector 𝑤 (and, in preference-guided formulations, the induced weights consistent with 𝑢) to be the same as an adjustable penalty value, because they are both coefficients inside the scalarized loss that increase or decrease the penalty assigned to each component of the loss vector and are adjusted during training to reach a target trade-off.”)
Kung, Chen, Kang, Geifman, Lin, and the instant application are analogous art because they are all directed to iterative machine-learning training that uses a loss-based objective with adjustable weighting.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the selective-prediction and model-updating framework disclosed by Kung, Chen, Kang, and Geifman to include the stopping criterion disclosed by Lin. One would be motivated to do so to efficiently ensure that updates repeat only until stabilization under a defined stability criterion, thereby improving convergence control and training reliability, as suggested by Lin (Lin, pages 4-5] “will be stopped once a feasible solution is found or a predefined number of iterations is met.”).
Regarding claim 23, Kung, Chen, Kang, Geifman, and Lin teaches The device of claim 22 (see rejection of claim 22).
Lin further teaches wherein the operations further comprise: in response to the adjustable penalty value being determined to be stabilized according to the defined stability criterion, ([Lin, page 3] “A MTL problem involves a set of m correlated tasks with a loss vector:..where Li(θ) is the loss of the i-th task. A MTL algorithm is to optimize all tasks simultaneously by exploiting the shared structure and information among them… combine the losses of all tasks into a single surrogate loss: [Eq.2] where wi is the weight for the i-th task” AND [Lin, page 4-5] “The update rule of the algorithm is ...where η is the step size and the search direction dt is obtained as follows:...and will be stopped once a feasible solution is found or a predefined number of iterations is met.”, wherein the examiner interprets the loss vector, and the update rule that will be stopped once a “pre-defined number of iterations”/stability is met to be the same as “defined stability criterion”. The examiner further interprets the task weighting vector 𝑤 (and, in preference-guided formulations, the induced weights consistent with 𝑢) to be the same as an adjustable penalty value, because they are both coefficients inside the scalarized loss that increase or decrease the penalty assigned to each component of the loss vector and are adjusted during training to reach a target trade-off.”).
Chen further teaches modifying the adjustable penalty value by an incremental value based on a difference between loss values associated with the loss vector and a defined preferred loss value. ([Chen, page 3] “Equation 1 gives a target for each task i’s gradient norms, and we update our loss weights w_i(t) to move gradient norms towards this target for each task. GradNorm is then implemented as an L1 loss function L_grad between the actual and target gradient norms at each timestep for each task, summed over all tasks.”, wherein the examiner interprets “update our loss weights w_i(t)” to be the same as “modifying the adjustable penalty value by an incremental value,” and interprets “between the actual and target” (with the target defined from loss-derived quantities) to be the same as “based on a difference between loss values associated with the loss vector and a defined preferred loss value,” because they are both directed to per-step adjustments of penalty-like weights using a measured difference to a defined target (i.e., a preferred level derived from the losses) for each component of the loss vector.)
Kung, Chen, Kang, Geifman, Lin, and the instant application are analogous art, because they are all directed to modifying a value based on loss.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the device of claim 22 disclosed by Kung, Chen, Kang, Geifman, and Lin to include “we update our loss weights w_i(t) to move gradient norms towards this target for each task. GradNorm is then implemented as an L1 loss function L_grad” disclosed by Chen One would be motivated to do so to use Carpenters approach of adjusting values based on loss results as suggested by Chen ([Chen, page 3] “we update our loss weights w_i(t) to move gradient norms towards this target for each task. GradNorm is then implemented as an L1 loss function L_grad”.) It would have further been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the device of claim 22 disclosed by Kung, Chen, Kang, Geifman, and Lin to include the multi-task loss-vector optimization with a scalarized, weighted objective disclosed by Lin. One would be motivated to do so to efficiently use the similar equation as the adjustable loss value as suggested by Lin ([Lin, page 3] “combine the losses of all tasks into a single surrogate loss: [Eq.2] where wi is the weight for the i-th task”.)
Regarding claim 24, Kung, Chen, Kang, Geifman, and Lin teaches The device of claim 22 (see rejection of claim 22).
Chen further teaches:
wherein the updating of the machine learning model comprises: setting the adjustable penalty value to a mean value of the conventional loss function as applied to a first batch of the second inputs processed before the first inputs, ([Chen, page 8] “We introduced GradNorm, an efficient algorithm for tuning loss weights in a multi-task learning setting based on balancing the training rates of different tasks.”, wherein the examiner interprets “tuning loss weights” based on task-training signals aggregated at the batch level to be the same as setting the adjustable penalty value to a mean value of the conventional loss function as applied to a first batch of the second inputs processed before the first inputs because they are both directed to determining a loss-side weight from batch-derived loss information obtained from inputs processed earlier in training.)
updating the loss vector based on the adjustable penalty value; ([Chen, page 3] “GradNorm is then implemented as an L1 loss function Lgrad between the actual and target gradient norms…The computed gradients ∇wiLgrad are then applied via standard update rules to update each wi”, wherein the examiner interprets “implemented as an L1 loss function Lgrad” and “applied via standard update rules to update each wi” to be the same as updating the loss vector based on the adjustable penalty value because they are both directed to modifying the loss-side quantities used in training as a function of the current adjustable penalty-related weight.)
repeating the setting of the adjustable penalty value and the updating of the loss vector for other batches of the second inputs, other than the first batch, ([Chen, page 3] “we update our loss weights wi(t) to move gradient norms towards this target for each task”, wherein the examiner interprets “update our loss weights wi(t)” to be the same as repeating the setting of the “adjustable penalty value and the updating of the loss vector for other batches of the second inputs, other than the first batch” because they are both directed to performing the loss-weight adjustment iteratively over successive training iterations/batches rather than only once.)
Kung, Chen, Kang, Geifman, and Lin do not teach until the adjustable penalty value is stabilized according to a defined stability criterion.
Han teaches until the adjustable penalty value is stabilized according to a defined stability criterion. ([Han, page 5, sec 3.2] “GNN is trained by joint losses of both the target and auxiliary tasks, given the weights of different tasks are determined by the weighting model, we define the multi-task objective of GNN as follows: … We train both networks in an iterative manner until convergence.”, wherein the examiner interprets “We train both networks in an iterative manner until convergence” to be the same as until the adjustable penalty value is stabilized according to a defined stability criterion because they are both directed to repeatedly updating a training-related adjustable weighting/penalty term under a defined iterative process until a termination condition reflecting stabilization or convergence is satisfied.)
Kung, Chen, Kang, Geifman, Lin, Han, and the instant application are analogous art because they are all directed to adaptive machine learning training using batch-based loss information to iteratively update weighting.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the device of claim 22 disclosed by Kung, Chen, Kang, Geifman, and Lin to include the adaptive tuning technique disclosed by Han. One would be motivated to do so to effectively ensure that the iteratively updated adjustable penalty value reaches a stable condition before termination of training updates, as suggested by Han ([Han, page 5, sec. 3.2] “We train both networks in an iterative manner until convergence.”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DEVAN KAPOOR whose telephone number is (703)756-1434. The examiner can normally be reached Monday - Friday: 9:00AM - 5:00 PM EST (times may vary).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DEVAN KAPOOR/Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights