DETAILED ACTION
This action is in response to the application field of 05/16/2024. Claims 1-20 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/09/2024. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more
Regarding claim 1:
Subject Matter of Eligibility Analysis Step 1:
Claim 1 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 1 recites
generating for an input a corresponding verification output that takes a first value or a second value, where the verification output taking the first value is statistically correlated with an agent output provided by a first computer-implemented agent in response to the input being valid for the input and the verification output taking the second value is statistically correlated with the agent output provided by the first computer-implemented agent in response to the input not being valid for the input (this limitation is a mental process as it encompasses a human mentally creating two values that depends on whether the output is valid or not).
the verifying comprising: employing a second computer-implemented agent, at least one of the first and second computer-implemented agents acting according to respective ones of a first and second protocol to verify, for a justification composed of a set of logical steps comprising one or more probabilistic agent outputs, whether the one or more probabilistic agent outputs correlate with probabilistic oracle values that would be output by a stochastic oracle (this limitation is a mental process as it encompasses a human mentally comparing outputs).
the first protocol comprising for each of successive ones of the set of logical steps, generating a corresponding probabilistic agent output which is statistically correlated with a corresponding probabilistic oracle value (this limitation is a mental process as it encompasses a human mentally calculating a probabilistic output for each logical step if the equation was given).
the second protocol comprising for each of the successive ones of the logical steps: determining whether the first computer-implemented agent has generated the probabilistic agent output in accordance with the first protocol (this limitation is a mental process as it encompasses a human mentally determining if an output meets the first protocol).
responsive to determining that the first computer-implemented agent has not generated the probabilistic agent output in accordance with the first protocol, generating a warning (this limitation is a mental process as it encompasses a human mentally creating a warning if the first protocol was not met).
a verification protocol comprising: if no warning is generated by the second computer-implemented agent, generating a verification output indicating that the probabilistic agent output is valid (this limitation is a mental process as it encompasses a human mentally creating a valid output when there is no warning).
generating the verification output having the first value when a correlation of the sampled results with the probabilistic agent output meets a first correlation criteria (this limitation is a mental process as it encompasses a human mentally determining a verification output based on a criteria).
generating the second verification output having the second value when the correlation of the sampled results with the probabilistic agent output does not meet the first correlation criteria (this limitation is a mental process as it encompasses a human mentally determining a verification output based on a criteria).
Therefore, claim 1 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 1 recites
if the second computer-implemented agent generates a warning for one of the successive probabilistic logical steps: sampling the stochastic oracle in respect of the probabilistic logical step (this element is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
Therefore, claim 1 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 2 do not provide significantly more than the abstract idea itself, taken alone and in combination because
if the second computer-implemented agent generates a warning for one of the successive probabilistic logical steps: sampling the stochastic oracle in respect of the probabilistic logical step is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
Therefore, claim 1 is subject matter ineligible.
Regarding claim 2:
Subject Matter of Eligibility Analysis Step 1:
Claim 2 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 2 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 2 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 2 further recites additional elements of
the first computer-implemented agent and the second computer-implemented agent are respective instances of the same computer-implemented agent (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
Therefore, claim 2 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 2 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the first computer-implemented agent and the second computer-implemented agent are respective instances of the same computer-implemented agent recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h). Therefore, claim 2 is subject-matter ineligible.
Regarding claim 3:
Subject Matter of Eligibility Analysis Step 1:
Claim 3 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 3 recites
determining a sample mean of the oracle answers (this limitation is a mental process as it encompasses a human mentally calculating the mean of oracle answers if an equation was given).
determining a correlation of the sampled results with the probabilistic agent output comprises determining a correlation between the sample mean and the probabilistic agent output (this limitation is a mental process as it encompasses a human mentally calculating the correlation between a mean and output if the equation was given).
Therefore, claim 3 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 3 further recites additional elements of
outputting a plurality of oracle queries and receiving a plurality of corresponding oracle answers (this limitation is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
Therefore, claim 3 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 3 do not provide significantly more than the abstract idea itself, taken alone and in combination because
outputting a plurality of oracle queries and receiving a plurality of corresponding oracle answers is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014))
Therefore, claim 3 is subject-matter ineligible.
Regarding claim 4:
Subject Matter of Eligibility Analysis Step 1:
Claim 4 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 4 recites
generating a probabilistic agent output which is statistically correlated with the corresponding probabilistic oracle value comprises outputting a probability for the logical step that is equal to a probability that would be generated by the stochastic oracle for the logical step (this limitation is a mathematical concept since it shows the mathematical relationship between the probability agent and the probability oracle).
Therefore, claim 4 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 4 does not further recite any additional elements. Therefore, claim 4 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
Since there are no additional elements, claim 4 does not provide significantly more than the abstract idea itself, taken alone or in combination. Therefore, claim 4 is subject matter ineligible.
Regarding claim 5:
Subject Matter of Eligibility Analysis Step 1:
Claim 5 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 5 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 5 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 5 further recites additional elements of
the first computer-implemented agent obtaining a first query input from an independent copy of the second computer-implemented agent (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
the second computer-implemented agent obtaining a second query input from an independent copy of the first computer-implemented agent (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 5 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 5 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the first computer-implemented agent obtaining a first query input from an independent copy of the second computer-implemented agent is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
the second computer-implemented agent obtaining a second query input from an independent copy of the first computer-implemented agent is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 5 is subject-matter ineligible.
Regarding claim 6:
Subject Matter of Eligibility Analysis Step 1:
Claim 6 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 6 recites
generating a probabilistic agent output which is statistically correlated with the corresponding probabilistic oracle value comprises outputting a probability for the logical step that is equal to a probability that would be generated by the stochastic oracle for the logical step (this limitation is a mathematical concept since it shows the mathematical relationship between the probability agent and the probability oracle).
generating the first protocol output comprises determining a single query input based on the first and second query inputs (this limitation is a mental process as it encompasses a human mentally combining two inputs into one).
setting the first protocol output based on whether the probability for the logical step is greater than the combined single query input (this limitation is a mental process as it encompasses a human mentally determining an output based on the comparison of a probability and input).
Therefore, claim 6 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 6 does not further recite any additional elements. Therefore, claim 6 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
Since there are no additional elements, claim 6 does not provide significantly more than the abstract idea itself, taken alone or in combination. Therefore, claim 6 is subject matter ineligible.
Regarding claim 7:
Subject Matter of Eligibility Analysis Step 1:
Claim 7 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 7 recites
a logical step t the probability for the logical step is given by
p
^
t
=
c
^
t
d, where
c
^
t
=0,…,d, d is a positive integer (this limitation is a mathematical concept since a mathematical equation is given).
Therefore, claim 7 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 7 further recites additional elements of
the first computer-implemented agent obtaining a first query input from an independent copy of the second computer-implemented agent comprises querying the independent copy of the second computer-implemented agent for a random integer value sampled uniformly from 0,…,d (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
the second computer-implemented agent obtaining a second query input from an independent copy of the first computer-implemented agent comprises querying the independent copy of the first computer-implemented agent for a random integer value sampled uniformly from 0,…,d (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 7 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 7 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the first computer-implemented agent obtaining a first query input from an independent copy of the second computer-implemented agent comprises querying the independent copy of the second computer-implemented agent for a random integer value sampled uniformly from 0,…,d is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
the second computer-implemented agent obtaining a second query input from an independent copy of the first computer-implemented agent comprises querying the independent copy of the first computer-implemented agent for a random integer value sampled uniformly from 0,…,d is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 7 is subject-matter ineligible.
Regarding claim 8:
Subject Matter of Eligibility Analysis Step 1:
Claim 8 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 8 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 8 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 8 further recites additional elements of
determining, by the second computer-implemented agent, whether the first computer-implemented agent has generated the probabilistic agent output in accordance with the first protocol comprises sampling the stochastic oracle in respect of the probabilistic logical step (this element is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
Therefore, claim 8 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 8 do not provide significantly more than the abstract idea itself, taken alone and in combination because
determining, by the second computer-implemented agent, whether the first computer-implemented agent has generated the probabilistic agent output in accordance with the first protocol comprises sampling the stochastic oracle in respect of the probabilistic logical step is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
Therefore, claim 8 is subject-matter ineligible.
Regarding claim 9:
Subject Matter of Eligibility Analysis Step 1:
Claim 9 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 9 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 9 is applied here. Therefore claim 1 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 9 further recites additional elements of
the first and second computer-implemented agents are sequence models (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
Therefore, claim 9 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 9 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the first and second computer-implemented agents are sequence models recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
Therefore, claim 9 is subject-matter ineligible.
Regarding claim 10:
Subject Matter of Eligibility Analysis Step 1:
Claim 10 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 10 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 10 is applied here. Therefore claim 1 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 10 further recites additional elements of
the first and second computer-implemented agents are multi-modal sequence models (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
Therefore, claim 10 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 10 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the first and second computer-implemented agents are multi-modal sequence models recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
Therefore, claim 10 is subject-matter ineligible.
Regarding claim 11:
Subject Matter of Eligibility Analysis Step 1:
Claim 11 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 11 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 11 is applied here. Therefore claim 1 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 11 further recites additional elements of
the verification protocol is performed by a verifier that is computationally limited compared to the first and second computer-implemented agents (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
Therefore, claim 11 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 11 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the verification protocol is performed by a verifier that is computationally limited compared to the first and second computer-implemented agents recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
Therefore, claim 11 is subject-matter ineligible.
Regarding claim 12:
Subject Matter of Eligibility Analysis Step 1:
Claim 12 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 12 recites
the stochastic oracle comprises one or more human agents (this limitation is a mental process since human agents are creating the stochastic oracle).
Therefore, claim 12 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 12 does not further recite any additional elements. Therefore, claim 12 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
Since there are no additional elements, claim 12 does not provide significantly more than the abstract idea itself, taken alone or in combination. Therefore, claim 12 is subject matter ineligible.
Regarding claim 13:
Subject Matter of Eligibility Analysis Step 1:
Claim 13 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 13 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 13 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 13 further recites additional elements of
the stochastic oracle comprises one or more sensors measuring a real-world environment ((this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).)
Therefore, claim 13 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 13 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the stochastic oracle comprises one or more sensors measuring a real-world environment recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
Therefore, claim 13 is subject-matter ineligible.
Regarding claim 14:
Subject Matter of Eligibility Analysis Step 1:
Claim 14 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 14 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 14 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 14 further recites additional elements of
the input indicates a security breach in a computing device or computer-network (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
the agent output comprises computer program code for execution by the device or computer network (this element does not integrate the abstract idea into a practical application because it a generic computing component on which to perform the abstract idea (see MPEP 2106.05(f))).
the verification output having the first value indicates that the output is configured to cause one or more computers to perform one or more actions configured to address the security breach (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 14 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 14 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the input indicates a security breach in a computing device or computer-network recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
the agent output comprises computer program code for execution by the device or computer network uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f))
the verification output having the first value indicates that the output is configured to cause one or more computers to perform one or more actions configured to address the security breach is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 14 is subject-matter ineligible.
Regarding claim 15:
Subject Matter of Eligibility Analysis Step 1:
Claim 15 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 15 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 15 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 15 further recites additional elements of
the input comprises input data derived from one or more sensors, each sensor input indicating one or more properties of one or more physical objects in a real-word environment
the input further comprising a request to output one or more instructions to complete a task including an action on or using the one or more physical objects (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
the agent output comprises one or more instructions for execution by a real-world agent interacting with the environment (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
the verification output having the first value indicates that execution of the one or more instructions by the real-world agent will result in completion of the action on or using the one or more physical objects (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 15 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 15 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the input comprises input data derived from one or more sensors, each sensor input indicating one or more properties of one or more physical objects in a real-word environment
the input further comprising a request to output one or more instructions to complete a task including an action on or using the one or more physical objects recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
the agent output comprises one or more instructions for execution by a real-world agent interacting with the environment is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
the verification output having the first value indicates that execution of the one or more instructions by the real-world agent will result in completion of the action on or using the one or more physical objects is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 15 is subject-matter ineligible.
Regarding claim 16:
Subject Matter of Eligibility Analysis Step 1:
Claim 16 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 16 is dependent on claim 15, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 15 is applied here. Therefore claim 16 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 16 further recites additional elements of
selectively controlling the real-world agent to perform action when the verification output has the first value and not controlling the real-world agent to perform the action when the verification output has the second value (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 16 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 16 do not provide significantly more than the abstract idea itself, taken alone and in combination because
selectively controlling the real-world agent to perform action when the verification output has the first value and not controlling the real-world agent to perform the action when the verification output has the second value is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 16 is subject-matter ineligible.
Regarding claim 17:
Subject Matter of Eligibility Analysis Step 1:
Claim 17 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 17 is dependent on claim 15, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 15 is applied here. Therefore claim 17 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 17 further recites additional elements of
the verification output having the first value indicates that completion of the action meets a safety criteria (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 17 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 17 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the verification output having the first value indicates that completion of the action meets a safety criteria is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 17 is subject-matter ineligible.
Regarding claim 18:
Subject Matter of Eligibility Analysis Step 1:
Claim 18 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 18 recites
the method further comprises generating a final verification output based on the plurality of verification outputs (this limitation is a mental process as it encompasses a human mentally creating an output based on other outputs).
Therefore, claim 18 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 18 further recites additional elements of
the verification output is one of a plurality of verification outputs obtained by performing the verifying a plurality of times (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
Therefore, claim 18 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 18 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the verification output is one of a plurality of verification outputs obtained by performing the verifying a plurality of times recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
Therefore, claim 18 is subject-matter ineligible.
Regarding claim 19:
Subject Matter of Eligibility Analysis Step 1:
Claim 19 recites a computer-readable storage medium, which may be interpreted as a signal or carrier wave, and thus is not one of the four statutory categories of patentable subject matter. Therefore, claim 19 is rejected as being directed towards signals per se.
Further, in the interest of compact prosecution, assuming claim 19 is amended to include statutory subject matter, claim 19 is further rejected as being directed towards an abstract idea as shown by the analysis below.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 19 recites
generating for an input a corresponding verification output that takes a first value or a second value, where the verification output taking the first value is statistically correlated with an agent output provided by a first computer-implemented agent in response to the input being valid for the input and the verification output taking the second value is statistically correlated with the agent output provided by the first computer-implemented agent in response to the input not being valid for the input (this limitation is a mental process as it encompasses a human mentally creating two values that depends on whether the output is valid or not).
the verifying comprising: employing a second computer-implemented agent, at least one of the first and second computer-implemented agents acting according to respective ones of a first and second protocol to verify, for a justification composed of a set of logical steps comprising one or more probabilistic agent outputs, whether the one or more probabilistic agent outputs correlate with probabilistic oracle values that would be output by a stochastic oracle (this limitation is a mental process as it encompasses a human mentally comparing outputs).
the first protocol comprising for each of successive ones of the set of logical steps, generating a corresponding probabilistic agent output which is statistically correlated with a corresponding probabilistic oracle value (this limitation is a mental process as it encompasses a human mentally calculating a probabilistic output for each logical step if the equation was given).
the second protocol comprising for each of the successive ones of the logical steps: determining whether the first computer-implemented agent has generated the probabilistic agent output in accordance with the first protocol (this limitation is a mental process as it encompasses a human mentally determining if an output meets the first protocol).
responsive to determining that the first computer-implemented agent has not generated the probabilistic agent output in accordance with the first protocol, generating a warning (this limitation is a mental process as it encompasses a human mentally creating a warning if the first protocol was not met).
a verification protocol comprising: if no warning is generated by the second computer-implemented agent, generating a verification output indicating that the probabilistic agent output is valid (this limitation is a mental process as it encompasses a human mentally creating a valid output when there is no warning).
generating the verification output having the first value when a correlation of the sampled results with the probabilistic agent output meets a first correlation criteria (this limitation is a mental process as it encompasses a human mentally determining a verification output based on a criteria).
generating the second verification output having the second value when the correlation of the sampled results with the probabilistic agent output does not meet the first correlation criteria (this limitation is a mental process as it encompasses a human mentally determining a verification output based on a criteria).
Therefore, claim 19 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 19 further recites additional elements of
One or more computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform (this element does not integrate the abstract idea into a practical application because it a generic computing component on which to perform the abstract idea (see MPEP 2106.05(f))).
if the second computer-implemented agent generates a warning for one of the successive probabilistic logical steps: sampling the stochastic oracle in respect of the probabilistic logical step (this element is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
Therefore, claim 19 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 19 do not provide significantly more than the abstract idea itself, taken alone and in combination because
One or more computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f))
if the second computer-implemented agent generates a warning for one of the successive probabilistic logical steps: sampling the stochastic oracle in respect of the probabilistic logical step is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
Therefore, claim 19 is subject-matter ineligible.
Regarding claim 20:
Subject Matter of Eligibility Analysis Step 1:
Claim 20 recites a system, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 20 recites
generating for an input a corresponding verification output that takes a first value or a second value, where the verification output taking the first value is statistically correlated with an agent output provided by a first computer-implemented agent in response to the input being valid for the input and the verification output taking the second value is statistically correlated with the agent output provided by the first computer-implemented agent in response to the input not being valid for the input (this limitation is a mental process as it encompasses a human mentally creating two values that depends on whether the output is valid or not).
the verifying comprising: employing a second computer-implemented agent, at least one of the first and second computer-implemented agents acting according to respective ones of a first and second protocol to verify, for a justification composed of a set of logical steps comprising one or more probabilistic agent outputs, whether the one or more probabilistic agent outputs correlate with probabilistic oracle values that would be output by a stochastic oracle (this limitation is a mental process as it encompasses a human mentally comparing outputs).
the first protocol comprising for each of successive ones of the set of logical steps, generating a corresponding probabilistic agent output which is statistically correlated with a corresponding probabilistic oracle value (this limitation is a mental process as it encompasses a human mentally calculating a probabilistic output for each logical step if the equation was given).
the second protocol comprising for each of the successive ones of the logical steps: determining whether the first computer-implemented agent has generated the probabilistic agent output in accordance with the first protocol (this limitation is a mental process as it encompasses a human mentally determining if an output meets the first protocol).
responsive to determining that the first computer-implemented agent has not generated the probabilistic agent output in accordance with the first protocol, generating a warning (this limitation is a mental process as it encompasses a human mentally creating a warning if the first protocol was not met).
a verification protocol comprising: if no warning is generated by the second computer-implemented agent, generating a verification output indicating that the probabilistic agent output is valid (this limitation is a mental process as it encompasses a human mentally creating a valid output when there is no warning).
generating the verification output having the first value when a correlation of the sampled results with the probabilistic agent output meets a first correlation criteria (this limitation is a mental process as it encompasses a human mentally determining a verification output based on a criteria).
generating the second verification output having the second value when the correlation of the sampled results with the probabilistic agent output does not meet the first correlation criteria (this limitation is a mental process as it encompasses a human mentally determining a verification output based on a criteria).
Therefore, claim 20 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 20 further recites additional elements of
A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform (this element does not integrate the abstract idea into a practical application because it a generic computing component on which to perform the abstract idea (see MPEP 2106.05(f))).
if the second computer-implemented agent generates a warning for one of the successive probabilistic logical steps: sampling the stochastic oracle in respect of the probabilistic logical step (this element is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
Therefore, claim 20 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 20 do not provide significantly more than the abstract idea itself, taken alone and in combination because
A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
if the second computer-implemented agent generates a warning for one of the successive probabilistic logical steps: sampling the stochastic oracle in respect of the probabilistic logical step is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
Therefore, claim 20 is subject-matter ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 3, 4, 9, and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anil et al. (Learning to Give Checkable Answers with Prover-Verifier Games) (hereafter referred to as Anil) in view of Uesato et al. (Solving math word problems with process and outcome-based feedback) (hereafter referred to as Uesato), Sauders et al. (Self-critiquing Models for Assisting Human Evaluators) (hereafter referred to as Sauders), and Madras et al. (Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer) (hereafter referred to as Madras) and Dawid et al. (The Well-Calibrated Bayesian) (hereafter referred to as Dawid).
Regarding claim 1, Anil teaches
A computer-implemented method of generating for an input a corresponding verification output that takes a first value or a second value, where the verification output taking the first value is statistically correlated with an agent output provided by a first computer-implemented agent in response to the input being valid for the input and the verification output taking the second value is statistically correlated with the agent output provided by the first computer-implemented agent in response to the input not being valid for the input (Anil, Section 3.3, “The verifier chooses a policy v which computes a convex combination of elements of V. (The verifier’s output can be interpreted as the probability it assigns to the label being 1)” and “Guarantee of Perfect Precision: There shouldn’t exist any prover which can trick the verifier into achieving non-perfect precision…implies that the verifier always has zero false positive rate regardless of which proof is used” (Anil, Section 3.1). Examiner notes that the verifier’s output is mapped to the first or second value.).
the verifying comprising employing a second computer-implemented agent, at least one of the first and second computer-implemented agents acting according to respective ones of a first and second protocol to verify, for a justification composed of a set of logical steps comprising one or more probabilistic agent outputs, whether the one or more probabilistic agent outputs correlate with probabilistic oracle values that would be output by a stochastic oracle (Anil, Abstract, “The PVG consists of two learners with competing objectives: a trusted verifier network tries to choose the correct answer, and a more powerful but untrusted prover network attempts to persuade the verifier of a particular answer, regardless of its correctness”. Examiner notes that the verifier network maps to the second agent).
the second protocol comprising for each of the successive ones of the logical steps: determining whether the first computer-implemented agent has generated the probabilistic agent output in accordance with the first protocol (Anil, Section 1, “Since the prover is untrustworthy, the verifier will only find its messages useful to the extent that it can independently verify the information”).
a verification protocol comprising: if no warning is generated by the second computer-implemented agent, generating a verification output indicating that the probabilistic agent output is valid (Anil, Figure 1)
PNG
media_image1.png
334
653
media_image1.png
Greyscale
Examiner notes that image 1b maps to the output being valid when there is no message.
Anil does not teach, but Uesato does teach
the verifying comprising employing a second computer-implemented agent, at least one of the first and second computer-implemented agents acting according to respective ones of a first and second protocol to verify, for a justification composed of a set of logical steps comprising one or more probabilistic agent outputs, whether the one or more probabilistic agent outputs correlate with probabilistic oracle values that would be output by a stochastic oracle (Uesato, Section 2.4, “For the process-supervised RM (PRM), the binary label after each step indicates whether the steps so far are correct ...A policy which maximizes the PRM score thus selects each step to maximize the RM-estimated probability of the steps so far being correct”).
the first protocol comprising for each of successive ones of the set of logical steps, generating a corresponding probabilistic agent output which is statistically correlated with a corresponding probabilistic oracle value (Uesato, Section 2.4, “For the process-supervised RM (PRM), the binary label after each step indicates whether the steps so far are correct ...A policy which maximizes the PRM score thus selects each step to maximize the RM-estimated probability of the steps so far being correct”).
Anil and Uesato are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil to use PRM from Uesato for justification. Uesato teaches that “this outperforms the approach from Li et al. (2022), which is similar to our PRM but replaces human evaluations with a heuristic based on string matching the results of the intermediate calculations” (Uesato, Section 2.4) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Anil and Uesato does not teach, but Saunders does teach
the first protocol comprising for each of successive ones of the set of logical steps, generating a corresponding probabilistic agent output which is statistically correlated with a corresponding probabilistic oracle value (Saunders, Section 2.3, “For these tasks, we don’t require human data collection because we have binary ground truth for both answer and critique validity. We use hand-coded oracles for each of the base, critiqueability, critique, and helpfulness tasks”).
responsive to determining that the first computer-implemented agent has not generated the probabilistic agent output in accordance with the first protocol, generating a warning (Saunders, Section 2.2.2, “the critiqueability score can be used to determine whether to ask it to critique in the first place, and the helpfulness score can be used to determine whether the critique is good after the fact”. Examiner notes that the critique maps to the warning).
Anil, Uesato, and Saunders are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil and Uesato to critique models from Saunders. Saunders teaches that “It’s possible that our model can identify and critique all of its mistakes. This motivates us to look at the percentage of the time poor outputs have helpful critiques. The higher this percentage, the easier it will be to assist humans in evaluation of the base task” (Saunders, Section 4.2) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Anil, Uesato, and Saunders do not teach, but Madras does teach
the verifying comprising employing a second computer-implemented agent, at least one of the first and second computer-implemented agents acting according to respective ones of a first and second protocol to verify, for a justification composed of a set of logical steps comprising one or more probabilistic agent outputs, whether the one or more probabilistic agent outputs correlate with probabilistic oracle values that would be output by a stochastic oracle (Madras, Section 2.3, “if the DM has constant loss (e.g. is an oracle), there exist values of γreject, γdefer for which the learning-to-defer and rejection learning objectives are equivalent” and (Madras, Section 2.1)).
PNG
media_image2.png
336
598
media_image2.png
Greyscale
Examiner notes that
y
^
_
D
maps to the probabilistic oracle values.
if the second computer-implemented agent generates a warning for one of the successive probabilistic logical steps: sampling the stochastic oracle in respect of the probabilistic logical step (Madras, Section 2.1, “The decision-making flow is modeled as a cascade, where the first-step model can either predict (positive/negative) or say pass. If it predicts, the DM will output the model’s prediction. However, if it says pass, the DM makes its own decision” and “We calculate these by sampling J times from the model, yielding J predictions zj ∈ [0,1]. Our prediction p is the sample mean…the system can threshold this uncertainty; any example with uncertainty beyond a threshold is rejected, and passed to the DM” (Appendix F)).
generating the verification output having the first value when a correlation of the sampled results with the probabilistic agent output meets a first correlation criteria and generating the second verification output having the second value when the correlation of the sampled results with the probabilistic agent output does not meet the first correlation criteria (Madras, Appendix F).
PNG
media_image3.png
152
590
media_image3.png
Greyscale
Examiner notes that the uncertainty threshold maps to the first correlation criteria and when that threshold is not reached, then the model’s output is used, which maps to the first value. If the threshold is reached, then DM’s output is used, which maps to the second value.
Anil, Uesato, Saunders, and Madras are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, and Saunders to sample stochastic oracles and learning to defer from Madras. Madras teaches that “Experiments demonstrate that learning to defer can make systems not only more accurate but also less biased. Even when working with inconsistent or biased users, we show that deferring models still greatly improve the accuracy and/or fairness of the entire system” (Madras, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Anil, Uesato, Saunders, and Madras do not teach, but Dawid does teach
generating the verification output having the first value when a correlation of the sampled results with the probabilistic agent output meets a first correlation criteria and generating the second verification output having the second value when the correlation of the sampled results with the probabilistic agent output does not meet the first correlation criteria (Dawid, Section 3, “One way of comparing forecasts with reality is to pick out some fairly arbitrary test set of days, and in it compare (a) the proportion p of days whose associated events in fact occur with (b) the average forecast probability π for those days.).
Anil, Uesato, Saunders, Madras, and Dawid are considered analogous to the claimed invention because they deal with validating outputs. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, Saunders, and Madras to compare forecasts with reality. One of the ordinary skill in the art would have known to apply the known technique of comparing ground truth data with the probabilistic output. Therefore, applying Dawid’s technique would yield the predicable result of determining the accuracy of a machine learning model (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Regarding claim 3, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 1, Anil, Uesatos, Saunders, and Madras do not teach, but Dawid does teach
outputting a plurality of oracle queries and receiving a plurality of corresponding oracle answers (Dawid, Section 3)
PNG
media_image4.png
77
341
media_image4.png
Greyscale
Examiner notes that day i maps to the oracle queries and Y_i maps to the corresponding oracle answers.
determining a sample mean of the oracle answers (Dawid, Section 3)
PNG
media_image5.png
79
430
media_image5.png
Greyscale
“That is, restricting attention to those days up to day k selected for inclusion in the test set, Vk is the number of such days, Pk the proportion for which the associated events in fact occur”. Examiner notes that the equation maps to the sample mean.
determining a correlation of the sampled results with the probabilistic agent output comprises determining a correlation between the sample mean and the probabilistic agent output (Dawid, Section 3, “One way of comparing forecasts with reality is to pick out some fairly arbitrary test set of days, and in it compare (a) the proportion p of days whose associated events in fact occur with (b) the average forecast probability πk for those days.” Examiner notes that (a) maps to the sample mean and (b) maps to the probabilistic ouput).
Anil, Uesato, Saunders, Madras, and Dawid are considered analogous to the claimed invention because they deal with validating outputs. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, Saunders, and Madras to compare forecasts with reality. One of the ordinary skill in the art would have known to apply the known technique of comparing ground truth data with the probabilistic output. Therefore, applying Dawid’s technique would yield the predicable result of determining the accuracy of a machine learning model (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Regarding claim 4, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 1, Anil, Uesatos, Saunders, and Dawid do not teach, but Madras does teach
generating a probabilistic agent output which is statistically correlated with the corresponding probabilistic oracle value comprises outputting a probability for the logical step that is equal to a probability that would be generated by the stochastic oracle for the logical step (Madras, Section 2.1)
PNG
media_image2.png
336
598
media_image2.png
Greyscale
Examiner notes that
Y
^
M
and
y
^
D
are probabilities on the same event (Y=1), which means they match each other
Anil, Uesato, Saunders, and Madras are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, and Saunders to sample stochastic oracles and learning to defer from Madras. Madras teaches that “Experiments demonstrate that learning to defer can make systems not only more accurate but also less biased. Even when working with inconsistent or biased users, we show that deferring models still greatly improve the accuracy and/or fairness of the entire system” (Madras, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Regarding claim 8, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 1, Anil further teaches
wherein determining, by the second computer-implemented agent, whether the first computer-implemented agent has generated the probabilistic agent output in accordance with the first protocol (Anil, Section 1, “Since the prover is untrustworthy, the verifier will only find its messages useful to the extent that it can independently verify the information”).
sampling the stochastic oracle in respect of the probabilistic logical step (Madras, Section 2.1, “The decision-making flow is modeled as a cascade, where the first-step model can either predict (positive/negative) or say pass. If it predicts, the DM will output the model’s prediction. However, if it says pass, the DM makes its own decision” and “We calculate these by sampling J times from the model, yielding J predictions zj ∈ [0,1]. Our prediction p is the sample mean…the system can threshold this uncertainty; any example with uncertainty beyond a threshold is rejected, and passed to the DM” (Appendix F)).
Anil, Uesato, Saunders, and Madras are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, and Saunders to sample stochastic oracles and learning to defer from Madras. Madras teaches that “Experiments demonstrate that learning to defer can make systems not only more accurate but also less biased. Even when working with inconsistent or biased users, we show that deferring models still greatly improve the accuracy and/or fairness of the entire system” (Madras, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Regarding claim 9, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 1, Anil, Uesatos, Saunders, and Madras do not teach, but Saunders does teach
the first and second computer-implemented agents are sequence models (Saunders, Abstract, “We fine-tune large language models to write natural language critiques”. Examiner notes that large language models are sequence models).
Anil, Uesatos, Saunders, Madras, and Dawid are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesatos, Madras, and Dawid to critique models from Saunders. Saunders teaches that “It’s possible that our model can identify and critique all of its mistakes. This motivates us to look at the percentage of the time poor outputs have helpful critiques. The higher this percentage, the easier it will be to assist humans in evaluation of the base task” (Saunders, Section 4.2) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Regarding claim 11, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 1, Anil further teaches
the verification protocol is performed by a verifier that is computationally limited compared to the first and second computer-implemented agents (Anil, Section 1, “Most of these notions can be thought of in terms of a game between a powerful but untrusted prover and a computationally limited but trusted verifier”).
Claim(s) 2 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anil in view of Uesato, Sauders, Madras, Dawid and Irving et al. (AI Safety via Debate) (hereafter referred to as Irving).
Regarding claim 2, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 1, Anil, Uesatos, Saunders, and Madras do not teach, but Irving does teach
the first computer-implemented agent and the second computer-implemented agent are respective instances of the same computer-implemented agent (Irving, Section 6.1, “Symmetry between the agents’ capabilities is easy to achieve, since we can use the same weights for both agents via self play”).
Anil, Uesatos, Saunders, Madras, Dawid, and Irving are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesatos, Madras, and Dawid to use self-play with shared weights from Irving. Irving teaches “Symmetry between the agents’ capabilities is easy to achieve” (Irving, Section 6.1) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Regarding claim 12, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 1, Anil, Uesatos, Saunders, and Madras do not teach, but Irving does teach
the stochastic oracle comprises one or more human agents (Irving, Figure 1)
PNG
media_image6.png
266
595
media_image6.png
Greyscale
Anil, Uesatos, Saunders, Madras, Dawid, and Irving are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesatos, Madras, and Dawid to include human judges from Irving. One of the ordinary skill in the art would have known to apply the known technique of including human agents in stochastic oracles when verifying outputs. Therefore, applying Irving’s technique would yield the predicable result of machine learning models learning from real-world applications (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anil in view of Uesato, Sauders, Madras, Dawid and Barnes et al. (Writeup: Progress on AI Safety via Debate) (hereafter referred to as Barnes).
Regarding claim 5, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 1, Anil, Uesatos, Saunders, and Madras do not teach, but Barnes does teach
the first computer-implemented agent obtaining a first query input from an independent copy of the second computer-implemented agent; and the second computer-implemented agent obtaining a second query input from an independent copy of the first computer-implemented agent (Barnes, Section: Progress so far, “we allow a debater to “cross-examine” multiple copies of the opposing debater who are not allowed to communicate” and “In lieu of making an argument, a debater may decide to cross-examine. They choose a prior claim, objection, or cross-ex answer to ask about, and write a question of at most 200 characters. We then create a copy of the debater who wrote the original text, at the time when they wrote it (i.e. directly after writing the argument, writing the objections, or writing the answer). That copy answers the question” (Barnes, Section: Comprehension rules) Examiner notes that debaters map to the agents).
Anil, Uesato, Saunders, Madras, Dawid, and Barnes are considered analogous to the claimed invention because they deal with validating outputs. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, Saunders, Madras, and Dawid to “cross-examine” other debaters from Barnes. One of the ordinary skill in the art would have known to apply the known technique of cross examining between different inputs. Therefore, applying Barnes’ technique would yield the predicable result of avoiding biases (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anil in view of Uesato, Sauders, Madras, Dawid, Barnes and Ben-Or et al. (US 4926479 A) (hereafter referred to as Ben-Or).
Regarding claim 6, Anil, Uesatos, Saunders, Madras, Dawid, and Barnes teach the method of claim 5, Anil, Uesatos, Saunders, and Dawid do not teach, but Madras does teach
generating a probabilistic agent output which is statistically correlated with the corresponding probabilistic oracle value comprises outputting a probability for the logical step that is equal to a probability that would be generated by the stochastic oracle for the logical step (Madras, Section 2.1)
PNG
media_image2.png
336
598
media_image2.png
Greyscale
Examiner notes that
Y
^
M
and
y
^
D
are probabilities on the same event (Y=1), which means they match each other
Anil, Uesato, Saunders, Madras, and Dawid are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, Saunders, and Dawid to sample stochastic oracles and learning to defer from Madras. Madras teaches that “Experiments demonstrate that learning to defer can make systems not only more accurate but also less biased. Even when working with inconsistent or biased users, we show that deferring models still greatly improve the accuracy and/or fairness of the entire system” (Madras, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Anil, Uesato, Saunders, Madras, and Dawid do not teach, but Ben-Or does teach
wherein generating the first protocol output comprises determining a single query input based on the first and second query inputs (Ben-Or, paragraph 0152, “Verifier V tosses all his coins and sends them to prover P.sub.1. In return, P.sub.1 sends V the entire history of communication that would have occurred for theses coin tosses between the real verifier V and the k real provers P.sub.i 's. If this is an accepting conversation for V, V now uses P.sub.2 to check the validity of the conversation. This is done by V selecting at random an original prover P.sub.i, and simulating with P.sub.2 the conversation between V and P.sub.i on these coin tosses”. Examiner notes that P.sub.1 and P.sub.2 maps to the first and second inputs, respectively).
setting the first protocol output based on whether the probability for the logical step is greater than the combined single query input (Ben-Or, paragraph 0152, “If the conversation does not match the conversation sent by P.sub.1 then V rejects, otherwise the protocol is repeated k times (in series) and finally V accepts”).
Anil, Uesato, Saunders, Madras, Dawid, Barnes, and Ben-Or are considered analogous to the claimed invention because they deal with validating outputs. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, Saunders, Madras, Dawid, and Ben-Or to combine different inputs from Ben-Or. One of the ordinary skill in the art would have known to apply the known technique of using multiple inputs to validate. Therefore, applying Ben-Or’s technique would yield the predicable result of maintaining accuracy (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anil in view of Uesato, Sauders, Madras, Dawid and Alayrac et al. (Flamingo: a Visual Language Model for Few-Shot Learning) (hereafter referred to as Alayrac).
Regarding claim 10, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 1, Anil further teaches
the first and second computer-implemented agents are multi-modal sequence models (Anil, Abstract, “The PVG consists of two learners with competing objectives: a trusted verifier network tries to choose the correct answer, and a more powerful but untrusted prover network attempts to persuade the verifier of a particular answer, regardless of its correctness”)
Anil, Uesatos, Saunders, Madras, and Dawid do not teach, but Alayrac does teach
the first and second computer-implemented agents are multi-modal sequence models (Alayrac, Abstract, “We propose key architectural innovations to: (i) bridge powerful pretrained vision-only and language-only models, (ii) handle sequences of arbitrarily interleaved visual and textual data, and (iii) seamlessly ingest images or videos as inputs”).
Anil, Uesato, Saunders, Madras, Dawid, and Flamingo are considered analogous to the claimed invention because they deal with large language models. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, Saunders, Madras, and Dawid to apply to multi-modal sequence models such as Flamingo from Alayrac. One of the ordinary skill in the art would have known to apply the known technique of verifying outputs for multi-modal sequences. Therefore, applying Alayrac’s technique would yield the predicable result of creating accurate outputs when combining different types of data (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anil in view of Uesato, Sauders, Madras, Dawid and Franco et al. (Interpretable Run-Time Monitoring and Replanning for Safe Autonomous Systems Operations) (hereafter referred to as Franco).
Regarding claim 13, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 1, Anil, Uesatos, Saunders, Madras, and Dawid do not teach, but Franco does teach
the stochastic oracle comprises one or more sensors measuring a real-world environment (Franco, abstract, “Autonomous robots, especially aerial vehicles, when subject to disturbances, uncertainties, and noises may experience variations from their desired states and deviations from the planned trajectory which may lead them into an unsafe state (e.g., a collision). It is thus necessary to monitor their states at run-time when operating in uncertain and cluttered environments and intervene to guarantee their and the surrounding’s safety…In this work we propose a novel approach for run-time monitoring that leverages a library of previously observed trajectories together with decision tree theory to predict if the system will be safe/unsafe and provide an explanation to understand the causes of the prediction”).
Anil, Uesatos, Saunders, Madras, Dawid, and Franco are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesatos, Madras, and Dawid to include environmental data from Franco. One of the ordinary skill in the art would have known to apply the known technique of including environmental data in stochastic oracles when verifying outputs. Therefore, applying Franco technique would yield the predicable result of machine learning models learning from real-world applications (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anil in view of Uesato, Sauders, Madras, Dawid and Pearce et al. (Can OpenAI Codex and Other Large Language Models Help Us Fix Security Bugs) (hereafter referred to as Pearce).
Regarding claim 14, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 1, Anil, Uesatos, Saunders, and Madras do not teach, but Pearce does teach
the input indicates a security breach in a computing device or computer-network (Pearce, Abstract, “Human developers can produce code with cyber security bugs. Can emerging ‘smart’ code completion tools help repair those bugs? In this work, we examine the use of large language models (LLMs) for code (such as OpenAI’s Codex and AI21’s Jurassic J-1) for zero-shot vulnerability repair...our experiments show that LLMs could collectively repair 100% of our synthetically generated and hand-crafted scenarios, as well as 58% of vulnerabilities in a selection of historical bugs in real-world open-source projects”).
the agent output comprises computer program code for execution by the device or computer network (Pearce, Figure 2)
PNG
media_image7.png
384
673
media_image7.png
Greyscale
Examiner notes that the potential repaired code maps to the computer program code.
the verification output having the first value indicates that the output is configured to cause one or more computers to perform one or more actions configured to address the security breach (Pearce, Section V-B, “With this information in hand, we can check out the vulnerable version of the project, build it with a sanitizer, and then attempt repair, using the sanitizer output as an oracle to test whether the vulnerability has been fixed and the regression tests to ensure that the fix does not break other functionality” and “we define a project ‘repaired’ when the replaced code results in a compiled program passes both the functional tests included with each program and when it no longer crashes with the ASAN/UBSAN triggering input” (Pearce, Section V-D)).
Anil, Uesatos, Saunders, Madras, Dawid, and Pearce are considered analogous to the claimed invention because they deal with large language models. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesatos, Madras, and Dawid to apply to fixing cyber security bugs from Pearce. One of the ordinary skill in the art would have known to apply the known technique of validating machine learning outputs in cyber security. Therefore, applying Irving’s technique would yield the predicable result of increasing the strength of cyber security programs (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 15 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anil in view of Uesato, Sauders, Madras, Dawid and Ahn et al. (Do As I Can, Not As I Say: Grounding Language in Robotic Affordances) (hereafter referred to as Ahn).
Regarding claim 15, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 1, Anil, Uesatos, Saunders, and Madras do not teach, but Ahn does teach
the input comprises input data derived from one or more sensors, each sensor input indicating one or more properties of one or more physical objects in a real-word environment (Ahn, Appendix C.1, “The RL models use an architecture similar to MT-Opt [14], with slight changes to support natural language inputs (see Fig. 9 for the network diagram). The camera image is first processed by 7 convolutional layers. The language instruction is embedded by the LLM, then concatenated with the robot action and non-image parts of the state, such as the gripper height” and (Ahn, Figure 2))
PNG
media_image8.png
308
931
media_image8.png
Greyscale
the input further comprising a request to output one or more instructions to complete a task including an action on or using the one or more physical objects (Ahn, Section 3, “Our system receives a user-provided natural language instruction i that de scribes a task that the robot should execute. The instruction can be long, abstract, or ambiguous”).
the agent output comprises one or more instructions for execution by a real-world agent interacting with the environment (Ahn, Section 5.1, “Returning to our initial example, “I spilled something, can you help?”, an ungrounded language model would respond with statements like “I can call you a cleaner” or “I can vacuum that up for you”, which given our robot are unreasonable. We have shown that PaLM-SayCan responds “I would: 1. find a sponge, 2. pick up the sponge, 3. bring it to you, 4. done” and is able execute this sequence on the robot in a real kitchen” and (Ahn, Figure 3))
PNG
media_image9.png
294
710
media_image9.png
Greyscale
verification output having the first value indicates that execution of the one or more instructions by the real-world agent will result in completion of the action on or using the one or more physical objects (Ahn, Section 3, “an affordance function p(cπ|s, lπ), which indicates the probability of c-ompleting the skill with description lπ successfully from state s. Intuitively, p(cπ|s, lπ) means “if I ask the robot to do lπ, will it do it?”. In RL terminology, p(cπ|s, lπ) is the value function for the skill if we take the reward to be 1 for successful completion and 0 otherwise”).
Anil, Uesatos, Saunders, Madras, Dawid, and Ahn are considered analogous to the claimed invention because they deal with large language models. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesatos, Madras, and Dawid to use the affordance function from Ahn. One of the ordinary skill in the art would have known to apply the known technique of validating large language models in robotics. Therefore, applying Ahn’s technique would yield the predicable result of increasing the strength of decision making in robotics (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Regarding claim 16, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 15, Anil, Uesatos, Saunders, and Madras do not teach, but Ahn does teach
selectively controlling the real-world agent to perform action when the verification output has the first value and not controlling the real-world agent to perform the action when the verification output has the second value (Ahn, Section 3, “For each skill, the affordance function and the LLM probability are then multiplied together and ultimately the most probable skill is selected, i.e. π = argmax π∈Π p(cπ|s, lπ)p(lπ|i). Once the skill is selected, the corresponding policy is executed by the agent”. Examiner notes that the highest skill maps to the first value and the lowest skill maps to the second value).
Anil, Uesatos, Saunders, Madras, Dawid, and Ahn are considered analogous to the claimed invention because they deal with large language models. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesatos, Madras, and Dawid to use the affordance function from Ahn. One of the ordinary skill in the art would have known to apply the known technique of validating large language models in robotics. Therefore, applying Ahn’s technique would yield the predicable result of increasing the strength of decision making in robotics (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anil in view of Uesato, Sauders, Madras, Dawid, Ahn and Franco.
Regarding claim 17, Anil, Uesatos, Saunders, Madras, Dawid, and Ahn teach the method of claim 15, Anil, Uesatos, Saunders, Madras, and Ahn do not teach, but Franco does teach
the verification output having the first value indicates that completion of the action meets a safety criteria (Franco, Abstract, “Autonomous robots, especially aerial vehicles, when subject to disturbances, uncertainties, and noises may experience variations from their desired states and deviations from the planned trajectory which may lead them into an unsafe state (e.g., a collision). It is thus necessary to monitor their states at run-time when operating in uncertain and cluttered environments and intervene to guarantee their and the surrounding’s safety”)
Anil, Uesatos, Saunders, Madras, Dawid, Ahn, and Franco are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesatos, Madras, Dawid, and Ahn to have the output indicate the safety of a trajectory made from Franco. One of the ordinary skill in the art would have known to apply the known technique of validating the safety of a decision. Therefore, applying Franco technique would yield the predicable result of determining the safest outcome from a machine learning model (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anil in view of Uesato, Sauders, Madras, Dawid, and Breiman et al. (Bagging Predictors) (hereafter referred to as Breiman).
Regarding claim 18, Anil, Uesatos, Saunders, Madras, and Dawid teach the method of claim 1, Anil, Uesatos, Saunders, and Madras do not teach, but Breiman does teach
the verification output is one of a plurality of verification outputs obtained by performing the verifying a plurality of times, and the method further comprises generating a final verification output based on the plurality of verification outputs (Breiman, Abstract, “Bagging predictors is a method for generating multiple versions of a predictor and using these to get an aggregated predictor. The aggregation averages over the versions when predicting a numerical outcome and does a plurality vote when predicting a class”).
Anil, Uesatos, Saunders, Madras, Dawid, and Breiman are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesatos, Madras, and Dawid to use the bagging predictors method from Breiman. Breiman teaches “The evidence so far indicates that bagged estimates are likely to be more accurate than the single estimates” (Breiman, Section 6.1) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Claim(s) 19 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anil in view of Uesato, Sauders, Madras, Dawid, and White et al. (US 11526622 B1) (hereafter referred to as White).
Regarding claim 19, Anil teaches
A computer-implemented method of generating for an input a corresponding verification output that takes a first value or a second value, where the verification output taking the first value is statistically correlated with an agent output provided by a first computer-implemented agent in response to the input being valid for the input and the verification output taking the second value is statistically correlated with the agent output provided by the first computer-implemented agent in response to the input not being valid for the input (Anil, Section 3.3, “The verifier chooses a policy v which computes a convex combination of elements of V. (The verifier’s output can be interpreted as the probability it assigns to the label being 1)” and “Guarantee of Perfect Precision: There shouldn’t exist any prover which can trick the verifier into achieving non-perfect precision…implies that the verifier always has zero false positive rate regardless of which proof is used” (Anil, Section 3.1). Examiner notes that the verifier’s output is mapped to the first or second value.).
the verifying comprising employing a second computer-implemented agent, at least one of the first and second computer-implemented agents acting according to respective ones of a first and second protocol to verify, for a justification composed of a set of logical steps comprising one or more probabilistic agent outputs, whether the one or more probabilistic agent outputs correlate with probabilistic oracle values that would be output by a stochastic oracle (Anil, Abstract, “The PVG consists of two learners with competing objectives: a trusted verifier network tries to choose the correct answer, and a more powerful but untrusted prover network attempts to persuade the verifier of a particular answer, regardless of its correctness”. Examiner notes that the verifier network maps to the second agent).
the second protocol comprising for each of the successive ones of the logical steps: determining whether the first computer-implemented agent has generated the probabilistic agent output in accordance with the first protocol (Anil, Section 1, “Since the prover is untrustworthy, the verifier will only find its messages useful to the extent that it can independently verify the information”).
a verification protocol comprising: if no warning is generated by the second computer-implemented agent, generating a verification output indicating that the probabilistic agent output is valid (Anil, Figure 1)
PNG
media_image1.png
334
653
media_image1.png
Greyscale
Examiner notes that image 1b maps to the output being valid when there is no message.
Anil does not teach, but Uesato does teach
the verifying comprising employing a second computer-implemented agent, at least one of the first and second computer-implemented agents acting according to respective ones of a first and second protocol to verify, for a justification composed of a set of logical steps comprising one or more probabilistic agent outputs, whether the one or more probabilistic agent outputs correlate with probabilistic oracle values that would be output by a stochastic oracle (Uesato, Section 2.4, “For the process-supervised RM (PRM), the binary label after each step indicates whether the steps so far are correct ...A policy which maximizes the PRM score thus selects each step to maximize the RM-estimated probability of the steps so far being correct”).
the first protocol comprising for each of successive ones of the set of logical steps, generating a corresponding probabilistic agent output which is statistically correlated with a corresponding probabilistic oracle value (Uesato, Section 2.4, “For the process-supervised RM (PRM), the binary label after each step indicates whether the steps so far are correct ...A policy which maximizes the PRM score thus selects each step to maximize the RM-estimated probability of the steps so far being correct”).
Anil and Uesato are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil to use PRM from Uesato for justification. Uesato teaches that “this outperforms the approach from Li et al. (2022), which is similar to our PRM but replaces human evaluations with a heuristic based on string matching the results of the intermediate calculations” (Uesato, Section 2.4) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Anil and Uesato does not teach, but Saunders does teach
the first protocol comprising for each of successive ones of the set of logical steps, generating a corresponding probabilistic agent output which is statistically correlated with a corresponding probabilistic oracle value (Saunders, Section 2.3, “For these tasks, we don’t require human data collection because we have binary ground truth for both answer and critique validity. We use hand-coded oracles for each of the base, critiqueability, critique, and helpfulness tasks”).
responsive to determining that the first computer-implemented agent has not generated the probabilistic agent output in accordance with the first protocol, generating a warning (Saunders, Section 2.2.2, “the critiqueability score can be used to determine whether to ask it to critique in the first place, and the helpfulness score can be used to determine whether the critique is good after the fact”. Examiner notes that the critique maps to the warning).
Anil, Uesato, and Saunders are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil and Uesato to critique models from Saunders. Saunders teaches that “It’s possible that our model can identify and critique all of its mistakes. This motivates us to look at the percentage of the time poor outputs have helpful critiques. The higher this percentage, the easier it will be to assist humans in evaluation of the base task” (Saunders, Section 4.2) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Anil, Uesato, and Saunders do not teach, but Madras does teach
the verifying comprising employing a second computer-implemented agent, at least one of the first and second computer-implemented agents acting according to respective ones of a first and second protocol to verify, for a justification composed of a set of logical steps comprising one or more probabilistic agent outputs, whether the one or more probabilistic agent outputs correlate with probabilistic oracle values that would be output by a stochastic oracle (Madras, Section 2.3, “if the DM has constant loss (e.g. is an oracle), there exist values of γreject, γdefer for which the learning-to-defer and rejection learning objectives are equivalent” and (Madras, Section 2.1)).
PNG
media_image2.png
336
598
media_image2.png
Greyscale
Examiner notes that
y
^
_
D
maps to the probabilistic oracle values.
if the second computer-implemented agent generates a warning for one of the successive probabilistic logical steps: sampling the stochastic oracle in respect of the probabilistic logical step (Madras, Section 2.1, “The decision-making flow is modeled as a cascade, where the first-step model can either predict (positive/negative) or say pass. If it predicts, the DM will output the model’s prediction. However, if it says pass, the DM makes its own decision” and “We calculate these by sampling J times from the model, yielding J predictions zj ∈ [0,1]. Our prediction p is the sample mean…the system can threshold this uncertainty; any example with uncertainty beyond a threshold is rejected, and passed to the DM” (Appendix F)).
generating the verification output having the first value when a correlation of the sampled results with the probabilistic agent output meets a first correlation criteria and generating the second verification output having the second value when the correlation of the sampled results with the probabilistic agent output does not meet the first correlation criteria (Madras, Appendix F).
PNG
media_image3.png
152
590
media_image3.png
Greyscale
Examiner notes that the uncertainty threshold maps to the first correlation criteria and when that threshold is not reached, then the model’s output is used, which maps to the first value. If the threshold is reached, then DM’s output is used, which maps to the second value.
Anil, Uesato, Saunders, and Madras are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, and Saunders to sample stochastic oracles and learning to defer from Madras. Madras teaches that “Experiments demonstrate that learning to defer can make systems not only more accurate but also less biased. Even when working with inconsistent or biased users, we show that deferring models still greatly improve the accuracy and/or fairness of the entire system” (Madras, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Anil, Uesato, Saunders, and Madras do not teach, but Dawid does teach
generating the verification output having the first value when a correlation of the sampled results with the probabilistic agent output meets a first correlation criteria and generating the second verification output having the second value when the correlation of the sampled results with the probabilistic agent output does not meet the first correlation criteria (Dawid, Section 3, “One way of comparing forecasts with reality is to pick out some fairly arbitrary test set of days, and in it compare (a) the proportion p of days whose associated events in fact occur with (b) the average forecast probability π for those days.).
Anil, Uesato, Saunders, Madras, and Dawid are considered analogous to the claimed invention because they deal with validating outputs. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, Saunders, and Madras to compare forecasts with reality. One of the ordinary skill in the art would have known to apply the known technique of comparing ground truth data with the probabilistic output. Therefore, applying Dawid’s technique would yield the predicable result of determining the accuracy of a machine learning model (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Anil, Uesato, Saunders, Madras, and Dawid do not teach, but White does teach
One or more computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform a method (White, Abstract, “Methods, non-transitory computer readable media, and query verification apparatuses are disclosed that receive data, store the data into a table of a database, and receive a query that is associated with the database table”).
Anil, Uesato, Saunders, Madras, Dawid, and White are considered analogous to the claimed invention because they deal with validating outputs. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, Saunders, and Madras to include a non-transitory computer readable medium from White. One of the ordinary skill in the art would have known to apply the known technique of applying methods on non-transitory computer readable mediums. Therefore, applying White’s technique would yield the predicable result of running instructions on a non-transitory computer readable medium (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Regarding claim 20, Anil teaches
A computer-implemented method of generating for an input a corresponding verification output that takes a first value or a second value, where the verification output taking the first value is statistically correlated with an agent output provided by a first computer-implemented agent in response to the input being valid for the input and the verification output taking the second value is statistically correlated with the agent output provided by the first computer-implemented agent in response to the input not being valid for the input (Anil, Section 3.3, “The verifier chooses a policy v which computes a convex combination of elements of V. (The verifier’s output can be interpreted as the probability it assigns to the label being 1)” and “Guarantee of Perfect Precision: There shouldn’t exist any prover which can trick the verifier into achieving non-perfect precision…implies that the verifier always has zero false positive rate regardless of which proof is used” (Anil, Section 3.1). Examiner notes that the verifier’s output is mapped to the first or second value.).
the verifying comprising employing a second computer-implemented agent, at least one of the first and second computer-implemented agents acting according to respective ones of a first and second protocol to verify, for a justification composed of a set of logical steps comprising one or more probabilistic agent outputs, whether the one or more probabilistic agent outputs correlate with probabilistic oracle values that would be output by a stochastic oracle (Anil, Abstract, “The PVG consists of two learners with competing objectives: a trusted verifier network tries to choose the correct answer, and a more powerful but untrusted prover network attempts to persuade the verifier of a particular answer, regardless of its correctness”. Examiner notes that the verifier network maps to the second agent).
the second protocol comprising for each of the successive ones of the logical steps: determining whether the first computer-implemented agent has generated the probabilistic agent output in accordance with the first protocol (Anil, Section 1, “Since the prover is untrustworthy, the verifier will only find its messages useful to the extent that it can independently verify the information”).
a verification protocol comprising: if no warning is generated by the second computer-implemented agent, generating a verification output indicating that the probabilistic agent output is valid (Anil, Figure 1)
PNG
media_image1.png
334
653
media_image1.png
Greyscale
Examiner notes that image 1b maps to the output being valid when there is no message.
Anil does not teach, but Uesato does teach
the verifying comprising employing a second computer-implemented agent, at least one of the first and second computer-implemented agents acting according to respective ones of a first and second protocol to verify, for a justification composed of a set of logical steps comprising one or more probabilistic agent outputs, whether the one or more probabilistic agent outputs correlate with probabilistic oracle values that would be output by a stochastic oracle (Uesato, Section 2.4, “For the process-supervised RM (PRM), the binary label after each step indicates whether the steps so far are correct ...A policy which maximizes the PRM score thus selects each step to maximize the RM-estimated probability of the steps so far being correct”).
the first protocol comprising for each of successive ones of the set of logical steps, generating a corresponding probabilistic agent output which is statistically correlated with a corresponding probabilistic oracle value (Uesato, Section 2.4, “For the process-supervised RM (PRM), the binary label after each step indicates whether the steps so far are correct ...A policy which maximizes the PRM score thus selects each step to maximize the RM-estimated probability of the steps so far being correct”).
Anil and Uesato are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil to use PRM from Uesato for justification. Uesato teaches that “this outperforms the approach from Li et al. (2022), which is similar to our PRM but replaces human evaluations with a heuristic based on string matching the results of the intermediate calculations” (Uesato, Section 2.4) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Anil and Uesato does not teach, but Saunders does teach
the first protocol comprising for each of successive ones of the set of logical steps, generating a corresponding probabilistic agent output which is statistically correlated with a corresponding probabilistic oracle value (Saunders, Section 2.3, “For these tasks, we don’t require human data collection because we have binary ground truth for both answer and critique validity. We use hand-coded oracles for each of the base, critiqueability, critique, and helpfulness tasks”).
responsive to determining that the first computer-implemented agent has not generated the probabilistic agent output in accordance with the first protocol, generating a warning (Saunders, Section 2.2.2, “the critiqueability score can be used to determine whether to ask it to critique in the first place, and the helpfulness score can be used to determine whether the critique is good after the fact”. Examiner notes that the critique maps to the warning).
Anil, Uesato, and Saunders are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil and Uesato to critique models from Saunders. Saunders teaches that “It’s possible that our model can identify and critique all of its mistakes. This motivates us to look at the percentage of the time poor outputs have helpful critiques. The higher this percentage, the easier it will be to assist humans in evaluation of the base task” (Saunders, Section 4.2) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Anil, Uesato, and Saunders do not teach, but Madras does teach
the verifying comprising employing a second computer-implemented agent, at least one of the first and second computer-implemented agents acting according to respective ones of a first and second protocol to verify, for a justification composed of a set of logical steps comprising one or more probabilistic agent outputs, whether the one or more probabilistic agent outputs correlate with probabilistic oracle values that would be output by a stochastic oracle (Madras, Section 2.3, “if the DM has constant loss (e.g. is an oracle), there exist values of γreject, γdefer for which the learning-to-defer and rejection learning objectives are equivalent” and (Madras, Section 2.1)).
PNG
media_image2.png
336
598
media_image2.png
Greyscale
Examiner notes that
y
^
_
D
maps to the probabilistic oracle values.
if the second computer-implemented agent generates a warning for one of the successive probabilistic logical steps: sampling the stochastic oracle in respect of the probabilistic logical step (Madras, Section 2.1, “The decision-making flow is modeled as a cascade, where the first-step model can either predict (positive/negative) or say pass. If it predicts, the DM will output the model’s prediction. However, if it says pass, the DM makes its own decision” and “We calculate these by sampling J times from the model, yielding J predictions zj ∈ [0,1]. Our prediction p is the sample mean…the system can threshold this uncertainty; any example with uncertainty beyond a threshold is rejected, and passed to the DM” (Appendix F)).
generating the verification output having the first value when a correlation of the sampled results with the probabilistic agent output meets a first correlation criteria and generating the second verification output having the second value when the correlation of the sampled results with the probabilistic agent output does not meet the first correlation criteria (Madras, Appendix F).
PNG
media_image3.png
152
590
media_image3.png
Greyscale
Examiner notes that the uncertainty threshold maps to the first correlation criteria and when that threshold is not reached, then the model’s output is used, which maps to the first value. If the threshold is reached, then DM’s output is used, which maps to the second value.
Anil, Uesato, Saunders, and Madras are considered analogous to the claimed invention because they deal with validating outputs in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, and Saunders to sample stochastic oracles and learning to defer from Madras. Madras teaches that “Experiments demonstrate that learning to defer can make systems not only more accurate but also less biased. Even when working with inconsistent or biased users, we show that deferring models still greatly improve the accuracy and/or fairness of the entire system” (Madras, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Anil, Uesato, Saunders, and Madras do not teach, but Dawid does teach
generating the verification output having the first value when a correlation of the sampled results with the probabilistic agent output meets a first correlation criteria and generating the second verification output having the second value when the correlation of the sampled results with the probabilistic agent output does not meet the first correlation criteria (Dawid, Section 3, “One way of comparing forecasts with reality is to pick out some fairly arbitrary test set of days, and in it compare (a) the proportion p of days whose associated events in fact occur with (b) the average forecast probability π for those days.).
Anil, Uesato, Saunders, Madras, and Dawid are considered analogous to the claimed invention because they deal with validating outputs. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, Saunders, and Madras to compare forecasts with reality. One of the ordinary skill in the art would have known to apply the known technique of comparing ground truth data with the probabilistic output. Therefore, applying Dawid’s technique would yield the predicable result of determining the accuracy of a machine learning model (See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Anil, Uesato, Saunders, Madras, and Dawid do not teach, but White does teach
A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform a method (White, paragraph 0006, “A query verification apparatus also is disclosed that includes a prover device including a first memory comprising first programmed instructions stored thereon, a first processor, and one or more GPUs.”).
Anil, Uesato, Saunders, Madras, Dawid, and White are considered analogous to the claimed invention because they deal with validating outputs. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Anil, Uesato, Saunders, and Madras to include the query verification apparatus from White. One of the ordinary skill in the art would have known to apply the known technique of applying methods on an apparatus. Therefore, applying White’s technique would yield the predicable result of running instructions on an apparatus(See MPEP 2141 (III)(D Applying a known technique to a known device ready for improvement to yield predicable results).
Allowable Subject Matter
Claim 7 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and the 101 rejection is overcome. The closest prior arts of record are Witten et al. (Arithmetic Coding for Data Compression) and Lund et al. (Algebraic Methods for Interactive Proof Systems).
Witten discloses
for logical step
t
the probability for the logical step is given by
p
^
t
=
c
^
t
/d, where
c
^
t
=0,…,d, d is a positive integer (Witten, Section: Representing the Model, “The probabilities in the model are represented as integer frequency counts, and cumulative counts are stored in the array cum- freq[]. As previously, this array is “backwards,” and the total frequency count, which is used to normalize all frequencies, appears in cum - fre9 [O]. Cumulative counts must not exceed a predetermined maximum, Max- frequency, and the model implementation must prevent overflow by scaling appropriately”).
Witten does not disclose
for logical step
t
the probability for the logical step is given by
p
^
t
=
c
^
t
/d, where
c
^
t
=0,…,d, d is a positive integer
the first computer-implemented agent obtaining a first query input from an independent copy of the second computer-implemented agent comprises querying the independent copy of the second computer-implemented agent for a random integer value sampled uniformly from 0,…,d
the second computer-implemented agent obtaining a second query input from an independent copy of the first computer-implemented agent comprises querying the independent copy of the first computer-implemented agent for a random integer value sampled uniformly from 0,…,d.
Lund discloses
the first computer-implemented agent obtaining a first query input from an independent copy of the second computer-implemented agent comprises querying the independent copy of the second computer-implemented agent for a random integer value sampled uniformly from 0,…,d and wherein the second computer-implemented agent obtaining a second query input from an independent copy of the first computer-implemented agent comprises querying the independent copy of the first computer-implemented agent for a random integer value sampled uniformly from 0,…,d (Lund, Section 3)
PNG
media_image10.png
193
497
media_image10.png
Greyscale
Lund does not disclose
the first computer-implemented agent obtaining a first query input from an independent copy of the second computer-implemented agent comprises querying the independent copy of the second computer-implemented agent for a random integer value sampled uniformly from 0,…,d
the second computer-implemented agent obtaining a second query input from an independent copy of the first computer-implemented agent comprises querying the independent copy of the first computer-implemented agent for a random integer value sampled uniformly from 0,…,d.
Therefore, the prior arts of record, individually or in combination, do not disclose the entirety of claim 7 as a whole.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Kalai et al. (Delegating Computation: Interactive Proofs for Muggles) discloses interactive proofs for tractable languages.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEVEN VO whose telephone number is (571)272-9622. The examiner can normally be reached Monday - Friday from 7-3 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.V./ Examiner, Art Unit 2148 /MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights