WODETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1, 19, 21, 23, 24, 26, 28, 29, and 32 have been amended by Applicant in preliminary amendment dated 11/27/2023. Claims 2-18 have been cancelled and no new claims have been added. Claims 1 and 19-32 are currently pending examination.
Information Disclosure Statement
The Information Disclosure Statement (IDS) submitted by Applicant on 11/27/2023 has been considered.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 and 19-32 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (abstract idea), without significantly more.
Regarding claim 1,
Step 1: Claim 1 is directed towards a method.
Step 2A, Prong 1: Claim 1 recites the following limitations:
classifying at least one of a plurality of inputs to the model as being supportive or resistant to a proposed action by the model; (i.e., a person can mentally classify inputs to a model as supporting or opposing a proposed action by a model)
comparing the classification of the at least one of the plurality of inputs to domain knowledge to determine whether or not the proposed action conflicts with the domain knowledge, wherein the domain knowledge is indicative of a relationship between the proposed action and the at least one of the plurality of inputs; (i.e., a person can mentally compare a classification of an input to domain knowledge and make a determination as to whether or not the classification conflicts with the domain knowledge.)
Hence, the claim recites an abstract idea.
Step 2A, Prong 2: Claim 1 recites the additional elements of “a computer-implemented method for determining whether to perform an action proposed by a model developed using a reinforcement learning process, the method comprising:” and “in response to determining that the proposed action does not conflict with the domain knowledge, initiating the proposed action”, amount to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)). Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 1 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of “a computer-implemented method for determining whether to perform an action proposed by a model developed using a reinforcement learning process, the method comprising:” and “in response to determining that the proposed action does not conflict with the domain knowledge, initiating the proposed action” amount to no more than mere instructions to apply the judicial exception using generic computer components (see MPEP 2106.05(f)). Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 19,
Step 1: Claim 19 is directed towards an apparatus.
Step 2A, Prong 1: Claim 19 recites the following limitations:
classify at least one of a plurality of inputs to the model as being supportive or resistant to a proposed action by the model; (i.e., a person can mentally classify inputs to a model as supporting or opposing a proposed action by a model)
compare the classification of the at least one of the plurality of inputs to domain knowledge to determine whether or not the proposed action conflicts with the domain knowledge, wherein the domain knowledge is indicative of a relationship between the proposed action and the at least one of the plurality of inputs; (i.e., a person can mentally compare a classification of an input to domain knowledge and make a determination as to whether or not the classification conflicts with the domain knowledge.)
Hence, the claim recites an abstract idea.
Step 2A, Prong 2: Claim 19 recites the additional elements of “an apparatus for determining whether to perform an action proposed by a model developed using a reinforcement learning process,”, the apparatus comprising a processor and a machine-readable medium, wherein the machine-readable medium contains instructions executable by the processor such that the apparatus is operable to:”, and “in response to determining that the proposed action does not conflict with the domain knowledge, initiate the proposed action”. These limitations are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)) Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 19 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of “an apparatus for determining whether to perform an action proposed by a model developed using a reinforcement learning process,”, the apparatus comprising a processor and a machine-readable medium, wherein the machine-readable medium contains instructions executable by the processor such that the apparatus is operable to:”, and “in response to determining that the proposed action does not conflict with the domain knowledge, initiate the proposed action” amount to no more than mere instructions to apply the judicial exception using generic computer components (see MPEP 2106.05(f)). Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 20,
Step 2A, Prong 1: Claim 20 recites an abstract idea as inherited from claim 19.
Step 2A, Prong 2: Claim 20 recites the additional element of wherein the apparatus is operable to classify at least one of the plurality of inputs as being supportive or resistant using an explainable artificial intelligence, XAI, process. This limitation is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)) Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 20 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of “wherein the apparatus is operable to classify at least one of the plurality of inputs as being supportive or resistant using an explainable artificial intelligence, XAI, process” is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)) Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 21,
Step 2A, Prong 1: Claim 21 recites an abstract idea as inherited from claim 19.
determine a relative importance of one of the plurality of inputs to the proposed action compared to at least one other input in the plurality of inputs; (i.e., a person can mentally, or with the aid of pen and paper, determine a relative importance of an input compared to at least one other input of a plurality of inputs to a model)
select the input for comparison with the domain knowledge based on its relative importance. (i.e., a person can mentally, or with the aid of pen and paper, select an input for comparison with a domain knowledge base on the determined relative importance)
Step 2A, Prong 2: Claim 21 recites the additional element of “,wherein the apparatus is further operable to:”, to perform the steps above, amounts to no more than mere instructions to apply the exception using a generic computer component. (see MPEP 2106.05(f)). Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 21 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of “,wherein the apparatus is further operable to:”, to perform the steps above, amounts to no more than mere instructions to apply the exception using a generic computer component. (see MPEP 2106.05(f)). Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 22,
Step 2A, Prong 1: Claim 22 recites an abstract idea as inherited from claims 19 and 21.
Step 2A, Prong 2: Claim 22 recites the additional elements of “wherein the apparatus is operable to determine the relative importance of the input using an XAI process”. This limitation is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)). Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 22 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of “wherein the apparatus is operable to determine the relative importance of the input using an XAI process” is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)) Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 23,
Step 2A, Prong 1: Claim 23 recites an abstract idea as inherited from claim 19. Claim 23 further recites the following limitations:
compare the classification of the at least one of the plurality of inputs to domain knowledge to determine whether or not the proposed action conflicts with the domain knowledge by: determining that a conflict occurs in response to determining that one or more inputs have a classification that contradicts the domain knowledge (i.e., a person can mentally compare a classification of at least one of a plurality of inputs to a model to domain knowledge to determine that a conflict occurs in response to determine that the input has a classification that contradicts the domain knowledge)
Step 2A, Prong 2: Claim 23 recites the additional elements of “wherein the apparatus is operable to” to perform the step stated above, is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)) Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 23 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of “wherein the apparatus is operable to” to perform the step stated above, is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)) Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 24,
Step 2A, Prong 1: Claim 24 recites an abstract idea as inherited from claim 19. Claim 24 further recites the following limitations:
map one or more other inputs to the model to one or more events for comparison with the domain knowledge; (i.e., a person can mentally map input data to a model other events for comparison with a domain knowledge)
compare the set of events with the domain knowledge to determine whether or not the proposed action conflicts with the domain knowledge (i.e., a person can mentally compare a set of events with a domain knowledge to determine whether a proposed action by a model conflicts or contradicts the domain knowledge)
Hence, the claim recites an abstract idea.
Step 2A, Prong 2: Claim 24 recites the additional element of “wherein the apparatus is further operable to:” to perform the steps stated above. This limitation is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05 (f)) Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 24 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of “wherein the apparatus is further operable to:”, to perform the steps stated above, is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05 (f)) Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 25,
Step 2A, Prong 1: Claim 24 recites an abstract idea as inherited from claims 19 and 24.
Step 2A, Prong 2: Claim 24 recites the additional element of “wherein the apparatus is operable to map one or more other inputs to one or more events using a mapping model developed using a machine learning process”. This limitation is recited at a high-level of generality, such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)) Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 25 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of “wherein the apparatus is operable to map one or more other inputs to one or more events using a mapping model developed using a machine learning process” is recited at a high-level of generality, such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)) Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 26,
Step 2A, Prong 1: Claim 26 recites an abstract idea as inherited from claim 19.
Step 2A, Prong 2: Claim 26 recites the additional element of “wherein the apparatus is a node in a communication network”. This limitation merely generally links the use of the judicial exception to a particular technological environment or field of use. (see MPEP 2106.05(h)) Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 26 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of “wherein the apparatus is a node in a communication network” merely generally links the use of the judicial exception to a particular technological environment or field of use. (see MPEP 2106.05(h)) Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 27,
Step 2A, Prong 1: Claim 27 recites an abstract idea as inherited from claim 19.
Step 2A, Prong 2: Claim 27 recites the additional elements of “wherein the communication network comprises a radio access network and the plurality of inputs comprises one or more metrics of the radio access network and the proposed action comprises configuring an operational parameter of the radio access network”. This limitation merely generally links the use of the judicial exception to a particular technological environment or field of use. (see MPEP 2106.05(h)) Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 27 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of “wherein the communication network comprises a radio access network and the plurality of inputs comprises one or more metrics of the radio access network and the proposed action comprises configuring an operational parameter of the radio access network” merely generally link the use of the judicial exception to a particular technological environment or field of use. (see MPEP 2106.05(h)) Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 28,
Step 2A, Prong 1: Claim 28 recites an abstract idea as inherited from claim 19.
Step 2A, Prong 2: Claim 28 recites the additional element of “wherein the node comprises a base station or a core network node”. This limitation merely generally links the use of the judicial exception to a particular technological environment or field of use. (see MPEP 2106.05(h)) Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 28 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of “wherein the node comprises a base station or a core network node” merely generally links the use of the judicial exception to a particular technological environment or field of use. (see MPEP 2106.05(h)) Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 29,
Step 2A, Prong 1: Claim 29 recites an abstract idea as inherited from claim 19. Claim 29 further recites the following limitations:
map other metrics of the radio access network that are input to the model to one or more events for comparison with the domain knowledge; (i.e., a person can mentally map metrics of a radio access network that are inputted to a model to one or more events for comparison with a domain knowledge.)
compare the set of events with the domain knowledge to determine whether or not the proposed action conflicts with the domain knowledge. (i.e., a person can mentally compare a set of events with a domain knowledge to determine whether the proposed action conflicts with the domain knowledge.)
Step 2A, Prong 2: Claim 29 recites the additional element of “wherein the apparatus is further operable to:”, to perform the steps stated above. This limitation is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)). Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 29 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of “wherein the apparatus is further operable to:”, to perform the steps stated above is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)). Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 30,
Step 2A, Prong 1: Claim 30 recites an abstract idea as inherited from claims 19 and 29.
Step 2A, Prong 2: Claim 30 recites the additional elements of “wherein the other metrics comprise data representing received signal power at a base station in the radio access network over a period of time and data representing a plurality of performance metrics for a cell served by the base station over the time period” and “wherein the apparatus is operable to map one or more other metrics to one or more events using a mapping model developed using a machine learning process.” The additional element of “wherein the other metrics comprise data representing received signal power at a base station in the radio access network over a period of time and data representing a plurality of performance metrics for a cell served by the base station over the time period” merely generally links the use of the judicial exception to a particular technological environment or field of use. (see MPEP 2106.05(h)). Furthermore, the additional element of “wherein the apparatus is operable to map one or more other metrics to one or more events using a mapping model developed using a machine learning process.” is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)) Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 30 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of “wherein the other metrics comprise data representing received signal power at a base station in the radio access network over a period of time and data representing a plurality of performance metrics for a cell served by the base station over the time period” merely generally links the use of the judicial exception to a particular technological environment or field of use. (see MPEP 2106.05(h)). Furthermore, the additional element of “wherein the apparatus is operable to map one or more other metrics to one or more events using a mapping model developed using a machine learning process.” is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. (see MPEP 2106.05(f)) Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 31,
Step 2A, Prong 1: Claim 31 recites an abstract idea as inherited from claim 19.
Step 2A, Prong 2: Claim 31 recites the additional element of “wherein the machine learning process is a multi-task learning process”. This limitation merely generally links the use of the judicial exception to a particular technological environment or field of use. (see MPEP 2106.05(h)). Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 31 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of “wherein the machine learning process is a multi-task learning process” merely generally links the use of the judicial exception to a particular technological environment or field of use. (see MPEP 2106.05(h)). Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 32,
Step 2A, Prong 1: Claim 32 recites an abstract idea as inherited from claim 19.
Step 2A, Prong 2: Claim 32 recites the additional element of “wherein the reinforcement learning process is a policy optimisation process or a q-learning process”. This limitation merely generally links the use of the judicial exception to a particular technological environment or field of use. (see MPEP 2106.05(f)) Hence the claim does not recite additional elements that integrate the judicial exception into a practical application. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Claim 32 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of “wherein the reinforcement learning process is a policy optimisation process or a q-learning process” merely generally links the use of the judicial exception to a particular technological environment or field of use. (see MPEP 2106.05(f)) Hence the claim lacks limitations which amount to significantly more than the judicial exception or an inventive concept, and is rejected. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim 1, 19, 21, 24, 26, and 32 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Taylor et al. (US 20190236458 A1, filed Jan. 31, 2019 and published Aug. 1, 2019)
Regarding claim 1, Taylor teaches a computer-implemented method for determining whether to perform an action proposed by a model developed using a reinforcement learning process (Taylor, [0010] teaches systems, methods, and computer readable media directed to interactive reinforcement learning with dynamic reuse of prior knowledge are described in various embodiments. In particular, computer implemented systems and methods configured for receiving third party actor demonstrator data sets (e.g., data sets representing states and/or actions of human actions or computer-based actors) and utilizing the third party actor data sets for conducting pre-training of an underlying machine learning mechanism (e.g., a neural network).) , the method comprising:
classifying at least one of a plurality of inputs to the model as being supportive or resistant to a proposed action by the model (Taylor, [0015] teaches as described in various embodiments, an interactive reinforcement learning mechanism is adapted for providing computer implemented systems for (i) identifying demonstration data that is contradictory, and not using it as the basis for a decision; and/or (ii) determining that insufficient demonstration data has been provided, and prompting for more data to be submitted.; Taylor, [0094] A data receiver 102 is configured for receiving data sets representative of the demonstrations for performing sequential tasks (e.g., playing games, trading stocks, sorting, association learning, image recognition, stock market transaction control). The demonstrator data sets are provided to classifier trainer, which trains a classifier model based on the demonstrator data stored on demonstrator classifier data storage 154.; Taylor [0095] teaches as there may be differences in quality as between demonstrators and their associated demonstrator data sets, as described in various embodiments, these potential contradictions arise in the form of differing actions that are suggested by at least one of the demonstrator data sets (e.g., from a demonstrator), or from the machine learning model itself.);
comparing the classification of the at least one of the plurality of inputs to domain knowledge to determine whether or not the proposed action conflicts with the domain knowledge, wherein the domain knowledge is indicative of a relationship between the proposed action and the at least one of the plurality of inputs (Taylor, [0014] teaches the system, of some embodiments, is adapted for comparisons with of the observations of the actors against the internal training of the machine learning mechanism of the system. Confidence data structures are tracked (e.g., maintained) for the actors, and their underlying demonstrator data sets, or portions thereof. A dynamic determination mechanism selects a source (e.g., actor-source determination) upon which an action should be selected. The choice of which actor-source was selected is utilized as a feedback to modify the confidence associated with their underlying demonstrator data sets, or portions thereof.; Taylor, [0015] further teaches as described in various embodiments, an interactive reinforcement learning mechanism is adapted for providing computer implemented systems for (i) identifying demonstration data that is contradictory, and not using it as the basis for a decision; and/or (ii) determining that insufficient demonstration data has been provided, and prompting for more data to be submitted.; Taylor, [0093] further teaches in an embodiment, system 100 is configured for processing one or more potential contradictions in demonstration data for machine learning, including at least one processor and computer readable memory.; Taylor [0095] further teaches as there may be differences in quality as between demonstrators and their associated demonstrator data sets, as described in various embodiments, these potential contradictions arise in the form of differing actions that are suggested by at least one of the demonstrator data sets (e.g., from a demonstrator), or from the machine learning model itself.); and
in response to determining that the proposed action does not conflict with the domain knowledge, initiating the proposed action (Taylor, [0104] teaches an action selection engine 104 is configured to provide a contradiction detection engine configured to process the one or more features by communicating the one or more features for processing by the neural network and receiving a signal output from the neural network indicative of the one or more potential contradictions.; Taylor [0105] further teaches these contradictions, for example, may be indicative of “best practices” that are contradictory. A demonstrator data set may indicate that a correct path to dodge a spike is to jump over it, while another data set may indicate that the correct path is to jump into it. Where there is contradictory actions, for example, the action selection engine 104 may generate a control signal indicating a specific action to be taken.[i.e., as in “in response to determining that the proposed action does not conflict with the domain knowledge”]; Taylor, [0098] teaches the confidence engine 105 tracks confidence scores associated with each demonstrator data source, and/or portions thereof. In some embodiments, the confidence scores are utilized by the action selection engine 104 which utilizes a selection function to determine an action for the machine learning model to take (e.g., one of the demonstrator classifier indicated actions, or an action indicated by its own internal policy function). [Note: understood as when ab action is taken when the confidence score is high – meaning that there is no conflict with the domain knowledge]; Taylor, [0025] the confidence data values of the one or more confidence data values based on the observed reward outcome; Taylor, [0030] teaches in another aspect, the selecting of the action-source is based upon an action selection mechanism including a soft decision model adapted for maximizing a current confidence expectation; Taylor, [0099] teaches the confidence scores are a distribution that, for example, may be modified based on feedback as obtained from the state observer after an action has been taken (e.g., the action suggested by the demonstrator led to an adverse result, reduce weight on the demonstrator's data sets so that it is less likely to be selected in the future); Taylor, [0100] in further embodiments, more than one confidence score is assigned to a demonstrator data set, and may be based upon different states/groups of states, and corresponding portions/sub-portions of the demonstrator data sets (e.g., where the demonstrator data set is unevenly adept at various sub-tasks, it may still be valuable for specific sub-tasks, such as an opening as opposed to an endgame).).
Regarding claim 19, Taylor teaches an apparatus for determining whether to perform an action proposed by a model developed using a reinforcement learning process, the apparatus comprising a processor and a machine-readable medium, wherein the machine-readable medium contains instructions executable by the processor such that the apparatus is operable to (Taylor, [0010] teaches systems, methods, and computer readable media directed to interactive reinforcement learning with dynamic reuse of prior knowledge are described in various embodiments. In particular, computer implemented systems and methods configured for receiving third party actor demonstrator data sets (e.g., data sets representing states and/or actions of human actions or computer-based actors) and utilizing the third party actor data sets for conducting pre-training of an underlying machine learning mechanism (e.g., a neural network).; Taylor, [0209] Embodiments of methods, systems, and apparatus are described through reference to the drawings.; Taylor, [0211] further teaches the embodiments of the devices, systems and methods described herein may be implemented in a combination of both hardware and software. These embodiments may be implemented on programmable computers, each computer including at least one processor, a data storage system (including volatile memory or non-volatile memory or other data storage elements or a combination thereof), and at least one communication interface.; Taylor, [0213] further teacher t54hroughout the foregoing discussion, numerous references will be made regarding servers, services, interfaces, portals, platforms, or other systems formed from computing devices. It should be appreciated that the use of such terms is deemed to represent one or more computing devices having at least one processor configured to execute software instructions stored on a computer readable tangible, non-transitory medium.):
classify at least one of a plurality of inputs to the model as being supportive or resistant to a proposed action by the model (Taylor, [0015] teaches as described in various embodiments, an interactive reinforcement learning mechanism is adapted for providing computer implemented systems for (i) identifying demonstration data that is contradictory, and not using it as the basis for a decision; and/or (ii) determining that insufficient demonstration data has been provided, and prompting for more data to be submitted.; Taylor, [0094] A data receiver 102 is configured for receiving data sets representative of the demonstrations for performing sequential tasks (e.g., playing games, trading stocks, sorting, association learning, image recognition, stock market transaction control). The demonstrator data sets are provided to classifier trainer, which trains a classifier model based on the demonstrator data stored on demonstrator classifier data storage 154.; Taylor [0095] teaches as there may be differences in quality as between demonstrators and their associated demonstrator data sets, as described in various embodiments, these potential contradictions arise in the form of differing actions that are suggested by at least one of the demonstrator data sets (e.g., from a demonstrator), or from the machine learning model itself.);
compare the classification of the at least one of the plurality of inputs to domain knowledge to determine whether or not the proposed action conflicts with the domain knowledge, wherein the domain knowledge is indicative of a relationship between the proposed action and the at least one of the plurality of inputs (Taylor, [0014] teaches the system, of some embodiments, is adapted for comparisons with of the observations of the actors against the internal training of the machine learning mechanism of the system. Confidence data structures are tracked (e.g., maintained) for the actors, and their underlying demonstrator data sets, or portions thereof. A dynamic determination mechanism selects a source (e.g., actor-source determination) upon which an action should be selected. The choice of which actor-source was selected is utilized as a feedback to modify the confidence associated with their underlying demonstrator data sets, or portions thereof.; Taylor, [0015] further teaches as described in various embodiments, an interactive reinforcement learning mechanism is adapted for providing computer implemented systems for (i) identifying demonstration data that is contradictory, and not using it as the basis for a decision; and/or (ii) determining that insufficient demonstration data has been provided, and prompting for more data to be submitted.; Taylor, [0093] further teaches in an embodiment, system 100 is configured for processing one or more potential contradictions in demonstration data for machine learning, including at least one processor and computer readable memory.; Taylor [0095] further teaches as there may be differences in quality as between demonstrators and their associated demonstrator data sets, as described in various embodiments, these potential contradictions arise in the form of differing actions that are suggested by at least one of the demonstrator data sets (e.g., from a demonstrator), or from the machine learning model itself.); and
in response to determining that the proposed action does not conflict with the domain knowledge, initiate the proposed action(Taylor, [0104] teaches an action selection engine 104 is configured to provide a contradiction detection engine configured to process the one or more features by communicating the one or more features for processing by the neural network and receiving a signal output from the neural network indicative of the one or more potential contradictions.; Taylor [0105] further teaches these contradictions, for example, may be indicative of “best practices” that are contradictory. A demonstrator data set may indicate that a correct path to dodge a spike is to jump over it, while another data set may indicate that the correct path is to jump into it. Where there is contradictory actions, for example, the action selection engine 104 may generate a control signal indicating a specific action to be taken.[i.e., as in “in response to determining that the proposed action does not conflict with the domain knowledge”]; Taylor, [0098] teaches the confidence engine 105 tracks confidence scores associated with each demonstrator data source, and/or portions thereof. In some embodiments, the confidence scores are utilized by the action selection engine 104 which utilizes a selection function to determine an action for the machine learning model to take (e.g., one of the demonstrator classifier indicated actions, or an action indicated by its own internal policy function). [Note: understood as when ab action is taken when the confidence score is high – meaning that there is no conflict with the domain knowledge]; Taylor, [0025] the confidence data values of the one or more confidence data values based on the observed reward outcome; Taylor, [0030] teaches in another aspect, the selecting of the action-source is based upon an action selection mechanism including a soft decision model adapted for maximizing a current confidence expectation; Taylor, [0099] teaches the confidence scores are a distribution that, for example, may be modified based on feedback as obtained from the state observer after an action has been taken (e.g., the action suggested by the demonstrator led to an adverse result, reduce weight on the demonstrator's data sets so that it is less likely to be selected in the future); Taylor, [0100] in further embodiments, more than one confidence score is assigned to a demonstrator data set, and may be based upon different states/groups of states, and corresponding portions/sub-portions of the demonstrator data sets (e.g., where the demonstrator data set is unevenly adept at various sub-tasks, it may still be valuable for specific sub-tasks, such as an opening as opposed to an endgame).).
Regarding claim 21, Taylor teaches all of the limitations of claim 19, and Taylor further teaches wherein the apparatus is further operable to: determine a relative importance of one of the plurality of inputs to the proposed action compared to at least one other input in the plurality of inputs (Taylor [0013] teaches the system may receive data sets representative of observations (e.g., action inputs/environmental states) of actors (e.g., workers or other trained robots) conducting these tasks. However, there may be varying levels of adeptness (e.g., ability to achieve a reward), and the levels may also vary as between tasks (e.g., an actor is good at grading, but not inspecting or placing eggs into cartons); Taylor, [0118] teaches the demonstrator data set based actions are dynamically applied, and in some situations, based on a tracked confidence level, the internal policy function 1508 begins to dominate over the trained classifiers in determining which actions to take. Accordingly, as the machine learning model 1506 improves performance, there may be an automatic down-weighting of the demonstrator data set based actions.; Taylor [0036] teaches in accordance with another aspect, the neural network is configured to integrate the one or more weights and the one or more data sets in a reinforcement learning loop.; Taylor [0099] teaches the confidence scores are a distribution that, for example, may be modified based on feedback as obtained from the state observer after an action has been taken (e.g., the action suggested by the demonstrator led to an adverse result, reduce weight on the demonstrator's data sets so that it is less likely to be selected in the future); Taylor [0100] teaches in further embodiments, more than one confidence score is assigned to a demonstrator data set, and may be based upon different states/groups of states, and corresponding portions/sub-portions of the demonstrator data sets (e.g., where the demonstrator data set is unevenly adept at various sub-tasks, it may still be valuable for specific sub-tasks, such as an opening as opposed to an endgame).); and select the input for comparison with the domain knowledge based on its relative importance (Taylor, [0118] teaches the demonstrator data set based actions are dynamically applied, and in some situations, based on a tracked confidence level, the internal policy function 1508 begins to dominate over the trained classifiers in determining which actions to take; Taylor [0100] teaches in further embodiments, more than one confidence score is assigned to a demonstrator data set, and may be based upon different states/groups of states, and corresponding portions/sub-portions of the demonstrator data sets (e.g., where the demonstrator data set is unevenly adept at various sub-tasks, it may still be valuable for specific sub-tasks, such as an opening as opposed to an endgame)).
Regarding claim 24, Taylor teaches all of the limitations of claim 19, and Taylor further teaches wherein the apparatus is further operable to: map one or more other inputs to the model to one or more events for comparison with the domain knowledge (Taylor, [0088] Demonstrator data sets can be provided in the form of encapsulated data structure elements, for example, as recorded by demonstrator computing unit 122, or observed through recorded and processed data sets of the agent associated with demonstrator computing unit 122 interacting with an environment, and the associated inputs indicative of the actions taken by the agent.; [Note: associated inputs indicative of actions taken by the agent (i.e., model) understood to read on inputs mapped to the model to one or more events]; Taylor [0095] As there may be differences in quality as between demonstrators and their associated demonstrator data sets, as described in various embodiments, these potential contradictions arise in the form of differing actions that are suggested by at least one of the demonstrator data sets (e.g., from a demonstrator), or from the machine learning model itself.;); and compare the set of events with the domain knowledge to determine whether or not the proposed action conflicts with the domain knowledge (Taylor, [0014] teaches the system, of some embodiments, is adapted for comparisons with of the observations of the actors against the internal training of the machine learning mechanism of the system. Confidence data structures are tracked (e.g., maintained) for the actors, and their underlying demonstrator data sets, or portions thereof. A dynamic determination mechanism selects a source (e.g., actor-source determination) upon which an action should be selected. The choice of which actor-source was selected is utilized as a feedback to modify the confidence associated with their underlying demonstrator data sets, or portions thereof.; Taylor, [0094] A data receiver 102 is configured for receiving data sets representative of the demonstrations for performing sequential tasks (e.g., playing games, trading stocks, sorting, association learning, image recognition, stock market transaction control). The demonstrator data sets are provided to classifier trainer, which trains a classifier model based on the demonstrator data stored on demonstrator classifier data storage 154. [Note: the demonstrator classifier data store understood as the “domain knowledge”]; Taylor [0095] As there may be differences in quality as between demonstrators and their associated demonstrator data sets, as described in various embodiments, these potential contradictions arise in the form of differing actions that are suggested by at least one of the demonstrator data sets (e.g., from a demonstrator), or from the machine learning model itself. [Note: in order to determine differences between demonstrators and their associated demonstrator sets a comparison must be made]; Taylor, [0113] further teaches the action selection engine 104 is configured to associate one or more weights with one or more data elements of the one or more data sets linked to the one or more contradictions, the one or more weights modifying the processing of the one or more data elements of the one or more data sets when training the machine learning model to improve the model over the training period.).
Regarding claim 26, Taylor teaches all of the limitations of claim 19, and Taylor further teaches wherein the apparatus is a node in a communication network (Taylor, [0212] teaches Program code is applied to input data to perform the functions described herein and to generate output information. The output information is applied to one or more output devices. In some embodiments, the communication interface may be a network communication interface. In embodiments in which elements may be combined, the communication interface may be a software communication interface, such as those for inter-process communication. In still other embodiments, there may be a combination of communication interfaces implemented as hardware, software, and combination thereof.; See also Fig. 1).
Regarding claim 32, Taylor teaches all of the limitations of claim 19, and Taylor further teaches wherein the reinforcement learning process is a policy optimisation process or a q-learning process (Taylor, [0127] as not all demonstrators are equally good at all sub-tasks that form part of the task being optimized by the learning model, in some embodiments, different confidence scores for the demonstration sets are assigned based upon specific states, features of states, or groups of states.; Taylor, [0128] further teaches for example, a demonstrator may be particularly good at opening states (which have a level of broad positional analysis, a challenging task for machine learning without bootstrapping based on demonstrators), but not so good at endgame states, where the machine-learning model is able to easily dominate by extending endgame tables.; Taylor [0129] further teaches accordingly, the mechanism of some embodiments is biased to prefer demonstrator source actions where the environment is in one of the opening states, and to prefer its own internal Q-learning policy where the environment is in one of the endgame states.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 20 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Taylor et al. in view of Dalli et al. (US 20210256377 A1, filed Feb. 12, 2021 and published Aug. 19, 2021)
Regarding claim 20, Taylor teaches all of the limitations of claim 19, however, Taylor does not distinctly disclose wherein the apparatus is operable to classify at least one of the plurality of inputs as being supportive or resistant using an explainable artificial intelligence, XAI, process.
Nevertheless, Dalli teaches wherein the apparatus is operable to classify at least one of the plurality of inputs as being supportive or resistant using an explainable artificial intelligence, XAI, process (Dalli, [0016] teaches receiving, from a user, a set of data corresponding to human knowledge; representing the set of data in the symbolic logic format, said representation comprising a localization trigger and an action; creating a new condition based on the localization trigger, wherein the explainable model is configured to execute the action upon detection of the new condition; receiving an input related to the human knowledge; inputting the input to the explainable model and receiving an output from the explainable model; receiving a localized explanation from the explainable model; summarizing the localized explanation and identifying relevant feature attributions; performing a control and quality check; validating, by a control node, the result of the control and quality check and determining if an exception should be triggered; generating an analysis of the partitions; generating a visualization of the summarized explanations and relevant feature attributions; and outputting the analysis of the partitions, visualization of the summarized explanations, and relevant feature attributions.; Dalli [0038] teaches the terms interpretable and explainable may have different meanings. Interpretability may be a characteristic that may need to be defined in terms of an interpreter. The interpreter may be an agent that interprets the system output or artifacts using a combination of (i) its own knowledge and beliefs; (ii) goal-action plans; (iii) context; and (iv) the world environment. An exemplary interpreter may be a knowledgeable human.; Dalli [0068] further teaches human knowledge may be incorporated directly into the XAI model using appropriate tools or interfaces in a similar manner to expert systems or workflow systems. When it comes to XNNs, it may be possible for human knowledge to be incorporated directly into an XNN due to its white-box nature. FIG. 2B shows an exemplary embodiment of such method. Step 120 shows how human knowledge injection may be applied directly. Additionally, FIG. 5 shows an exemplary embodiment of an XNN with human knowledge injection applied, which could have been applied directly or indirectly from an expert system via the necessary conversions. Dalli, [0085] teaches when justifications are supported by the explainable model, the rule format may be in the form: [0086] If <Localization Trigger> then (<Answer>, <Explanation>, <Justification>) where <Localization Trigger> may represent the partition condition which activates a partition containing a rule; <Answer> represents the answer when the partition rule is applied to the input and may be a classification, probability/value, or some other result; and <Explanation> may provide an underlying explanation, such as in the form of text or an image which explains which features of the input produced the corresponding answer, and may correspond to the model explanation produced by an explainable model. The <Justification> of the answer and/or its model explanation may be an explanation of the model explanation (i.e., a meta-explanation) that gives additional information about the assumptions, processes and decisions taken by the explainable or interpretable system and/or model when outputting the answer and/or model explanation.; Dalli [0088] Expert systems typically include a set of triggers and actions. A trigger may be represented by one or more partition conditions. In this example, x≤20 is the localization trigger for the first rule (Rule 0), and x>10∧x≤20 is the localization trigger for the second rule (Rule 1). According to the generalized rule format, the <Localization Trigger> represents the trigger of the expert system, and the action includes the combination of <Answer> and <Explanation>. The <Localization Trigger> may also represent a condition based a combination of one or more application logic, taxonomical knowledge, ontological knowledge, behavioral constraint model, attention model, semantical model, syntactical model, or business logic that represents the precise condition when an expert system rule triggers or workflow node is activated. [Note: Examiner is interpreting Dalli to teach input classification using XIA and justifications supported by the explainable model as taught in [0085] and [0086]])
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the method and system for construction of a machine learning model for structured inputs, as taught by Taylor, with the incorporation of the XIA model for explainable actions or answers, as taught by Dalli. Using an exemplary embodiment, humans or other appropriate automated systems such as logical reasoning systems may add rules that will give rise to a potential hybrid implementation that may use XNNs for most computational needs and then can use the XNN action trigger and distributed implementation options to selectively use the XAI model rules to incorporate results that are currently beyond the computational expressivity of neural networks. Notably, such a framework may thus make it possible to incorporate any type of XAI model and rule in a complete 1:1 correspondence, allowing the XNN (or other ANN) to benefit from the more complete (and more Turing-complete) functionality of the XAI or XAIs. (Dalli, [0061])
Regarding claim 22, Taylor teaches all of the limitations of claim 21, however Taylor does not distinctly disclose wherein the apparatus is operable to determine the relative importance of the input using an XAI process.
Nevertheless, Dalli teaches wherein the apparatus is operable to determine the relative importance of the input using an XAI process (Dalli, [0147] teaches the globality of the XAI model and/or XNN enables the user to analyze for any potential bias. It may be analyzed by a human user, that the XAI model and/or XNN is biased towards a specific feature, or group of features. Such bias may be either in the training data or in the model itself. In order to circumvent the problem, human knowledge injection may be applied to enforce specific rules. For example, human rules and/or a combination of processes using workflows may be applied in order to ensure gender equality in a hiring system, ensuring that country-specific or jurisdiction-specific legislation is complied with. Bias detection may be determined through the coefficients of the rules or by aggregating the coefficients of specific explanations. In general, the higher the coefficient absolute values, the more important the feature is. Such feature importance helps determine where the bias is. Bias detection may also be determined for the partition conditions.; Dalli [0162], teaches in an exemplary embodiment, XAI, XNNs, rule-based models, workflows and logically equivalent explainable models may allow for selective deletion of particular logical rules or selective deletion of specific components from an XAI, XNN, rule-based model, workflow or other explainable model. In an exemplary application, customer records may have to be deleted due to data protection issues and the right to be forgotten (GDPR). The white-box nature of explainable models may easily identify which partitions, components or connections could potentially be impacted by a removing a specific data point. Analysis may be performed such that the impact is examined locally (on a specific partition) as well as globally (on the entire explainable model). The analysis may incorporate frequency analysis of a specific path trace along partitions, connections and features in order to identify the rarity or commonality of such data point. The trace path analysis may be in the form of a backmap process whereby the output of the neural network is projected back to the input in order to analyze and perform an impact assessment of the partition, feature importance, and data in the explainable model and data via HKI processes, against a number of criteria and thresholds and values set against those criteria. If the impact assessment concludes that such data points will result into different model behavior, various mitigation strategies may be applied. The first strategy may involve updating of weights to minimize or take out a path without the need for re-training. A second strategy may involve updating weights along the connection to minimize or reduce the effect of the data point without needing re-training. A third strategy may involve using Fast XAI extensions to achieve the two other strategies in real-time by updating the explainable model in real-time without need for re-training. A fourth strategy may also involve re-training parts of the model using a modified version of the original dataset which now excludes the selected data points. Unlike black-box models, XAI/XNN models may eliminate the need to retrain the entire model from scratch, which in some cases may not be practically possible.).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the method and system for construction of a machine learning model for structured inputs, as taught by Taylor, with the incorporation of the XIA model for explainable actions or answers, as taught by Dalli. Using an exemplary embodiment, humans or other appropriate automated systems such as logical reasoning systems may add rules that will give rise to a potential hybrid implementation that may use XNNs for most computational needs and then can use the XNN action trigger and distributed implementation options to selectively use the XAI model rules to incorporate results that are currently beyond the computational expressivity of neural networks. Notably, such a framework may thus make it possible to incorporate any type of XAI model and rule in a complete 1:1 correspondence, allowing the XNN (or other ANN) to benefit from the more complete (and more Turing-complete) functionality of the XAI or XAIs. (Dalli, [0061])
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Taylor et al., as applied to claim 19, and further in view of Takahashi et al. (US 20190042625 A1, filed Jul. 31, 2018 and published Feb. 7, 2019)
Regarding claim 23, Taylor teaches all of the limitations of claim 19, and Taylor further teaches wherein the apparatus is operable to compare the classification of the at least one of the plurality of inputs to domain knowledge to determine whether or not the proposed action conflicts with the domain knowledge … (Taylor, [0094] A data receiver 102 is configured for receiving data sets representative of the demonstrations for performing sequential tasks (e.g., playing games, trading stocks, sorting, association learning, image recognition, stock market transaction control). The demonstrator data sets are provided to classifier trainer, which trains a classifier model based on the demonstrator data stored on demonstrator classifier data storage 154.; Taylor [0095] teaches as there may be differences in quality as between demonstrators and their associated demonstrator data sets, as described in various embodiments, these potential contradictions arise in the form of differing actions that are suggested by at least one of the demonstrator data sets (e.g., from a demonstrator), or from the machine learning model itself.; Taylor, [0015] further teaches as described in various embodiments, an interactive reinforcement learning mechanism is adapted for providing computer implemented systems for (i) identifying demonstration data that is contradictory, and not using it as the basis for a decision; and/or (ii) determining that insufficient demonstration data has been provided, and prompting for more data to be submitted.)
However, Taylor does not distinctly disclose …determine whether or not the proposed action conflicts with the domain knowledge by: determining that a conflict occurs in response to determining that one or more inputs have a classification that contradicts the domain knowledge.
Nevertheless, Takahashi teaches wherein the apparatus is operable to compare the classification of the at least one of the plurality of inputs to domain knowledge to determine whether or not the proposed action conflicts with the domain knowledge by: determining that a conflict occurs in response to determining that one or more inputs have a classification that contradicts the domain knowledge (Takahashi, [0121] teaches a detection program in the present embodiment causes a computer to execute processing of determining whether or not an input class corresponds to any of specific classes in individual subject classes of a knowledge base including a hierarchical structure and classes corresponding to specific classes in negation classes that contradict the subject classes. If it is determined that the input class corresponds to none of the specific classes and the corresponding classes, the detection program causes the computer to execute processing of outputting information relating to the determined specific class. This allows refinement detection of the class.; Takahashi, [0031] teaches in one aspect of the present disclosure, providing a detection method, a detection apparatus, and a recording medium that may support proper classification of the input class is intended.; Takahashi, [0123] teaches if it is determined that the input class corresponds to any of the specific classes and the corresponding classes, lower-level classes of the class determined as corresponding and classes equal to or lower than the contradicting class corresponding to the class determined as corresponding may be excluded from the target of the subsequent processing of inference. This may reduce the number of classes deemed as the target of the inference processing)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the method and system for construction of a machine learning model for structured inputs, as taught by Taylor, with the detection program, as taught by Takahashi, in order to reduce the number of classes deemed as the target of the inference processing. (Takahashi, [0123])
Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over Taylor et al., as applied to claim 24, and further in view of Dalli et al. (US 20210232915 A1, hereinafter Dalli ‘915, filed Jan. 22, 2021 and published Jul. 29, 2021)
Regarding claim 25, Taylor teaches all of the limitations of claim 24, however, Taylor does not distinctly disclose wherein the apparatus is operable to map one or more other inputs to one or more events using a mapping model developed using a machine learning process.
Nevertheless, Dalli ‘915 teaches wherein the apparatus is operable to map one or more other inputs to one or more events using a mapping model developed using a machine learning process (Dalli ‘915, [0019] teaches another main advantage of CNN-XNNs over ACE is that the CNN-XNN architecture is fully white-box which enables the explainability not just of the key concepts, but also of the key symbols and kernels along the entire network via the reverse indexing mechanism (Backmap); Dalli ‘915, [0059] further teaches the reverse indexing mechanism (Backmap) is something unique to the CNN-XNN architecture which is possible due to the white-box nature of CNN-XNNs. Since all layers are white-box, including the CNN layers, it is possible to apply reverse indexing in order to backtrack the output all the way to the original input.; Dalli ‘915, [0063] further teaches the output from the prediction network may serve as the basis for generating explanations in a CNN-XNN. The prediction network may weigh the input features by using an equation where each feature is weighed by a coefficient. Each coefficient represents the importance of the final convoluted features. In order to create meaningful explanations, the convoluted features along with their importance may be mapped back to the original input. Since the middle layers of CNN-XNNs are not fully connected dense layers (black-box) but rather sparsely connected (white-box) layers of the XNN, it is possible to apply a reverse indexing mechanism (Backmap) that maps the output of the convoluted features back to the original input. CNN-XNNs are unique in their implementation of a reverse indexing mechanism, which allows fast output of explanations together with fast association of the explanations with the answer and the precise activation path followed by the CNN-XNN during processing.; Dalli ‘915, [0065] teaches the Backmap may be processed as an external process or as a neural network, or logically equivalent, which performs the inverse function. The neural network may be embedded within the CNN-XNN or kept as a separate network specifically designed for performing the inverse indexing mechanism.).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the method and system for construction of a machine learning model for structured inputs, as taught by Taylor, with the Backmap process/neural network within the XNN, as taught by Dalli ‘915, in order to allows fast output of explanations together with fast association of the explanations with the answer and the precise activation path followed by the CNN-XNN during processing. (Dalli ‘915, [0063])
Claim 27 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Taylor et al., as applied to claim 26, and further in view of Wang et al. (WO2020/239203A1, published Dec. 3, 2020)
Regarding claim 27, Taylor teaches all of the limitations of claim 26, however, Taylor does not distinctly disclose wherein the communication network comprises a radio access network and the plurality of inputs comprises one or more metrics of the radio access network and the proposed action comprises configuring an operational parameter of the radio access network.
Nevertheless, Wang teaches wherein the communication network comprises a radio access network and the plurality of inputs comprises one or more metrics of the radio access network and the proposed action comprises configuring an operational parameter of the radio access network (Wang, Abstract, teaches a technique for generating synthetic data as input for a machine learning process that recommends radio access network, RAN, configurations is presented… The synthetic data is in the same form as the non-synthetic data and comprises synthetic configuration management, CM, parameter values, synthetic RAN characteristic parameter values and synthetic performance indicator values.; Wang, Background, lines 15-20 teaches in radio access networks (RANs), each radio base station (RBS) has a large number of configurable parameters to control the behavior of individual RBS functions. The maintenance and optimization of these parameters are the tasks of configuration management (CM), and the parameters are therefore also called CM parameters; Wang, pg. 7, lines 14-22 teaches the apparatus, in particular in the context of the machine learning process, may be configured to process the synthetic data and the non-synthetic data in matrix form using a matrix factorization approach. The apparatus may be configured to output, for a given RAN or RAN cell type associated with one or more given RAN characteristic parameter values, the one or more CM parameter values associated that fulfil a predefined criterion in regard to one or more performance indicators. The criterion may be a criterion selected such that the one or more CM parameter values associated with the best performance in regard to one or more performance indicators are output. Alternatively or in addition, a thresholding criterion may be applied. [Note: predefined criterion reading on inputted “metric”]).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the construction of a machine learning model for structured inputs, as taught by Taylor, with the radio access network configuration recommendation method and system, as taught by Wang. The proposed technique provides a solution to a key challenge when building a per- RAN or per-cell CM recommender system. By generating synthetic data from non- synthetic data, the recommender system can give recommendations on a wider range of CM parameter settings with higher accuracy. Since the synthetic data can be generated offline using an ML model, it reduces risk, time and cost compared with trying the new settings in a live mobile network. The resulting CM recommender system is an important function for automated network optimization and zero-touch network management. (Wang, pg. 23, lines 35-38 and pg. 24, lines 1-4)
Regarding claim 28, Taylor teaches all of the limitations of claim 26, however, Taylor does not distinctly disclose wherein the node comprises a base station or a core network node.
Nevertheless, Wang teaches wherein the node comprises a base station or a core network node (Wang, pg. 1, lines 16-20 teaches in radio access networks (RANs), each radio base station (RBS) has a large number of configurable parameters to control the behavior of individual RBS functions. The maintenance and optimization of these parameters are the tasks of configuration management (CM), and the parameters are therefore also called CM parameters.; Wang, pg. 5, lines 33-38 and pg. 6, lines 1-22 teaches one or more of these processing steps, in particular the classifying steps, may be performed using NN processing. NN processing in one or more of these processing steps may be performed by a dedicated NN. The first classification result may be based on a result of the evaluation of the input and the second classification result may be based on the evaluation of the matching between the input and the conditional values. In some implementations, the result of the first classification is a first confidence parameter indicative of whether the synthetic data can be regarded as an authentic, or realistic, input for the machine learning process. The result of the second classification may be a second confidence parameter indicative of whether the synthetic data satisfies the conditional values. The apparatus may be configured to determine if the result of the first classification (e.g., the first confidence parameter) to meet a predefined first condition (e.g., based on a first threshold decision) to decide whether or not to use the synthetic data as input for the machine learning process. The apparatus may also be configured to subject the result of the second classification (e.g., the first confidence parameter) to meet a predefined second condition (e.g., based on a second threshold decision) to decide whether or not to use the synthetic data as input for the machine learning process. The trained generative machine learning model may have been trained using a gradient-based optimization process. In some variants, the trained generative machine learning model is configured to generate one or more discrete data values comprised by the synthetic data. The apparatus may be configured to be operated offline in regard to a given RAN for which the RAN configuration is to be determined. The apparatus may be located at site remote from a site at which RBSs of the given RAN are located.).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the construction of a machine learning model for structured inputs, as taught by Taylor, with the radio access network configuration recommendation method and system, as taught by Wang. The proposed technique provides a solution to a key challenge when building a per- RAN or per-cell CM recommender system. By generating synthetic data from non- synthetic data, the recommender system can give recommendations on a wider range of CM parameter settings with higher accuracy. Since the synthetic data can be generated offline using an ML model, it reduces risk, time and cost compared with trying the new settings in a live mobile network. The resulting CM recommender system is an important function for automated network optimization and zero-touch network management. (Wang, pg. 23, lines 35-38 and pg. 24, lines 1-4)
Claim 29 is rejected under 35 U.S.C. 103 as being unpatentable over Taylor et al. in view of Wang et al., as applied to claim 27, and further in view of John et al. (US 20210132947 A1, filed Nov. 2, 2020 and published May 6, 2021)
Regarding claim 29, the combination of Taylor in view of Wang teaches all of the limitations of claim 27, and however the combination does not distinctly disclose wherein the apparatus is further operable to: map other metrics of the radio access network that are input to the model to one or more events for comparison with the domain knowledge; and compare the set of events with the domain knowledge to determine whether or not the proposed action conflicts with the domain knowledge.
Nevertheless, John teaches wherein the apparatus is further operable to: map other metrics of the radio access network that are input to the model to one or more events for comparison with the domain knowledge (John, [0074] teaches the optimizer system 800 can include a training module 802 (e.g., as described with reference to FIG. 1A). In particular, the training module 802 may contain several different models (e.g., regression model, deep reinforcement q-learning model as described in FIG. 5, etc.) that have learned the relationship [i.e., mapping] between different combinations between predictors and responses from possible inputs 804 (e.g., predict runtime, cost of a particular configuration, optimal configuration, etc.).; and compare the set of events with the domain knowledge to determine whether or not the proposed action conflicts with the domain knowledge (John, [0083] teaches The deep reinforcement q-learning model can, in some implementations, use q-learning methods to learn what actions to take under what circumstances.;; John, [0106] teaches in some implementations, the WMS further includes: one or more data access objects (“DAOs”) for each ML model comprising a train DAO, a validation DAO and/or a test DAO stored in an application database on external distributed storage and made available by the CSP interface module to the training component of the optimizer module for performing training and evaluation of each of the ML models of the optimizer module, wherein the training component of the optimizer module achieves improvements of the ML models with respect to a cost function defined for each ML model by executing an optimization algorithm on the internal weights and/or parameters of the ML model using data from the train DAO, and using data from the validation DAO to gain performance metrics to be used in making decisions regarding hyperparameter tuning of the ML model, and using data from the test DAO to maintain a set of performance metrics for each ML model that describe the performance of the current version of the ML model.; John, [0108] teaches the configuration module contains functionality to simulate runs of existing workflows by means of data collected from previous runs, user-provided estimates of or constraints placed upon task execution time or resource usage, or estimates obtained by means of heuristic algorithms in order to inform the determination of optimal task execution configurations for the workflow in question, and whereby the configuration module may further launch redundant tasks and workflows with different execution configurations in order to obtain more data for the training and validation of models to improve prediction performance of the ML models of the optimization module.; John, [claim 11] further teaches the configuration subsystem is configured to simulate runs of existing workflows by means of data collected from previous runs, user-provided estimates of constraints placed upon task execution time or resource usage, or estimates obtained by means of heuristic algorithms in order to inform a determination of optimal task execution configurations for the workflow in question, and wherein the configuration subsystem further launches redundant tasks and workflows with different execution configurations in order to obtain more data for training and validation of models to improve prediction performance of the ML models of the optimization subsystem; [Note: Wang is relied upon to teach “metrics of the radio access network” while John teaches validation process to gain performance metrics to be used in tuning of the ML model and test data to maintain a set of performance metrics for each ML model. And, John further teaching learned the relationship [i.e., mapping] between different combinations between predictors and responses from possible inputs).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the construction of a machine learning model for structured inputs, as taught by Taylor in view of Wang, with the validation/test process of artificial intelligence (AI-) enabled workflow management system, as taught by John, in order to improve prediction performance of the ML models of the optimization module. (John, [0108])
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Hildebrandt et al. (US 20220374730 A1) teaching a computer-implemented method and system for assigning at least one query triplet to at least one respective class. The at least one respective class is true or false. The method includes the steps of providing the at least one query triplet and a knowledge graph with a plurality of triples and extracting at least one affirmative argument using reinforcement learning on the basis of the at least one query triplet and the knowledge graph. The at least one affirmative argument indicates that the at least one query triplet is true. The method further includes extracting at least one opposing argument using reinforcement learning on the basis of the at least one query triplet and the knowledge graph. The at least one opposing argument indicates that the at least one query triplet is false. The method further includes assigning the at least one query triplet to the at least one respective class using supervised machine learning depending on the at least two arguments. See also [0013], [0014], and [0015].
Kodish-Wachs et al. (US 20190197428 A1) - [0040] The model structure differentiation tool 510 receives one or more knowledge models from the control center 508. For example, the model structure differentiation tool 510 identifies a structure of a first knowledge model and identifies a structure of a second knowledge model. The term “structure” refers to one or more of the rules, axioms, taxonomic hierarchy of an ontology, rule maps, and a master document. In embodiments, the model structure differentiation tool 510 identifies the rules, axioms, taxonomic hierarchy, a rules maps, and a master document for each model received. When the model structure differentiation tool 510 receives two or more models, the model structure differentiation tool 510 may further determines differences between two or more of the models received. See also, [0046], [0047], and [0051].
Steyerberg et al., “Prediction models need appropriate internal, internal-external, and external validation” (2016)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BEATRIZ RAMIREZ BRAVO whose telephone number is 571-272-2156. The examiner can normally be reached Mon. - Fri. 7:30a.m.-5:00p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, USMAAN SAEED can be reached at 571-272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.R.B./Examiner, Art Unit 2146
/SHOURJO DASGUPTA/Primary Examiner, Art Unit 2144
Prosecution Insights