Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
The present application is being examined under the claims filed on 5/15/2024.
Claims 1-20 are rejected.
Claims 1-20 are pending.
Specification
The specification filed on 5/15/2024 is acceptable for examination purposes.
Drawings
The drawings filed on 5/15/2024 are acceptable for examination purposes.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: “the executed application engines causing the at least one processor to perform operations […]” in claims 1, 10, 13, 17 and 20, “the subset of the executed application engines causing the at least one processor to perform operations […]” in claims 12 and 19.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Objections
Claims 1, 2, 5, 10, 11, 13 and 17-20 are objected to because of the following informalities:
In claim 1, line 11, “a trained, machine-learning” should read “a trained machine-learning”
In claim 2, lines 2-3, “the trained, machine-learning” should read “the trained machine-learning”
In claim 5, lines 7-8, “corresponding ones of element of the first configuration data” should read “corresponding elements of the first configuration data”
In claim 10, lines 6-7, “the trained, machine-learning” should read “the trained machine-learning”
In claim 10, lines 8-9, “being subsequent to the first inferencing data” should read “being subsequent to the first inferencing date”
In claim 11, line 2, “the trained, machine-learning” should read “the trained machine-learning”
In claim 11, lines 2-3, “a trained, forward-in-time machine-learning” should read “a trained forward-in-time machine-learning”
In claim 13, line 6, “a trained, machine-learning” should read “a trained machine-learning”
In claim 17, line 1, “The apparatus of claim 13” should read “The computer-implemented method of claim 13”
In claim 17, line 7, “the trained, machine-learning” should read “the trained machine-learning”
In claim 17, line 9, “being subsequent to the first inferencing data” should read “being subsequent to the first inferencing date”
In claim 18, line 1, “The apparatus of claim 13” should read “The computer-implemented method of claim 13”
In claim 18, line 2, “the trained, machine-learning” should read “the trained machine-learning”
In claim 19, line 1, “The apparatus of claim 18” should read “The computer-implemented method of claim 18”
In claim 20, line 7, “a trained, machine-learning” should read “a trained machine-learning”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as failing to set forth the subject matter which the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the applicant regards as the invention.
Claim limitations “the executed application engines causing the at least one processor to perform operations […]” in claims 1, 10, 13, 17 and 20, and “the subset of the executed application engines causing the at least one processor to perform operations […]” in claims 12 and 19 invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. In particular, the specification, at best, describes “a default training pipeline of a sequentially ordered subset of the application engines that includes, but is not limited to, retrieval engine 156, preprocessing engine 158, indexing engine 160, target-generation engine 162, splitting engine 164, featurizer engine 166, training engine 168, and reporting engine 172” (Spec. Paragraph [044]) and does not provide any specific software or hardware to support the claimed application engines. Thus, the disclosure provides no association between the structure and the function can be found in the specification. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
In reference to dependent claims 2-12 and 14-19, claims 2-12 do not cure the deficiencies noted in the rejection of independent claim 1, and claims 14-19 do not cure the deficiencies noted in the rejection of independent claim 13. Therefore, these claims are rejected under the same rationale as claim 1 and claim 13.
For the purpose of this examination, limitations “application engines” are interpreted as software module loaded in a component data store 138. See MPEP 2173.06(I).
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1,
Step 1: Claim 1 is an apparatus claim. Therefore, Claims 1-12 are directed to a machine.
Step 2A Prong 1:
execute sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data, the executed application engines causing the at least one processor to perform operations that apply a trained, machine-learning process to an input dataset on a first inferencing date (mental process – executing sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data and performing operations that apply a trained, machine-learning process to an input dataset on a first inferencing date may be performed manually by a user with the aid of pen and paper by observing/analyzing the first configuration data, a trained, machine-learning process, and an input dataset on a first inferencing date, and using a judgement to execute sequentially a plurality of application engines and apply the machine-learning process to the input dataset. See MPEP 2106.04(a)(2)(III)(C).)
perform operations that populate a first data record with at least an identifier of the inferencing pipeline, the first inferencing date, and the elements of artifact data, and that store the first data record within a corresponding portion of the memory, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date (mental process – performing operations that populate a first data record with at least an identifier of the inferencing pipeline, the first inferencing date, and the elements of artifact data, and that store the first data record within a corresponding portion of the memory may be performed manually by a user with the aid of pen and paper by observing/analyzing the configuration data and the first inferencing date, and using a judgement to create a first data record with at least an identifier of the inferencing pipeline and store the first data record in memory. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
a memory storing instructions; a communications interface; and at least one processor coupled to the memory and the communications interface, the at least one processor being configured to execute the instructions to: (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
obtain elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional elements recite generic computer elements and programs at a high-level of generality to perform the judicial exception as well as recitation of generic computer functionality such as a memory, a communications interface and a processor.
Additional Elements:
a memory storing instructions; a communications interface; and at least one processor coupled to the memory and the communications interface, the at least one processor being configured to execute the instructions to: (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
obtain elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
For the reasons above, Claim 1 is rejected as being directed to an abstract idea without significantly more. This rejection applies equally to dependent claims 2-12. The additional limitations of the dependent claims are addressed below.
Regarding Claim 2,
Step 2A Prong 1:
See the rejection of Claim 1 above, which Claim 2 depends on.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
wherein the elements of artifact data comprise explainability data that characterizes the application of the trained, machine- learning process to the input dataset on the first inferencing date, the explainability data being generated by a corresponding one of the executed application engines within the inferencing pipeline (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
wherein the elements of artifact data comprise explainability data that characterizes the application of the trained, machine- learning process to the input dataset on the first inferencing date, the explainability data being generated by a corresponding one of the executed application engines within the inferencing pipeline (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
Regarding Claim 3,
Step 2A Prong 1:
See the rejection of Claim 2 above, which Claim 3 depends on.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
wherein the explainability data comprises at least one of (i) a first value characterizing an importance of one or more features to an output of the trained machine-learning process or (ii) a second value characterizing a performance of the trained machine-learning process (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
wherein the explainability data comprises at least one of (i) a first value characterizing an importance of one or more features to an output of the trained machine-learning process or (ii) a second value characterizing a performance of the trained machine-learning process (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
Regarding Claim 4,
Step 2A Prong 1:
See the rejection of Claim 2 above, which Claim 4 depends on.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
wherein the elements of artifact data further comprise elements of performance data characterizing an operation of each of the executed application engines within the inferencing pipeline (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
wherein the elements of artifact data further comprise elements of performance data characterizing an operation of each of the executed application engines within the inferencing pipeline (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
Regarding Claim 5,
Step 2A Prong 1:
the at least one processor is further configured to execute the instructions to execute sequentially each of the application engines within the inferencing pipeline based on corresponding ones of element of the first configuration data (mental process – executing the instructions to execute sequentially each of the application engines within the inferencing pipeline based on corresponding ones of element of the first configuration data may be performed manually by a user with the aid of pen and paper by observing/analyzing the corresponding ones of element of the first configuration data and using a judgement to execute sequentially each of the application engines. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
wherein: the first configuration data comprises a plurality of elements, the elements of the first configuration data being associated with corresponding ones of the application engines (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
the at least one processor is further configured to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
wherein: the first configuration data comprises a plurality of elements, the elements of the first configuration data being associated with corresponding ones of the application engines (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
the at least one processor is further configured to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Regarding Claim 6,
Step 2A Prong 1:
execute the inferencing pipeline script, the executed inferencing pipeline script causing the at least one processor to execute sequentially each of the application engines at a corresponding position within the execution flow (mental process – executing the inferencing pipeline script and executing sequentially each of the application engines at a corresponding position within the execution flow may be performed manually by a user with the aid of pen and paper by observing/analyzing the inferencing pipeline script, a corresponding position within the execution flow and using a judgement to execute the inferencing pipeline script and execute sequentially each of the application engines. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
wherein: obtain pipeline data associated with the inferencing pipeline from the memory, the pipeline data comprising a inferencing pipeline script that establishes an execution flow for the sequential execution of the application engines within the inferencing pipeline (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
the executed inferencing pipeline script causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
wherein: obtain pipeline data associated with the inferencing pipeline from the memory, the pipeline data comprising a inferencing pipeline script that establishes an execution flow for the sequential execution of the application engines within the inferencing pipeline (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
the executed inferencing pipeline script causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Regarding Claim 7,
Step 2A Prong 1:
See the rejection of Claim 1 above, which Claim 7 depends on.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
the plurality of data records comprising the first data record and one or more second data records, each of the second data records establishing an additional configuration of the executed application engines of the inferencing pipeline at a second inferencing date, and the first inferencing date being subsequent to each of the second inferencing dates (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
wherein the corresponding portion of the memory comprises a plurality of data records (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
the plurality of data records comprising the first data record and one or more second data records, each of the second data records establishing an additional configuration of the executed application engines of the inferencing pipeline at a second inferencing date, and the first inferencing date being subsequent to each of the second inferencing dates (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
wherein the corresponding portion of the memory comprises a plurality of data records (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Regarding Claim 8,
Step 2A Prong 1:
based on at least elements of temporal data maintained within the data records, determine that a subset of the data records are consistent with the query data, the subset of the data records comprising the first data record and at least one of the second data records (mental process – based on at least elements of temporal data maintained within the data records, determine that a subset of the data records are consistent with the query data, the subset of the data records comprising the first data record and at least one of the second data records may be performed manually by a user with the aid of pen and paper by observing/analyzing the elements of temporal data within the data records and the query data, and using a judgement to determine that a subset of the data records are consistent with the query data. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
receive elements of query data from a computing system via the communications interface (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
wherein the at least one processor is further configured to execute the instructions to: (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
transmit elements of response data that include the subset of the data records to the computing system via the communications interface (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
receive elements of query data from a computing system via the communications interface (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
wherein the at least one processor is further configured to execute the instructions to: (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
transmit elements of response data that include the subset of the data records to the computing system via the communications interface (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
Regarding Claim 9,
Step 2A Prong 1:
wherein: the computing system is configured to execute one or more scripts, the one or more scripts causing the computing system to process portions of the first data record and the at least one of the second data record (mental process – executing one or more scripts and the one or more scripts processing portions of the first data record and the at least one of the second data record may be performed manually by a user with the aid of pen and paper by observing/analyzing the one or more scripts and using a judgement to execute them to process portions of the first data record and the at least one of the second data record. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
the computing system is configured to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
causing the computing system to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
present a graphical representation of at least the processed portion within a digital interface (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
the computing system is further configured to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
the computing system is configured to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
causing the computing system to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
present a graphical representation of at least the processed portion within a digital interface (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
the computing system is further configured to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Regarding Claim 10,
Step 2A Prong 1:
wherein the at least one processor is further configured to execute the instructions to: execute sequentially at least a subset of the application engines within the inferencing pipeline in accordance with second configuration data, the executed application engines causing the at least one processor to perform operations that apply the trained, machine- learning process to an additional input dataset on a second inferencing date, the second inferencing date being subsequent to the first inferencing data (mental process – executing sequentially at least a subset of the application engines within the inferencing pipeline in accordance with second configuration data and performing operations that apply the trained, machine-learning process to an additional input dataset on a second inferencing date, the second inferencing date being subsequent to the first inferencing data may be performed manually by a user with the aid of pen and paper by observing/analyzing the second configuration data, the trained, machine-learning process, and an additional input dataset on a second inferencing date, and using a judgement to execute sequentially at least a subset of the application engines and apply the machine-learning process to the additional input dataset. See MPEP 2106.04(a)(2)(III)(C).)
detect a failure in the sequential execution of at least one of the subset of the application engines based on elements of additional artifact data (mental process – detecting a failure in the sequential execution of at least one of the subset of the application engines based on elements of additional artifact data may be performed manually by a user with the aid of pen and paper by observing/analyzing the elements of additional artifact data, the subset of the application engines, and using a judgement to detect a failure in the sequential execution of at least one of the subset of the application engines. See MPEP 2106.04(a)(2)(III)(C).)
perform operations that initiate an additional sequential execution of the plurality of application engines within the inferencing pipeline in accordance with at least a portion of the first configuration data (mental process – performing operations that initiate an additional sequential execution of the plurality of application engines within the inferencing pipeline in accordance with at least a portion of the first configuration data may be performed manually by a user with the aid of pen and paper by observing/analyzing the first configuration data and using a judgement to perform operations that initiate an additional sequential execution of the application engines. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
wherein the at least one processor is further configured to execute the instructions to: (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
based on the detected failure, obtain the first data record from the corresponding portion of the memory, and obtain the first configuration data from the elements of artifact data (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
wherein the at least one processor is further configured to execute the instructions to: (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
based on the detected failure, obtain the first data record from the corresponding portion of the memory, and obtain the first configuration data from the elements of artifact data (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
Regarding Claim 11,
Step 2A Prong 1:
the executed application engines further cause the at least one processor to perform operations that apply the trained, forward-in-time machine-learning process to the input dataset, and based on the application of the trained, forward-in-time machine-learning process to the input dataset, generate output data indicating a predicted likelihood of an occurrence of a target event during a future temporal interval (mental process – performing operations that apply the trained, forward-in-time machine-learning process to the input dataset, and based on the application of the trained, forward-in-time machine-learning process to the input dataset, generating output data indicating a predicted likelihood of an occurrence of a target event during a future temporal interval may be performed manually by a user with the aid of pen and paper by observing/analyzing the trained, forward-in-time machine-learning process, the input dataset, a target event, and using a judgement to apply the trained, forward-in-time machine-learning process to the input dataset and create output data indicating a predicted likelihood of an occurrence of a target event. See MPEP 2106.04(a)(2)(III)(C).)
generate elements of evaluation data that characterize a predictive performance of the trained, forward-in-time machine-learning process (mental process – generating elements of evaluation data that characterize a predictive performance of the trained, forward-in-time machine-learning process may be performed manually by a user with the aid of pen and paper by observing/analyzing the trained, forward-in-time machine-learning process and using a judgement to create elements of evaluation data that characterize a predictive performance of the trained, forward-in-time machine-learning process. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
wherein: the trained, machine-learning process comprises a trained, forward-in-time machine-learning process (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
the executed application engines further cause the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
obtain labelling data associated with the input dataset, the labelling data specifying the occurrence, or a non- occurrence, of the target event during the future temporal interval (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
the at least one processor is further configured to execute the instructions to: (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
transmit the elements of evaluation data to a computing system (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
wherein: the trained, machine-learning process comprises a trained, forward-in-time machine-learning process (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
the executed application engines further cause the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
obtain labelling data associated with the input dataset, the labelling data specifying the occurrence, or a non- occurrence, of the target event during the future temporal interval (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
the at least one processor is further configured to execute the instructions to: (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
transmit the elements of evaluation data to a computing system (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
Regarding Claim 12,
Step 2A Prong 1:
wherein the at least one processor is further configured to execute the instructions to execute sequentially at least a subset of the application engines within a target-generation pipeline in accordance with second configuration data, the subset of the executed application engines causing the at least one processor to perform operations that generate the labelling data associated with the input dataset (mental process – executing sequentially at least a subset of the application engines within a target-generation pipeline in accordance with second configuration data and performing operations that generate the labelling data associated with the input dataset may be performed manually by a user with the aid of pen and paper by observing/analyzing the second configuration data, the input dataset, and using a judgement to execute sequentially at least a subset of the application engines and create the labelling data associated with the input dataset. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
wherein the at least one processor is further configured to execute the instructions to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the subset of the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
wherein the at least one processor is further configured to execute the instructions to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the subset of the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Regarding Claim 13,
Step 1: Claim 13 is a method claim. Therefore, Claims 13-19 are directed to a process.
Step 2A Prong 1:
using at least one processor, executing sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data, the executed application engines causing the at least one processor to perform operations that apply a trained, machine-learning process to an input dataset on a first inferencing date (mental process – executing sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data and performing operations that apply a trained, machine-learning process to an input dataset on a first inferencing date may be performed manually by a user with the aid of pen and paper by observing/analyzing the first configuration data, a trained, machine-learning process, and an input dataset on a first inferencing date, and using a judgement to execute sequentially a plurality of application engines and apply the machine-learning process to the input dataset. See MPEP 2106.04(a)(2)(III)(C).)
performing operations, using the at least one processor, that populate a first data record with at least an identifier of the inferencing pipeline, the first inferencing date, and the elements of artifact data, and that store the first data record within a corresponding portion of a data repository, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date (mental process – performing operations that populate a first data record with at least an identifier of the inferencing pipeline, the first inferencing date, and the elements of artifact data, and that store the first data record within a corresponding portion of a data repository may be performed manually by a user with the aid of pen and paper by observing/analyzing the configuration data and the first inferencing date, and using a judgement to create a first data record with at least an identifier of the inferencing pipeline and store the first data record in a data repository. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
using at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
obtaining, using the at least one processor, elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
using the at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional elements recite generic computer elements and programs at a high-level of generality to perform the judicial exception as well as recitation of generic computer functionality such as a processor and application engines.
Additional Elements:
using at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
obtaining, using the at least one processor, elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
using the at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
For the reasons above, Claim 13 is rejected as being directed to an abstract idea without significantly more. This rejection applies equally to dependent claims 14-19. The additional limitations of the dependent claims are addressed below.
Regarding Claim 14,
Step 2A Prong 1:
See the rejection of Claim 13 above, which Claim 14 depends on.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
wherein the elements of artifact data comprise explainability data that characterizes the application of the trained, machine- learning process to the input dataset on the first inferencing date and performance data characterizing an operation of each of the executed application engines within the inferencing pipeline, the explainability and performance data being generated by a corresponding one of the executed application engines within the inferencing pipeline (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
wherein the elements of artifact data comprise explainability data that characterizes the application of the trained, machine- learning process to the input dataset on the first inferencing date and performance data characterizing an operation of each of the executed application engines within the inferencing pipeline, the explainability and performance data being generated by a corresponding one of the executed application engines within the inferencing pipeline (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
Regarding Claim 15,
Step 2A Prong 1:
executing the inferencing pipeline script using the at least one processor, the executed inferencing pipeline script causing the at least one processor to execute sequentially each of the application engines at a corresponding position within the execution flow (mental process – executing the inferencing pipeline script and executing sequentially each of the application engines at a corresponding position within the execution flow may be performed manually by a user with the aid of pen and paper by observing/analyzing the inferencing pipeline script, a corresponding position within the execution flow and using a judgement to execute the inferencing pipeline script and execute sequentially each of the application engines. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
obtaining, using the at least one processor, pipeline data associated with the inferencing pipeline from the memory, the pipeline data comprising a inferencing pipeline script that establishes an execution flow for the sequential execution of the application engines within the inferencing pipeline (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
using the at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the executed inferencing pipeline script causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
obtaining, using the at least one processor, pipeline data associated with the inferencing pipeline from the memory, the pipeline data comprising a inferencing pipeline script that establishes an execution flow for the sequential execution of the application engines within the inferencing pipeline (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
using the at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the executed inferencing pipeline script causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Regarding Claim 16,
Step 2A Prong 1:
based on at least elements of temporal data maintained within the data records, determining, using the at least one processor, that a subset of the data records are consistent with the query data, the subset of the data records comprising the first data record and at least one of the second data records (mental process – based on at least elements of temporal data maintained within the data records, determining that a subset of the data records are consistent with the query data, the subset of the data records comprising the first data record and at least one of the second data records may be performed manually by a user with the aid of pen and paper by observing/analyzing the elements of temporal data within the data records and the query data, and using a judgement to determine that a subset of the data records are consistent with the query data. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
the plurality of data records comprising the first data record and one or more second data records, each of the second data records establishing an additional configuration of the executed application engines of the inferencing pipeline at a second inferencing date, and the first inferencing date being subsequent to each of the second inferencing dates (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
wherein the corresponding portion of the data repository comprises a plurality of data records (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
receiving, using the at least one processor, elements of query data from a computing system (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
using the at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
transmitting, using the at least one processor, elements of response data that include the subset of the data records to the computing system (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
the plurality of data records comprising the first data record and one or more second data records, each of the second data records establishing an additional configuration of the executed application engines of the inferencing pipeline at a second inferencing date, and the first inferencing date being subsequent to each of the second inferencing dates (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
wherein the corresponding portion of the data repository comprises a plurality of data records (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
receiving, using the at least one processor, elements of query data from a computing system (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
using the at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
transmitting, using the at least one processor, elements of response data that include the subset of the data records to the computing system (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
Regarding Claim 17,
Step 2A Prong 1:
wherein the at least one processor is further configured to execute the instructions to: using the at least one processor, execute sequentially at least a subset of the application engines within the inferencing pipeline in accordance with second configuration data, the executed application engines causing the at least one processor to perform operations that apply the trained, machine- learning process to an additional input dataset on a second inferencing date, the second inferencing date being subsequent to the first inferencing data (mental process – executing sequentially at least a subset of the application engines within the inferencing pipeline in accordance with second configuration data and performing operations that apply the trained, machine-learning process to an additional input dataset on a second inferencing date, the second inferencing date being subsequent to the first inferencing data may be performed manually by a user with the aid of pen and paper by observing/analyzing the second configuration data, the trained, machine-learning process, and an additional input dataset on a second inferencing date, and using a judgement to execute sequentially at least a subset of the application engines and apply the machine-learning process to the additional input dataset. See MPEP 2106.04(a)(2)(III)(C).)
detecting, using the at least one processor, a failure in the sequential execution of at least one of the subset of the application engines based on elements of additional artifact data (mental process – detecting a failure in the sequential execution of at least one of the subset of the application engines based on elements of additional artifact data may be performed manually by a user with the aid of pen and paper by observing/analyzing the elements of additional artifact data, the subset of the application engines, and using a judgement to detect a failure in the sequential execution of at least one of the subset of the application engines. See MPEP 2106.04(a)(2)(III)(C).)
performing operations, using the at least one processor, that initiate an additional sequential execution of the plurality of application engines within the inferencing pipeline in accordance with at least a portion of the first configuration data (mental process – performing operations that initiate an additional sequential execution of the plurality of application engines within the inferencing pipeline in accordance with at least a portion of the first configuration data may be performed manually by a user with the aid of pen and paper by observing/analyzing the first configuration data and using a judgement to perform operations that initiate an additional sequential execution of the application engines. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
wherein the at least one processor is further configured to execute the instructions to: (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
using the at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
based on the detected failure, and using the at least one processor, obtaining the first data record from the corresponding portion of the memory, and obtaining the first configuration data from the elements of artifact data (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
wherein the at least one processor is further configured to execute the instructions to: (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
using the at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
based on the detected failure, and using the at least one processor, obtaining the first data record from the corresponding portion of the memory, and obtaining the first configuration data from the elements of artifact data (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
Regarding Claim 18,
Step 2A Prong 1:
the executed application engines further cause the at least one processor to perform operations that apply the trained, forward-in-time machine-learning process to the input dataset, and based on the application of the trained, forward-in-time machine-learning process to the input dataset, generate output data indicating a predicted likelihood of an occurrence of a target event during a future temporal interval (mental process – performing operations that apply the trained, forward-in-time machine-learning process to the input dataset, and based on the application of the trained, forward-in-time machine-learning process to the input dataset, generating output data indicating a predicted likelihood of an occurrence of a target event during a future temporal interval may be performed manually by a user with the aid of pen and paper by observing/analyzing the trained, forward-in-time machine-learning process, the input dataset, a target event, and using a judgement to apply the trained, forward-in-time machine-learning process to the input dataset and create output data indicating a predicted likelihood of an occurrence of a target event. See MPEP 2106.04(a)(2)(III)(C).)
generating, using the at least one processor, elements of evaluation data that characterize a predictive performance of the trained, forward-in-time machine-learning process (mental process – generating elements of evaluation data that characterize a predictive performance of the trained, forward-in-time machine-learning process may be performed manually by a user with the aid of pen and paper by observing/analyzing the trained, forward-in-time machine-learning process and using a judgement to create elements of evaluation data that characterize a predictive performance of the trained, forward-in-time machine-learning process. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
wherein: the trained, machine-learning process comprises a trained, forward-in-time machine-learning process (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
the executed application engines further cause the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
obtaining, using the at least one processor, labelling data associated with the input dataset, the labelling data specifying the occurrence, or a non- occurrence, of the target event during the future temporal interval (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
the at least one processor is further configured to execute the instructions to: (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
using the at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
transmitting, using the at least one processor, the elements of evaluation data to a computing system (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
wherein: the trained, machine-learning process comprises a trained, forward-in-time machine-learning process (merely reciting the words "apply it" (or an equivalent) with the judicial exception. See MPEP 2106.05(f).)
the executed application engines further cause the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
obtaining, using the at least one processor, labelling data associated with the input dataset, the labelling data specifying the occurrence, or a non- occurrence, of the target event during the future temporal interval (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
the at least one processor is further configured to execute the instructions to: (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
using the at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
transmitting, using the at least one processor, the elements of evaluation data to a computing system (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
Regarding Claim 19,
Step 2A Prong 1:
wherein the at least one processor is further configured to execute the instructions to, using the at least one processor, executing sequentially at least a subset of the application engines within a target-generation pipeline in accordance with second configuration data, the subset of the executed application engines causing the at least one processor to perform operations that generate the labelling data associated with the input dataset (mental process – executing sequentially at least a subset of the application engines within a target-generation pipeline in accordance with second configuration data and performing operations that generate the labelling data associated with the input dataset may be performed manually by a user with the aid of pen and paper by observing/analyzing the second configuration data, the input dataset, and using a judgement to execute sequentially at least a subset of the application engines and create the labelling data associated with the input dataset. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
wherein the at least one processor is further configured to execute the instructions to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
using the at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the subset of the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
wherein the at least one processor is further configured to execute the instructions to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
using the at least one processor (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the subset of the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Regarding Claim 20,
Step 1: Claim 20 is a non-transitory computer-readable medium claim. Therefore, Claim 20 is directed to a manufacture.
Step 2A Prong 1:
executing sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data, the executed application engines causing the at least one processor to perform operations that apply a trained, machine-learning process to an input dataset on a first inferencing date (mental process – executing sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data and performing operations that apply a trained, machine-learning process to an input dataset on a first inferencing date may be performed manually by a user with the aid of pen and paper by observing/analyzing the first configuration data, a trained, machine-learning process, and an input dataset on a first inferencing date, and using a judgement to execute sequentially a plurality of application engines and apply the machine-learning process to the input dataset. See MPEP 2106.04(a)(2)(III)(C).)
performing operations that populate a first data record with at least an identifier of the inferencing pipeline, the first inferencing date, and the elements of artifact data, and that store the first data record within a corresponding portion of the memory, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date (mental process – performing operations that populate a first data record with at least an identifier of the inferencing pipeline, the first inferencing date, and the elements of artifact data, and that store the first data record within a corresponding portion of the memory may be performed manually by a user with the aid of pen and paper by observing/analyzing the configuration data and the first inferencing date, and using a judgement to create a first data record with at least an identifier of the inferencing pipeline and store the first data record in memory. See MPEP 2106.04(a)(2)(III)(C).)
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
Additional Elements:
the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
obtaining elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines (Adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional elements recite generic computer elements and programs at a high-level of generality to perform the judicial exception as well as recitation of generic computer functionality such as a processor and application engines.
Additional Elements:
the executed application engines causing the at least one processor to (merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
obtaining elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
For the reasons above, Claim 20 is rejected as being directed to an abstract idea without significantly more.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5-10, 13, 15-17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US 10452992 B2) (hereinafter Lee) and in view of Karl (EP 3961411 A1).
Regarding Claim 1,
Lee teaches:
“An apparatus, comprising: a memory storing instructions; a communications interface; and at least one processor coupled to the memory and the communications interface, the at least one processor being configured to execute the instructions to:” (Lee, Col. 10, Lines 55-58, “In some embodiments, a pool of compute servers and/or storage servers may be pre-configured for the MLS, and the resources for a given job may be selected from such a pool.”; Lee, Col. 11, Lines 9-12, “In system 100, the MLS may implement a set of programmatic interfaces 161 (e.g., APIs, command-line tools, web pages, or standalone GUIs) that can be used by clients 164 […]”; Examiner’s note: a pool of compute servers and/or storage servers pre-configured for the MLS are at least one processor coupled to a memory storing instructions. A set of programmatic interfaces are the communications interface.)
“execute sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data, the executed application engines causing the at least one processor to perform operations that apply a trained, machine-learning process to an input dataset on a first inferencing date” (Lee, Col. 17, Lines 35-39, “In one case, termed “completion dependency”, the execution of one job Jp cannot be started until another job Jq is completed successfully (e.g., because the final output of Jq is required as input for Jp).”; Lee, Col. 28, Lines 44-46, “a text version 1101 of a transformation recipe may be passed as a parameter in a “createRecipe” MLS API call by a client.”; Lee, Col. 7, Lines 59-63 and Col. 8, Lines 5-9, “a number of different types of entities related to machine learning tasks may be generated, modified, read, executed, and/or queried/searched via MLS programmatic interfaces. Supported entity types in one embodiment may include […] model execution results such as predictions or evaluations, online access points for models that are to be used on streaming or real-time data, and/or aliases (e.g., pointers to model versions that have been “published” for use as described below).”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”; Examiner’s note: execute sequentially (i.e., completion dependency, the execution of one job cannot be started until another job is completed successfully) a plurality of application engines (i.e., computing service 258) within an inferencing pipeline (i.e., a pipeline of successive transformations) in accordance with first configuration data (i.e., transformation recipe passed as a parameter to MLS), the executed application engines (i.e., computing service) causing the at least one processor (i.e., the one or more processors) to perform operations that apply a trained, machine-learning process to an input dataset (i.e., model execution results such as predictions or evaluations) on a first inferencing date (i.e., the first prediction date or the first evaluation date) is taught.)
“obtain elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines” (Lee, Col. 7, Lines 62-67 and Col. 8, Lines 1-12, “Supported entity types in one embodiment may include, among others, data sources (e.g., descriptors of locations or objects from which input records for machine learning can be obtained), sets of statistics generated by analyzing the input data, recipes (e.g., descriptors of feature processing transformations to be applied to input data for training models), processing plans (e.g., templates for executing various machine learning tasks), models (which may also be referred to as predictors), parameter sets to be used for recipes and/or models, model execution results such as predictions or evaluations, online access points for models that are to be used on streaming or real-time data, and/or aliases (e.g., pointers to model versions that have been “published” for use as described below). Instances of these entity types may be referred to as machine learning artifacts herein—for example, a specific recipe or a specific model may each be considered an artifact.”; Lee, Col. 17, Lines 35-39, “In one case, termed “completion dependency”, the execution of one job Jp cannot be started until another job Jq is completed successfully (e.g., because the final output of Jq is required as input for Jp).”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Examiner’s note: obtain elements of artifact data (i.e., supported entity types such as data sources, sets of statistics, recipes, processing plans, models, parameter sets, model execution results, etc.) associated with the sequential execution (i.e., one job cannot be started until another job is completed successfully) of the application engines (i.e., the computing service 258), the elements of artifact data (i.e., entity types) comprising at least one input artifact ingested by (i.e., data sources, input records), and at least one output artifact generated by (i.e., predictions or evaluations), corresponding ones of the executed application engines (i.e., the computing service) is taught.)
perform operations that populate a first data record with at least an identifier of the inferencing pipeline, and the elements of artifact data, and that store the first data record within a corresponding portion of the memory (Lee, Col. 11, Lines 30-35 and 60-62, “As mentioned earlier, each job object may indicate one or more operations that are to be performed as a result of the invocation of a programmatic interface 161, and the scheduling of a given job may in some cases depend upon the successful completion of at least a subset of the operations of an earlier-generated job […] Results of some jobs may be stored as MLS artifacts within repository 120 in some embodiments […]”; Examiner’s note: perform operations that populate a first data record with at least an identifier of the inferencing pipeline, and the elements of artifact data (i.e., job performing operations as a result of the invocation of a programmatic interface and the scheduling of a given job may depend upon the successful completion of an earlier-generated job), and that store the first data record within a corresponding portion of the memory (i.e., results of some jobs stored as MLS artifacts within repository) is taught.)
Lee does not explicitly teach:
perform operations that populate a first data record with at least an identifier of the first inferencing date, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date
Karl teaches:
perform operations that populate a first data record with at least an identifier of the first inferencing date, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date (Karl, Paragraph [0014], “metadata records are added to the artifact structure and accumulated over time, creating an audit trail of the processing performed with and on any other information stored within the same artifact […] For example a first chain of records in Figure 2a could include the standard model information generated by the trainer (R3, R5) followed by records related to the training activity such as time it was performed, loss or fitness information […]”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Examiner’s note: perform operations that populate a first data record with at least an identifier of the first inferencing date (i.e., metadata records are accumulated over time and an audit trail is created. Therefore, the first data record with the first inferencing date is populated.), the first data record specifying a configuration of the executed application engines (i.e., the computing service) of the inferencing pipeline (i.e., a pipeline of successive transformations) at the first inferencing date (i.e., a first chain of records including the standard model information teaches the first inferencing date) is taught.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the interactive interfaces for machine learning model evaluations in Lee, and the method for decentralized accessioning for distributed machine learning and other applications as taught in Karl. Lee teaches executing sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data, obtaining elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines, and performing operations that populate a first data record with at least an identifier of the inferencing pipeline, and the elements of artifact data, and that store the first data record within a corresponding portion of the memory. Karl teaches performing operations that populate a first data record with at least an identifier of the first inferencing date, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date. One of ordinary skill would have motivation to combine Lee and Karl to “allow a time-based ordering to be honored when reading out the data” (Karl, Paragraph [0018]).
Regarding Claim 5,
The combination of Lee and Karl teaches:
“The apparatus of claim 1,” (preamble)
“wherein: the first configuration data comprises a plurality of elements, the elements of the first configuration data being associated with corresponding ones of the application engines” (Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Examiner’s note: wherein: the first configuration data comprises a plurality of elements, the elements of the first configuration data (i.e., a pipeline of successive transformations teaches the elements of the first configuration data) being associated with corresponding ones of the application engines (i.e., the computing service) is taught.)
“the at least one processor is further configured to execute the instructions to execute sequentially each of the application engines within the inferencing pipeline based on corresponding ones of element of the first configuration data” (Lee, Col. 17, Lines 36-38, “the execution of one job Jp cannot be started until another job Jq is completed successfully […]”; Lee, Col. 21, Lines 8-14, “the MLS may support recurring scheduling of related jobs. For example, a client may create an artifact such as a model, and may want that same model to be re-trained and/or re-executed for different input data sets (e.g., using the same configuration of resources for each of the training or prediction iterations) at specified points in time.”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”; Examiner’s note: the at least one processor (i.e., the one or more processors) is further configured to execute the instructions to execute sequentially (i.e., one job cannot be started until another job is completed successfully) each of the application engines (i.e., the computing service 258) within the inferencing pipeline (i.e., a pipeline of successive transformations) based on corresponding ones of element of the first configuration data (i.e., the same model being re-trained and/or re-executed for different input data sets (e.g., using the same configuration of resources for each of the training or prediction iterations) at specified points in time) is taught.)
The reasons of obviousness have been noted in the rejection of Claim 1 above and applicable herein.
Regarding Claim 6,
The combination of Lee and Karl teaches:
“The apparatus of claim 1,” (preamble)
“wherein: obtain pipeline data associated with the inferencing pipeline from the memory, the pipeline data comprising a inferencing pipeline script that establishes an execution flow for the sequential execution of the application engines within the inferencing pipeline” (Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 24, Lines 59-64, “A workload distribution strategy and processing plan may be identified for Jk—e.g., the number of processing passes or phases to be used, the degree of parallelism to be used, an iterative convergence criterion to be used for completing Jk (element 954).”; Lee, Col. 17, Lines 36-38, “the execution of one job Jp cannot be started until another job Jq is completed successfully (e.g., because the final output of Jq is required as input for Jp).”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Examiner’s note: wherein: obtain pipeline data associated with the inferencing pipeline from the memory (i.e., a recipe indicating a pipeline of successive transformations), the pipeline data comprising a inferencing pipeline script that establishes an execution flow (i.e., a workload distribution strategy and processing plan identified for Jk) for the sequential execution (i.e., one job cannot be started until another job is completed successfully) of the application engines (i.e., the computing service) within the inferencing pipeline (i.e., a pipeline of successive transformations) is taught.)
“execute the inferencing pipeline script, the executed inferencing pipeline script causing the at least one processor to execute sequentially each of the application engines at a corresponding position within the execution flow” (Lee, Col. 17, Lines 36-38, “the execution of one job Jp cannot be started until another job Jq is completed successfully […]”; Lee, Col. 39, Lines 5-11, “A chunk-level filtering plan 1850 may be generated for the chunked data set 1810 in some embodiments, e.g., based on contents of a filtering descriptor (which may also be referred to as a retrieval descriptor) included in the client's request. The chunk-level filtering plan may indicate, for example, the sequence in which a plurality of in-memory filtering operations 1870”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”; Examiner’s note: execute the inferencing pipeline script (i.e., generating a chunk-level filtering plan), the executed inferencing pipeline script causing the at least one processor (i.e., the one or more processors) to execute sequentially (i.e., one job cannot be started until another job is completed successfully) each of the application engines (i.e., the computing service 258) at a corresponding position within the execution flow (i.e., the chunk-level filtering plan indicating the sequence in which a plurality of in-memory filtering operations) is taught.)
The reasons of obviousness have been noted in the rejection of Claim 1 above and applicable herein.
Regarding Claim 7,
The combination of Lee and Karl teaches:
“The apparatus of claim 1,” (preamble)
“wherein the corresponding portion of the memory comprises a plurality of data records, the plurality of data records comprising the first data record and one or more second data records, each of the second data records establishing an additional configuration of the executed application engines of the inferencing pipeline at a second inferencing date, and the first inferencing date being subsequent to each of the second inferencing dates” (Karl, Paragraph [0014], “metadata records are added to the artifact structure and accumulated over time, creating an audit trail of the processing performed with and on any other information stored within the same artifact […] For example a first chain of records in Figure 2a could include the standard model information generated by the trainer (R3, R5) followed by records related to the training activity such as time it was performed, loss or fitness information […] Similarly, a second chain of records in Figure 2b includes various records including structured (R6, R7), accession (R8) and other data (R9), as well as model data (R10)”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Examiner’s note: wherein the corresponding portion of the memory comprises a plurality of data records, the plurality of data records comprising the first data record (i.e., a first chain of records) and one or more second data records (i.e., a second chain of records), each of the second data records establishing additional configuration of the executed application engines (i.e., the computing service) of the inferencing pipeline (i.e., a pipeline of successive transformations) at a second inferencing date (i.e., a second chain of records including structured, accession, other data and model data teaches a second inferencing date), and the first inferencing date being subsequent to each of the second inferencing dates (i.e., an audit trail of the processing performed teaches that the first inferencing date is subsequent to each of the second inferencing dates) is taught.)
The reasons of obviousness have been noted in the rejection of Claim 1 above and applicable herein.
Regarding Claim 8,
The combination of Lee and Karl teaches:
“The apparatus of claim 7,” (preamble)
“wherein the at least one processor is further configured to execute the instructions to:” (Lee, Col. 10, Lines 55-58, “In some embodiments, a pool of compute servers and/or storage servers may be pre-configured for the MLS, and the resources for a given job may be selected from such a pool.”; Examiner’s note: a pool of compute servers pre-configured for the MLS teaches the at least one processor is further configured to execute the instructions to.)
“receive elements of query data from a computing system via the communications interface” (Karl, Paragraph [0034], “The supervising entity 45 of a processing entity framework 5 includes a function S55 for receiving notifications or receiving responses to polling inquiries (e.g., SQL queries) S45b from one or more database, or storage platforms 15 via a query engine 60, of the existence of new versions of one or more existing artifacts and/or a new artifacts. The supervising entity 45 then extracts the metadata from the artifact(s) and presents the extracted metadata to the query engine S45c”; Examiner’s note: wherein the at least one processor is further configured to execute the instructions to (i.e., the supervising entity 45 of a processing entity framework 5): receive elements of query data from a computing system (i.e., receiving responses to polling inquiries (e.g., SQL queries)) via the communications interface (i.e., via a query engine 60) is taught.)
“based on at least elements of temporal data maintained within the data records, determine that a subset of the data records are consistent with the query data, the subset of the data records comprising the first data record and at least one of the second data records” (Karl, Paragraph [0014], “metadata records are added to the artifact structure and accumulated over time, creating an audit trail of the processing performed with and on any other information stored within the same artifact […]”; Lee, Col. 39, Lines 14-22 and Lines 59-64, “the machine learning model may support parallelized training of models, in which for example respective (and potentially partially overlapping) subsets of an input data set may be used to train a given model in parallel. The duration of one training operation may overlap at least partly with the duration of another in such a scenario, and the input data set may be partitioned for the parallel training sessions using a chunk-level filtering operation […] For example, the in-memory results of the first filtering operation may serve as the input data set for the second filtering operation, the in-memory results of the second filtering operation may serve as the input data set for the third filtering operation, and so on.”; Examiner’s note: based on at least elements of temporal data maintained within the data records (i.e., records accumulated over time and an audit trail teach the elements of temporal data in the data records), determine that a subset of the data records are consistent with the query data (i.e., respective and potentially partially overlapping subsets of an input data set), the subset of the data records comprising the first data record and at least one of the second data records (i.e., results of the first filtering operation serving as the input data set for the second filtering operation and results of the second filtering operation serving as the input data set for the third filtering operation, and so on) is taught.)
“transmit elements of response data that include the subset of the data records to the computing system via the communications interface” (Lee, Col. 65, Lines 65-67, “including the FPT's processed variable(s) as inputs for the model's execution for a prediction/evaluation run […]”; Examiner’s note: transmit elements of response data that include the subset of the data records to the computing system via the communications interface (i.e., transmitting the FPT’s processed variable(s) as inputs for the model’s execution for a prediction/evaluation) is taught.)
The reasons of obviousness have been noted in the rejection of Claim 7 above and applicable herein.
Regarding Claim 9,
The combination of Lee and Karl teaches:
“The apparatus of claim 8,” (preamble)
“wherein: the computing system is configured to execute one or more scripts, the one or more scripts causing the computing system to process portions of the first data record and the at least one of the second data record” (Lee, Col. 39, Lines 5-11, “A chunk-level filtering plan 1850 may be generated for the chunked data set 1810 in some embodiments, e.g., based on contents of a filtering descriptor (which may also be referred to as a retrieval descriptor) included in the client's request. The chunk-level filtering plan may indicate, for example, the sequence in which a plurality of in-memory filtering operations 1870”; Lee, Col. 39, Lines 59-64, “For example, the in-memory results of the first filtering operation may serve as the input data set for the second filtering operation, the in-memory results of the second filtering operation may serve as the input data set for the third filtering operation, and so on.”; Examiner’s note: wherein: the computing system is configured to execute one or more scripts (i.e., generating a chunk-level filtering plan), the one or more scripts causing the computing system to process portions of the first data record and the at least one of the second data record (i.e., the chunk-level filtering plan indicating the sequence in which a plurality of in-memory filtering operations, and results of the first filtering operation serving as the input data set for the second filtering operation and results of the second filtering operation serving as the input data set for the third filtering operation, and so on) is taught.)
“the computing system is further configured to present a graphical representation of at least the processed portion within a digital interface” (Lee, Col. 108, Lines 5-10, “computing device 9000 includes one or more processors 9010 coupled to a system memory 9020 (which may comprise both non-volatile and volatile memory modules) via an input/output (I/O) interface 9030. Computing device 9000 further includes a network interface 9040 coupled to I/O interface 9030.”; Examiner’s note: the computing system (i.e., computing device 9000) is further configured to present a graphical representation of at least the processed portion within a digital interface (i.e., I/O interface 9030 presents a graphical representation of at least the processed portion) is taught.)
The reasons of obviousness have been noted in the rejection of Claim 8 above and applicable herein.
Regarding Claim 10,
The combination of Lee and Karl teaches:
“The apparatus of claim 1,” (preamble)
“wherein the at least one processor is further configured to execute the instructions to:” (Lee, Col. 10, Lines 55-58, “In some embodiments, a pool of compute servers and/or storage servers may be pre-configured for the MLS, and the resources for a given job may be selected from such a pool.”; Examiner’s note: a pool of compute servers pre-configured for the MLS teaches the at least one processor is further configured to execute the instructions to.)
“execute sequentially at least a subset of the application engines within the inferencing pipeline in accordance with second configuration data, the executed application engines causing the at least one processor to perform operations that apply the trained, machine-learning process to an additional input dataset on a second inferencing date, the second inferencing date being subsequent to the first inferencing data” (Lee, Col. 17, Lines 35-39, “In one case, termed “completion dependency”, the execution of one job Jp cannot be started until another job Jq is completed successfully (e.g., because the final output of Jq is required as input for Jp).”; Lee, Col. 28, Lines 44-46, “a text version 1101 of a transformation recipe may be passed as a parameter in a “createRecipe” MLS API call by a client.”; Lee, Col. 7, Lines 59-63 and Col. 8, Lines 5-9, “a number of different types of entities related to machine learning tasks may be generated, modified, read, executed, and/or queried/searched via MLS programmatic interfaces. Supported entity types in one embodiment may include […] model execution results such as predictions or evaluations, online access points for models that are to be used on streaming or real-time data, and/or aliases (e.g., pointers to model versions that have been “published” for use as described below).”; Karl, Paragraph [0014], “metadata records are added to the artifact structure and accumulated over time, creating an audit trail of the processing performed with and on any other information stored within the same artifact […]”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”; Examiner’s note: execute sequentially (i.e., completion dependency, the execution of one job cannot be started until another job is completed successfully) at least a subset of the application engines (i.e., the computing service) within the inferencing pipeline (i.e., a pipeline of successive transformations) in accordance with second configuration data (i.e., second transformation recipe passed as a parameter to MLS), the executed application engines (i.e., the computing service) causing the at least one processor (i.e., the one or more processors) to perform operations that apply the trained, machine-learning process to an additional input dataset (i.e., model execution results such as predictions or evaluations) on a second inferencing date (i.e., the second prediction date or the second evaluation date), the second inferencing date being subsequent to the first inferencing date (i.e., an audit trail of the processing performed teaches that the second inferencing date is subsequent to the first inferencing date) is taught.)
“detect a failure in the sequential execution of at least one of the subset of the application engines based on elements of additional artifact data” (Lee, Col. 7, Lines 46-53, “The term “MLS control plane” may be used herein to refer to a collection of hardware and/or software entities that are responsible for implementing various types of machine learning functionality on behalf of clients of the MLS, and for administrative tasks not necessarily visible to external MLS clients, such as ensuring that an adequate set of resources is provisioned to meet client demands, detecting and recovering from failures, generating bills, and so on.”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Examiner’s note: detecting a failure in the sequential execution of at least one of the subset of the application engines (i.e., the computing service) based on elements of additional artifact data (i.e., MLS control plane referring to a collection of hardware and/or software entities responsible for implementing various types of machine learning functionality can include detecting a failure in the sequential execution of at least one of the subset of the application engines) is taught.)
“based on the detected failure, obtain the first data record from the corresponding portion of the memory, and obtain the first configuration data from the elements of artifact data” (Lee, Col. 7, Lines 46-53, “The term “MLS control plane” may be used herein to refer to a collection of hardware and/or software entities that are responsible for implementing various types of machine learning functionality on behalf of clients of the MLS, and for administrative tasks not necessarily visible to external MLS clients, such as ensuring that an adequate set of resources is provisioned to meet client demands, detecting and recovering from failures, generating bills, and so on.”; Karl, Paragraph [0014], “metadata records are added to the artifact structure and accumulated over time, creating an audit trail of the processing performed with and on any other information stored within the same artifact […] For example a first chain of records in Figure 2a could include the standard model information generated by the trainer (R3, R5) followed by records related to the training activity such as time it was performed, loss or fitness information […]”; Examiner’s note: based on the detected failure (i.e., MLS control plane detected the failure), obtain the first data record from the corresponding portion of the memory (i.e., the first data record retrieved from an audit trail), and obtain the first configuration data from elements of artifact data (i.e., a first chain of records including the standard model information) is taught.)
“perform operations that initiate an additional sequential execution of the plurality of application engines within the inferencing pipeline in accordance with at least a portion of the first configuration data” (Lee, Col. 17, Lines 35-39, “In one case, termed “completion dependency”, the execution of one job Jp cannot be started until another job Jq is completed successfully (e.g., because the final output of Jq is required as input for Jp).”; Lee, Col. 28, Lines 44-46, “a text version 1101 of a transformation recipe may be passed as a parameter in a “createRecipe” MLS API call by a client.”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Examiner’s note: perform operations that initiate an additional sequential execution (i.e., the execution of one job cannot be started until another job is completed successfully) of the plurality of application engines (i.e., the computing service) within the inferencing pipeline (i.e., a pipeline of successive transformations) in accordance with at least a portion of the first configuration data (i.e., transformation recipe passed as a parameter to MLS) is taught.)
The reasons of obviousness have been noted in the rejection of Claim 1 above and applicable herein.
Regarding Claim 13,
Lee teaches:
“A computer-implemented method, comprising: using at least one processor” (Lee, Col. 10, Lines 55-58, “In some embodiments, a pool of compute servers and/or storage servers may be pre-configured for the MLS, and the resources for a given job may be selected from such a pool.”; Examiner’s note: a pool of compute servers pre-configured for the MLS teaches at least one processor.)
“executing sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data, the executed application engines causing the at least one processor to perform operations that apply a trained, machine-learning process to an input dataset on a first inferencing date” (Lee, Col. 17, Lines 35-39, “In one case, termed “completion dependency”, the execution of one job Jp cannot be started until another job Jq is completed successfully (e.g., because the final output of Jq is required as input for Jp).”; Lee, Col. 28, Lines 44-46, “a text version 1101 of a transformation recipe may be passed as a parameter in a “createRecipe” MLS API call by a client.”; Lee, Col. 7, Lines 59-63 and Col. 8, Lines 5-9, “a number of different types of entities related to machine learning tasks may be generated, modified, read, executed, and/or queried/searched via MLS programmatic interfaces. Supported entity types in one embodiment may include […] model execution results such as predictions or evaluations, online access points for models that are to be used on streaming or real-time data, and/or aliases (e.g., pointers to model versions that have been “published” for use as described below).”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”; Examiner’s note: executing sequentially (i.e., completion dependency, the execution of one job cannot be started until another job is completed successfully) a plurality of application engines (i.e., the computing service) within an inferencing pipeline (i.e., a pipeline of successive transformations) in accordance with first configuration data (i.e., transformation recipe passed as a parameter to MLS), the executed application engines (i.e., the computing service) causing the at least one processor (i.e., the one or more processors) to perform operations that apply a trained, machine-learning process to an input dataset (i.e., model execution results such as predictions or evaluations) on a first inferencing date (i.e., the first prediction date or the first evaluation date) is taught.)
“obtaining, using the at least one processor, elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines” (Lee, Col. 7, Lines 62-67 and Col. 8, Lines 1-12, “Supported entity types in one embodiment may include, among others, data sources (e.g., descriptors of locations or objects from which input records for machine learning can be obtained), sets of statistics generated by analyzing the input data, recipes (e.g., descriptors of feature processing transformations to be applied to input data for training models), processing plans (e.g., templates for executing various machine learning tasks), models (which may also be referred to as predictors), parameter sets to be used for recipes and/or models, model execution results such as predictions or evaluations, online access points for models that are to be used on streaming or real-time data, and/or aliases (e.g., pointers to model versions that have been “published” for use as described below). Instances of these entity types may be referred to as machine learning artifacts herein—for example, a specific recipe or a specific model may each be considered an artifact.”; Lee, Col. 17, Lines 35-39, “In one case, termed “completion dependency”, the execution of one job Jp cannot be started until another job Jq is completed successfully (e.g., because the final output of Jq is required as input for Jp).”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”; Examiner’s note: obtaining, using the at least one processor (i.e., the one or more processors), elements of artifact data (i.e., supported entity types such as data sources, sets of statistics, recipes, processing plans, models, parameter sets, model execution results, etc.) associated with the sequential execution (i.e., one job cannot be started until another job is completed successfully) of the application engines (i.e., the computing service), the elements of artifact data comprising at least one input artifact ingested by (i.e., data sources, input records), and at least one output artifact generated by (i.e., predictions or evaluations), corresponding ones of the executed application engines (i.e., the computing service) is taught.)
performing operations, using the at least one processor, that populate a first data record with at least an identifier of the inferencing pipeline, and the elements of artifact data, and that store the first data record within a corresponding portion of a data repository (Lee, Col. 11, Lines 30-35 and 60-62, “As mentioned earlier, each job object may indicate one or more operations that are to be performed as a result of the invocation of a programmatic interface 161, and the scheduling of a given job may in some cases depend upon the successful completion of at least a subset of the operations of an earlier-generated job […] Results of some jobs may be stored as MLS artifacts within repository 120 in some embodiments […]”; Examiner’s note: performing operations, using the at least one processor, that populate a first data record with at least an identifier of the inferencing pipeline, and the elements of artifact data (i.e., job performing operations as a result of the invocation of a programmatic interface and the scheduling of a given job may depend upon the successful completion of an earlier-generated job), and that store the first data record within a corresponding portion of a data repository (i.e., results of some jobs stored as MLS artifacts within repository) is taught.)
Lee does not explicitly teach:
performing operations, using the at least one processor, that populate a first data record with at least an identifier of the first inferencing date, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date
Karl teaches:
performing operations, using the at least one processor, that populate a first data record with at least an identifier of the first inferencing date, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date (Karl, Paragraph [0014], “metadata records are added to the artifact structure and accumulated over time, creating an audit trail of the processing performed with and on any other information stored within the same artifact […] For example a first chain of records in Figure 2a could include the standard model information generated by the trainer (R3, R5) followed by records related to the training activity such as time it was performed, loss or fitness information […]”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”; Examiner’s note: performing operations, using the at least one processor (i.e., the one or more processors), that populate a first data record with at least an identifier of the first inferencing date (i.e., metadata records are accumulated over time and an audit trail is created. Therefore, the first data record with the first inferencing date is populated.), the first data record specifying a configuration of the executed application engines (i.e., the computing service) of the inferencing pipeline (i.e., a pipeline of successive transformations) at the first inferencing date (i.e., a first chain of records including the standard model information teaches the first inferencing date.) is taught.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the interactive interfaces for machine learning model evaluations in Lee, and the method for decentralized accessioning for distributed machine learning and other applications as taught in Karl. Lee teaches executing sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data, obtaining elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines, and performing operations that populate a first data record with at least an identifier of the inferencing pipeline, and the elements of artifact data, and that store the first data record within a corresponding portion of the memory. Karl teaches performing operations that populate a first data record with at least an identifier of the first inferencing date, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date. One of ordinary skill would have motivation to combine Lee and Karl to “allow a time-based ordering to be honored when reading out the data” (Karl, Paragraph [0018]).
Regarding Claim 15,
The combination of Lee and Karl teaches:
“The computer-implemented method of claim 13, further comprising:” (preamble)
“obtaining, using the at least one processor, pipeline data associated with the inferencing pipeline from the memory, the pipeline data comprising a inferencing pipeline script that establishes an execution flow for the sequential execution of the application engines within the inferencing pipeline” (Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 24, Lines 59-64, “A workload distribution strategy and processing plan may be identified for Jk—e.g., the number of processing passes or phases to be used, the degree of parallelism to be used, an iterative convergence criterion to be used for completing Jk (element 954).”; Lee, Col. 17, Lines 36-38, “the execution of one job Jp cannot be started until another job Jq is completed successfully (e.g., because the final output of Jq is required as input for Jp).”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”; Examiner’s note: obtaining, using the at least one processor (i.e., the one or more processors), pipeline data associated with the inferencing pipeline from the memory (i.e., a recipe indicating a pipeline of successive transformations), the pipeline data comprising a inferencing pipeline script that establishes an execution flow (i.e., a workload distribution strategy and processing plan identified for Jk) for the sequential execution (i.e., one job cannot be started until another job is completed successfully) of the application engines (i.e., the computing service) within the inferencing pipeline (i.e., a pipeline of successive transformations) is taught.)
“executing the inferencing pipeline script using the at least one processor, the executed inferencing pipeline script causing the at least one processor to execute sequentially each of the application engines at a corresponding position within the execution flow” (Lee, Col. 17, Lines 36-38, “the execution of one job Jp cannot be started until another job Jq is completed successfully […]”; Lee, Col. 39, Lines 5-11, “A chunk-level filtering plan 1850 may be generated for the chunked data set 1810 in some embodiments, e.g., based on contents of a filtering descriptor (which may also be referred to as a retrieval descriptor) included in the client's request. The chunk-level filtering plan may indicate, for example, the sequence in which a plurality of in-memory filtering operations 1870”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”; Examiner’s note: executing the inferencing pipeline script (i.e., generating a chunk-level filtering plan) using the at least one processor (i.e., the one or more processors), the executed inferencing pipeline script (i.e., generated a chunk-level filtering plan) causing the at least one processor (i.e., the one or more processors) to execute sequentially (i.e., the execution of one job cannot be started until another job is completed successfully) each of the application engines (i.e., the computing service) at a corresponding position within the execution flow (i.e., the chunk-level filtering plan indicating the sequence in which a plurality of in-memory filtering operations) is taught.)
The reasons of obviousness have been noted in the rejection of Claim 13 above and applicable herein.
Regarding Claim 16,
The combination of Lee and Karl teaches:
“The computer-implemented method of claim 13,” (preamble)
“wherein: the corresponding portion of the data repository comprises a plurality of data records, the plurality of data records comprising the first data record and one or more second data records, each of the second data records establishing an additional configuration of the executed application engines of the inferencing pipeline at a second inferencing date, and the first inferencing date being subsequent to each of the second inferencing dates” (Karl, Paragraph [0014], “metadata records are added to the artifact structure and accumulated over time, creating an audit trail of the processing performed with and on any other information stored within the same artifact […] For example a first chain of records in Figure 2a could include the standard model information generated by the trainer (R3, R5) followed by records related to the training activity such as time it was performed, loss or fitness information […] Similarly, a second chain of records in Figure 2b includes various records including structured (R6, R7), accession (R8) and other data (R9), as well as model data (R10)”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Examiner’s note: wherein: the corresponding portion of the data repository comprises a plurality of data records, the plurality of data records comprising the first data record (i.e., a first chain of records) and one or more second data records (i.e., a second chain of records), each of the second data records establishing additional configuration of the executed application engines (i.e., the computing service) of the inferencing pipeline (i.e., a pipeline of successive transformations) at a second inferencing date (i.e., a second chain of records including structured, accession, other data and model data teaches a second inferencing date), and the first inferencing date being subsequent to each of the second inferencing dates (i.e., an audit trail of the processing performed teaches that the first inferencing date is subsequent to each of the second inferencing dates) is taught.)
“receiving, using the at least one processor, elements of query data from a computing system” (Karl, Paragraph [0034], “The supervising entity 45 of a processing entity framework 5 includes a function S55 for receiving notifications or receiving responses to polling inquiries (e.g., SQL queries) S45b from one or more database, or storage platforms 15 via a query engine 60, of the existence of new versions of one or more existing artifacts and/or a new artifacts. The supervising entity 45 then extracts the metadata from the artifact(s) and presents the extracted metadata to the query engine S45c”; Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”; Examiner’s note: receiving, using the at least one processor (i.e., the one or more processors), elements of query data from a computing system (i.e., receiving responses to polling inquiries (e.g., SQL queries)) is taught.)
“based on at least elements of temporal data maintained within the data records, determining, using the at least one processor, that a subset of the data records are consistent with the query data, the subset of the data records comprising the first data record and at least one of the second data records” (Karl, Paragraph [0014], “metadata records are added to the artifact structure and accumulated over time, creating an audit trail of the processing performed with and on any other information stored within the same artifact […]”; Lee, Col. 39, Lines 14-22 and Lines 59-64, “the machine learning model may support parallelized training of models, in which for example respective (and potentially partially overlapping) subsets of an input data set may be used to train a given model in parallel. The duration of one training operation may overlap at least partly with the duration of another in such a scenario, and the input data set may be partitioned for the parallel training sessions using a chunk-level filtering operation […] For example, the in-memory results of the first filtering operation may serve as the input data set for the second filtering operation, the in-memory results of the second filtering operation may serve as the input data set for the third filtering operation, and so on.”; Examiner’s note: based on at least elements of temporal data maintained within the data records (i.e., records accumulated over time and an audit trail teach the elements of temporal data in the data records), determining, using the at least one processor, that a subset of the data records are consistent with the query data (i.e., respective and potentially partially overlapping subsets of an input data set), the subset of the data records comprising the first data record and at least one of the second data records (i.e., results of the first filtering operation serving as the input data set for the second filtering operation and results of the second filtering operation serving as the input data set for the third filtering operation, and so on) is taught.)
“transmitting, using the at least one processor, elements of response data that include the subset of the data records to the computing system” (Lee, Col. 65, Lines 65-67, “including the FPT's processed variable(s) as inputs for the model's execution for a prediction/evaluation run […]”; Examiner’s note: transmitting, using the at least one processor, elements of response data that include the subset of the data records to the computing system (i.e., transmitting the FPT’s processed variable(s) as inputs for the model’s execution for a prediction/evaluation) is taught.)
The reasons of obviousness have been noted in the rejection of Claim 13 above and applicable herein.
Regarding Claim 17,
Claim 17 recites substantially the same limitations as Claim 10, in the form of a computer-implemented method; therefore, it is rejected under the same rationale.
Regarding Claim 20,
Lee teaches:
“A tangible, non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform a method, comprising:” (Lee, Col. 112, Lines 15-17, “A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors […]”)
“executing sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data, the executed application engines causing the at least one processor to perform operations that apply a trained, machine-learning process to an input dataset on a first inferencing date” (Lee, Col. 17, Lines 35-39, “In one case, termed “completion dependency”, the execution of one job Jp cannot be started until another job Jq is completed successfully (e.g., because the final output of Jq is required as input for Jp).”; Lee, Col. 28, Lines 44-46, “a text version 1101 of a transformation recipe may be passed as a parameter in a “createRecipe” MLS API call by a client.”; Lee, Col. 7, Lines 59-63 and Col. 8, Lines 5-9, “a number of different types of entities related to machine learning tasks may be generated, modified, read, executed, and/or queried/searched via MLS programmatic interfaces. Supported entity types in one embodiment may include […] model execution results such as predictions or evaluations, online access points for models that are to be used on streaming or real-time data, and/or aliases (e.g., pointers to model versions that have been “published” for use as described below).”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”; Examiner’s note: executing sequentially (i.e., completion dependency, the execution of one job cannot be started until another job is completed successfully) a plurality of application engines (i.e., the computing service) within an inferencing pipeline (i.e., a pipeline of successive transformations) in accordance with first configuration data (i.e., transformation recipe passed as a parameter to MLS), the executed application engines (i.e., the computing service) causing the at least one processor (i.e., the one or more processors) to perform operations that apply a trained, machine-learning process to an input dataset (i.e., model execution results such as predictions or evaluations) on a first inferencing date (i.e., the first prediction date or the first evaluation date) is taught.)
“obtaining elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines” (Lee, Col. 7, Lines 62-67 and Col. 8, Lines 1-12, “Supported entity types in one embodiment may include, among others, data sources (e.g., descriptors of locations or objects from which input records for machine learning can be obtained), sets of statistics generated by analyzing the input data, recipes (e.g., descriptors of feature processing transformations to be applied to input data for training models), processing plans (e.g., templates for executing various machine learning tasks), models (which may also be referred to as predictors), parameter sets to be used for recipes and/or models, model execution results such as predictions or evaluations, online access points for models that are to be used on streaming or real-time data, and/or aliases (e.g., pointers to model versions that have been “published” for use as described below). Instances of these entity types may be referred to as machine learning artifacts herein—for example, a specific recipe or a specific model may each be considered an artifact.”; Lee, Col. 17, Lines 35-39, “In one case, termed “completion dependency”, the execution of one job Jp cannot be started until another job Jq is completed successfully (e.g., because the final output of Jq is required as input for Jp).”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Examiner’s note: obtaining elements of artifact data (i.e., supported entity types such as data sources, sets of statistics, recipes, processing plans, models, parameter sets, model execution results, etc.) associated with the sequential execution (i.e., one job cannot be started until another job is completed successfully) of the application engines (i.e., the computing service), the elements of artifact data comprising at least one input artifact ingested by (i.e., data sources, input records), and at least one output artifact generated by (i.e., predictions or evaluations), corresponding ones of the executed application engines (i.e., the computing service) is taught.)
performing operations that populate a first data record with at least an identifier of the inferencing pipeline, and the elements of artifact data, and that store the first data record within a corresponding portion of the memory (Lee, Col. 11, Lines 30-35 and 60-62, “As mentioned earlier, each job object may indicate one or more operations that are to be performed as a result of the invocation of a programmatic interface 161, and the scheduling of a given job may in some cases depend upon the successful completion of at least a subset of the operations of an earlier-generated job […] Results of some jobs may be stored as MLS artifacts within repository 120 in some embodiments […]”; Examiner’s note: performing operations that populate a first data record with at least an identifier of the inferencing pipeline, and the elements of artifact data (i.e., job performing operations as a result of the invocation of a programmatic interface and the scheduling of a given job may depend upon the successful completion of an earlier-generated job), and that store the first data record within a corresponding portion of the memory (i.e., results of some jobs stored as MLS artifacts within repository) is taught.)
Lee does not explicitly teach:
performing operations that populate a first data record with at least an identifier of the first inferencing date, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date
Karl teaches:
performing operations that populate a first data record with at least an identifier of the first inferencing date, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date (Karl, Paragraph [0014], “metadata records are added to the artifact structure and accumulated over time, creating an audit trail of the processing performed with and on any other information stored within the same artifact […] For example a first chain of records in Figure 2a could include the standard model information generated by the trainer (R3, R5) followed by records related to the training activity such as time it was performed, loss or fitness information […]”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Examiner’s note: performing operations that populate a first data record with at least an identifier of the first inferencing date (i.e., metadata records are accumulated over time and an audit trail is created. Therefore, the first data record with the first inferencing date is populated.), the first data record specifying a configuration of the executed application engines (i.e., the computing service) of the inferencing pipeline (i.e., a pipeline of successive transformations) at the first inferencing date (i.e., a first chain of records including the standard model information teaches the first inferencing date) is taught.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the interactive interfaces for machine learning model evaluations in Lee, and the method for decentralized accessioning for distributed machine learning and other applications as taught in Karl. Lee teaches executing sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data, obtaining elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines, and performing operations that populate a first data record with at least an identifier of the inferencing pipeline, and the elements of artifact data, and that store the first data record within a corresponding portion of the memory. Karl teaches performing operations that populate a first data record with at least an identifier of the first inferencing date, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date. One of ordinary skill would have motivation to combine Lee and Karl to “allow a time-based ordering to be honored when reading out the data” (Karl, Paragraph [0018]).
Claims 2-4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Karl as applied in claim 1, in view of Bavly et al. (US 20210334693 A1) (hereinafter Bavly).
Regarding Claim 2,
The combination of Lee and Karl teaches:
“The apparatus of claim 1,” (preamble)
the input dataset on the first inferencing date (Karl, Paragraph [0014], “metadata records are added to the artifact structure and accumulated over time, creating an audit trail of the processing performed with and on any other information stored within the same artifact […] For example a first chain of records in Figure 2a could include the standard model information generated by the trainer (R3, R5) followed by records related to the training activity such as time it was performed, loss or fitness information […]”; Examiner’s note: the input dataset on the first inferencing date (i.e., a first chain of records with an audit trail of the processing) is taught.)
The combination of Lee and Karl does not explicitly teach:
wherein the elements of artifact data comprise explainability data that characterizes the application of the trained, machine- learning process to the input dataset, the explainability data being generated by a corresponding one of the executed application engines within the inferencing pipeline
Bavly teaches:
wherein the elements of artifact data comprise explainability data that characterizes the application of the trained, machine- learning process to the input dataset, the explainability data being generated by a corresponding one of the executed application engines within the inferencing pipeline (Bavly, Paragraph [0038], “the model training results may provide an explainability description of the trained model. The explainability description may describe explainability properties of the outputs of the models that the expert should be able to understand and use for selecting one or more models. The explainability of the output of a trained model may be evaluated based on various properties and or parameters, such as feature transparency on the model output, interpretability, feature inclemency on a prediction, etc.”; Examiner’s note: wherein the elements of artifact data comprise explainability data that characterizes the application of the trained, machine-learning process to the input dataset (i.e., the model training results providing an explainability description of the trained model that describes explainability properties of the outputs of the models), the explainability data being generated by a corresponding one of the executed application engines within the inferencing pipeline (i.e., explainability properties provided by the model training results) is taught.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the interactive interfaces for machine learning model evaluations in Lee, the method for decentralized accessioning for distributed machine learning and other applications in Karl, and the automated generation of explainable machine learning as taught in Bavly. Lee teaches executing sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data, obtaining elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines, and performing operations that populate a first data record with at least an identifier of the inferencing pipeline, and the elements of artifact data, and that store the first data record within a corresponding portion of the memory. Karl teaches performing operations that populate a first data record with at least an identifier of the first inferencing date, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date, and the input dataset on the first inferencing date. Bavly teaches that the elements of artifact data comprise explainability data that characterizes the application of the trained, machine- learning process to the input dataset. One of ordinary skill would have motivation to combine Lee, Karl and Bavly to “effectively improve decision-making reliability based on explainable machine learning solutions” (Bavly, Paragraph [0054]).
Regarding Claim 3,
The combination of Lee, Karl and Bavly teaches:
“The apparatus of claim 2,” (preamble)
“wherein the explainability data comprises at least one of (i) a first value characterizing an importance of one or more features to an output of the trained machine-learning process or (ii) a second value characterizing a performance of the trained machine-learning process” (Bavly, Paragraphs [0036] and [0037], “application server 120 may execute the auto-XAI module 212 to generate a respective accuracy and provide a respective explainability for each trained model. The model training results may describe model performance of each model in comparison to respective original model […] The model performance may be represented by an accuracy indicative of an accuracy value or performance score (e.g., F1 score) and explainability (also referred to herein as explainability properties).”; Examiner’s note: wherein the explainability data comprises at least one of (i) a first value characterizing an importance of one or more features to an output of the trained machine-learning process (i.e., a respective accuracy for each trained model) or (ii) a second value characterizing a performance of the trained machine-learning process (i.e., performance score (e.g., F1 score) for each trained model) is taught.)
The reasons of obviousness have been noted in the rejection of Claim 2 above and applicable herein.
Regarding Claim 4,
The combination of Lee, Karl and Bavly teaches:
“The apparatus of claim 2,” (preamble)
“wherein the elements of artifact data further comprise elements of performance data characterizing an operation of each of the executed application engines within the inferencing pipeline” (Lee, Col. 27, Lines 7-10, “the MLS control plane may comprise a set of monitoring agents that collect performance and other metrics from the resources used for the various phases of machine learning operations (element 1054).”; Lee, Col. 21, Lines 22-23 and Lines 29-35, “A respective job may be placed in the MLS job queue for each recurring training or execution iteration […] Such programmatic interfaces may be referred to as “pipelining APIs” in some embodiments. In addition to the artifact types shown in FIG. 6, pipeline artifacts may be stored in the MLS artifact repository in some embodiments, with each instance of a pipeline artifact representing a named set of recurring operations requested via such APIs.”; Examiner’s note: wherein the elements of artifact data further comprise elements of performance data (i.e., performance from the resources used for the various phases of machine learning operations) characterizing an operation of each of the executed application engines within the inferencing pipeline (i.e., recurring training or execution iteration, pipelining APIs and each instance of a pipeline artifact representing a set of recurring operations via such APIs) is taught.)
The reasons of obviousness have been noted in the rejection of Claim 2 above and applicable herein.
Regarding Claim 14,
The combination of Lee, Karl and Bavly teaches:
“The computer-implemented method of claim 13,” (preamble)
“wherein the elements of artifact data comprise explainability data that characterizes the application of the trained, machine-learning process to the input dataset on the first inferencing date and performance data characterizing an operation of each of the executed application engines within the inferencing pipeline, the explainability and performance data being generated by a corresponding one of the executed application engines within the inferencing pipeline” (Bavly, Paragraph [0038], “the model training results may provide an explainability description of the trained model. The explainability description may describe explainability properties of the outputs of the models that the expert should be able to understand and use for selecting one or more models. The explainability of the output of a trained model may be evaluated based on various properties and or parameters, such as feature transparency on the model output, interpretability, feature inclemency on a prediction, etc.”; Karl, Paragraph [0014], “metadata records are added to the artifact structure and accumulated over time, creating an audit trail of the processing performed with and on any other information stored within the same artifact […] For example a first chain of records in Figure 2a could include the standard model information generated by the trainer (R3, R5) followed by records related to the training activity such as time it was performed, loss or fitness information […]”; Lee, Col. 27, Lines 7-10, “the MLS control plane may comprise a set of monitoring agents that collect performance and other metrics from the resources used for the various phases of machine learning operations (element 1054).”; Lee, Col. 21, Lines 22-23 and Lines 29-35, “A respective job may be placed in the MLS job queue for each recurring training or execution iteration […] Such programmatic interfaces may be referred to as “pipelining APIs” in some embodiments. In addition to the artifact types shown in FIG. 6, pipeline artifacts may be stored in the MLS artifact repository in some embodiments, with each instance of a pipeline artifact representing a named set of recurring operations requested via such APIs.”; Examiner’s note: wherein the elements of artifact data comprise explainability data that characterizes the application of the trained, machine-learning process to the input dataset (i.e., the model training results providing an explainability description of the trained model that describes explainability properties of the outputs of the models) on the first inferencing date (i.e., a first chain of records with an audit trail of the processing teaches the first inferencing date) and performance data (i.e., performance from the resources used for the various phases of machine learning operations) characterizing an operation of each of the executed application engines within the inferencing pipeline (i.e., recurring training or execution iteration, pipelining APIs and each instance of a pipeline artifact representing a set of recurring operations via such APIs), the explainability and performance data being generated by a corresponding one of the executed application engines within the inferencing pipeline (i.e., explainability properties provided by the model training results and performance used for the machine learning operations) is taught.)
The reasons of obviousness have been noted in the rejection of Claim 13 above and applicable herein.
Claims 11-12 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Karl as applied in claim 1, in view of Watters et al. (US 20210152835 A1) (hereinafter Watters).
Regarding Claim 11,
The combination of Lee and Karl teaches:
“The apparatus of claim 1,” (preamble)
the executed application engines further cause the at least one processor to perform operations that apply the trained, machine-learning process to the input dataset, and based on the application of the trained, machine-learning process to the input dataset, generate output data indicating a predicted likelihood of an occurrence of a target event during a future temporal interval (Lee, Col. 28, Lines 44-46, “a text version 1101 of a transformation recipe may be passed as a parameter in a “createRecipe” MLS API call by a client.”; Lee, Col. 7, Lines 59-63 and Col. 8, Lines 5-9, “a number of different types of entities related to machine learning tasks may be generated, modified, read, executed, and/or queried/searched via MLS programmatic interfaces. Supported entity types in one embodiment may include […] model execution results such as predictions or evaluations, online access points for models that are to be used on streaming or real-time data, and/or aliases (e.g., pointers to model versions that have been “published” for use as described below).”; Lee, Col. 102, Lines 38-41 and Lines 47-51, “the MLS (or the client on whose behalf the model is trained and evaluated) may have initially selected the default names “0” and “1” for the classes into which observation records are to be placed […] such as “Won't buy” (replacing the label “0”) and “Will-buy” (replacing the label “1”) indicating that the model is classifying shoppers based on predictions about the likelihood that the shoppers will make a purchase (the “1” class) or will not make a purchase (the “0” class)”; Lee, Col. 82, “pruning iterations 5334 may be scheduled at regular intervals, e.g., once every X seconds, regardless of the rate at which observation records are received or examined.”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”; Examiner’s note: the executed application engines (i.e., the computing service) further causing the at least one processor (i.e., the one or more processors) to perform operations that apply the trained, machine-learning process to the input dataset (i.e., model execution results such as predictions or evaluations), and based on the application of the trained, machine-learning process to the input dataset (i.e., based on predictions or evaluations), generate output data indicating a predicted likelihood of an occurrence of a target event (i.e., the MLS (or the client on whose behalf the model is trained and evaluated) selecting 0 or 1 based on predictions about the likelihood that the shoppers will make a purchase) during a future temporal interval (i.e., pruning iterations scheduled at regular intervals) is taught.)
“the at least one processor is further configured to execute the instructions to:” (Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”)
“obtain labelling data associated with the input dataset, the labelling data specifying the occurrence, or a non- occurrence, of the target event during the future temporal interval” (Lee, Col. 102, Lines 38-41 and Lines 47-51, “the MLS (or the client on whose behalf the model is trained and evaluated) may have initially selected the default names “0” and “1” for the classes into which observation records are to be placed […] such as “Won't buy” (replacing the label “0”) and “Will-buy” (replacing the label “1”) indicating that the model is classifying shoppers based on predictions about the likelihood that the shoppers will make a purchase (the “1” class) or will not make a purchase (the “0” class)”; Lee, Col. 82, “pruning iterations 5334 may be scheduled at regular intervals, e.g., once every X seconds, regardless of the rate at which observation records are received or examined.”; Examiner’s note: obtain labelling data associated with the input dataset (i.e., the label “0” or “1”), the labelling data specifying the occurrence, or a non-occurrence, of the target event (i.e., “1” indicating the occurrence, “0” indicating the non-occurrence) during the future temporal interval (i.e., pruning iterations scheduled at regular intervals) is taught.)
generate elements of evaluation data that characterize a predictive performance of the trained, machine-learning process (Lee, Col. 20, Lines 52-56, “model evaluations 610 (measures of the accuracy of a model, computed when the predictions of the model can be compared to known values of dependent variables) may also be stored as artifacts by the MLS in some embodiments.”; Examiner’s note: generate elements of evaluation data (i.e., model evaluations 610) that characterizes a predictive performance of the trained, machine-learning process (i.e., measuring the accuracy of a model evaluates a predictive performance of the trained, machine-learning process) is taught.)
“transmit the elements of evaluation data to a computing system” (Lee, Col. 65, Lines 65-67, “including the FPT's processed variable(s) as inputs for the model's execution for a prediction/evaluation run […]”; Examiner’s note: transmit elements of evaluation data to a computing system (i.e., transmitting the FPT’s processed variable(s) as inputs for the model’s execution for a prediction/evaluation run) is taught.)
The combination of Lee and Karl does not explicitly teach:
“wherein: the trained, machine-learning process comprises a trained, forward-in-time machine-learning process“
Watters teaches:
“wherein: the trained, machine-learning process comprises a trained, forward-in-time machine-learning process“ (Watters, Paragraph [0006], “The dynamics predictor learns to roll the representations forward in time by computing their interactions and dynamics, producing a predicted physical trajectory of arbitrary length.”; Examiner’s note: wherein: the trained, machine-learning process comprises a trained, forward-in-time machine-learning process (i.e., the dynamic predictor rolling the representations forward in time) is taught.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the interactive interfaces for machine learning model evaluations in Lee, the method for decentralized accessioning for distributed machine learning and other applications in Karl, and the making object-level predictions of the future state of a physical system as taught in Watters. Lee teaches executing sequentially a plurality of application engines within an inferencing pipeline in accordance with first configuration data, obtaining elements of artifact data associated with the sequential execution of the application engines, the elements of artifact data comprising at least one input artifact ingested by, and at least one output artifact generated by, corresponding ones of the executed application engines, and performing operations that populate a first data record with at least an identifier of the inferencing pipeline, and the elements of artifact data, and that store the first data record within a corresponding portion of the memory. Karl teaches performing operations that populate a first data record with at least an identifier of the first inferencing date, the first data record specifying a configuration of the executed application engines of the inferencing pipeline at the first inferencing date, and the input dataset on the first inferencing date. Watters teaches the trained, forward-in-time machine-learning process. One of ordinary skill would have motivation to combine Lee, Karl and Watters to “take as input a sequence of state codes, for example from the visual encoder, and predict a “final”, i.e., predicted, state code for a next unobserved frame” (Watters, Paragraph [0007]).
Regarding Claim 12,
The combination of Lee, Karl and Watters teaches:
“The apparatus of claim 11,” (preamble)
“wherein the at least one processor is further configured to execute the instructions to” (Lee, Col. 10, Lines 55-58, “In some embodiments, a pool of compute servers and/or storage servers may be pre-configured for the MLS, and the resources for a given job may be selected from such a pool.”; Examiner’s note: a pool of compute servers pre-configured for the MLS teaches the at least one processor is further configured to execute the instructions to.)
“execute sequentially at least a subset of the application engines within a target-generation pipeline in accordance with second configuration data, the subset of the executed application engines causing the at least one processor to perform operations that generate the labelling data associated with the input dataset” (Lee, Col. 17, Lines 35-39, “In one case, termed “completion dependency”, the execution of one job Jp cannot be started until another job Jq is completed successfully (e.g., because the final output of Jq is required as input for Jp).”; Lee, Col. 28, Lines 44-46, “a text version 1101 of a transformation recipe may be passed as a parameter in a “createRecipe” MLS API call by a client.”; Lee, Col. 7, Lines 59-63 and Col. 8, Lines 5-9, “a number of different types of entities related to machine learning tasks may be generated, modified, read, executed, and/or queried/searched via MLS programmatic interfaces. Supported entity types in one embodiment may include […] model execution results such as predictions or evaluations, online access points for models that are to be used on streaming or real-time data, and/or aliases (e.g., pointers to model versions that have been “published” for use as described below).”; Lee, Col. 102, Lines 38-41 and Lines 47-51, “the MLS (or the client on whose behalf the model is trained and evaluated) may have initially selected the default names “0” and “1” for the classes into which observation records are to be placed […] such as “Won't buy” (replacing the label “0”) and “Will-buy” (replacing the label “1”) indicating that the model is classifying shoppers based on predictions about the likelihood that the shoppers will make a purchase (the “1” class) or will not make a purchase (the “0” class)”; Lee, Col. 14, Lines 30-32, “one or more servers 262 from computing service 258 to execute the job's computational operations.”; Lee, Col. 28, Lines 35-37, “a pipeline of successive transformations to be performed starting with a given input data set may be indicated within a single recipe.”; Lee, Col. 112, Lines 50-51, “the instructions when executed at the one or more processors”; Examiner’s note: execute sequentially (i.e., completion dependency, the execution of one job cannot be started until another job is completed successfully and the MLS with “0” and “1” selected for the classes) at least a subset of the application engines (i.e., the computing service) within a target-generation pipeline (i.e., a pipeline of successive transformations) in accordance with second configuration data (i.e., second transformation recipe passed as a parameter to MLS), the subset of the executed application engines (i.e., the computing service) causing the at least one processor (i.e., the one or more processors) to perform operations that generate the labelling data associated with the input dataset (i.e., model execution results such as predictions or evaluations including the label “0” and “1”) is taught.)
Regarding Claim 18,
Claim 18 recites substantially the same limitations as Claim 11, in the form of a computer-implemented method; therefore, it is rejected under the same rationale.
Regarding Claim 19,
Claim 19 recites substantially the same limitations as Claim 12, in the form of a computer-implemented method; therefore, it is rejected under the same rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Baylor et al. teaches the TensorFlow-based production-scale machine learning platform. Lundberg et al. teaches the unified approach to interpreting model predictions.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YONG D RHO whose telephone number is (571)270-0194. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at 5712705871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YONG DOO RHO/Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights