DETAILED ACTION
This is the initial Office action based on the application filed on August 15, 2024.
Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Internet Communications
Without a written authorization for Internet communications by the Applicant in place, the USPTO cannot communicate with the Applicant via email and will not respond via email to any Internet correspondence which contains information subject to the confidentiality requirement as set forth in 35 U.S.C. § 122, such as claimed subject matter in an interview agenda or proposed claim amendments for an Examiner’s Amendment.
Therefore, in the interest of facilitating compact prosecution, the Examiner kindly asks the Applicant to authorize Internet communications with the USPTO by using Form PTO/SB/439 (available at https://www.uspto.gov/patents/apply/forms). The form may be submitted via the USPTO patent electronic filing system (Patent Center) using the document description “Internet Communications Authorized” to facilitate processing. The written authorization for Internet communications must be submitted on a separate paper to be entitled to acceptance in accordance with 37 CFR § 1.4(c). The separate paper will facilitate processing and avoid confusion. The written authorization for Internet communications may not be submitted via an email. See MPEP § 502.03(II).
Claim Interpretation
During patent examination, the pending claims must be “given their broadest reasonable interpretation consistent with the specification.” See MPEP § 2111. Under a broadest reasonable interpretation (BRI), words of the claim must be given their plain meaning, unless such meaning is inconsistent with the specification. The plain meaning of a term means the ordinary and customary meaning given to the term by those of ordinary skill in the art at the relevant time. The ordinary and customary meaning of a term may be evidenced by a variety of sources, including the words of the claims themselves, the specification, the drawings, and the prior art. See MPEP § 2111.01(I).
Applicant is entitled to be their own lexicographer and may rebut the presumption that claim terms are to be given their ordinary and customary meaning by clearly setting forth a definition of the term that is different from its ordinary and customary meaning(s) in the specification at the relevant time. Where an explicit definition is provided by the Applicant for a term, that definition will control interpretation of the term as it is used in the claim. See MPEP § 2111.01(IV)(A). Any such lexicographic definition for a term will be expressly noted by the Examiner in the prior art rejections of the claims.
Claim Mapping
For clarity of the prosecution history record, the Examiner has provided annotations in the prior art rejections of the claims to aid the Applicant in understanding the Examiner’s interpretations of the claimed invention and the prior art, such as emphasizing notable and relevant portions of the prior art citations, using item-to-item matching to the prior art citations, pairing exact claim language to particular language used in the prior art citations, and/or clearly explaining the Examiner’s interpretation as to how a prior art citation maps to the claim language, especially when there is no one-to-one matching of terms. Furthermore, the annotations are provided in the prior art rejections of the claims at the Examiner’s discretion where the Examiner deemed to be appropriate and necessary.
Specification
The disclosure is objected to because of the following informalities:
On page 1, paragraph [0001] of the specification, “U.S. Patent Application No. 18/048,287 filed on October 20, 2022” should read -- U.S. Patent Application No. 18/048,287 (issued as U.S. Patent No. 12,086,054) filed on October 20, 2022 --.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
On page 1, paragraph [0001] of the specification, “U.S. Patent Application No. 17/805,388 filed on June 3, 2022” should read -- U.S. Patent Application No. 17/805,388 (issued as U.S. Patent No. 12,086,053) filed on June 3, 2022 --.
Appropriate correction is required.
Claim Objections
Claims 11 and 15-20 are objected to because of the following informalities:
Claim 11 recites “a validation system.” It should read -- the validation system --.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 11 recites “the testing module.” It should read -- the test module --.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claims 15-20 recite “[n]on-transitory computer-readable media.” It should read -- A non-transitory computer-readable medium --.
Appropriate correction is required.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement.
Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 3.73(b).
Claims 1-3, 8, 9, and 15-17 are rejected on the ground of non-statutory obviousness-type double patenting as being unpatentable over Claims 1, 6, 7, 16, and 17 of U.S. Patent No. 12,086,053 (hereinafter “‘053”). Although the conflicting claims are not identical, they are not patentably distinct from each other because Claims 1-3, 8, 9, and 15-17 of the instant application define an obvious variation of the invention claimed in ‘053.
Examiner respectfully submits the relevant portions of MPEP §§ 804(II)(B), 804(II)(B)(3), and 804(II)(B)(4) with emphasis added for purposes of convenience in discussion and illustration:
MPEP § 804(II)(B) Nonstatutory Double Patenting
A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985).
MPEP § 804(II)(B)(3) Obviousness Analysis
Any nonstatutory double patenting rejection made under the obviousness analysis should make clear:
(A) The differences between the inventions defined by the conflicting claims – a claim in the patent compared to a claim in the application; and
(B) The reasons why a person of ordinary skill in the art would conclude that the invention defined in the claim at issue would have been an obvious variation of the invention defined in a claim in the patent.
MPEP § 804(II)(B)(4) One-Way Test for Distinctness
If the patent term filing date of an application under examination is the same or later than that of a reference application or patent, only a one-way determination of distinctness is needed in resolving the issue of double patenting, i.e., whether the invention claimed in the application would have been anticipated by, or an obvious variation of, the invention claimed in the reference application or patent. See, e.g., In re Berg, 140 F.3d 1428, 1435, 46 USPQ2d 1226, 1231-32 (Fed. Cir. 1998). The court in Berg applied a one-way test where an applicant filed two separate applications even though all claims could have been filed in a single application, because the applicant’s action could have resulted in an improper timewise extension of rights if one patent expired later than the other. If a claimed invention in the application would have been obvious over a claimed invention in the patent, there would be an unjustified timewise extension of the patent and a nonstatutory double patenting rejection is proper. According to the Berg court, improperly extending the patent term “is precisely the result that the doctrine of obviousness-type double patenting was created to prevent.” Id.
Similarly, even if the application under examination has the earlier patent term filing date, only a one-way determination of distinctness is needed to support a double patenting rejection in the absence of a finding: (A) that “the PTO is solely responsible for any delays” in prosecution of that application (In re Hubbell, 709 F.3d 1140, 1150, 106 USPQ2d 1032, 1039 (Fed. Cir. 2013)); and (B) that the applicant could not have filed the conflicting claims in a single (i.e., the earlier-filed) application (In re Kaplan, 789 F.2d 1574, 229 USPQ 678 (Fed. Cir. 1986)).
It is noted that the instant application is a later-filed continuation of a continuation of ‘053. It is also noted that both the instant application and ‘053 were filed by the same inventive entity and by a common assignee/owner. Claims 1, 6, 7, 16, and 17 of ‘053 recite all the limitations of Claims 1-3, 8, 9, and 15-17 of the instant application, while also recite further limitations, and thus anticipate the claims of the instant application. The claims of the instant application therefore are not patentably distinct from the earlier patent claims and as such are unpatentable for obviousness-type double patenting. A later claim is not patentably distinct from an earlier claim if the later claim is anticipated by the earlier claim.
Claim 17 of ‘053 as shown in table 1 recites all the limitations of Claim 1 of the instant application, while also recite further limitations, and as such anticipates Claim 1 of the instant application. The further limitations recited in Claim 17 of ‘053 are boldfaced for the Applicant’s convenience. Claims 1, 6, 7, and 16 of ‘053 are not shown with Claims 2, 3, 8, 9, and 15-17 of the instant application for the purpose of brevity.
TABLE 1. Claim 17 of ‘053 v. Claim 1 of the Instant Application
U.S. Patent No. 12,086,053
Instant Application No. 18/806,516
[AltContent: textbox (17.)]A validation computing system comprising processing circuitry and a storage device, wherein the processing circuitry has access to the storage device and is configured to:
[AltContent: textbox (1.)]A computing system comprising processing circuitry and a storage device, wherein the processing circuitry has access to the storage device and is configured to:
receive, from a development system, a request to perform a test on a model configured to execute on the development system, wherein to receive the request, the processing circuitry is further configured to receive requests from a plurality of development systems;
receive, from a development system, a request to validate a model configured to execute on the development system;
output, to each of the plurality of development systems and in response to each respective request, a test instruction;
interact with each of the plurality of development systems to enable each of the plurality of development systems to execute the test instruction output to each respective development system and thereby perform a validation of the model based on the test instruction output to each respective development system;
interact with the development system to perform tests to validate the model;
enable the development system to generate test response data generated in response to the tests performed to validate the model;
receive, from each respective development system, test response data; and
receive, from the development system, the test response data; and
evaluate the test response data.
evaluate the test response data.
Claim Rejections - 35 U.S.C. § 112
The following is a quotation of 35 U.S.C. § 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-7 and 15-20 are rejected under 35 U.S.C. § 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1 and 15 recite the limitation “enable the development system to generate test response data generated” at line 6. The claims are rendered vague and indefinite because of the awkward claim language used. In the interest of compact prosecution, the Examiner subsequently interprets this limitation as reading “enable the development system to generate test response data” for the purpose of further examination.
Claims 2-7 depend on Claim 1. Therefore, Claims 2-7 suffer the same deficiency as Claim 1.
Claims 16-20 depend on Claim 15. Therefore, Claims 16-20 suffer the same deficiency as Claim 15.
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Claim Interpretation: It is essential that the broadest reasonable interpretation (BRI) of a claim as a whole be established prior to examining the claim for eligibility. Under the BRI, the limitations of Claim 1 are presumed to have their plain meaning consistent with the specification as it would be interpreted by one of ordinary skill in the art. See MPEP § 2111.
The BRI of Claim 1 is a computing system for receiving, from a development system, a request to validate a model configured to execute on the development system; interacting with the development system to perform tests to validate the model; enabling the development system to generate test response data generated in response to the tests performed to validate the model; receiving, from the development system, the test response data; and evaluating the test response data.
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory category. See MPEP § 2106.03. Claim 1 is directed to a computing system, which is a machine, and falls within one of the statutory categories of invention. (Step 1: YES).
Step 2A, Prong One: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP § 2106.04(II), a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim.
Claim 1 recites the limitations:
(a) […] validate the model;
(b) […] generate test response data generated in response to the tests performed to validate the model; and
(c) evaluate the test response data.
These recited steps, under the BRI, cover performance of the steps in the human mind alone or with the aid of pen and paper. That is, other than reciting:
(1) [a] computing system comprising processing circuitry and a storage device, wherein the processing circuitry has access to the storage device and is configured to.
Nothing in the claim precludes the steps from practically being performed in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper. For example, the limitation (a) in the context of the claim encompasses a human evaluating a model in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper to validate the model. And the limitation (b) in the context of the claim encompasses a human evaluating tests in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper to generate test response data. And the limitation (c) in the context of the claim encompasses a human evaluating the test response data in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper. See MPEP § 2106.04(a)(2)(III).
If a claim limitation, under its BRI, covers a practical performance in the human mind alone or with the aid of pen and paper but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. (Step 2A, Prong One: YES).
Step 2A, Prong Two: This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the judicial exception or whether the claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the judicial exception into a practical application. See MPEP § 2106.04(d).
This judicial exception is not integrated into a practical application. In particular, the claim recites the additional element:
(1) [a] computing system comprising processing circuitry and a storage device, wherein the processing circuitry has access to the storage device and is configured to.
The additional element (1) is recited at a high level of generality such that it amounts to no more than mere instructions to apply the judicial exception using generic computer components. The processing circuitry and storage device are used as a tool to perform the various steps of the claim. See MPEP § 2106.05(f).
Also, the claim recites the additional elements:
(2) interact with the development system to perform tests […]; and
(3) enable the development system […].
The additional elements (2) and (3) fail to meaningfully limit the claim because they do not require any particular application of the judicial exception and are, at best, the equivalent of merely adding the words “apply it” (or an equivalent) to the judicial exception. See MPEP § 2106.05(f). The additional elements recite only the ideas of interacting with a development system and enabling the development system without details on how they are accomplished. The claim omits any details as to how interacting with the development system and enabling the development system solve a technical problem, and instead recite only the idea of a solution or outcome. Therefore, the additional elements attempt to cover any solution to the identified problems of interacting with the development system and enabling the development system with no restriction on how they are accomplished and no description of the mechanism for accomplishing them, and do not integrate the judicial exception into a practical application because this type of recitation is equivalent to the words “apply it.”
Also, the claim recites the additional elements:
(4) receive, from a development system, a request to validate a model configured to execute on the development system; and
(5) receive, from the development system, the test response data.
The additional elements (4) and (5) are mere data gathering recited at a high level of generality and thus, are an insignificant extra-solution activity. See MPEP § 2106.05(g). Furthermore, all uses of the judicial exception require such data gathering, and, as such, the additional elements do not impose any meaningful limits on the claim. The additional elements amount to necessary data gathering. See MPEP § 2106.05(g).
Accordingly, even when viewed in combination, the additional elements do not integrate the recited judicial exception into a practical application because they do not impose any meaningful limits on practicing the judicial exception. (Step 2A, Prong Two: NO). The claim is directed to an abstract idea. (Step 2A: YES).
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amounts to significantly more than the recited judicial exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP § 2106.05.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as a combination do not amount to significantly more than the judicial exception. As discussed above with respect to integration of the judicial exception into a practical application, the claim recites the additional element:
(1) [a] computing system comprising processing circuitry and a storage device, wherein the processing circuitry has access to the storage device and is configured to.
The additional element (1) amounts to no more than mere instructions to apply the judicial exception using generic computer components. The analysis under Step 2A, Prong Two is carried through to Step 2B. The use of a computer or other machinery in its ordinary capacity does not integrate a judicial exception into a practical application or provide significantly more.
Also, the claim recites the additional elements:
(2) interact with the development system to perform tests […]; and
(3) enable the development system […].
The additional elements (2) and (3) do not require any particular application of the judicial exception and are, at best, the equivalent of merely adding the words “apply it” (or an equivalent) to the judicial exception. The analysis under Step 2A, Prong Two is carried through to Step 2B. Therefore, the additional elements attempt to cover any solution to the identified problems of interacting with the development system and enabling the development system with no restriction on how they are accomplished and no description of the mechanism for accomplishing them, and do not provide significantly more because this type of recitation is equivalent to the words “apply it.”
Also, the claim recites the additional elements:
(4) receive, from a development system, a request to validate a model configured to execute on the development system; and
(5) receive, from the development system, the test response data.
The additional elements (4) and (5) simply append well-understood, routine, and conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception and thus, are not indicative of an inventive concept. MPEP § 2106.05(d)(II) expressly states that the courts have recognized the computer function of receiving or transmitting data over a network, e.g., using the Internet to gather data as a well‐understood, routine, and conventional computer function when it is claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activities. Thus, a person of ordinary skill in the art would readily comprehend that it is well-understood, routine, and conventional in the computing art to receive a request and test response data. Therefore, the limitations remain insignificant extra-solution activities even upon reconsideration and do not amount to significantly more.
Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the additional elements as a combination adds nothing that is not already present when looking at the additional elements taken individually. Even when considered in combination, the additional elements represent mere instructions to apply a judicial exception using generic computer components, only the idea of a solution or outcome, and insignificant extra-solution activities, and therefore do not provide an inventive concept. (Step 2B: NO). The claim is not patent eligible.
Claims 2-7 are dependent on Claim 1, but do not add any feature or subject matter that would solve the judicial exception deficiencies of Claim 1.
Claims 2-7 are as follows:
Claim 2 recites the limitation(s):
(a) wherein to evaluate the test response data, the processing circuitry is further configured to:
(b) determine that the model cannot be validated.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 3 recites the limitation(s):
(a) enable the development system to modify the model; and
(b) after the development system modifies the model, evaluate the modified model.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 4 recites the limitation(s):
(a) output a request for model telemonitoring information over a network to the development system; and
(b) receive, over the network and from the development system and responsive to outputting the request for model telemonitoring information, responsive model telemonitoring data.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 5 recites the limitation(s):
(a) wherein the responsive model telemonitoring data is a first set of model telemonitoring data, and wherein the processing circuitry is further configured to:
(b) receive a second set of model telemonitoring data from the development system without outputting a specific request for the second set of model telemonitoring data to the development system.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 6 recites the limitation(s):
(a) wherein to interact with the development system, the processing circuitry is further configured to:
(b) determine, based on the request, an appropriate test type to use for the model;
(c) select, from a repository of test scripts, a test script having the appropriate test type to use for the model; and
(d) interact with the development system based on the test script.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 7 recites the limitation(s):
(a) wherein to interact with the development system, the processing circuitry is further configured to:
(b) generate, based on development activity from a validator, a new test script of the appropriate test type to use for the model.
Claims 2, 3, 6, and 7 recite further mental steps which can be practically performed in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper and thus, fail to make the claim any less abstract under Step 2A, Prong One (see MPEP § 2106.04(a)(2)(III)).
Claims 3 and 6 recite further additional elements that do not integrate the judicial exception into a practical application of the judicial exception because they do not require any particular application of the judicial exception and are, at best, the equivalent of merely adding the words “apply it” (or an equivalent) to the judicial exception under Step 2A, Prong Two (see MPEP § 2106.05(f)) and thus, are also not significantly more than the abstract idea under Step 2B.
Claims 4 and 5 recite further additional elements that do not integrate the judicial exception into a practical application of the judicial exception because they are mere data gathering/transmitting/outputting recited at a high level of generality and thus, are insignificant extra-solution activities under Step 2A, Prong Two (see MPEP § 2106.05(g)) and thus, are also not significantly more than the abstract idea under Step 2B.
Thus, Claims 2-7 do not add any steps or additional elements, when considered both individually and as a combination, that would convert Claim 1 into patent-eligible subject matter.
Therefore, Claims 1-7 are not drawn to patent-eligible subject matter as they are directed to an abstract idea without significantly more.
<<>> • × • <<>> • × • <<>> • × • <<>> • + • <<>> • × • <<>> • × • <<>> • × • <<>>
Claim Interpretation: It is essential that the broadest reasonable interpretation (BRI) of a claim as a whole be established prior to examining the claim for eligibility. Under the BRI, the limitations of Claim 8 are presumed to have their plain meaning consistent with the specification as it would be interpreted by one of ordinary skill in the art. See MPEP § 2111.
The BRI of Claim 8 is a computing system for integrating a test module into a model under development at the computing system; outputting, by the test module integrated into the model and over a network to a validation system, a request for validation services; receiving, by the test module and from the validation system, signals in response to the request for validation services; executing, based on the signals received from the validation system, a plurality of test instructions to validate the model; outputting, over the network, test response data generated in response to executing the plurality of test instructions; and enabling the validation system to evaluate the test response data.
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory category. See MPEP § 2106.03. Claim 8 is directed to a computing system, which is a machine, and falls within one of the statutory categories of invention. (Step 1: YES).
Step 2A, Prong One: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP § 2106.04(II), a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim.
Claim 8 recites the limitations:
(a) […] validate the model; and
(b) […] evaluate the test response data.
These recited steps, under the BRI, cover performance of the steps in the human mind alone or with the aid of pen and paper. That is, other than reciting:
(1) [a] computing system comprising processing circuitry and a storage device, wherein the processing circuitry has access to the storage device and is configured to.
Nothing in the claim precludes the steps from practically being performed in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper. For example, the limitation (a) in the context of the claim encompasses a human evaluating a model in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper to validate the model. And the limitation (b) in the context of the claim encompasses a human evaluating tests in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper to generate test response data. And the limitation (c) in the context of the claim encompasses a human evaluating the test response data in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper. See MPEP § 2106.04(a)(2)(III).
If a claim limitation, under its BRI, covers a practical performance in the human mind alone or with the aid of pen and paper but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. (Step 2A, Prong One: YES).
Step 2A, Prong Two: This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the judicial exception or whether the claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the judicial exception into a practical application. See MPEP § 2106.04(d).
This judicial exception is not integrated into a practical application. In particular, the claim recites the additional element:
(1) [a] computing system comprising processing circuitry and a storage device, wherein the processing circuitry has access to the storage device and is configured to.
The additional element (1) is recited at a high level of generality such that it amounts to no more than mere instructions to apply the judicial exception using generic computer components. The processing circuitry and storage device are used as a tool to perform the various steps of the claim. See MPEP § 2106.05(f).
Also, the claim recites the additional elements:
(2) integrate a test module into a model under development at the computing system;
(3) execute, based on the signals received from the validation system, a plurality of test instructions to validate the model; and
(4) enable the validation system […].
The additional elements (2) to (4) fail to meaningfully limit the claim because they do not require any particular application of the judicial exception and are, at best, the equivalent of merely adding the words “apply it” (or an equivalent) to the judicial exception. See MPEP § 2106.05(f). The additional elements recite only the ideas of integrating a test module, executing test instructions, and enabling a validation system without details on how they are accomplished. The claim omits any details as to how integrating the test module, executing the test instructions, and enabling the validation system solve a technical problem, and instead recite only the idea of a solution or outcome. Therefore, the additional elements attempt to cover any solution to the identified problems of integrating a test module, executing test instructions, and enabling a validation system with no restriction on how they are accomplished and no description of the mechanism for accomplishing them, and do not integrate the judicial exception into a practical application because this type of recitation is equivalent to the words “apply it.”
Also, the claim recites the additional elements:
(5) output, by the test module integrated into the model and over a network to a validation system, a request for validation services;
(6) receive, by the test module and from the validation system, signals in response to the request for validation services; and
(7) output, over the network, test response data generated in response to executing the plurality of test instructions.
The additional elements (5) to (7) are mere data gathering/outputting recited at a high level of generality and thus, are an insignificant extra-solution activity. See MPEP § 2106.05(g). Furthermore, all uses of the judicial exception require such data gathering/outputting, and, as such, the additional elements do not impose any meaningful limits on the claim. The additional elements amount to necessary data gathering/outputting. See MPEP § 2106.05(g).
Accordingly, even when viewed in combination, the additional elements do not integrate the recited judicial exception into a practical application because they do not impose any meaningful limits on practicing the judicial exception. (Step 2A, Prong Two: NO). The claim is directed to an abstract idea. (Step 2A: YES).
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amounts to significantly more than the recited judicial exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP § 2106.05.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as a combination do not amount to significantly more than the judicial exception. As discussed above with respect to integration of the judicial exception into a practical application, the claim recites the additional element:
(1) [a] computing system comprising processing circuitry and a storage device, wherein the processing circuitry has access to the storage device and is configured to.
The additional element (1) amounts to no more than mere instructions to apply the judicial exception using generic computer components. The analysis under Step 2A, Prong Two is carried through to Step 2B. The use of a computer or other machinery in its ordinary capacity does not integrate a judicial exception into a practical application or provide significantly more.
Also, the claim recites the additional elements:
(2) integrate a test module into a model under development at the computing system;
(3) execute, based on the signals received from the validation system, a plurality of test instructions to validate the model; and
(4) enable the validation system […].
The additional elements (2) to (4) do not require any particular application of the judicial exception and are, at best, the equivalent of merely adding the words “apply it” (or an equivalent) to the judicial exception. The analysis under Step 2A, Prong Two is carried through to Step 2B. Therefore, the additional elements attempt to cover any solution to the identified problems of integrating the test module, executing the test instructions, and enabling the validation system with no restriction on how they are accomplished and no description of the mechanism for accomplishing them, and do not provide significantly more because this type of recitation is equivalent to the words “apply it.”
Also, the claim recites the additional elements:
(5) output, by the test module integrated into the model and over a network to a validation system, a request for validation services;
(6) receive, by the test module and from the validation system, signals in response to the request for validation services; and
(7) output, over the network, test response data generated in response to executing the plurality of test instructions.
The additional elements (5) to (7) simply append well-understood, routine, and conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception and thus, are not indicative of an inventive concept. MPEP § 2106.05(d)(II) expressly states that the courts have recognized the computer function of receiving or transmitting data over a network, e.g., using the Internet to gather data as a well‐understood, routine, and conventional computer function when it is claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activities. Thus, a person of ordinary skill in the art would readily comprehend that it is well-understood, routine, and conventional in the computing art to receive signals and output a request/test response data. Therefore, the limitations remain insignificant extra-solution activities even upon reconsideration and do not amount to significantly more.
Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the additional elements as a combination adds nothing that is not already present when looking at the additional elements taken individually. Even when considered in combination, the additional elements represent mere instructions to apply a judicial exception using generic computer components, only the idea of a solution or outcome, and insignificant extra-solution activities, and therefore do not provide an inventive concept. (Step 2B: NO). The claim is not patent eligible.
Claims 9-14 are dependent on Claim 8, but do not add any feature or subject matter that would solve the judicial exception deficiencies of Claim 8.
Claims 9-14 are as follows:
Claim 9 recites the limitation(s):
(a) wherein to enable the validation system to evaluate the test response data, the processing circuitry is further configured to:
(b) enable the validation system to determine that the model cannot be validated.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 10 recites the limitation(s):
(a) modify the model based on development activity to generate a modified model; and
(b) enable the validation system to evaluate the modified model.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 11 recites the limitation(s):
(a) receive a request for model telemonitoring information over the network from a validation system; and
(b) output, over the network to the validation system by the testing module and during development of the model, responsive model telemonitoring data.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 12 recites the limitation(s):
(a) wherein the responsive model telemonitoring data is a first set of model telemonitoring data, and wherein the processing circuitry is further configured to:
(b) output a second set of model telemonitoring data over the network to the validation system having received a specific request for the second set of model telemonitoring data from the validation system.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 13 recites the limitation(s):
(a) wherein to receive signals in response to the request for validation services, the processing circuitry is further configured to enable the validation system to:
(b) determine, based on the request, an appropriate test type to use for the model;
(c) select, from a repository of test scripts, a test script having the appropriate test type to use for the model; and
(d) interact with the computing system based on the test script.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 14 recites the limitation(s):
(a) wherein the processing circuitry is further configured to enable the validation system to:
(b) generate, based on development activity from a validator, a new test script of the appropriate test type to use for the model.
Claims 9, 10, 13, and 14 recite further mental steps which can be practically performed in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper and thus, fail to make the claim any less abstract under Step 2A, Prong One (see MPEP § 2106.04(a)(2)(III)).
Claims 9, 10, and 13 recite further additional elements that do not integrate the judicial exception into a practical application of the judicial exception because they do not require any particular application of the judicial exception and are, at best, the equivalent of merely adding the words “apply it” (or an equivalent) to the judicial exception under Step 2A, Prong Two (see MPEP § 2106.05(f)) and thus, are also not significantly more than the abstract idea under Step 2B.
Claims 11 and 12 recite further additional elements that do not integrate the judicial exception into a practical application of the judicial exception because they are mere data gathering/transmitting/outputting recited at a high level of generality and thus, are insignificant extra-solution activities under Step 2A, Prong Two (see MPEP § 2106.05(g)) and thus, are also not significantly more than the abstract idea under Step 2B.
Thus, Claims 9-14 do not add any steps or additional elements, when considered both individually and as a combination, that would convert Claim 8 into patent-eligible subject matter.
Therefore, Claims 8-14 are not drawn to patent-eligible subject matter as they are directed to an abstract idea without significantly more.
<<>> • × • <<>> • × • <<>> • × • <<>> • + • <<>> • × • <<>> • × • <<>> • × • <<>>
Claim Interpretation: It is essential that the broadest reasonable interpretation (BRI) of a claim as a whole be established prior to examining the claim for eligibility. Under the BRI, the limitations of Claim 15 are presumed to have their plain meaning consistent with the specification as it would be interpreted by one of ordinary skill in the art. See MPEP § 2111.
The BRI of Claim 15 is a non-transitory computer-readable media for receiving, from a development system, a request to validate a model configured to execute on the development system; interacting with the development system to perform tests to validate the model; enabling the development system to generate test response data generated in response to the tests performed to validate the model; receiving, from the development system, the test response data; and evaluating the test response data.
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory category. See MPEP § 2106.03. Claim 15 is directed to a non-transitory computer-readable media, which is an article of manufacture, and falls within one of the statutory categories of invention. (Step 1: YES).
Step 2A, Prong One: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP § 2106.04(II), a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim.
Claim 15 recites the limitations:
(a) […] validate the model;
(b) […] generate test response data generated in response to the tests performed to validate the model; and
(c) evaluate the test response data.
These recited steps, under the BRI, cover performance of the steps in the human mind alone or with the aid of pen and paper. That is, other than reciting:
(1) [n]on-transitory computer-readable media comprising instructions that, when executed, cause processing circuitry of a computing system to.
Nothing in the claim precludes the steps from practically being performed in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper. For example, the limitation (a) in the context of the claim encompasses a human evaluating a model in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper to validate the model. And the limitation (b) in the context of the claim encompasses a human evaluating tests in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper to generate test response data. And the limitation (c) in the context of the claim encompasses a human evaluating the test response data in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper. See MPEP § 2106.04(a)(2)(III).
If a claim limitation, under its BRI, covers a practical performance in the human mind alone or with the aid of pen and paper but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. (Step 2A, Prong One: YES).
Step 2A, Prong Two: This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the judicial exception or whether the claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the judicial exception into a practical application. See MPEP § 2106.04(d).
This judicial exception is not integrated into a practical application. In particular, the claim recites the additional element:
(1) [n]on-transitory computer-readable media comprising instructions that, when executed, cause processing circuitry of a computing system to.
The additional element (1) is recited at a high level of generality such that it amounts to no more than mere instructions to apply the judicial exception using generic computer components. The non-transitory computer-readable medium and processing circuitry are used as a tool to perform the various steps of the claim. See MPEP § 2106.05(f).
Also, the claim recites the additional elements:
(2) interact with the development system to perform tests […]; and
(3) enable the development system […].
The additional elements (2) and (3) fail to meaningfully limit the claim because they do not require any particular application of the judicial exception and are, at best, the equivalent of merely adding the words “apply it” (or an equivalent) to the judicial exception. See MPEP § 2106.05(f). The additional elements recite only the ideas of interacting with a development system and enabling the development system without details on how they are accomplished. The claim omits any details as to how interacting with the development system and enabling the development system solve a technical problem, and instead recite only the idea of a solution or outcome. Therefore, the additional elements attempt to cover any solution to the identified problems of interacting with the development system and enabling the development system with no restriction on how they are accomplished and no description of the mechanism for accomplishing them, and do not integrate the judicial exception into a practical application because this type of recitation is equivalent to the words “apply it.”
Also, the claim recites the additional elements:
(4) receive, from a development system, a request to validate a model configured to execute on the development system; and
(5) receive, from the development system, the test response data.
The additional elements (4) and (5) are mere data gathering recited at a high level of generality and thus, are an insignificant extra-solution activity. See MPEP § 2106.05(g). Furthermore, all uses of the judicial exception require such data gathering, and, as such, the additional elements do not impose any meaningful limits on the claim. The additional elements amount to necessary data gathering. See MPEP § 2106.05(g).
Accordingly, even when viewed in combination, the additional elements do not integrate the recited judicial exception into a practical application because they do not impose any meaningful limits on practicing the judicial exception. (Step 2A, Prong Two: NO). The claim is directed to an abstract idea. (Step 2A: YES).
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amounts to significantly more than the recited judicial exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP § 2106.05.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as a combination do not amount to significantly more than the judicial exception. As discussed above with respect to integration of the judicial exception into a practical application, the claim recites the additional element:
(1) [n]on-transitory computer-readable media comprising instructions that, when executed, cause processing circuitry of a computing system to.
The additional element (1) amounts to no more than mere instructions to apply the judicial exception using generic computer components. The analysis under Step 2A, Prong Two is carried through to Step 2B. The use of a computer or other machinery in its ordinary capacity does not integrate a judicial exception into a practical application or provide significantly more.
Also, the claim recites the additional elements:
(2) interact with the development system to perform tests […]; and
(3) enable the development system […].
The additional elements (2) and (3) do not require any particular application of the judicial exception and are, at best, the equivalent of merely adding the words “apply it” (or an equivalent) to the judicial exception. The analysis under Step 2A, Prong Two is carried through to Step 2B. Therefore, the additional elements attempt to cover any solution to the identified problems of interacting with the development system and enabling the development system with no restriction on how they are accomplished and no description of the mechanism for accomplishing them, and do not provide significantly more because this type of recitation is equivalent to the words “apply it.”
Also, the claim recites the additional elements:
(4) receive, from a development system, a request to validate a model configured to execute on the development system; and
(5) receive, from the development system, the test response data.
The additional elements (4) and (5) simply append well-understood, routine, and conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception and thus, are not indicative of an inventive concept. MPEP § 2106.05(d)(II) expressly states that the courts have recognized the computer function of receiving or transmitting data over a network, e.g., using the Internet to gather data as a well‐understood, routine, and conventional computer function when it is claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activities. Thus, a person of ordinary skill in the art would readily comprehend that it is well-understood, routine, and conventional in the computing art to receive a request and test response data. Therefore, the limitations remain insignificant extra-solution activities even upon reconsideration and do not amount to significantly more.
Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the additional elements as a combination adds nothing that is not already present when looking at the additional elements taken individually. Even when considered in combination, the additional elements represent mere instructions to apply a judicial exception using generic computer components, only the idea of a solution or outcome, and insignificant extra-solution activities, and therefore do not provide an inventive concept. (Step 2B: NO). The claim is not patent eligible.
Claims 16-20 are dependent on Claim 15, but do not add any feature or subject matter that would solve the judicial exception deficiencies of Claim 15.
Claims 16-20 are as follows:
Claim 16 recites the limitation(s):
(a) wherein the instructions that cause the processing circuitry to evaluate the test response data further include instructions that, when executed, further cause the processing circuitry to:
(b) determine that the model cannot be validated.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 17 recites the limitation(s):
(a) enable the development system to modify the model; and
(b) after the development system modifies the model, evaluate the modified model.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 18 recites the limitation(s):
(a) output a request for model telemonitoring information over a network to the development system; and
(b) receive, over the network and from the development system and responsive to outputting the request for model telemonitoring information, responsive model telemonitoring data.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 19 recites the limitation(s):
(a) wherein the responsive model telemonitoring data is a first set of model telemonitoring data, and wherein the instructions include instructions that, when executed, further cause the processing circuitry to:
(b) receive a second set of model telemonitoring data from the development system without outputting a specific request for the second set of model telemonitoring data to the development system.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 20 recites the limitation(s):
(a) determine, based on the request, an appropriate test type to use for the model;
(b) select, from a repository of test scripts, a test script having the appropriate test type to use for the model; and
(c) interact with the development system based on the test script.
Claims 16, 17, and 20 recite further mental steps which can be practically performed in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper and thus, fail to make the claim any less abstract under Step 2A, Prong One (see MPEP § 2106.04(a)(2)(III)).
Claim 17 recites further additional elements that do not integrate the judicial exception into a practical application of the judicial exception because they do not require any particular application of the judicial exception and are, at best, the equivalent of merely adding the words “apply it” (or an equivalent) to the judicial exception under Step 2A, Prong Two (see MPEP § 2106.05(f)) and thus, are also not significantly more than the abstract idea under Step 2B.
Claims 18-20 recite further additional elements that do not integrate the judicial exception into a practical application of the judicial exception because they are mere data gathering/transmitting/outputting recited at a high level of generality and thus, are insignificant extra-solution activities under Step 2A, Prong Two (see MPEP § 2106.05(g)) and thus, are also not significantly more than the abstract idea under Step 2B.
Claim 19 recites further additional elements that do not integrate the judicial exception into a practical application of the judicial exception because they merely indicate a field of use or technological environment in which the judicial exception is performed and thus, fail to add an inventive concept to the claims under Step 2A, Prong Two (see MPEP § 2106.05(h)) and thus, are also not significantly more than the abstract idea under Step 2B.
Thus, Claims 16-20 do not add any steps or additional elements, when considered both individually and as a combination, that would convert Claim 15 into patent-eligible subject matter.
Therefore, Claims 15-20 are not drawn to patent-eligible subject matter as they are directed to an abstract idea without significantly more.
Claim Rejections - 35 U.S.C. § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 8, and 15 are rejected under 35 U.S.C. § 102(a)(2) as being anticipated by US 11,314,620 (hereinafter “Lin”).
As per Claim 1, Lin discloses:
A computing system comprising processing circuitry and a storage device (col. 16 lines 5-8, “A system comprising: one or more processors; and memory storing instructions […].”), wherein the processing circuitry has access to the storage device and is configured to:
receive, from a development system, a request to validate a model configured to execute on the development system (col. 9 lines 20-23, “[…] process 500 may represent the steps taken by one or more devices as shown in FIG. 4 when integrating model development control systems and model validation platforms [execute on the development system].” and lines 30-35, “At step 502, process 500 receives (e.g., using control circuitry of one or more components of system 400 (FIG. 4)) a user request to perform a model validation assessment. For example, the system may receive (e.g., via a user interface 100 (FIG. 1)) a user request to perform a model validation assessment using a model validation platform [receive, from a development system, a request to validate a model].”);
interact with the development system to perform tests to validate the model (col. 4 lines 18-22, “[…] the first model may comprise a series of steps that the model validation platform iterates through to test the validating of any inputted model. The series of steps may include one or more tasks (e.g., specific operations, functions, etc.) applied while testing an inputted model [perform tests to validate the model] (emphasis added).”; col. 9 lines 43-48, “At step 504, process 500 retrieves (e.g., using control circuitry of one or more components of system 400 (FIG. 4)) a first model for a model validation platform. For example, the system may retrieve a first model for a model validation platform [interact with the development system to perform tests to validate the model], wherein the first model defines a first hierarchy of tasks for validating inputted models.”);
enable the development system to generate test response data in response to the tests performed to validate the model (col. 10 lines 41-53, “At step 510, process 500 generates (e.g., using control circuitry of one or more components of system 400 (FIG. 4)) the model validation assessment based on the first model and the second model [enable the development system to generate test response data]. For example, the system may generate the model validation assessment based on the first model and the second model, wherein the model validation assessment indicates a performance level of the second model. For example, the system may receive (e.g., via user interface 100 (FIG. 1)), a user selection of the model validation assessment [in response to the tests performed to validate the model]. Each model validation assessment may contain native data and also may link to other models. For example, the model validation assessment may be represented by a non-linear or linear data model of nodes and edges.”);
receive, from the development system, the test response data (col. 10 lines 41-53, “At step 510, process 500 generates (e.g., using control circuitry of one or more components of system 400 (FIG. 4)) the model validation assessment based on the first model and the second model. For example, the system may generate the model validation assessment based on the first model and the second model, wherein the model validation assessment indicates a performance level of the second model [receive, from the development system, the test response data]. For example, the system may receive (e.g., via user interface 100 (FIG. 1)), a user selection of the model validation assessment. Each model validation assessment may contain native data and also may link to other models. For example, the model validation assessment may be represented by a non-linear or linear data model of nodes and edges.”); and
evaluate the test response data (col. 11 lines 15-17, “At step 514, process 500 generates (e.g., using control circuitry of one or more components of system 400 (FIG. 4)) for display native data and assessment data.”; col. 12 lines 3-8, “[…] the system may allow a user to update and/or edit the assessment data [evaluate the test response data]. For example, the system may receive a user edit to the assessment data and then store the edited assessment data. The system may then generate for display the edited assessment data subsequently.”).
As per Claim 8, Lin discloses:
A computing system comprising processing circuitry and a storage device (col. 16 lines 5-8, “A system comprising: one or more processors; and memory storing instructions […].”), wherein the processing circuitry has access to the storage device and is configured to:
integrate a test module into a model under development at the computing system (col. 4 lines 35-37, “Through this structure, the model validation platform may receive different types of models and provide validations therefor [integrate a test module into a model under development at the computing system].”);
output, by the test module integrated into the model and over a network to a validation system, a request for validation services (col. 9 lines 20-23, “[…] process 500 may represent the steps taken by one or more devices as shown in FIG. 4 when integrating model development control systems and model validation platforms.” and lines 30-35, “At step 502, process 500 receives (e.g., using control circuitry of one or more components of system 400 (FIG. 4)) a user request to perform a model validation assessment. For example, the system may receive (e.g., via a user interface 100 (FIG. 1)) a user request to perform a model validation assessment using a model validation platform [output, by the test module integrated into the model and over a network to a validation system, a request for validation services].”);
receive, by the test module and from the validation system, signals in response to the request for validation services (col. 9 lines 43-48, “At step 504, process 500 retrieves (e.g., using control circuitry of one or more components of system 400 (FIG. 4)) a first model for a model validation platform. For example, the system may retrieve a first model for a model validation platform, wherein the first model defines a first hierarchy of tasks for validating inputted models [receive, by the test module and from the validation system, signals in response to the request for validation services].”);
execute, based on the signals received from the validation system, a plurality of test instructions to validate the model (col. 4 lines 18-22, “[…] the first model may comprise a series of steps that the model validation platform iterates through to test the validating of any inputted model. The series of steps may include one or more tasks (e.g., specific operations, functions, etc.) applied while testing an inputted model [execute, based on the signals received from the validation system, a plurality of test instructions to validate the model] (emphasis added).”; col. 9 lines 43-48, “At step 504, process 500 retrieves (e.g., using control circuitry of one or more components of system 400 (FIG. 4)) a first model for a model validation platform. For example, the system may retrieve a first model for a model validation platform, wherein the first model defines a first hierarchy of tasks for validating inputted models.”);
output, over the network, test response data generated in response to executing the plurality of test instructions (col. 10 lines 41-53, “At step 510, process 500 generates (e.g., using control circuitry of one or more components of system 400 (FIG. 4)) the model validation assessment based on the first model and the second model. For example, the system may generate the model validation assessment based on the first model and the second model, wherein the model validation assessment indicates a performance level of the second model [output, over the network, test response data generated in response to executing the plurality of test instructions]. For example, the system may receive (e.g., via user interface 100 (FIG. 1)), a user selection of the model validation assessment. Each model validation assessment may contain native data and also may link to other models. For example, the model validation assessment may be represented by a non-linear or linear data model of nodes and edges.”); and
enable the validation system to evaluate the test response data (col. 11 lines 15-17, “At step 514, process 500 generates (e.g., using control circuitry of one or more components of system 400 (FIG. 4)) for display native data and assessment data.”; col. 12 lines 3-8, “[…] the system may allow a user to update and/or edit the assessment data [enable the validation system to evaluate the test response data]. For example, the system may receive a user edit to the assessment data and then store the edited assessment data. The system may then generate for display the edited assessment data subsequently.”).
Claim 15 is a non-transitory computer-readable media claim corresponding to the computing system claim hereinabove (Claim 1). Therefore, Claim 15 is rejected for the same reason set forth in the rejection of Claim 1.
Claim Rejections - 35 U.S.C. § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 9, and 16 are rejected under 35 U.S.C. § 103 as being unpatentable over Lin in view of US 2004/0078180 (hereinafter “Allen”).
EXAMINER’S REMARKS
In order for a reference to be proper for use in an obviousness rejection under 35 U.S.C. § 103, the reference must be analogous art to the claimed invention. In re Bigio, 381 F.3d 1320, 1325, 72 USPQ2d 1209, 1212 (Fed. Cir. 2004). A reference is analogous art to the claimed invention if: (1) the reference is from the same field of endeavor as the claimed invention (even if it addresses a different problem); or (2) the reference is reasonably pertinent to the problem faced by the inventor (even if it is not in the same field of endeavor as the claimed invention). See MPEP § 2141.01(a)(I).
The claimed invention is generally directed to evaluating models, including machine learning and/or statistical models (specification, paragraph [0002]). As for the “same field of endeavor” test, Lin is generally directed to integrating model development control systems and model validation platforms (specification, col. 1 lines 8 and 9). And Allen is generally directed to automatically decomposing a dynamic system model into separate submodels for ultimate execution on diverse target platforms (Abstract). Thus, Lin and Allen are both analogous art to the claimed invention (even if they address different problems).
As per Claim 2, the rejection of Claim 1 is incorporated; and Lin does not explicitly disclose:
determine that the model cannot be validated.
However, Allen discloses:
determine that the model cannot be validated (paragraph [0033], “The decomposition process (step 440 of FIG. 4, shown in greater detail in FIG. 5) begins with a validation of the decorated system model 100 to ensure that it can be decomposed properly into system submodels (step 510). If the decorated system model fails the validation, the decomposition process halts.”).
As pointed out hereinabove, Lin and Allen are both analogous art to the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Allen into the teaching of Lin to include “determine that the model cannot be validated.” The modification would be obvious because one of ordinary skill in the art would be motivated to allow a user to update and/or edit model assessment data if a model cannot be validated.
Claim 9 is a computing system claim corresponding to the computing system claim hereinabove (Claim 2). Therefore, Claim 9 is rejected for the same reason set forth in the rejection of Claim 2.
Claim 16 is a non-transitory computer-readable media claim corresponding to the computing system claim hereinabove (Claim 2). Therefore, Claim 16 is rejected for the same reason set forth in the rejection of Claim 2.
Claims 3, 10, and 17 are rejected under 35 U.S.C. § 103 as being unpatentable over Lin in view of Allen as applied to Claims 2, 9, and 16 above, and further in view of US 2020/0117430 (hereinafter “Shanahin”).
EXAMINER’S REMARKS
In order for a reference to be proper for use in an obviousness rejection under 35 U.S.C. § 103, the reference must be analogous art to the claimed invention. In re Bigio, 381 F.3d 1320, 1325, 72 USPQ2d 1209, 1212 (Fed. Cir. 2004). A reference is analogous art to the claimed invention if: (1) the reference is from the same field of endeavor as the claimed invention (even if it addresses a different problem); or (2) the reference is reasonably pertinent to the problem faced by the inventor (even if it is not in the same field of endeavor as the claimed invention). See MPEP § 2141.01(a)(I).
The claimed invention is generally directed to evaluating models, including machine learning and/or statistical models (specification, paragraph [0002]). As for the “same field of endeavor” test, Shanahin is generally directed to model configuration (specification, paragraph [0001]). Thus, Shanahin is an analogous art to the claimed invention (even if it addresses a different problem).
As per Claim 3, the rejection of Claim 2 is incorporated; and the combination of Lin and Allen does not explicitly disclose:
enable the development system to modify the model; and
after the development system modifies the model, evaluate the modified model.
However, Shanahin discloses:
enable the development system to modify the model (paragraph [0034], “Starting at the point where a user has provided configuration inputs via a graphical user interface to configure the model as presented on a display of a user device 106 by the product configurator UI 108, and submitted those options, the product configurator UI 108 sends the changes (e.g., updated options, etc.) in operation 201 send a partial model (e.g., a partial data set representing the changes to the model, such as the model diff) to the modeling platform 118.”); and
after the development system modifies the model, evaluate the modified model (paragraph [0034], “The modeling platform 118 integrates the changed options into the configuration model and uses the segmenting engine 128 to segment the configuration model into a segmented data model, assemble an appropriate set of chunks from the segmented data model into a set of partial structured data, in this case, a partial configuration engine model, to send to the configuration engine 126 for evaluation in operation 202 send data for evaluation.”).
As pointed out hereinabove, Shanahin is an analogous art to the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Shanahin into the combined teachings of Lin and Allen to include “enable the development system to modify the model; and after the development system modifies the model, evaluate the modified model.” The modification would be obvious because one of ordinary skill in the art would be motivated to receive configuration input configuring a configurable model (Shanahin, paragraph [0005]).
Claim 10 is a computing system claim corresponding to the computing system claim hereinabove (Claim 3). Therefore, Claim 10 is rejected for the same reason set forth in the rejection of Claim 3.
Claim 17 is a non-transitory computer-readable media claim corresponding to the computing system claim hereinabove (Claim 3). Therefore, Claim 17 is rejected for the same reason set forth in the rejection of Claim 3.
Claims 4, 11, and 18 are rejected under 35 U.S.C. § 103 as being unpatentable over Lin in view of US 2019/0050505 (hereinafter “Nendorf”).
EXAMINER’S REMARKS
In order for a reference to be proper for use in an obviousness rejection under 35 U.S.C. § 103, the reference must be analogous art to the claimed invention. In re Bigio, 381 F.3d 1320, 1325, 72 USPQ2d 1209, 1212 (Fed. Cir. 2004). A reference is analogous art to the claimed invention if: (1) the reference is from the same field of endeavor as the claimed invention (even if it addresses a different problem); or (2) the reference is reasonably pertinent to the problem faced by the inventor (even if it is not in the same field of endeavor as the claimed invention). See MPEP § 2141.01(a)(I).
The claimed invention is generally directed to evaluating models, including machine learning and/or statistical models (specification, paragraph [0002]). As for the “same field of endeavor” test, Lin is generally directed to integrating model development control systems and model validation platforms (specification, col. 1 lines 8 and 9). And Nendorf is generally directed to processing of multi-platform models and management of execution of these model (specification, paragraph [0001]). Thus, Lin and Nendorf are both analogous art to the claimed invention (even if they address different problems).
As per Claim 4, the rejection of Claim 1 is incorporated; and Lin does not explicitly disclose:
output a request for model telemonitoring information over a network to the development system; and
receive, over the network and from the development system and responsive to outputting the request for model telemonitoring information, responsive model telemonitoring data.
However, Nendorf discloses:
output a request for model telemonitoring information over a network to the development system (paragraph [0060], “[…] model data monitoring and analysis engine 700 may be configured to periodically transmit requests for data to multi-model execution module 711. The periodic requests may be automatically transmitted by model data monitoring and analysis engine 700, and may be sent every few seconds, every minute, every hour, daily, weekly, and/or the like.”); and
receive, over the network and from the development system and responsive to outputting the request for model telemonitoring information, responsive model telemonitoring data (paragraph [0060], “In response, multi-model execution module 711 may transmit any new data received from internal models (i.e., since a last request for data was received from model data monitoring and analysis engine 700) to model data monitoring and analysis engine 700.”).
As pointed out hereinabove, Nendorf is an analogous art to the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Nendorf into the teaching of Lin to include “output a request for model telemonitoring information over a network to the development system; and receive, over the network and from the development system and responsive to outputting the request for model telemonitoring information, responsive model telemonitoring data.” The modification would be obvious because one of ordinary skill in the art would be motivated to monitor data generated within a multi-platform model processing and execution management engine (Nendorf, paragraph [0059]).
As per Claim 11, the rejection of Claim 8 is incorporated; and Lin does not explicitly disclose:
receive a request for model telemonitoring information over the network from a validation system; and
output, over the network to the validation system by the testing module and during development of the model, responsive model telemonitoring data.
However, Nendorf discloses:
receive a request for model telemonitoring information over the network from a validation system (paragraph [0060], “[…] model data monitoring and analysis engine 700 may be configured to periodically transmit requests for data to multi-model execution module 711. The periodic requests may be automatically transmitted by model data monitoring and analysis engine 700, and may be sent every few seconds, every minute, every hour, daily, weekly, and/or the like.”); and
output, over the network to the validation system by the testing module and during development of the model, responsive model telemonitoring data (paragraph [0060], “In response, multi-model execution module 711 may transmit any new data received from internal models (i.e., since a last request for data was received from model data monitoring and analysis engine 700) to model data monitoring and analysis engine 700.”).
As pointed out hereinabove, Nendorf is an analogous art to the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Nendorf into the teaching of Lin to include “receive a request for model telemonitoring information over the network from a validation system; and output, over the network to the validation system by the testing module and during development of the model, responsive model telemonitoring data.” The modification would be obvious because one of ordinary skill in the art would be motivated to monitor data generated within a multi-platform model processing and execution management engine (Nendorf, paragraph [0059]).
Claim 18 is a non-transitory computer-readable media claim corresponding to the computing system claim hereinabove (Claim 4). Therefore, Claim 18 is rejected for the same reason set forth in the rejection of Claim 4.
Claims 5, 12, and 19 are rejected under 35 U.S.C. § 103 as being unpatentable over Lin in view of Nendorf as applied to Claims 4, 11, and 18 above, and further in view of US 2018/0309634 (hereinafter “Makovsky”).
EXAMINER’S REMARKS
In order for a reference to be proper for use in an obviousness rejection under 35 U.S.C. § 103, the reference must be analogous art to the claimed invention. In re Bigio, 381 F.3d 1320, 1325, 72 USPQ2d 1209, 1212 (Fed. Cir. 2004). A reference is analogous art to the claimed invention if: (1) the reference is from the same field of endeavor as the claimed invention (even if it addresses a different problem); or (2) the reference is reasonably pertinent to the problem faced by the inventor (even if it is not in the same field of endeavor as the claimed invention). See MPEP § 2141.01(a)(I).
The claimed invention is generally directed to evaluating models, including machine learning and/or statistical models (specification, paragraph [0002]). As for the “reasonably pertinent” test, Makovsky is generally directed to monitoring computerized services (specification, paragraph [0002]). Thus, Makovsky is an analogous art to the claimed invention (even if it is not in the same field of endeavor as the claimed invention).
As per Claim 5, the rejection of Claim 4 is incorporated; and the combination of Lin and Nendorf does not explicitly disclose:
receive a second set of model telemonitoring data from the development system without outputting a specific request for the second set of model telemonitoring data to the development system.
However, Makovsky discloses:
receive a second set of model telemonitoring data from the development system without outputting a specific request for the second set of model telemonitoring data to the development system (paragraph [0066], “[…] the agent device 316 can push the data indicative of the event to the management system 314 for processing, such as without first receiving a request for that from the management system 314.”; paragraph [0139], “The monitoring mechanism can receive, from the management system, the service model, a subset of the service model, a transformation of the service model, or information based on the service model to the monitoring mechanism. For example, the service model can be generated using a service model generator of the management system.”).
As pointed out hereinabove, Makovsky is an analogous art to the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Makovsky into the combined teachings of Lin and Nendorf to include “receive a second set of model telemonitoring data from the development system without outputting a specific request for the second set of model telemonitoring data to the development system.” The modification would be obvious because one of ordinary skill in the art would be motivated to generate service models based on data received from an agent device (Makovsky, paragraph [0071]).
As per Claim 12, the rejection of Claim 11 is incorporated; and the combination of Lin and Nendorf does not explicitly disclose:
output a second set of model telemonitoring data over the network to the validation system having received a specific request for the second set of model telemonitoring data from the validation system.
However, Makovsky discloses:
output a second set of model telemonitoring data over the network to the validation system having received a specific request for the second set of model telemonitoring data from the validation system (paragraph [0138], “[…] the command may include an indication of a type of response to monitor, such as based on a request included in the test message.”; paragraph [0139], “The monitoring mechanism can receive, from the management system, the service model, a subset of the service model, a transformation of the service model, or information based on the service model to the monitoring mechanism. For example, the service model can be generated using a service model generator of the management system.”).
As pointed out hereinabove, Makovsky is an analogous art to the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Makovsky into the combined teachings of Lin and Nendorf to include “output a second set of model telemonitoring data over the network to the validation system having received a specific request for the second set of model telemonitoring data from the validation system.” The modification would be obvious because one of ordinary skill in the art would be motivated to generate service models based on data received from an agent device (Makovsky, paragraph [0071]).
Claim 19 is a non-transitory computer-readable media claim corresponding to the computing system claim hereinabove (Claim 5). Therefore, Claim 19 is rejected for the same reason set forth in the rejection of Claim 5.
Claims 6, 13, and 20 are rejected under 35 U.S.C. § 103 as being unpatentable over Lin in view of US 2015/0113331 (hereinafter “Bhattacharya”).
EXAMINER’S REMARKS
In order for a reference to be proper for use in an obviousness rejection under 35 U.S.C. § 103, the reference must be analogous art to the claimed invention. In re Bigio, 381 F.3d 1320, 1325, 72 USPQ2d 1209, 1212 (Fed. Cir. 2004). A reference is analogous art to the claimed invention if: (1) the reference is from the same field of endeavor as the claimed invention (even if it addresses a different problem); or (2) the reference is reasonably pertinent to the problem faced by the inventor (even if it is not in the same field of endeavor as the claimed invention). See MPEP § 2141.01(a)(I).
The claimed invention is generally directed to evaluating models, including machine learning and/or statistical models (specification, paragraph [0002]). As for the “same field of endeavor” test, Lin is generally directed to integrating model development control systems and model validation platforms (specification, col. 1 lines 8 and 9). As for the “reasonably pertinent” test, Bhattacharya is generally directed to improved software testing project execution (specification, paragraph [0001]). Thus, Lin and Bhattacharya are both analogous art to the claimed invention (even if they address different problems or are not in the same field of endeavor as the claimed invention).
As per Claim 6, the rejection of Claim 1 is incorporated; and Lin discloses “the model,” but Lin does not explicitly disclose:
determine, based on the request, an appropriate test type to use for the model;
select, from a repository of test scripts, a test script having the appropriate test type to use for the model; and
interact with the development system based on the test script.
However, Bhattacharya discloses:
determine, based on the request, an appropriate test type to use […] (paragraph [0045], “With reference to FIG. 4A, in some embodiments, one or more users 401a may identify a test type […]” and “Upon receiving a new test execution request (see 414), the test scheduling computer 402 may add the test script execution request to a request queue, 416.”);
select, from a repository of test scripts, a test script having the appropriate test type to use […] (paragraph [0045], “Upon receiving a new test execution request (see 414), the test scheduling computer 402 may add the test script execution request to a request queue, 416. The test scheduling computer 402 may select a test script set for scheduling, 417, and parse the test execution request, 418. Based on the parsing, the test scheduling computer 402 may identify test script parameters (e.g., number of execution computers, their geographic location(s), software/protocol compatibility requirements, timing requirements, manual intervention requirements, etc.) for the text script corresponding to the selected test execution request, 419.”); and
interact with the development system based on the test script (paragraph [0049], “With reference to FIG. 4E, in some embodiments, the script test execution computer 405 may select an operation to perform as part of test script execution, 445.”).
As pointed out hereinabove, Bhattacharya is an analogous art to the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Bhattacharya into the teaching of Lin to include “determine, based on the request, an appropriate test type to use for the model; select, from a repository of test scripts, a test script having the appropriate test type to use for the model; and interact with the development system based on the test script.” The modification would be obvious because one of ordinary skill in the art would be motivated to test a model to achieve reliability and quality of the model (Bhattacharya, paragraph [0002]).
Claim 13 is a computing system claim corresponding to the computing system claim hereinabove (Claim 6). Therefore, Claim 13 is rejected for the same reason set forth in the rejection of Claim 6.
Claim 20 is a non-transitory computer-readable media claim corresponding to the computing system claim hereinabove (Claim 6). Therefore, Claim 20 is rejected for the same reason set forth in the rejection of Claim 6.
Claims 7 and 14 are rejected under 35 U.S.C. § 103 as being unpatentable over Lin in view of Bhattacharya as applied to Claims 6 and 13 above, and further in view of US 2019/0138427 (hereinafter “Purushothaman”).
EXAMINER’S REMARKS
In order for a reference to be proper for use in an obviousness rejection under 35 U.S.C. § 103, the reference must be analogous art to the claimed invention. In re Bigio, 381 F.3d 1320, 1325, 72 USPQ2d 1209, 1212 (Fed. Cir. 2004). A reference is analogous art to the claimed invention if: (1) the reference is from the same field of endeavor as the claimed invention (even if it addresses a different problem); or (2) the reference is reasonably pertinent to the problem faced by the inventor (even if it is not in the same field of endeavor as the claimed invention). See MPEP § 2141.01(a)(I).
The claimed invention is generally directed to evaluating models, including machine learning and/or statistical models (specification, paragraph [0002]). As for the “reasonably pertinent” test, Purushothaman is generally directed to controlling data processing operations pertaining to the development, maintenance, and installation of software programs (specification, paragraph [0001]). Thus, Purushothaman is an analogous art to the claimed invention (even if it is not in the same field of endeavor as the claimed invention).
As per Claim 7, the rejection of Claim 6 is incorporated; and Lin discloses “the model” and the combination of Lin and Bhattacharya discloses “the appropriate test type,” but the combination of Lin and Bhattacharya does not explicitly disclose:
generate, based on development activity from a validator, a new test script of the appropriate test type to use for the model.
However, Purushothaman discloses:
generate, based on development activity from a validator, a new test script […] to use […] (paragraph [0054], “[…] the pre-deployment validation computing platform 110 may generate a script to test the new feature and/or update in an isolated portion of the production environment.”).
As pointed out hereinabove, Purushothaman is an analogous art to the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Purushothaman into the combined teachings of Lin and Bhattacharya to include “generate, based on development activity from a validator, a new test script of the appropriate test type to use for the model.” The modification would be obvious because one of ordinary skill in the art would be motivated to test a new feature of a model to determine errors corresponding to the model (Purushothaman, paragraph [0037]).
Claim 14 is a computing system claim corresponding to the computing system claim hereinabove (Claim 7). Therefore, Claim 14 is rejected for the same reason set forth in the rejection of Claim 7.
Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to the Applicant’s disclosure. They are as follows:
US 2017/0277624 (hereinafter “Avadhanula”) discloses creating test case model by utilizing underlying model driven UI generation framework.
Conclusion
Any inquiry concerning this communication or earlier communications from the Examiner should be directed to Qing Chen whose telephone number is 571-270-1071. The Examiner can normally be reached on Monday through Friday from 9:00 AM to 5:00 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, the Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at https://www.uspto.gov/ interviewpractice.
If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s supervisor, Wei Mui, can be reached at 571-272-3708. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for more information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO customer service representative, call 800-786-9199 (in USA or Canada) or 571-272-1000.
/Qing Chen/
Primary Examiner, Art Unit 2191
Prosecution Insights