DETAILED ACTION
This is the initial Office action based on the preliminary amendment filed on December 21, 2023.
Claims 1, 2, and 4-8 are pending.
Claim 3 is canceled.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Internet Communications
Without a written authorization for Internet communications by the Applicant in place, the USPTO cannot communicate with the Applicant via email and will not respond via email to any Internet correspondence which contains information subject to the confidentiality requirement as set forth in 35 U.S.C. § 122, such as claimed subject matter in an interview agenda or proposed claim amendments for an Examiner’s Amendment.
Therefore, in the interest of facilitating compact prosecution, the Examiner kindly asks the Applicant to authorize Internet communications with the USPTO by using Form PTO/SB/439 (available at https://www.uspto.gov/patents/apply/forms). The form may be submitted via the USPTO patent electronic filing system (Patent Center) using the document description “Internet Communications Authorized” to facilitate processing. The written authorization for Internet communications must be submitted on a separate paper to be entitled to acceptance in accordance with 37 CFR § 1.4(c). The separate paper will facilitate processing and avoid confusion. The written authorization for Internet communications may not be submitted via an email. See MPEP § 502.03(II).
Claim Interpretation
During patent examination, the pending claims must be “given their broadest reasonable interpretation consistent with the specification.” See MPEP § 2111. Under a broadest reasonable interpretation (BRI), words of the claim must be given their plain meaning, unless such meaning is inconsistent with the specification. The plain meaning of a term means the ordinary and customary meaning given to the term by those of ordinary skill in the art at the relevant time. The ordinary and customary meaning of a term may be evidenced by a variety of sources, including the words of the claims themselves, the specification, the drawings, and the prior art. See MPEP § 2111.01(I).
Applicant is entitled to be their own lexicographer and may rebut the presumption that claim terms are to be given their ordinary and customary meaning by clearly setting forth a definition of the term that is different from its ordinary and customary meaning(s) in the specification at the relevant time. Where an explicit definition is provided by the Applicant for a term, that definition will control interpretation of the term as it is used in the claim. See MPEP § 2111.01(IV)(A). Any such lexicographic definition for a term will be expressly noted by the Examiner in the prior art rejections of the claims.
Claim Interpretation Under 35 U.S.C. § 112(f)
For clarity of the prosecution history record, Claims 2, 4, and 5 recite various “means-plus-function” limitations. However, in light of the Applicant’s preliminary amendments to Claim 1 (by deleting the recited “means-plus-function” limitations) and further consideration of the claims by the Examiner, it does not appear that the Applicant intend on treating these “means-plus-function” limitations in accordance with 35 U.S.C. § 112(f). Therefore, Claims 2, 4, and 5 are not being interpreted under 35 U.S.C. § 112(f) for the purpose of further examination (see 35 U.S.C. § 112(b) rejections of Claims 2, 4, and 5 hereinbelow).
Claim Mapping
For clarity of the prosecution history record, the Examiner has provided annotations in the prior art rejections of the claims to aid the Applicant in understanding the Examiner’s interpretations of the claimed invention and the prior art, such as emphasizing notable and relevant portions of the prior art citations, using item-to-item matching to the prior art citations, pairing exact claim language to particular language used in the prior art citations, and/or clearly explaining the Examiner’s interpretation as to how a prior art citation maps to the claim language, especially when there is no one-to-one matching of terms. Furthermore, the annotations are provided in the prior art rejections of the claims at the Examiner’s discretion where the Examiner deemed to be appropriate and necessary.
Claim Objections
Claims 1, 2, 4, and 6-8 are objected to because of the following informalities:
Claim 1 contains a typographical error: the semicolon (;) after the limitation “execute the instructions to” should be replaced with a colon (:).
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 1 contains a typographical error: a semicolon (;) should be added after the “display” step.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claims 1, 7, and 8 recite “the cause of accuracy degradation.” It should read -- the cause of the accuracy degradation --.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claims 2 and 6 recite “the accuracy degradation.” It should read -- the accuracy degradation of the machine learning model --.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 4 recites “comprising.” It should read -- further comprising --.
Appropriate correction is required.
Claim Rejections - 35 U.S.C. § 112
The following is a quotation of 35 U.S.C. § 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1, 2, and 4-8 are rejected under 35 U.S.C. § 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1, 7, and 8 recite the limitation “the result” at line 17. There is insufficient antecedent basis for this limitation in the claims. In the interest of compact prosecution, the Examiner subsequently interprets this limitation as reading “a result” for the purpose of further examination.
Claims 2 and 4-6 depend on Claim 1. Therefore, Claims 2 and 4-6 suffer the same deficiency as Claim 1.
Claims 2, 4, and 5 recite various “means-plus-function” limitations. However, the parent claim of Claims 2, 4, and 5 (Claim 1) does not recite any “means-plus-function” limitations. Thus, the claims are rendered vague and indefinite because of the inconsistency between Claim 1 and Claims 2, 4, and 5. In the interest of compact prosecution, the Examiner subsequently interprets Claims 2, 4, and 5 as not reciting the various “means-plus-function” limitations for the purpose of further examination.
Claim 6 depends on Claim 4. Therefore, Claim 6 suffers the same deficiency as Claim 4 (see 35 U.S.C. § 112(b) rejection of Claim 6 hereinbelow).
Claim 5 recites the limitation “the display means” at line 4. There is insufficient antecedent basis for this limitation in the claim. In the interest of compact prosecution, the Examiner subsequently interprets this limitation as reading “a display” for the purpose of further examination.
Claim 5 recites the limitation “the number” at line 7. There is insufficient antecedent basis for this limitation in the claim. In the interest of compact prosecution, the Examiner subsequently interprets this limitation as reading “a number” for the purpose of further examination.
Claim 6 recites the limitation “the information” at line 2. There is insufficient antecedent basis for this limitation in the claim. In the interest of compact prosecution, the Examiner subsequently interprets Claim 6 as depending on Claim 4 for the purpose of further examination. Note that such dependency order would provide sufficient antecedent basis for this limitation in the claim.
Claim 6 recites the limitation “the prediction” at line 3. There is insufficient antecedent basis for this limitation in the claim. In the interest of compact prosecution, the Examiner subsequently interprets this limitation as reading “a prediction” for the purpose of further examination.
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 2, and 4-8 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Claim Interpretation: It is essential that the broadest reasonable interpretation (BRI) of a claim as a whole be established prior to examining the claim for eligibility. Under the BRI, the limitations of Claim 1 are presumed to have their plain meaning consistent with the specification as it would be interpreted by one of ordinary skill in the art. See MPEP § 2111.
The BRI of Claim 1 is an information processing apparatus for acquiring a cause of accuracy degradation of a machine learning model; acquiring an improvement measure of the learning model that corresponds to the cause of the accuracy degradation of the machine learning model; displaying the improvement measure of the machine learning model; acquiring a degradation cause classification that corresponds to the cause of accuracy degradation of the machine learning model; acquiring a recommendation template that corresponds to the degradation cause classification; checking whether or not search target data in the recommendation template satisfies display conditions in the recommendation template; and acquiring a message that corresponds to the result of the check in the recommendation template as the improvement measure of the machine learning model.
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory category. See MPEP § 2106.03. Claim 1 is directed to an information processing apparatus, which is a machine, and falls within one of the statutory categories of invention. (Step 1: YES).
Step 2A, Prong One: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP § 2106.04(II), a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim.
Claim 1 recites the limitation:
(a) checking whether or not search target data in the recommendation template satisfies display conditions in the recommendation template.
These recited steps, under the BRI, cover performance of the steps in the human mind alone or with the aid of pen and paper. That is, other than reciting:
(1) at least one memory storing instructions; and
(2) at least one processor configured to execute the instructions to.
Nothing in the claim precludes the steps from practically being performed in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper. For example, the limitation (a) in the context of the claim encompasses a human evaluating search target data in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper. See MPEP § 2106.04(a)(2)(III).
If a claim limitation, under its BRI, covers a practical performance in the human mind alone or with the aid of pen and paper but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. (Step 2A, Prong One: YES).
Step 2A, Prong Two: This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the judicial exception or whether the claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the judicial exception into a practical application. See MPEP § 2106.04(d).
This judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements:
(1) at least one memory storing instructions; and
(2) at least one processor configured to execute the instructions to.
The additional elements (1) and (2) are recited at a high level of generality such that they amount to no more than mere instructions to apply the judicial exception using generic computer components. The memory and processor are used as a tool to perform the various steps of the claim. See MPEP § 2106.05(f).
Also, the claim recites the additional elements:
(3) acquire a cause of accuracy degradation of a machine learning model;
(4) acquire an improvement measure of the learning model that corresponds to the cause of the accuracy degradation of the machine learning model;
(5) display the improvement measure of the machine learning model;
(6) acquire a degradation cause classification that corresponds to the cause of accuracy degradation of the machine learning model;
(7) acquire a recommendation template that corresponds to the degradation cause classification; and
(8) acquire a message that corresponds to the result of the check in the recommendation template as the improvement measure of the machine learning model.
The additional elements (3) to (8) are mere data gathering/outputting recited at a high level of generality and thus, are insignificant extra-solution activities. See MPEP § 2106.05(g). Furthermore, all uses of the judicial exception require such data gathering/outputting, and, as such, the additional elements do not impose any meaningful limits on the claim. The additional elements amount to necessary data gathering/outputting. See MPEP § 2106.05(g).
Accordingly, even when viewed in combination, the additional elements do not integrate the recited judicial exception into a practical application because they do not impose any meaningful limits on practicing the judicial exception. (Step 2A, Prong Two: NO). The claim is directed to an abstract idea. (Step 2A: YES).
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amounts to significantly more than the recited judicial exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP § 2106.05.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as a combination do not amount to significantly more than the judicial exception. As discussed above with respect to integration of the judicial exception into a practical application, the claim recites the additional elements:
(1) at least one memory storing instructions; and
(2) at least one processor configured to execute the instructions to.
The additional elements (1) and (2) amount to no more than mere instructions to apply the judicial exception using generic computer components. The analysis under Step 2A, Prong Two is carried through to Step 2B. The use of a computer or other machinery in its ordinary capacity does not integrate a judicial exception into a practical application or provide significantly more.
Also, the claim recites the additional elements:
(3) acquire a cause of accuracy degradation of a machine learning model;
(4) acquire an improvement measure of the learning model that corresponds to the cause of the accuracy degradation of the machine learning model;
(5) display the improvement measure of the machine learning model;
(6) acquire a degradation cause classification that corresponds to the cause of accuracy degradation of the machine learning model;
(7) acquire a recommendation template that corresponds to the degradation cause classification; and
(8) acquire a message that corresponds to the result of the check in the recommendation template as the improvement measure of the machine learning model.
The additional elements (3) to (8) simply append well-understood, routine, and conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception and thus, are not indicative of an inventive concept. MPEP § 2106.05(d)(II) expressly states that the courts have recognized the computer functions of receiving or transmitting data over a network, e.g., using the Internet to gather data and presenting offers and gathering statistics as well‐understood, routine, and conventional computer functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activities. Thus, a person of ordinary skill in the art would readily comprehend that it is well-understood, routine, and conventional in the computing art to acquire/display data/information. Therefore, the limitations remain insignificant extra-solution activities even upon reconsideration and do not amount to significantly more.
Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the additional elements as a combination adds nothing that is not already present when looking at the additional elements taken individually. Even when considered in combination, the additional elements represent mere instructions to apply a judicial exception using generic computer components and insignificant extra-solution activities, and therefore do not provide an inventive concept. (Step 2B: NO). The claim is not patent eligible.
Claims 2 and 4-6 are dependent on Claim 1, but do not add any feature or subject matter that would solve the judicial exception deficiencies of Claim 1.
Claim 2 recites the limitations:
(a) determination means for determining whether or not the machine learning model satisfies evaluation criteria;
(b) extraction means for extracting information that is required to estimate the cause of the accuracy degradation from data of the machine learning model that has been determined not to satisfy the evaluation criteria; and
(c) input means for inputting information that is required to estimate the cause of the accuracy degradation to a degradation cause estimation apparatus,
(d) wherein the accuracy degradation cause acquisition means acquires an improvement measure of the machine learning model that corresponds to the cause of the accuracy degradation of the machine learning model output from the degradation cause estimation apparatus.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 4 recites the limitations:
(a) registration means for registering a result of an operation performed by a user on the improvement measure of the machine learning model; and
(b) parameter update means for updating a parameter, which is information that is required to estimate the cause of the accuracy degradation of the machine learning model based on the result of the operation performed by the user.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 5 recites the limitations:
(a) wherein the registration means issues a problem when the user has selected the improvement measure of the machine learning model displayed on the display means, and
(b) wherein the parameter update means updates the parameter in accordance with a percentage that it is determined to be effective in view of case effectiveness in the issued problem when the number of issued problems is equal to or larger than a predetermined number.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
Claim 6 recites the limitation:
(a) wherein the information that is required to estimate the cause of the accuracy degradation includes a result of the prediction, prediction target data, and a degradation determination threshold.
Claim 2 recites further mental steps which can be practically performed in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper and thus, fail to make the claim any less abstract under Step 2A, Prong One (see MPEP § 2106.04(a)(2)(III)).
Claims 4 and 5 recite further additional elements that do not integrate the judicial exception into a practical application of the judicial exception because they do not require any particular application of the judicial exception and are, at best, the equivalent of merely adding the words “apply it” (or an equivalent) to the judicial exception under Step 2A, Prong Two (see MPEP § 2106.05(f)) and thus, are also not significantly more than the abstract idea under Step 2B.
Claim 2 recites further additional elements that do not integrate the judicial exception into a practical application of the judicial exception because they are mere data gathering/transmitting/outputting recited at a high level of generality and thus, are insignificant extra-solution activities under Step 2A, Prong Two (see MPEP § 2106.05(g)) and thus, are also not significantly more than the abstract idea under Step 2B.
Claim 6 recites further additional elements that do not integrate the judicial exception into a practical application of the judicial exception because they merely indicate a field of use or technological environment in which the judicial exception is performed and thus, fail to add an inventive concept to the claims under Step 2A, Prong Two (see MPEP § 2106.05(h)) and thus, are also not significantly more than the abstract idea under Step 2B.
Thus, Claims 2 and 4-6 do not add any steps or additional elements, when considered both individually and as a combination, that would convert Claim 1 into patent-eligible subject matter.
Therefore, Claims 1, 2, and 4-6 are not drawn to patent-eligible subject matter as they are directed to an abstract idea without significantly more.
<<>> • × • <<>> • × • <<>> • × • <<>> • + • <<>> • × • <<>> • × • <<>> • × • <<>>
Claim Interpretation: It is essential that the broadest reasonable interpretation (BRI) of a claim as a whole be established prior to examining the claim for eligibility. Under the BRI, the limitations of Claim 7 are presumed to have their plain meaning consistent with the specification as it would be interpreted by one of ordinary skill in the art. See MPEP § 2111.
The BRI of Claim 7 is an information processing method for acquiring a cause of accuracy degradation of a machine learning model; acquiring an improvement measure of the learning model that corresponds to the cause of the accuracy degradation of the machine learning model; displaying the improvement measure of the machine learning model; acquiring a degradation cause classification that corresponds to the cause of accuracy degradation of the machine learning model; acquiring a recommendation template that corresponds to the degradation cause classification; checking whether or not search target data in the recommendation template satisfies display conditions in the recommendation template; and acquiring a message that corresponds to the result of the check in the recommendation template as the improvement measure of the machine learning model.
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory category. See MPEP § 2106.03. Claim 7 is directed to an information processing method, which is a process (a series of steps or acts), and falls within one of the statutory categories of invention. (Step 1: YES).
Step 2A, Prong One: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP § 2106.04(II), a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim.
Claim 7 recites the limitation:
(a) checking whether or not search target data in the recommendation template satisfies display conditions in the recommendation template.
These recited steps, under the BRI, cover performance of the steps in the human mind alone or with the aid of pen and paper. That is, nothing in the claim precludes the steps from practically being performed in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper. For example, the limitation (a) in the context of the claim encompasses a human evaluating search target data in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper. See MPEP § 2106.04(a)(2)(III).
If a claim limitation, under its BRI, covers a practical performance in the human mind alone or with the aid of pen and paper but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. (Step 2A, Prong One: YES).
Step 2A, Prong Two: This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the judicial exception or whether the claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the judicial exception into a practical application. See MPEP § 2106.04(d).
This judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements:
(1) acquiring a cause of accuracy degradation of a machine learning model;
(2) acquiring an improvement measure of the learning model that corresponds to the cause of the accuracy degradation of the machine learning model;
(3) displaying the improvement measure of the machine learning model;
(4) acquiring a degradation cause classification that corresponds to the cause of accuracy degradation of the machine learning model;
(5) acquiring a recommendation template that corresponds to the degradation cause classification; and
(6) acquiring a message that corresponds to the result of the check in the recommendation template as the improvement measure of the machine learning model.
The additional elements (1) to (6) are mere data gathering/outputting recited at a high level of generality and thus, are insignificant extra-solution activities. See MPEP § 2106.05(g). Furthermore, all uses of the judicial exception require such data gathering/outputting, and, as such, the additional elements do not impose any meaningful limits on the claim. The additional elements amount to necessary data gathering/outputting. See MPEP § 2106.05(g).
Accordingly, even when viewed in combination, the additional elements do not integrate the recited judicial exception into a practical application because they do not impose any meaningful limits on practicing the judicial exception. (Step 2A, Prong Two: NO). The claim is directed to an abstract idea. (Step 2A: YES).
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amounts to significantly more than the recited judicial exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP § 2106.05.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as a combination do not amount to significantly more than the judicial exception. As discussed above with respect to integration of the judicial exception into a practical application, the claim recites the additional elements:
(1) acquiring a cause of accuracy degradation of a machine learning model;
(2) acquiring an improvement measure of the learning model that corresponds to the cause of the accuracy degradation of the machine learning model;
(3) displaying the improvement measure of the machine learning model;
(4) acquiring a degradation cause classification that corresponds to the cause of accuracy degradation of the machine learning model;
(5) acquiring a recommendation template that corresponds to the degradation cause classification; and
(6) acquiring a message that corresponds to the result of the check in the recommendation template as the improvement measure of the machine learning model.
The additional elements (1) to (6) simply append well-understood, routine, and conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception and thus, are not indicative of an inventive concept. MPEP § 2106.05(d)(II) expressly states that the courts have recognized the computer functions of receiving or transmitting data over a network, e.g., using the Internet to gather data and presenting offers and gathering statistics as well‐understood, routine, and conventional computer functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activities. Thus, a person of ordinary skill in the art would readily comprehend that it is well-understood, routine, and conventional in the computing art to acquire/display data/information. Therefore, the limitations remain insignificant extra-solution activities even upon reconsideration and do not amount to significantly more.
Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the additional elements as a combination adds nothing that is not already present when looking at the additional elements taken individually. Even when considered in combination, the additional elements represent insignificant extra-solution activities, and therefore do not provide an inventive concept. (Step 2B: NO). The claim is not patent eligible.
<<>> • × • <<>> • × • <<>> • × • <<>> • + • <<>> • × • <<>> • × • <<>> • × • <<>>
Claim Interpretation: It is essential that the broadest reasonable interpretation (BRI) of a claim as a whole be established prior to examining the claim for eligibility. Under the BRI, the limitations of Claim 8 are presumed to have their plain meaning consistent with the specification as it would be interpreted by one of ordinary skill in the art. See MPEP § 2111.
The BRI of Claim 8 is a non-transitory computer readable medium for acquiring a cause of accuracy degradation of a machine learning model; acquiring an improvement measure of the learning model that corresponds to the cause of the accuracy degradation of the machine learning model; displaying the improvement measure of the machine learning model; acquiring a degradation cause classification that corresponds to the cause of accuracy degradation of the machine learning model; acquiring a recommendation template that corresponds to the degradation cause classification; checking whether or not search target data in the recommendation template satisfies display conditions in the recommendation template; and acquiring a message that corresponds to the result of the check in the recommendation template as the improvement measure of the machine learning model.
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory category. See MPEP § 2106.03. Claim 8 is directed to a non-transitory computer readable medium, which is an article of manufacture, and falls within one of the statutory categories of invention. (Step 1: YES).
Step 2A, Prong One: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP § 2106.04(II), a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim.
Claim 8 recites the limitation:
(a) checking whether or not search target data in the recommendation template satisfies display conditions in the recommendation template.
These recited steps, under the BRI, cover performance of the steps in the human mind alone or with the aid of pen and paper. That is, other than reciting:
(1) [a] non-transitory computer readable medium storing a program for executing an information processing method, wherein the information processing method comprises.
Nothing in the claim precludes the steps from practically being performed in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper. For example, the limitation (a) in the context of the claim encompasses a human evaluating search target data in the human mind alone using observation, evaluation, judgment, and opinion or with the aid of pen and paper. See MPEP § 2106.04(a)(2)(III).
If a claim limitation, under its BRI, covers a practical performance in the human mind alone or with the aid of pen and paper but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. (Step 2A, Prong One: YES).
Step 2A, Prong Two: This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the judicial exception or whether the claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the judicial exception into a practical application. See MPEP § 2106.04(d).
This judicial exception is not integrated into a practical application. In particular, the claim recites the additional element:
(1) [a] non-transitory computer readable medium storing a program for executing an information processing method, wherein the information processing method comprises.
The additional element (1) is recited at a high level of generality such that it amounts to no more than mere instructions to apply the judicial exception using generic computer components. The non-transitory computer readable medium is used as a tool to perform the various steps of the claim. See MPEP § 2106.05(f).
Also, the claim recites the additional elements:
(2) acquiring a cause of accuracy degradation of a machine learning model;
(3) acquiring an improvement measure of the learning model that corresponds to the cause of the accuracy degradation of the machine learning model;
(4) displaying the improvement measure of the machine learning model;
(5) acquiring a degradation cause classification that corresponds to the cause of accuracy degradation of the machine learning model;
(6) acquiring a recommendation template that corresponds to the degradation cause classification; and
(7) acquiring a message that corresponds to the result of the check in the recommendation template as the improvement measure of the machine learning model.
The additional elements (2) to (7) are mere data gathering/outputting recited at a high level of generality and thus, are insignificant extra-solution activities. See MPEP § 2106.05(g). Furthermore, all uses of the judicial exception require such data gathering/outputting, and, as such, the additional elements do not impose any meaningful limits on the claim. The additional elements amount to necessary data gathering/outputting. See MPEP § 2106.05(g).
Accordingly, even when viewed in combination, the additional elements do not integrate the recited judicial exception into a practical application because they do not impose any meaningful limits on practicing the judicial exception. (Step 2A, Prong Two: NO). The claim is directed to an abstract idea. (Step 2A: YES).
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amounts to significantly more than the recited judicial exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP § 2106.05.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as a combination do not amount to significantly more than the judicial exception. As discussed above with respect to integration of the judicial exception into a practical application, the claim recites the additional element:
(1) [a] non-transitory computer readable medium storing a program for executing an information processing method, wherein the information processing method comprises.
The additional element (1) amounts to no more than mere instructions to apply the judicial exception using generic computer components. The analysis under Step 2A, Prong Two is carried through to Step 2B. The use of a computer or other machinery in its ordinary capacity does not integrate a judicial exception into a practical application or provide significantly more.
Also, the claim recites the additional elements:
(2) acquiring a cause of accuracy degradation of a machine learning model;
(3) acquiring an improvement measure of the learning model that corresponds to the cause of the accuracy degradation of the machine learning model;
(4) displaying the improvement measure of the machine learning model;
(5) acquiring a degradation cause classification that corresponds to the cause of accuracy degradation of the machine learning model;
(6) acquiring a recommendation template that corresponds to the degradation cause classification; and
(7) acquiring a message that corresponds to the result of the check in the recommendation template as the improvement measure of the machine learning model.
The additional elements (2) to (7) simply append well-understood, routine, and conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception and thus, are not indicative of an inventive concept. MPEP § 2106.05(d)(II) expressly states that the courts have recognized the computer functions of receiving or transmitting data over a network, e.g., using the Internet to gather data and presenting offers and gathering statistics as well‐understood, routine, and conventional computer functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activities. Thus, a person of ordinary skill in the art would readily comprehend that it is well-understood, routine, and conventional in the computing art to acquire/display data/information. Therefore, the limitations remain insignificant extra-solution activities even upon reconsideration and do not amount to significantly more.
Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the additional elements as a combination adds nothing that is not already present when looking at the additional elements taken individually. Even when considered in combination, the additional elements represent mere instructions to apply a judicial exception using generic computer components and insignificant extra-solution activities, and therefore do not provide an inventive concept. (Step 2B: NO). The claim is not patent eligible.
Claim Rejections - 35 U.S.C. § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 7, and 8 are rejected under 35 U.S.C. § 103 as being unpatentable over US 2021/0232950 (hereinafter “Kono”) in view of US 2022/0019936 (hereinafter “Sarda”).
Examiner’s Remarks: In order for a reference to be proper for use in an obviousness rejection under 35 U.S.C. § 103, the reference must be analogous art to the claimed invention. In re Bigio, 381 F.3d 1320, 1325, 72 USPQ2d 1209, 1212 (Fed. Cir. 2004). A reference is analogous art to the claimed invention if: (1) the reference is from the same field of endeavor as the claimed invention (even if it addresses a different problem); or (2) the reference is reasonably pertinent to the problem faced by the inventor (even if it is not in the same field of endeavor as the claimed invention). See MPEP § 2141.01(a)(I).
Note that the claimed invention is generally directed to presenting an appropriate improvement measure to a machine learning model (specification, paragraph [0005]). As for the “same field of endeavor” test, Kono is generally directed to performing inference by utilizing machine learning (specification, paragraph [0002]). And Sarda is generally directed to selecting machine learning features (specification, paragraph [0011]). Thus, Kono and Sarda are both analogous art to the claimed invention (even if they address different problems).
As per Claim 1, Kono discloses:
An information processing apparatus (Figure 4) comprising:
at least one memory (Figure 4: 12) storing instructions, and
at least one processor (Figure 4: 11) configured to execute the instructions to;
acquire a cause of accuracy degradation of a machine learning model (paragraph [0029], “[…] in the following description, the ‘machine learning model’ is also referred to as ‘inference model’ (emphasis added).”; paragraph [0078], “The factor determination unit 750 determines a factor that causes degradation of the inference accuracy when the inference unit 520 detects the degradation of the inference accuracy [acquire a cause of accuracy degradation of a machine learning model].”; paragraph [0084], “[…] the inference unit 520 acquires a result of the inference of the API by inputting the inference data to the inference model (S1314) (emphasis added).”);
acquire an improvement measure of the learning model that corresponds to the cause of the accuracy degradation of the machine learning model (paragraph [0111], “FIGS. 18 and 19 are diagrams schematically illustrating an example of the accuracy degradation countermeasure [an improvement measure of the learning model] determination processing S1600 of FIG. 16 (emphasis added).”; paragraph [0112], “FIG. 18 shows a case where the inference accuracy of the inference model having an inference model ID of ‘model002’ is degraded, and as a result, it is determined that the change of the effective feature amount in S1611 of FIG. 16 is a factor that causes degradation of the inference accuracy. In this example, in S1621 of FIG. 16, a new inference model having an inference model ID of ‘model002'’ corresponding to the change of the effective feature amount is generated, and the generated new inference model is deployed on the inference server 500 of the inference environment 2a and the inference server 500 of the inference environment 2b. In addition, ‘model002′’ is allocated to the clients ‘client002’, ‘client003’, and ‘client004’, to which the ‘model002’ having the degraded inference accuracy is allocated [acquire an improvement measure of the learning model that corresponds to the cause of the accuracy degradation of the machine learning model] (emphasis added).”);
display the improvement measure of the machine learning model (paragraph [0038], “The management server 700 determines an application method (countermeasure method) for the new inference model [the improvement measure of the machine learning model] depending on the factor of the inference accuracy degradation, and applies the new inference model to the inference environment 2 using the determined method.”; paragraph [0065]1, “[…] the memory unit 710 may store a program that displays a trend of the inference data and a temporal change of the inference accuracy (emphasis added).”); and
1Examiner’s Remarks: Note that Kono discloses displaying a trend of the inference data and a temporal change of the inference accuracy. Thus, one of ordinary skill in the art would readily comprehend that displaying the trend of the inference data and the temporal change of the inference accuracy includes displaying the inference model data after the accuracy degradation countermeasure is applied.
acquire a degradation cause classification that corresponds to the cause of accuracy degradation of the machine learning model (paragraph [0112], “FIG. 18 shows a case where the inference accuracy of the inference model having an inference model ID of ‘model002’ is degraded, and as a result, it is determined that the change of the effective feature amount in S1611 of FIG. 16 is a factor that causes degradation of the inference accuracy [acquire a degradation cause classification that corresponds to the cause of accuracy degradation of the machine learning model]. In this example, in S1621 of FIG. 16, a new inference model having an inference model ID of ‘model002'’ corresponding to the change of the effective feature amount is generated, and the generated new inference model is deployed on the inference server 500 of the inference environment 2a and the inference server 500 of the inference environment 2b. In addition, ‘model002′’ is allocated to the clients ‘client002’, ‘client003’, and ‘client004’, to which the ‘model002’ having the degraded inference accuracy is allocated (emphasis added).”).
Kono does not explicitly disclose:
acquire a recommendation template that corresponds to the degradation cause classification;
checking whether or not search target data in the recommendation template satisfies display conditions in the recommendation template; and
acquire a message that corresponds to the result of the check in the recommendation template as the improvement measure of the machine learning model.
However, Sarda discloses:
acquire a recommendation template that corresponds to the degradation cause classification (paragraph [0014]2, “Using the provided data and desired target field, server 121 recommends a set of features that predict with a high degree of accuracy the desired target field. A customer can select a subset of the recommended features [acquire a recommendation template] from which to train a machine learning model. In some embodiments, the model is trained using the provided customer data. In some embodiments, as part of the feature selection process, the customer is provided with a performance metric of each recommended feature. The performance metric provides the customer with a quantified value related to how much a specific feature improves the prediction accuracy of a model [corresponds to the degradation cause classification]. In some embodiments, the recommended features are ranked based on impact on prediction accuracy.”);
2Examiner’s Remarks: Note that Sarda discloses that the performance metric provides the customer with a quantified value related to how much a specific feature improves the prediction accuracy of a model. Thus, one of ordinary skill in the art would readily comprehend that the quantified value shows the customer how much the prediction accuracy of the model would improve from a prior degradation state of the model.
checking whether or not search target data in the recommendation template satisfies display conditions in the recommendation template (paragraph [0021], “At 205, features are selected based on the recommended input features. For example, using an interactive user interface, a set of recommended machine learning features for use in building a machine learning model are presented to a user. In some embodiments, the example user interface is implemented as a web application or web service. A user can select from the displayed recommended features to determine the set of features to use for training the machine learning model [display conditions in the recommendation template] (emphasis added).”; paragraph [0034], “At 407, eligible machine learning features are evaluated. For example, the eligible machine learning features are evaluated for impact on training an accurate machine learning model. In some embodiments, the eligible machine learning features are evaluated using an evaluation pipeline to successively filter out features by usefulness in predicting the desired target value [checking whether or not search target data in the recommendation template satisfies display conditions in the recommendation template]. For example, in some embodiments, a first evaluation step can determine an impact score such as a relief score to identify the distinction a column brings to a classification model. Columns with a relief score below a threshold value can be removed from recommendation.”); and
acquire a message that corresponds to the result of the check in the recommendation template as the improvement measure of the machine learning model (paragraph [0021], “At 205, features are selected based on the recommended input features. For example, using an interactive user interface, a set of recommended machine learning features for use in building a machine learning model are presented to a user. In some embodiments, the example user interface is implemented as a web application or web service. A user can select from the displayed recommended features to determine the set of features to use for training the machine learning model.”; paragraph [0036]3, “At 409, recommended features are provided. For example, the remaining features are recommended as input features [corresponds to the result of the check in the recommendation template as the improvement measure of the machine learning model]. In some embodiments, the set of recommended features is provided to the user via a graphical user interface of a web application [acquire a message]. The recommended features can be provided with quantified metrics related to how much impact each of the features has on model accuracy. In some embodiments, the features are provided in a ranked order allowing a user to select the most impactful features for training a machine learning model.”).
3Examiner’s Remarks: Note that Sarda discloses that the set of recommended features is provided to the user via a graphical user interface of a web application. Thus, one of ordinary skill in the art would readily comprehend that the web application provides messages to the user via the graphical user interface regarding the user-specified target field and training data of the improvement features for model accuracy.
As pointed out hereinabove, Kono and Sarda are both analogous art to the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Sarda into the teaching of Kono to include “acquire a recommendation template that corresponds to the degradation cause classification; checking whether or not search target data in the recommendation template satisfies display conditions in the recommendation template; and acquire a message that corresponds to the result of the check in the recommendation template as the improvement measure of the machine learning model.” The modification would be obvious because one of ordinary skill in the art would be motivated to automatically recommend and select machine learning features that result in significant improvement in the prediction accuracy of a machine learning model (Sarda, paragraph [0011]).
Claim 7 is an information processing method claim corresponding to the information processing apparatus claim hereinabove (Claim 1). Therefore, Claim 7 is rejected for the same reason set forth in the rejection of Claim 1.
Claim 8 is a non-transitory computer readable medium claim corresponding to the information processing apparatus claim hereinabove (Claim 1). Therefore, Claim 8 is rejected for the same reason set forth in the rejection of Claim 1.
Allowable Subject Matter
Claims 2 and 4-6 are objected to as being dependent upon a rejected base claim under 35 U.S.C. § 103, but would be allowable over the cited prior art if rewritten in independent form including all of the limitations of the base claim and any intervening claims, and overcome any corresponding objections and/or rejections set forth hereinabove.
The following is an Examiner’s statement of reasons for the indication of allowable subject matter:
As per Claim 2, the closest cited prior art, the combination of Kono and Sarda, fails to teach or suggest, among the other claimed limitations, “determination means for determining whether or not the machine learning model satisfies evaluation criteria; extraction means for extracting information that is required to estimate the cause of the accuracy degradation from data of the machine learning model that has been determined not to satisfy the evaluation criteria; and input means for inputting information that is required to estimate the cause of the accuracy degradation to a degradation cause estimation apparatus, wherein the accuracy degradation cause acquisition means acquires an improvement measure of the machine learning model that corresponds to the cause of the accuracy degradation of the machine learning model output from the degradation cause estimation apparatus.” These claimed limitations, in combination with the other claimed limitations, are neither taught nor suggested by the combination of Kono and Sarda.
As per Claim 4, the closest cited prior art, the combination of Kono and Sarda, fails to teach or suggest, among the other claimed limitations, “registration means for registering a result of an operation performed by a user on the improvement measure of the machine learning model; and parameter update means for updating a parameter, which is information that is required to estimate the cause of the accuracy degradation of the machine learning model based on the result of the operation performed by the user.” These claimed limitations, in combination with the other claimed limitations, are neither taught nor suggested by the combination of Kono and Sarda.
As per Claims 5 and 6, these claims depend directly or indirectly from Claim 4, and encompass the required limitations recited in Claim 4 as stated hereinabove. Note that Claim 6 is interpreted as depending on Claim 4 (see 35 U.S.C. § 112(b) rejection of Claim 6 hereinabove).
Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to the Applicant’s disclosure. They are as follows:
US 2020/0311601 (hereinafter “Robinson”) discloses generating a predictive output based on a predictive input.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
US 2020/0327371 (hereinafter “Sharma”) discloses providing an edge computing platform with an executable machine learning model that has been adapted or “edge-ified” to operate within the constraints of the edge computing environment to receive and process one or more streams of sensor data and produce one or more streams of inferences in real-time.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
US 2021/0064372 (hereinafter “Sun”) discloses tuning machine learning models using hardware configured with mixed precision capability.
<<>> + <<>> + <<>> • × • <<>> + <<>> + <<>>
US 2021/0334695 (hereinafter “Raj”) discloses detecting, alerting, and correcting machine learning model drift and integrating it in a model governance tool and/or a machine learning application.
Conclusion
Any inquiry concerning this communication or earlier communications from the Examiner should be directed to Qing Chen whose telephone number is 571-270-1071. The Examiner can normally be reached on Monday through Friday from 9:00 AM to 5:00 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, the Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at https://www.uspto.gov/ interviewpractice.
If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s supervisor, Wei Mui, can be reached at 571-272-3708. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for more information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO customer service representative, call 800-786-9199 (in USA or Canada) or 571-272-1000.
/Qing Chen/
Primary Examiner, Art Unit 2191
Prosecution Insights