DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is a First Action Non-Final on the merits. Claims 1-20 as originally filed on January 5, 2024, are currently pending and have been considered below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites method for evaluation platform.
Step 2A – Prong 1
Independent Claims 1, 11 and 19 as a whole recite a method of organizing human activity. The limitations from exemplary Claim 1 reciting “receiving an evaluation request, from an evaluation client, requesting evaluation of a generative; parsing the evaluation request to identify evaluation parameters; selecting an evaluation template, of a plurality of evaluation templates, based on the evaluation parameters, each of the plurality of evaluation templates corresponding to a different evaluation pipeline; and triggering the selected evaluation template to evaluate the generative” is a method of managing interactions between people, which falls into the certain methods of organizing human activity grouping, additionally mathematical concepts such as mathematical relationships, mathematical formulas or equations and mathematical calculations as the machine learning model can be computed using pen and paper to provide an evaluation utilizing evaluation parameters. The mere recitation of a generic computer (computer implemented method, AI system of claim 1; computer system, AI system, repository, processor of claim 11; computer system, processor, AI system of claim 19) does not take the claim out of the methods of organizing human activity grouping. Thus, the claim recites an abstract idea.
Step 2A - Prong 2: Claims 1-20 and their underlining limitations, steps, features and terms, are further inspected by the Examiner under the current examining guidelines, and found, both individually and as a whole, not to include additional elements that are sufficient to integrate the abstract idea into a practical application. The limitations are directed to limitations referenced in MPEP 2106.05 that are not enough to integrate the abstract idea into a practical application. Limitations that are not enough include, as a non-limiting or non-exclusive examples, such as: (i) adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, e.g., a claim to an abstract idea requiring no more than a generic computer to perform generic computer functions, (ii) insignificant extra solution activity, and/or (iii) generally linking the use of the judicial exception to a particular technological environment or field of use.
This judicial exception is not integrated into a practical application because the claim recites the additional elements of (computer implemented method, AI system of claim 1; computer system, AI system, repository, processor of claim 11; computer system, processor, AI system of claim 19). The computer implemented method, AI system of claim 1; computer system, AI system, repository, processor of claim 11; computer system, processor, AI system of claim 19, are recited at a high level of generality and are generically recited computer elements. The generically recited computer elements amount to simply implementing the abstract idea on a computer. The combination of these additional elements are additional elements do no more than generally link the use of the judicial exception to a particular technological environment or field of use. Accordingly, in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
The claim do not include additional elements that are sufficient to amount to significantly more than the judicial exception because, as discussed above, the additional elements do no more than generally link the use of the judicial exception to a particular technological environment or field of use. Thus, even when viewed as an ordered combination, nothing in the claims add significantly more (i.e. an inventive concept) to the abstract idea. The claims are ineligible.
Dependent claims 2-10, 12-18 and 20 are also directed to same grouping of methods of organizing human activity. The additional elements of the computer implemented method in claims 2-10; computer system in claims 12-18 and 20; repository in claims 7-9 and 16; processor of claim 15 and 17; user interface in claims 3, 12 and 20; computing system in claims 16 and 20; monitoring system and propagation system in claim 16; client computing system in claim 20, are additional elements do no more than generally link the use of the judicial exception to a particular technological environment or field of use. Accordingly, in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-5, 7-9 and 11-20 are rejected under 35 U.S.C. 102 (a)(2) as being anticipated by Bodigutla et al (US Patent Application Publication No. 20230140702 - hereinafter Bodi).
Re. claim 1, 11 and 19, Bodi discloses:
A computer implemented method, comprising:
receiving an evaluation request, from an evaluation client, requesting evaluation of a generative artificial intelligence (AI) system; [Bodi; ¶32 shows evaluation request and request of machine learning (AI) such as for example “RL evaluates several objectives, as described in more detail below with reference to FIG. 4, such as naturalness, relatedness, and user feedback, but other embodiments may use other or different objectives for calculating the reward. Naturalness refers to generating query suggestions that are natural, that is, query suggestions that would be entered by a person instead of strange query suggestions that would not be entered by a person (e.g., machine learning learning). Relatedness refers to the degree by which the query suggestion is related to the current search query entered by the user. Further, user feedback measures the engagement of the user within the online service, such as selecting suggested queries and amount of time the user is engaged with the online service in a session”].
parsing the evaluation request to identify evaluation parameters; [Bodi; ¶44].
selecting an evaluation template, of a plurality of evaluation templates, based on the evaluation parameters, each of the plurality of evaluation templates corresponding to a different evaluation pipeline; and [Bodi; ¶44 and ¶61].
triggering the selected evaluation template to evaluate the generative AI system. [Bodi; ¶44]
Re. claim 2 and 15, Bodi further discloses:
further comprising:
running an evaluation pipeline corresponding to the triggered evaluation template to obtain evaluation results; and outputting the evaluation results for access by the evaluation client. [Bodi; ¶44 and ¶122]
Re. claim 3 and 12, Bodi further discloses:
further comprising: exposing a user interface (UI) with the evaluation client, the UI having user actuatable parameter selection mechanisms for interaction by a user; and detecting user interaction with the user actuatable parameter selection mechanisms to identify the evaluation parameters. [Bodi; ¶135 shows utilization of user interface and ¶120 shows one example of a user selection through search-query logs].
Re. claim 4 and 13, Bodi further discloses:
further comprising: generating the evaluation request at the evaluation client based on the identified evaluation parameters. [Bodi; ¶32].
Re. claim 5 and 17, Bodi further discloses:
wherein selecting an evaluation template comprises: prompting an AI classifier with the evaluation parameters; and selecting the evaluation template with the AI classifier. [Bodi; ¶122 shows “class prediction performance of the contextual naturalness estimator was evaluated using F15 score and Accuracy metrics. The mean of the following metrics calculated on the test set, to evaluate the relevance, engagement, accuracy and heterogeneity of generated queries, were used: [0123] Sessions with positive user-action (Sessions+@6): Long-term binary engagement metric indicating if recommended queries lead to a successful session. Its value is “1”, if any of the six generated queries belong to a search-session in test-data with an associated down-stream positive user action (Section 2.2.2)”].
Re. claim 7 and 16, Bodi further discloses:
further comprising: detecting a modified pipeline definition generated at an evaluation engineer computing system, the modified pipeline definition being different from pipeline definitions corresponding to the plurality of evaluation templates; and storing the modified pipeline definition, as one of the plurality of pipeline templates, in a pipeline repository. [Bodi; ¶65 shows identifying if the query was natural such as “annotators were asked to identify if the query was “natural”. In some experiments, an average 58% of model-generated queries and 74% of real-user queries were identified as natural. The Inter Annotator Agreement (IAA) was poor (0.04) when the users evaluated model-generated sentences. In comparison, when they evaluated queries entered by real users, IAA was better (0.34) between the three annotators' ratings. Higher IAA and higher percentage of queries identified as “natural” imply that real-user queries are more natural and distinguishable than queries sampled from the pre-trained Seq2SeqNMT mode”. Meanwhile, ¶138 shows storing such as “any medium that is capable of storing, encoding, or carrying instructions 824 for execution by the machine 800 and that cause the machine 800 to perform any one or more of the techniques of the present disclosure, or that is capable of storing, encoding, or carrying data structures used by or associated with such instructions 824”].
Re. claim 8, Bodi further discloses:
further comprising: detecting that the additional pipeline definition has been stored in the pipeline repository; and generating an endpoint in the evaluation client to the pipeline template corresponding to the pipeline definition. [Bodi; ¶99-¶101 and ¶130 shows iterations in the detection such as “current approach bootstraps the method by starting with the supervised model and then iteratively fine tuning the model with reinforcement learning. Starting without the supervised model would mean that sampling starts without prior information (e.g., random word selections) and the model would take much longer to converge. Instead, we take the supervised model is used at the beginning and convergence is monitored, and a reward signal is designed to achieve the desired objectives”].
Re. claim 9, Bodi further discloses:
further comprising: monitoring the pipeline repository to detect addition of further pipeline definitions; and for each further pipeline definition added to the pipeline repository, generating an endpoint in the evaluation client to a pipeline template corresponding to the further pipeline definition. [Bodi; ¶90, ¶99-¶101 and ¶130].
Re. claim 14, Bodi further discloses:
further comprising: an evaluation result data store configured to store evaluation results generated by the selected evaluation pipeline. [Bodi; ¶138 shows storing such as “any medium that is capable of storing, encoding, or carrying instructions 824 for execution by the machine 800 and that cause the machine 800 to perform any one or more of the techniques of the present disclosure, or that is capable of storing, encoding, or carrying data structures used by or associated with such instructions 824”].
Re. claim 18, Bodi further discloses:
wherein the evaluation request is generated according to a predefined schema and wherein the evaluation request parsing system parses the evaluation request to identify the evaluation parameters in the predefined schema. [Bodi; ¶44, ¶61 and ¶65 shows identifying if the query was natural such as “annotators were asked to identify if the query was “natural”. In some experiments, an average 58% of model-generated queries and 74% of real-user queries were identified as natural. The Inter Annotator Agreement (IAA) was poor (0.04) when the users evaluated model-generated sentences. In comparison, when they evaluated queries entered by real users, IAA was better (0.34) between the three annotators' ratings. Higher IAA and higher percentage of queries identified as “natural” imply that real-user queries are more natural and distinguishable than queries sampled from the pre-trained Seq2SeqNMT mode”].
Re. claim 20, Bodi further discloses:
further comprising: a client computing system configured to generate a user interface with selection actuators and detect actuation of the selection actuators to identify the user-selected parameters and to generate a results user interface to display evaluation results generated by the selected evaluation template. [Bodi; ¶135 shows utilization of user interface and ¶120 shows one example of a user selection through search-query logs].
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating
obviousness or nonobviousness.
Claims 6 is rejected under 35 U.S.C. 103 as being unpatentable over Bodi in view of Oskabe et al (US Patent Application Publication No. 20240078290 - hereinafter Oskabe).
Re. claim 6, Bodi teaches the computer implemented method of claim 1.
Bodi doesn’t teach, Oskabe teaches:
wherein selecting an evaluation template comprises: running a set of heuristics based on the evaluation parameters; and selecting the evaluation template with the set of heuristics. [Osakabe; ¶129-¶131].
It would have been obvious to one of ordinary skill in the art before the effective filing date to include limitation(s) as taught by Oskabe in the system of Bodi, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Claims 10 is rejected under 35 U.S.C. 103 as being unpatentable over Bodi in view of Fisher et al (US Patent Application Publication No. 20220400251 - hereinafter Fisher).
Re. claim 10, Bodi teaches the computer implemented method of claim 1.
Bodi doesn’t teach, Fisher teaches:
wherein triggering the selected evaluation template comprises: scheduling triggering of the evaluation template on a trigger schedule based on trigger criteria; and triggering the evaluation template based on the trigger schedule. [Fisher; ¶35].
It would have been obvious to one of ordinary skill in the art before the effective filing date to include limitation(s) as taught by Fisher in the system of Bodi, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IBRAHIM EL-BATHY whose telephone number is (571)272-7545. The examiner can normally be reached Monday - Friday 9am - 7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Shannon Campbell can be reached at 5712725587. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IBRAHIM N EL-BATHY/Primary Examiner, Art Unit 3626
Prosecution Insights