Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Detailed Action
The following action is in response to the communication(s) received on 12/16/2022.
As of the claims filed 12/16/2022:
Claims 1-21 are pending.
Claims 1, 8, and 15 are independent claims.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 03/01/2023 and 07/21/2023 were filed in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 11, 12, 18, and 19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 11 and 18 recite the limitation "the second ML model". There is insufficient antecedent basis for this limitation in the claim.
Claim 11 and 18 recite the limitation "the second inference environment". There is insufficient antecedent basis for this limitation in the claim.
Claims 12 and 19 are rejected by virtue of dependency to their respective parent claims.
For examination purposes, Claims 11 and 18 are read as dependent to claims 10 and 17, respectively.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites method, thus a process, one of the four statutory categories of patentable subject matter (Step 1). However, Claim 1 further recites:
defining a use case type at a labeling platform, which is an evaluation or judgement that can be performed in the human mind;
associating, at the labeling platform, a plurality of ML platforms with the use case type, which is an evaluation or judgement that can be performed in the human mind;
mapping the ML model inference configuration to a first inference environment to configure the first inference environment to use a first ML model, the first inference environment provided by a first ML platform from the plurality of ML platforms, which is an evaluation or judgement that can be performed in the human mind;
Thus, the claim recites an abstract idea under Step 2A Prong 1.
Under Step 2A Prong 2, the claim does not include any additional elements which integrate the abstract idea into a practical application, since the additional elements consist of:
providing a set of adapters to map an ML platform agnostic format to a plurality of ML platform specific formats, which is merely an insignificant extra-solution activity of data gathering, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application;
receiving, at the labeling platform, a use case associated with the use case type, the use case comprising an ML model inference configuration, wherein the ML model inference configuration is ML platform agnostic, which is merely an insignificant extra-solution activity of data gathering, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application;
computer-implemented, as the performance of an abstract idea on a computer is not more than instructions to "apply it" on a computer, which by MPEP 2106.05(f) cannot integrate an abstract idea into a practical application.
and routing labeling requests to the first inference environment for labeling by the first ML model, which is merely an insignificant extra-solution activity of data transfer, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application;
Thus, the claim is directed towards an abstract idea.
Further, the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself, because implementation on a computer (MPEP 2106.05(f)), and the activity of data gathering/transfer (MPEP 2106.05(g)) cannot provide significantly more, as storing and retrieving information in memory is well understood, routine, and conventional (MPEP 2106.05(d)(II)(iv)), and the activity of data transfer (MPEP 2106.05(g)) cannot provide significantly more, as receiving or transmitting data over a network is well understood, routine, and conventional (MPEP 2106.05(d)(II)(i), buySAFE, Inc. v. Google, Inc), and the combination of additional elements does not provide an inventive concept. Thus, the claim is ineligible.
Claim 2, dependent upon Claim 1, further recites
the ML model inference configuration comprises a declaration of an ML algorithm, which is merely a detail of an abstract idea (mapping the ML model inference configuration to a first inference environment…);
the first inference environment is selected from among several that support the first ML model, which is an evaluation or judgement that can be performed in the human mind;
Thus, the claim recites an abstract idea under Step 2A Prong 1.
Under Step 2A Prong 2 and 2B, the claim does not recite any new additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself. Thus, the claim is ineligible.
Claim 3, dependent upon Claim 1, further recites
mapping the ML model inference configuration to a second inference environment to configure the second inference environment to use a second ML model, which is an evaluation or judgement that can be performed in the human mind;
routing labeling requests to the second inference environment for labeling by the second ML model, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1.
Under Step 2A Prong 2 and 2B, the claim does not recite any new additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself. Thus, the claim is ineligible.
Claim 4, dependent upon Claim 3, further recites
based on a determination that the second ML model is more accurate than the first ML model for the use case, switching using the first inference environment to the second inference environment for inference related to the use case, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1.
Under Step 2A Prong 2 and 2B, the claim does not recite any new additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself. Thus, the claim is ineligible.
Claim 5, dependent upon Claim 4, further recites
the second inference environment is provided by a second ML platform of the plurality of ML platforms, which is merely a detail of an abstract idea (mapping the ML model inference configuration to a second inference environment…).
Thus, the claim recites an abstract idea under Step 2A Prong 1.
Under Step 2A Prong 2 and 2B, the claim does not recite any new additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself. Thus, the claim is ineligible.
Claim 6, dependent upon Claim 1, further recites
the ML model inference configuration characterizes an expected label space for inferences, which is merely a detail of an abstract idea (mapping the ML model inference configuration to a first inference environment…).
Thus, the claim recites an abstract idea under Step 2A Prong 1.
Under Step 2A Prong 2 and 2B, the claim does not recite any new additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself. Thus, the claim is ineligible.
Claim 7, dependent upon Claim 1, further recites
the ML model inference configuration comprises one or more of: an input conditioning configuration, a target deconditioning configuration, a request pipe configuration, a result pipe configuration, or a target conditioning configuration, which is merely a detail of an abstract idea (mapping the ML model inference configuration to a first inference environment…).
Thus, the claim recites an abstract idea under Step 2A Prong 1.
Under Step 2A Prong 2 and 2B, the claim does not recite any new additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself. Thus, the claim is ineligible.
Claims 8-14 recite A computer program product, thus an article of manufacture, one of the four statutory categories of patentable subject matter. However, Claims 8-14 recite the computer program product comprising a non-transitory, computer-readable medium having stored thereon a set of computer executable instructions, the set of computer- executable instructions comprising instructions perform precisely the abstract ideas and additional elements of Claims 1-7, respectively. Therefore, Step 2A Prong 1 analysis remains the same. As for Step 2A Prong 2 and Step 2B: performance on a computer cannot integrate an abstract idea into a practical application (Step 2A Prong 2) nor provide significantly more than the abstract idea itself (Step 2B) (MPEP 2106.05(f)), Claims 8-14 are rejected as subject-matter ineligible for reasons set forth in the rejections of Claims 1-7, respectively.
Claims 15-21 recite A labeling platform, thus a machine, one of the four statutory categories of patentable subject matter. However, Claims 15-21 recite comprising… a processor; a non-transitory computer readable medium having stored thereon a set of computer executable instructions, the set of computer-executable instructions comprising instructions perform precisely the abstract ideas and additional elements of Claims 1-7, respectively. Therefore, Step 2A Prong 1 analysis remains the same. As for Step 2A Prong 2 and Step 2B: performance on a computer cannot integrate an abstract idea into a practical application (Step 2A Prong 2) nor provide significantly more than the abstract idea itself (Step 2B) (MPEP 2106.05(f)), Claims 15-21 are rejected as subject-matter ineligible for reasons set forth in the rejections of Claims 1-7, respectively.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-21 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kotthoff et al., “Algorithm Selection for Combinatorial Search Problems: A Survey” (hereinafter Kotthoff).
Regarding Claim 1, Kotthoff teaches:
A computer-implemented method for ML platform-agnostic machine learning (ML) inference, the method comprising: defining a use case type at a labeling platform; (Kotthoff [fig.2]
PNG
media_image1.png
482
948
media_image1.png
Greyscale
) (Note: x corresponds to the use case type; the algorithm selection problem model corresponds to the use case type)
associating, at the labeling platform, a plurality of ML platforms with the use case type; (Kotthoff [fig.2] Algorithm space)
providing a set of adapters to map an ML platform agnostic format to a plurality of ML platform specific formats; (Kotthoff [p.2 left ¶2] given a space of instances and a space of algorithms, map each instance-algorithm pair to its performance)
receiving, at the labeling platform, a use case associated with the use case type, the use case comprising an ML model inference configuration, wherein the ML model inference configuration is ML platform agnostic; (Koffhoff [p.2 left ¶2] This mapping can then be used to select the best algorithm for a given instance.)
mapping the ML model inference configuration to a first inference environment to configure the first inference environment to use a first ML model, the first inference environment provided by a first ML platform from the plurality of ML platforms; (Koffhoff [p.2 left ¶2] …given a space of instances and a space of algorithms, map each instance-algorithm pair to its performance. This mapping can then be used to select the best algorithm for a given instance.) (Note: each instance corresponds to each model inference configuration; the selected instance-algorithm pair from the space of algorithms for the given instance corresponds to the first inference environment from the plurality; the selected best algorithm from the space of algorithms corresponds to the first ML platform from the plurality of ML platforms)
and routing labeling requests to the first inference environment for labeling by the first ML model. (Kotthoff [abstract] The algorithm selection problem is concerned with selecting the best algorithm to solve a given problem instance on a case-by-case basis.) (Note: solving the given problem instance with the selected best algorithm corresponds to routing the labeling requests to the selected algorithm (first inference environment) for labeling.)
Regarding Claim 2, Kotthoff respectively teaches and incorporates the claimed limitations and rejections of Claim 1. Kotthoff further teaches:
The computer-implemented method of claim 1, wherein the ML model inference configuration comprises a declaration of an ML algorithm and wherein the first inference environment is selected from among several that support the first ML model. (Kotthoff [p.2 left ¶2] given a space of instances and a space of algorithms, map each instance-algorithm pair to its performance. This mapping can then be used to select the best algorithm for a given instance)
Regarding Claim 6, Kotthoff respectively teaches and incorporates the claimed limitations and rejections of Claim 1. Kotthoff further teaches:
The computer-implemented method of claim 1, wherein the ML model inference configuration characterizes an expected label space for inferences. (Kotthoff [fig.2]
PNG
media_image1.png
482
948
media_image1.png
Greyscale
) (Note: the prediction corresponds to the expected label space)
Regarding Claim 7, Kotthoff respectively teaches and incorporates the claimed limitations and rejections of Claim 1. Kotthoff further teaches:
The computer-implemented method of claim 1, wherein the ML model inference configuration comprises one or more of: an input conditioning configuration, a target deconditioning configuration, a request pipe configuration, a result pipe configuration, or a target conditioning configuration. (Kotthoff [fig.2]
PNG
media_image1.png
482
948
media_image1.png
Greyscale
) (Note: extracting features in the feature extraction phase corresponds to conditioning inputs and thus input conditioning configuration)
Claims 9, 13, and 14, dependent on Claim 8, also recite the system configured to perform precisely the methods of Claims 2, 6, and 7, respectively. Thus, Claims 9, 13, and 14,are rejected for reasons set forth in Claims 2-7, respectively.
Independent Claim 15 recites A labeling platform comprising… a processor; a non-transitory computer readable medium having stored thereon a set of computer executable instructions, the set of computer-executable instructions comprising instructions (Kotthoff [p.11 left last ¶] Another way to get started is to use one of the systems which are available as open source on the web. Several versions of SATzilla, along with data sets and documentation, can be downloaded.) to perform precisely the methods of Claim 1. Thus, Claim 15 is rejected for reasons set forth in Claim 1.
Claims 16, 20, and 21, dependent on Claim 8, also recite the system configured to perform precisely the methods of Claims 2, 6, and 7, respectively. Thus, Claims 16, 20, and 21 are rejected for reasons set forth in Claims 2, 6, and 7, respectively.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 3-5, 10-12, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Kotthoff in view of Matricon et al., "Statistical Comparison of Algorithm Performance Through Instance Selection" (hereinafter Matricon).
Regarding Claim 3, Kotthoff respectively teaches and incorporates the claimed limitations and rejections of Claim 1. Kotthoff does not teach, but Matricon further teaches:
The computer-implemented method of claim 1, further comprising: mapping the ML model inference configuration to a second inference environment to configure the second inference environment to use a second ML model; (Matricon [p.2 ¶3] We introduce the per-set efficient algorithm selection problem (PSEAS): Given two algorithms, an incumbent Ainc and a challenger Ach, and a set of problem instances I, how can we minimise the computational resources (here: CPU time) required to determine, at a required level of confidence, whether Ach performs better than Ainc on I?
[p.5, Algorithm 1]
PNG
media_image2.png
329
856
media_image2.png
Greyscale
) (Note: the challenger Ach on instance I (i.e. the challenger instance-algorithm pair) corresponds to the second inference environment; the challenger Ach corresponds to the second ML model.)
Matricon and Kotthoff are analogous to the present invention because both are from the same field of endeavor of model selection methods. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement the challenger-incumbent comparison method from Matricon into Kotthoff’s algorithm selection method. The motivation would be to “obtain a probabilistic statement on which algorithm performs best, trading off between the computational cost of running algorithms and the confidence in the result.” (Matricon [abstract]).
Kotthoff, via Kotthoff/Matricon, further teaches:
and routing labeling requests to the second inference environment for labeling by the second ML model. (Kotthoff [abstract] The algorithm selection problem is concerned with selecting the best algorithm to solve a given problem instance on a case-by-case basis.) (Note: solving the given problem instance with the selected best algorithm corresponds to routing the labeling requests to the selected algorithm (first inference environment) for labeling.)
Regarding Claim 4, Kotthoff/Matricon respectively teaches and incorporates the claimed limitations and rejections of Claim 3. Matricon, via Kotthof/Matricon, further teaches:
The computer-implemented method of claim 3, further comprising: based on a determination that the second ML model is more accurate than the first ML model for the use case, switching using the first inference environment to the second inference environment for inference related to the use case. (Matricon [p.5, Algorithm 1]
PNG
media_image2.png
329
856
media_image2.png
Greyscale
) (Note: returning Ach from running Algorithm 1 corresponds to switching from the first inference environment to the second inference environment for inference related to the use case)
Regarding Claim 5, Kotthoff/Matricon respectively teaches and incorporates the claimed limitations and rejections of Claim 4. Kotthoff, via Kotthoff/Matricon, further teaches:
The computer-implemented method of claim 4, wherein the second inference environment is provided by a second ML platform of the plurality of ML platforms. (Kotthoff [fig.2] Algorithm space
[p.8 left bottom 2nd ¶] The most recent version of SATzilla… uses models for pairs of algorithms to predict one which is going to have better performance.) (Note: the selected algorithm corresponds to the second inference environment; the selected algorithm in the algorithm space corresponds to the second ML platform)
Claims 10-12, dependent on Claim 8, also recite the system configured to perform precisely the methods of Claims 3-5, respectively. Thus, Claims 10-12 are rejected for reasons set forth in Claims 3-5, respectively.
Claims 17-19, dependent on Claim 15, also recite the system configured to perform precisely the methods of Claims 3-5, respectively. Thus, Claims 17-19 are rejected for reasons set forth in Claims 3-5, respectively.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSEP HAN whose telephone number is (703)756-1346. The examiner can normally be reached Mon-Fri 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached on (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.H./Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122