Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Application is a continuation of PCT Application No. PCT/EP2021/083220, filed on November 26, 2021.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 08/06/2024 is in compliance
with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being
considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Regarding claim 1, in Step 1 of the 101 analyses set forth in MPEP 2106, the claim recites A computer-implemented machine learning (ML) method, the method comprising. A method is one of the four statutory categories.
In Step 2a Prong 1 of the 101 analyses set forth in the MPEP 2106, the examiner has determined that the following limitations recite a process that, under the broadest reasonable interpretation, covers a [ mental process/mathematical concept] but for recitation of generic computer components:
a) computing a labeling matrix by applying a set of labeling functions (LFs), to data points of an unlabeled dataset; (Applying labeling functions are mathematical calculation which falls within the mathematical concept grouping of abstract ideas (MPEP 2106.04(a)(2)). Further A person can mentally compute a label by applying a set of labeling functions to unlabeled datasets by a process of simply evaluating the unlabeled data and making a judgement on how the labeling function should be applied. (MPEP 2106))
b) generating a projected labels matrix by computing, based on the labeling matrix, LFs labels projections to undefined labels; (Applying labeling functions are mathematical calculation which falls within the mathematical concept grouping of abstract ideas (MPEP 2106.04(a)(2)). Further A person can mentally generate a projected label matrix by a process of simply evaluating the labeling matric and label projections and making a judgement on what the projected labels matrix should be.)
c) estimating, for each labeled data point, an uncertainty of a respective label of the each labeled data point based on an output of the LFs and the LFs labels projections; (Applying labeling functions are mathematical calculation which falls within the mathematical concept grouping of abstract ideas (MPEP 2106.04(a)(2)). Further A person can mentally estimate uncertainty of label outputs and label projections by a process of simply evaluating the label outputs and the label projections and making a judgement on what the uncertainty is.)
d) selecting data points depending on the uncertainty estimated for the respective label of the each data point; (Applying labeling functions are mathematical calculation which falls within the mathematical concept grouping of abstract ideas (MPEP 2106.04(a)(2)). Further A person can mentally select a label for each data point by a process of simply evaluating the uncertainty and making a judgement on what the label should be.)
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a [ mental process/mathematical concept] but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. According, the claim “recites” an abstract idea.
In Step 2a Prong 2 of the 101 analyses set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
and e) submitting labeling request for the selected data points to an oracle (Adding insignificant extra-solution activity (mere data output) to the judicial exception (MPEP 2106.05(g))).
and updating the labeling matrix according to responses of the oracle; (Adding insignificant extra-solution activity (mere data gathering) to the judicial exception (MPEP 2106.05(g))).
Since the claim does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In Step 2b of the 101 analyses set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
and e) submitting labeling request for the selected data points to an oracle (Adding insignificant extra-solution activity (mere data output) to the judicial exception (MPEP 2106.05(g)), Furthermore, the additional element is directed to receiving or transmitting data over a network / performing repetitive calculations / electronic recordkeeping / storing and retrieving information in memory / electronically scanning or extracting data from a physical document, which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II).).
and updating the labeling matrix according to responses of the oracle; (Adding insignificant extra-solution activity (mere data gathering) to the judicial exception (MPEP 2106.05(g)), Furthermore, the additional element is directed to receiving or transmitting data over a network / performing repetitive calculations / electronic recordkeeping / storing and retrieving information in memory / electronically scanning or extracting data from a physical document, which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II).).
Claims 14 and 15 are rejected on the same grounds as Claim 1.
Regarding claim 2 it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 2 recites further comprising: iteratively repeating steps b)-e) until the projected labels matrix comprises a number of labels above a configurable threshold number. (In step 2A, prong 2, this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))). In step 2B, generally linking the use of the judicial exception to a particular technological environment is not indicative of significantly more.)
Regarding claim 3 it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 3 recites further comprising: generating probabilistic labels by aggregating labels from the projected labels matrix. (In step 2A, prong 1, this recites a mental process without significantly more. A person can mentally generate probabilistic labels by a process of simply evaluating the aggregated labels judgement on what the probability labels should be. (MPEP 2106).)
Regarding claim 4 it is dependent upon claim 3, and thereby incorporates the limitations of, and corresponding analysis applied to claim 3. Further, claim 4 recites further comprising: training an end classifier with the probabilistic labels after active labeling according to steps b)-e) is completed. (In step 2A prong 2, merely training a generic machine learning operation constitutes “applying” the machine learning operation (MPEP 2106.05(f)). In step 2B, merely applying a generic machine learning operation is not indicative of significantly more.)
Regarding claim 5 it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 5 recites further comprising computing the LFs labels projection to undefined labels by applying ML techniques or heuristics. (In step 2A prong 2, merely training a generic machine learning operation constitutes “applying” the machine learning operation (MPEP 2106.05(f)). In step 2B, merely applying a generic machine learning operation is not indicative of significantly more.)
Regarding claim 6 it is dependent upon claim 5, and thereby incorporates the limitations of, and corresponding analysis applied to claim 5. Further, claim 6 recites wherein a heuristic comprises applying labels based on a number of stochastic encounters between a labeled data point and non-labeled data point by a LF and setting, based thereupon, a value close to the label given to the labeled data point. (In step 2A prong 2, merely training a generic machine learning operation constitutes “applying” the machine learning operation (MPEP 2106.05(f)). In step 2B, merely applying a generic machine learning operation is not indicative of significantly more.)
Regarding claim 7 it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 7 recites wherein the generating the projected labels matrix is performed by calculating probabilities based on data features of non-labeled data points and outputs of the LFs. (In step 2A, prong 1, this recites a mental process without significantly more. A person can mentally calculate probabilities based on data features by a process of simply evaluating the data features of non-labeled data and making a judgement on what the probabilities should be. (MPEP 2106).)
Regarding claim 8 it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 8 recites wherein the generating the projected labels matrix of step b) is performed depending on a distance function between data points. (In step 2A prong 2, this recites mere instructions to apply an exception (MPEP 2106.05(f)). In step 2B, merely describing what calculations to use is not indicative of significantly more.)
Regarding claim 9 it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 9 recites wherein the uncertainty of the respective label of the each labeled data point is estimated by machine learning algorithms, comprising using a decision tree or random forest, or heuristics. (In step 2A prong 2, this recites mere instructions to apply an exception (MPEP 2106.05(f)). In step 2B, merely describing what machine learning algorithm to use is not indicative of significantly more.)
Regarding claim 10 it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 10 recites projecting labels with a confidence estimation for undefined labels after the LFs application using the computed labels from other data points and their features. (In step 2A, prong 2, this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))). In step 2B, generally linking the use of the judicial exception to a particular technological environment is not indicative of significantly more.)
Regarding claim 11 it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 11 recites wherein the selection of data points according to step d) is performed by: ranking the labeled data points according to the estimated uncertainty of the respective labels, and selecting, in each iteration, a predefined number of the highest ranked labeled data points. (A person can mentally rank and select the label data points by a process of simply evaluating the estimated uncertainty and making a judgement on what the ranking should be. (MPEP 2106).)
Regarding claim 12 it is dependent upon claim 11, and thereby incorporates the limitations of, and corresponding analysis applied to claim 11. Further, claim 12 recites using labels that are acquired from the oracle to re-calculate uncertainty estimations; (Applying labeling functions are mathematical calculation which falls within the mathematical concept grouping of abstract ideas (MPEP 2106.04(a)(2)). Further A person can mentally select a label for each data point by a process of simply evaluating the uncertainty and making a judgement on what the label should be.)and update the existing ranking of the labeled data points according to the re-calculated uncertainty estimations. (A person can mentally rank and select the label data points by a process of simply evaluating the estimated uncertainty and making a judgement on what the ranking should be. (MPEP 2106).)
Regarding claim 13 it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 13 recites wherein the set of LFs are configured to provide a confidence of their annotation. (In step 2A, prong 2, this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))). In step 2B, generally linking the use of the judicial exception to a particular technological environment is not indicative of significantly more.)
Regarding claim 16 it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 16 recites wherein the responses of the oracle are generated through usage of data features in an optimization process. (In step 2A, prong 2, this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))). In step 2B, generally linking the use of the judicial exception to a particular technological environment is not indicative of significantly more.)
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-5, 8, and 16 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Chang et al. Pub No.: US 20210201076 A1.
Regarding claim 1 Chang teaches A computer-implemented machine learning (ML) method, the method comprising: a) computing a labeling matrix by applying a set of labeling functions (LFs), to data points of an unlabeled dataset; (Cheng, paragraph 0069, teaches computing a matrix (i.e. labeling matrix) by applying a set of labeling functions.)
b) generating a projected labels matrix by computing, based on the labeling matrix, LFs labels projections to undefined labels; (Cheng, paragraph 0072, teaches the projection of the labeling matrix by a generative model to undefined labels to generate a projected labels matrix.)
c) estimating, for each labeled data point, an uncertainty of a respective label of the each labeled data point based on an output of the LFs and the LFs labels projections; (Cheng, paragraph 0072- 0073, teaches the generation of probabilities for labels (i.e. uncertainties of labels) these probabilities represent how certain the model is that the label assigned to the data is correct.)
d) selecting data points depending on the uncertainty estimated for the respective label of the each data point; (Cheng, paragraph 0072- 0073, teaches filtering out of some of the label based on the probabilities.)
and e) submitting labeling request for the selected data points to an oracle and updating the labeling matrix according to responses of the oracle; (Cheng, paragraph 0078- 0079, teaches the use of a domain expert that manually assigns correct labels and sends the correct labels back to the existing labeling functions that are then updated with the correct labels.)
Claims 14 and 15 are rejected on the same grounds as Claim 1.
Regarding claim 2 Chang teaches The method according to claim 1, further comprising: iteratively repeating steps b)-e) until the projected labels matrix comprises a number of labels above a configurable threshold number. (Chang, paragraph 0070, teaches the iterative training of generative model by the maximizing of the label matrix meaning that the matrix is iteratively repeating the updating process until a threshold maximization value is met.)
Regarding claim 3 Chang teaches The method according to claim 1, further comprising: generating probabilistic labels by aggregating labels from the projected labels matrix. (Chang, paragraphs 0069-0072, teaches the aggregation of labels to create a labels from a labels matrix to produce probabilistic labels.)
Regarding claim 4 Chang teaches The method according to claim 3, further comprising: training an end classifier with the probabilistic labels after active labeling according to steps b)-e) is completed. (Chang, paragraph 0049-0051, 0084, teaches the training of a classifier that is based on the probabilistic labels that are created by the label-producing process.)
Regarding claim 5 Chang teaches The method according to claim l, further comprising computing the LFs labels projection to undefined labels by applying ML techniques or heuristics. (Chang, paragraph 0068-0072, teaches computing labels for undefined labels using a generative model (i.e. machine learning model))
Regarding claim 8 Chang teaches The method according to claim l, wherein the generating the projected labels matrix of step b) is performed depending on a distance function between data points. (Chang, paragraph 0082-0085, teaches the use of distance functions to determine the generating of a label comparison matrix (i.e. projected labels matrix))
Regarding claim 16 Chang teaches The method according to claim 1, wherein the responses of the oracle are generated through usage of data features in an optimization process. (Chang, paragraph 0051, teaches the use of experts (i.e. oracle) to help reduce the effort and time for the tasks (i.e. optimization).)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6-7, and 9-13 are rejected under 35 U.S.C. 103 as being unpatentable over Chang et al. Pub No.: US 20210201076 A1 in view of Quader et al. Pub No.: US 20210209412 A1.
Regarding claim 6 Chang teaches The method according to claim 5,
Chang does not teach wherein a heuristic comprises applying labels based on a number of stochastic encounters between a labeled data point and non-labeled data point by a LF and setting, based thereupon, a value close to the label given to the labeled data point. However, Quader in analogous art teaches this limitation (Quader, paragraph 0037, teaches the use of a heuristic generator module that is configured to use a labeled data set and a heuristic to apply labels to data in the unlabeled dataset in which the heuristic can be stochastic.)
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Quader’s teaching of heuristic based weak supervision learning with Changs’s teaching of learning function based weak supervision learning. The motivation to do so would be to improve the speed and lower the resource usage in approximating the labels by using defined heuristics in the process.
Regarding claim 7 Chang teaches The method according to claim 1,
Chang does not teach wherein the generating the projected labels matrix is performed by calculating probabilities based on data features of non-labeled data points and outputs of the LFs. However, Quader in analogous art teaches this limitation (Quader, paragraph 0043 - 0045, teaches the use of classification models that are based on the heuristics and use the data properties of the non-labeled data’s features along with the heuristic (i.e. label function) to calculate the probability distribution over a set of classes for each unlabeled data point.)
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Quader’s teaching of heuristic based weak supervision learning with Changs’s teaching of learning function based weak supervision learning. The motivation to do so would be to improve the speed and lower the resource usage in approximating the labels by using defined heuristics in the process.
Regarding claim 9 Chang teaches The method according to claim l,
Chang does not teach wherein the uncertainty of the respective label of the each labeled data point is estimated by machine learning algorithms, comprising using a decision tree or random forest, or heuristics. However, Quader in analogous art teaches this limitation (Quader, paragraph 0043 - 0045, teaches the use of classification models that are based on heuristics to create a probability distribution (i.e. uncertainty))
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Quader’s teaching of heuristic based weak supervision learning with Changs’s teaching of learning function based weak supervision learning. The motivation to do so would be to improve the speed and lower the resource usage in approximating the labels by using defined heuristics in the process.
Regarding claim 10 Chang teaches The method according to claim l,
Chang does not teach further comprising: projecting labels with a confidence estimation for undefined labels after the LFs application using the computed labels from other data points and their features. However, Quader in analogous art teaches this limitation (Quader, paragraph 0049-0050, teaches the assigning of confidence levels to labels created by the system where the heuristics disagree on data points and features.)
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Quader’s teaching of heuristic based weak supervision learning with Changs’s teaching of learning function based weak supervision learning. The motivation to do so would be to improve the speed and lower the resource usage in approximating the labels by using defined heuristics in the process.
Regarding claim 11 Chang teaches The method according to claim l,
Chang does not teach wherein the selection of data points according to step d) is performed by: ranking the labeled data points according to the estimated uncertainty of the respective labels, and selecting, in each iteration, a predefined number of the highest ranked labeled data points. However, Quader in analogous art teaches this limitation (Quader, paragraph 0048-0059, teaches the ranking of the labeled data using a confidence score based on the probabilistic labels and determines high confidence and low confidence labeled data where the low confidence labeled data is able to be presented to a user for further consideration.)
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Quader’s teaching of heuristic based weak supervision learning with Changs’s teaching of learning function based weak supervision learning. The motivation to do so would be to improve the speed and lower the resource usage in approximating the labels by using defined heuristics in the process.
Regarding claim 12 Chang teaches The method according to claim l,
Chang does not teach further comprising: using labels that are acquired from the oracle to re-calculate uncertainty estimations; and update the existing ranking of the labeled data points according to the re-calculated uncertainty estimations. However, Quader in analogous art teaches this limitation (Quader, paragraph 0048-0059, teaches the ranking of the labeled data using a confidence score based on the probabilistic labels and determines high confidence and low confidence labeled data where the low confidence labeled data is able to be presented to a user for further consideration. Further it teaches the process being iterative where after the user (i.e. the oracle) assigns new labels the system is able to iteratively restart the process and update the rankings based on the new information.)
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Quader’s teaching of heuristic based weak supervision learning with Changs’s teaching of learning function based weak supervision learning. The motivation to do so would be to improve the speed and lower the resource usage in approximating the labels by using defined heuristics in the process.
Regarding claim 13 Chang teaches The method according to claim l,
Chang does not teach wherein the set of LFs are configured to provide a confidence of their annotation. However, Quader in analogous art teaches this limitation (Quader, paragraph 0049-0050, teaches the assigning of confidence levels to labels created by the system where the heuristics disagree on data points and feature.)
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Quader’s teaching of heuristic based weak supervision learning with Changs’s teaching of learning function based weak supervision learning. The motivation to do so would be to improve the speed and lower the resource usage in approximating the labels by using defined heuristics in the process.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS B LANE whose telephone number is (571)272-1872. The examiner can normally be reached M-Th: 7:20am-5:20pm; F: Out of Office.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MARIELA REYES can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS BERNARD LANE/Examiner, Art Unit 2142
/HAIMEI JIANG/Primary Examiner, Art Unit 2142
Prosecution Insights