DETAILED ACTION
The present application, filed on 7/12/2023 is being examined under the AIA first inventor to file provisions.
The following is a FINAL Office Action in response to Applicant’s amendments filed on 8/27/2026.
a. Claims 1-2, 5, 8, 11-12, 14, 16, 18 are amended
Overall, claims 1-20 are pending and have been considered below.
Claim Rejections - 35 USC § 101
35 USC 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 USC 101 because the claimed invention is not directed to patent eligible subject matter. The claimed matter is directed to a judicial exception, i.e. an abstract idea, not integrated into a practical application, and without significantly more.
Per Step 1 of the multi-step eligibility analysis, claims 1-10 are directed to a computer implemented method, claims 11-13 are directed to a computer implemented method, claims 14 are directed to computer executable instructions stored on a non-transitory storage medium, and claims 15-20 are directed to a system.
Thus, on its face, each independent claim and the associated dependent claims are directed to a statutory category of invention.
[INDEPENDENT CLAIMS]
Per Step 2A.1. Independent claim 1, (which is representative of independent claims 14) is rejected under 35 USC 101 because the independent claim is directed to an abstract idea, a judicial exception, without reciting additional elements that integrate the judicial exception into a practical application.
The limitations of the independent claim 1 (which is representative of independent claims 14) recite an abstract idea, shown in bold below:
[A] A processor-implemented method comprising:
[B] receiving a training dataset comprising image data and labels corresponding to defect codes, the image data comprising a scanning electron microscopy (SEM) image of a semiconductor obtained by an image sensor;
[C] determining a confidence comprising a first probability of each of the labels being correct and a second probability of each of the labels being incorrect;
[D] classifying, based on a distribution of the confidences, data samples in the training dataset into a first data sample comprising an incorrect label and a second data sample comprising a correct label;
[E] training a first model to predict the confidences of labels for the data samples, including
[F] using a corrected data sample obtained by correcting the incorrect label based on a corresponding confidence detected by the first model and an estimated corrected label generated by a second model;
[G] training the second model to estimate a correct other label corresponding to another incorrect label detected based on a corresponding confidence generated by the first model with respect to the other incorrect label; and
[H] automatically correcting the other incorrect label with the estimated correct other label to generate corrected data samples;
[I] constructing, using the corrected data samples, data for training a semiconductor defect classification system.
Independent claim 1 (which is representative of independent claims 14) recites: determining a confidence into data samples correctness and classifying the data samples based on the determined label ([C], [D]); training a first model to predict the confidence in the data labels and using a corrected data sample and training a second model ([F], [G]); correcting the incorrect labels and constructing data for training a data classification system ([H], [I]), which, based on the claim language and in view of the application disclosure, represents a process aimed at: assessing labels, correcting them, classifying them and automating labeling.
This is a combination that, under its broadest reasonable interpretation, covers performance of limitations expressing following rules or instructions. These fall under the Certain Methods of Organizing Human Activity, i.e., Managing Personal Behavior or Relationships, or Interactions Between People grouping of abstract ideas (see MPEP 2106.04(a)(2)).
Accordingly, it is concluded that independent claim 1 (which is representative of independent claims 14) recites an abstract idea that corresponds to a judicial exception.
In addition, or alternatively, this is a combination that, under its broadest reasonable interpretation, covers reasonable performance of limitations expressing observation, judgement in the human mind. Nothing in the claim elements precludes the steps from being practically performed in the human mind. For example, the step “determining a confidence”, as drafted in the context of this claim, encompasses the user manually or mentally assessing the confidence of data, without physical aid. Further, the step “classifying data samples”, as drafted in the context of this claim, encompasses the user manually or mentally classifying data samples, without physical aid. Further, the step “training a first model”, as drafted in the context of this claim, encompasses the user manually or mentally training a model, without physical aid. Further, the step “correcting an incorrect label”, as drafted in the context of this claim, encompasses the user manually or mentally correcting labels, without physical aid. Further, the step “training a second model”, as drafted in the context of this claim, encompasses the user manually or mentally training a model, without physical aid. Further, the step “correcting other incorrect labels”, as drafted in the context of this claim, encompasses the user manually or mentally correcting labels, without physical aid. Further, the step “constructing training data”, as drafted in the context of this claim, encompasses the user manually or mentally creating training data, without physical aid. These limitations fall under the Mental Processes, i.e., Concepts Performed in the Human Mind grouping of abstract ideas (see MPEP 2106.04(a)(2)). The use of a physical aid would not negate the mental nature of this limitation (see MPEP 2106.04(a)(2) iii B)
Accordingly, it is concluded that independent claim 1 (which is representative of independent claims 14) recites an abstract idea that corresponds to a judicial exception.
[INDEPENDENT CLAIMS – Additional Elements]
Per Step 2A.2. The identified abstract idea is not integrated into a practical application because the additional elements in the independent claims only amount to instructions to apply the judicial exception to a computer, or are a general link to a technological environment (see MPEP 2106.05(f); MPEP 2106.05(h)).
For example, the added elements “storage medium” recite computing elements at a high level of generality, generally linking the use of a judicial exception to a particular technological environment (see MPEP 2106.05(h)), or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
These additional elements of the independent claims do not preclude from carrying out the identified abstract idea assessing labels, correcting them, classifying them and automating labeling, and do not serve to integrate the identified abstract idea into a practical application.
The additional elements in the independent claims, shown not bolded above, recite: receiving a training dataset ([B]). When considered individually, they amount to nothing more than receiving data, processing data, storing results or transmitting data that serves merely to implement the abstract idea using computing components for performing computer functions (corresponding to the words “apply it” or an equivalent), or merely uses a computer as a tool to perform the identified abstract idea. Thus, it is concluded that these claim elements do not integrate the identified abstract idea (assessing labels, correcting them, classifying them and automating labeling) into a practical application (see MPEP 2106.05(f)(2)).
Therefore, the additional claim elements of independent claim 1, (which is representative of independent claims 14), evaluated individually, as well as a whole, as an ordered combination, do not integrate the identified abstract idea into a practical application and the claims are directed to the recited judicial exception.
Per Step 2B. Independent claim 1 (which is representative of claims independent 14) does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when the independent claim is reevaluated as a whole, as an ordered combination under the considerations of Step 2B, the outcome is the same like under Step 2A.2.
Overall, it is concluded that independent claims 1, 14 are deemed ineligible.
Per Step 2A.1. Independent claim 11, is rejected under 35 USC 101 because the independent claim is directed to an abstract idea, a judicial exception, without reciting additional elements that integrate the judicial exception into a practical application.
The limitations of the independent claim 11 recite an abstract idea, shown in bold below:
[A] A processor-implemented automatic labeling method;
[B] receiving a data sample comprising input data and a label corresponding to the input data, the input data comprising a scanning electron microscopy (SEM) image of a semiconductor obtained by an image sensor;
[C] determining a confidence of the label, comprising a first probability of the label being correct and a second probability of the label being incorrect, by applying the data sample to a first model;
[D] detecting whether the label is an incorrect label based on the determined confidence,
[E] wherein the first model comprises a first neural network that is trained to detect the incorrect label comprised in the data sample based on the confidence of the label; and
[F] generating, in response to the label being detected as the incorrect label, a corrected data sample for constructing data for training a semiconductor defect classification system.
Independent claim 11 recites: determining a confidence of a label and identifying the label correctness ([C], [D]); generating a corrected data sample ([F]), which, based on the claim language and in view of the application disclosure, represents a process aimed at: assessing labels, correcting them, classifying them and automating labeling.
This is a combination that, under its broadest reasonable interpretation, covers performance of limitations expressing following rules or instructions. These fall under the Certain Methods of Organizing Human Activity, i.e., Managing Personal Behavior or Relationships, or Interactions Between People grouping of abstract ideas (see MPEP 2106.04(a)(2)).
Accordingly, it is concluded that independent claim 11 recites an abstract idea that corresponds to a judicial exception.
In addition, or alternatively, this is a combination that, under its broadest reasonable interpretation, covers reasonable performance of limitations expressing observation, judgement in the human mind. Nothing in the claim elements precludes the steps from being practically performed in the human mind. For example, the step “determining the confidence of a label”, as drafted in the context of this claim, encompasses the user manually or mentally assessing the confidence level of label, without physical aid. Further, the step “determining the correctness of a label”, as drafted in the context of this claim, encompasses the user manually or mentally determining the correctness of a label, without physical aid. Further, the step “generating a corrected data sample for training”, as drafted in the context of this claim, encompasses the user manually or mentally creating a corrected data sample, without physical aid. These limitations fall under the Mental Processes, i.e., Concepts Performed in the Human Mind grouping of abstract ideas (see MPEP 2106.04(a)(2)). The use of a physical aid would not negate the mental nature of this limitation (see MPEP 2106.04(a)(2) iii B)
Accordingly, it is concluded that independent claim 11 recites an abstract idea that corresponds to a judicial exception.
[INDEPENDENT CLAIMS – Additional Elements]
Per Step 2A.2. The identified abstract idea is not integrated into a practical application because the additional elements in the independent claims only amount to instructions to apply the judicial exception to a computer, or are a general link to a technological environment (see MPEP 2106.05(f); MPEP 2106.05(h)).
For example, the additional elements “wherein the first model comprises a first neural network that is trained to detect the incorrect label comprised in the data sample based on the confidence of the label” as applied to the first model, are nothing more than (a) descriptive limitations of claim elements, such as describing the nature, structure and/or content of other claim elements, or (b) general links to the computing environment, which amount to instructions to “apply it,” or equivalent (MPEP 2106.05(f)).
These additional elements of the independent claims do not preclude from carrying out the identified abstract idea assessing labels, correcting them, classifying them and automating labeling, and do not serve to integrate the identified abstract idea into a practical application.
The additional elements in the independent claims, shown not bolded above, recite: receiving data samples ([B]). When considered individually, they amount to nothing more than receiving data, processing data, storing results or transmitting data that serves merely to implement the abstract idea using computing components for performing computer functions (corresponding to the words “apply it” or an equivalent), or merely uses a computer as a tool to perform the identified abstract idea. Thus, it is concluded that these claim elements do not integrate the identified abstract idea (assessing labels, correcting them, classifying them and automating labeling) into a practical application (see MPEP 2106.05(f)(2)).
Therefore, the additional claim elements of independent claim 11, evaluated individually, as well as a whole, as an ordered combination, do not integrate the identified abstract idea into a practical application and the claims are directed to the recited judicial exception.
Per Step 2B. Independent claim 11 does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when the independent claim is reevaluated as a whole, as an ordered combination under the considerations of Step 2B, the outcome is the same like under Step 2A.2.
Overall, it is concluded that independent claims 11 are deemed ineligible.
Per Step 2A.1. Independent claim 15 is rejected under 35 USC 101 because the independent claim is directed to an abstract idea, a judicial exception, without reciting additional elements that integrate the judicial exception into a practical application.
The limitations of the independent claim 15 recite an abstract idea, shown in bold below:
[A] An electronic device, comprising: a communication system; and a processor
[B] iteratively train a first model to detect incorrect labels in the training dataset and a second model to estimate correct labels corresponding to the incorrect labels, and
[C] generate a data sample in which an incorrect label is corrected using at least one of the first model or the second model,
[D] wherein the training dataset comprises image data comprising a scanning electron microscopy (SEM) image of a semiconductor obtained by an image sensor and labels corresponding to defect codes, and
wherein the processor is further configured to:
[E] determine a confidence comprising a first probability of each of the labels being correct and a second probability of each of the labels being incorrect;
[F] classify, based on a distribution of the confidences, the data samples into a first data sample comprising an incorrect label and a second data sample comprising a correct label;
[G] train the first model to predict the confidences, including using the corrected data sample generated by correcting the incorrect label based on a corresponding confidence detected by the first model and an estimated corrected label generated by the second model;
[H] train the second model to estimate correct labels for the data samples, including estimating a corrected other label corresponding to another incorrect label detected based on a corresponding confidence generated by the first model with respect to the other incorrect and
[I] automatically correct the other incorrect label with the estimated correct other label to generate corrected data samples used for constructing data for training a semiconductor defect classification system.
Independent claim 15 recites: train a first model and generate a data sample ([B], [C]); determine a confidence level and classify the data sample ([E], [F]); train a first model and train a second model ([G], [H]); and correct incorrect labels ([I]), which, based on the claim language and in view of the application disclosure, represents a process aimed at: assessing labels, correcting them, classifying them and automating labeling.
This is a combination that, under its broadest reasonable interpretation, covers performance of limitations expressing following rules or instructions. These fall under the Certain Methods of Organizing Human Activity, i.e., Managing Personal Behavior or Relationships, or Interactions Between People grouping of abstract ideas (see MPEP 2106.04(a)(2)).
Accordingly, it is concluded that independent claim 15 recites an abstract idea that corresponds to a judicial exception.
In addition, or alternatively, this is a combination that, under its broadest reasonable interpretation, covers reasonable performance of limitations expressing observation, judgement in the human mind. Nothing in the claim elements precludes the steps from being practically performed in the human mind. For example, the step “iteratively train a first model to detect incorrect labels in the training dataset and a second model”, as drafted in the context of this claim, encompasses the user manually or mentally training a model, without physical aid. Further, the step “generate a data sample in which an incorrect label is corrected”, as drafted in the context of this claim, encompasses the user manually or mentally generating a data sample, without physical aid. Further, the step “determine a confidence comprising a first probability of each of the labels being correct and a second probability of each of the labels being incorrect”, as drafted in the context of this claim, encompasses the user manually or mentally determining a confidence in the data sample, without physical aid. Further, the step “classify the data samples into a first data sample comprising an incorrect label and a second data sample comprising a correct label”, as drafted in the context of this claim, encompasses the user manually or mentally classifying the data sample, without physical aid. Further, the step “train the first model to predict the confidences”, as drafted in the context of this claim, encompasses the user manually or mentally training a first model, without physical aid. Further, the step “train the second model to estimate correct labels for the data samples”, as drafted in the context of this claim, encompasses the user manually or mentally training a second model, without physical aid. Further, the step “automatically correct the other incorrect label with the estimated correct other label”, as drafted in the context of this claim, encompasses the user manually or mentally correcting incorrect labels, without physical aid. These limitations fall under the Mental Processes, i.e., Concepts Performed in the Human Mind grouping of abstract ideas (see MPEP 2106.04(a)(2)). The use of a physical aid would not negate the mental nature of this limitation (see MPEP 2106.04(a)(2) iii B)
Accordingly, it is concluded that independent claim 15 recites an abstract idea that corresponds to a judicial exception.
[INDEPENDENT CLAIMS – Additional Elements]
Per Step 2A.2. The identified abstract idea is not integrated into a practical application because the additional elements in the independent claims only amount to instructions to apply the judicial exception to a computer, or are a general link to a technological environment (see MPEP 2106.05(f); MPEP 2106.05(h)).
For example, the added elements “communication system,” “processor” recite computing elements at a high level of generality, generally linking the use of a judicial exception to a particular technological environment (see MPEP 2106.05(h)), or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). Further, the additional elements “wherein the training dataset comprises image data comprising a scanning electron microscopy (SEM) image of a semiconductor obtained by an image sensor and labels corresponding to defect codes” as applied to the training dataset, are nothing more than (a) descriptive limitations of claim elements, such as describing the nature, structure and/or content of other claim elements, or (b) general links to the computing environment, which amount to instructions to “apply it,” or equivalent (MPEP 2106.05(f)).
These additional elements of the independent claims do not preclude from carrying out the identified abstract idea assessing labels, correcting them, classifying them and automating labeling, and do not serve to integrate the identified abstract idea into a practical application.
Therefore, the additional claim elements of independent claim 15, evaluated individually, as well as a whole, as an ordered combination, do not integrate the identified abstract idea into a practical application and the claims are directed to the recited judicial exception.
Per Step 2B. Independent claim 15 does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when the independent claim is reevaluated as a whole, as an ordered combination under the considerations of Step 2B, the outcome is the same like under Step 2A.2.
Overall, it is concluded that independent claims 15 are deemed ineligible.
[DEPENDENT CLAIMS]
Dependent claim 2, which is representative of dependent claims 16, recites:
wherein the respective trainings of the first and second models are iterative trainings that, based on the confidences of the labels in the training dataset,
iteratively trains the first model to detect incorrect labels and the second model to estimate the correct labels;
the iterative training further comprises:
training the first model by updating first parameters of the first model, to predict confidence, using the corrected data samples obtained by correcting the incorrect labels in the first data sample.
When considered individually, these added claim elements further elaborate on the abstract idea identified in the independent claims, because the dependent claim continues to recite the identified abstract idea: assessing labels, correcting them, classifying them and automating labeling. The elements in this dependent claim are comparable to performance of limitations expressing mathematical concepts like mathematical relationships, mathematical calculations. These fall under Mathematical Concepts. i.e., mathematical relationships, mathematical formulas or equations, or mathematical calculations grouping of abstract ideas (see MPEP 2106.04(a)(2) I).
The dependent claim elements have the same relationship to the underlying abstract idea (assessing labels, correcting them, classifying them and automating labeling) as outlined in the independent claims analysis above. Thus, the dependent claim elements are not directed to any specific improvements of the independent claims and do not practically or significantly alter how the identified abstract idea would be performed. When considered as a whole, as an ordered combination, the dependent claim further elaborates on the previously identified abstract idea (assessing labels, correcting them, classifying them and automating labeling).
Therefore, dependent claim 2 (which is representative of dependent claims 16) is deemed ineligible.
Dependent claim 3 recites:
sampling a second data sample comprising the correct labels based on a Bernoulli distribution.
When considered individually, these added claim elements further elaborate on the abstract idea identified in the independent claims, because the dependent claim continues to recite the identified abstract idea: assessing labels, correcting them, classifying them and automating labeling. The elements in this dependent claim are comparable to “receiving or transmitting data over a network, e.g., using the Internet to gather or provide data”, which has been recognized by a controlling court as "well-understood, routine and conventional computing functions" when claimed generically as they are in these dependent claims. Thus, it is concluded that these claim elements do not integrate the identified abstract idea (assessing labels, correcting them, classifying them and automating labeling) into a practical application (see MPEP 2106.05(d) II)).
The dependent claim elements have the same relationship to the underlying abstract idea (assessing labels, correcting them, classifying them and automating labeling) as outlined in the independent claims analysis above. Thus, the dependent claim elements are not directed to any specific improvements of the independent claims and do not practically or significantly alter how the identified abstract idea would be performed. When considered as a whole, as an ordered combination, the dependent claim further elaborates on the previously identified abstract idea (assessing labels, correcting them, classifying them and automating labeling).
Therefore, dependent claim 3 is deemed ineligible.
Dependent claim 4, which is representative of dependent claims 17, recites:
updating the first parameters of the first model based on a maximum likelihood corresponding to the corrected data samples.
When considered individually, these added claim elements further elaborate on the abstract idea identified in the independent claims, because the dependent claim continues to recite the identified abstract idea: assessing labels, correcting them, classifying them and automating labeling. The elements in this dependent claim are comparable to receiving/transmitting data, processing data, storing results or transmitting data that serves merely to implement the abstract idea using computing components for performing computer functions (corresponding to the words “apply it” or an equivalent), or merely uses a computer as a tool to perform the identified abstract idea. Thus, it is concluded that these claim elements do not integrate the identified abstract idea (assessing labels, correcting them, classifying them and automating labeling) into a practical application (see MPEP 2106.05(f)(2)).
The dependent claim elements have the same relationship to the underlying abstract idea (assessing labels, correcting them, classifying them and automating labeling) as outlined in the independent claims analysis above. Thus, the dependent claim elements are not directed to any specific improvements of the independent claims and do not practically or significantly alter how the identified abstract idea would be performed. When considered as a whole, as an ordered combination, the dependent claim further elaborates on the previously identified abstract idea (assessing labels, correcting them, classifying them and automating labeling).
Therefore, dependent claim 4 (which is representative of dependent claims 17) is deemed ineligible.
Dependent claim 5, which is representative of dependent claims 18, recites:
determining a regularization penalty for the confidences; and
training the first model by applying tie regularization penalty for the confidences to the updated first parameters.
When considered individually, these added claim elements further elaborate on the abstract idea identified in the independent claims, because the dependent claim continues to recite the identified abstract idea: assessing labels, correcting them, classifying them and automating labeling. The elements in this dependent claim are comparable to performance of limitations expressing mathematical concepts like mathematical relationships, mathematical calculations. These fall under Mathematical Concepts. i.e., mathematical relationships, mathematical formulas or equations, or mathematical calculations grouping of abstract ideas (see MPEP 2106.04(a)(2) I).
The dependent claim elements have the same relationship to the underlying abstract idea (assessing labels, correcting them, classifying them and automating labeling) as outlined in the independent claims analysis above. Thus, the dependent claim elements are not directed to any specific improvements of the independent claims and do not practically or significantly alter how the identified abstract idea would be performed. When considered as a whole, as an ordered combination, the dependent claim further elaborates on the previously identified abstract idea (assessing labels, correcting them, classifying them and automating labeling).
Therefore, dependent claim 5 (which is representative of dependent claims 18) is deemed ineligible.
Dependent claim 6, which is representative of dependent claims 19, recites:
determining initial parameter values of the first model based on a calculated cross-entropy loss.
When considered individually, these added claim elements further elaborate on the abstract idea identified in the independent claims, because the dependent claim continues to recite the identified abstract idea: assessing labels, correcting them, classifying them and automating labeling. The elements in this dependent claim are comparable to performance of limitations expressing mathematical concepts like mathematical relationships, mathematical calculations. These fall under Mathematical Concepts. i.e., mathematical relationships, mathematical formulas or equations, or mathematical calculations grouping of abstract ideas (see MPEP 2106.04(a)(2) I).
The dependent claim elements have the same relationship to the underlying abstract idea (assessing labels, correcting them, classifying them and automating labeling) as outlined in the independent claims analysis above. Thus, the dependent claim elements are not directed to any specific improvements of the independent claims and do not practically or significantly alter how the identified abstract idea would be performed. When considered as a whole, as an ordered combination, the dependent claim further elaborates on the previously identified abstract idea (assessing labels, correcting them, classifying them and automating labeling).
Therefore, dependent claim 6 (which is representative of dependent claims 19) is deemed ineligible.
Dependent claim 7, which is representative of dependent claims 20, recites:
estimating a probability of the correct other label corresponding to the other incorrect label of the first data sample;
training the second model by updating second parameters of the second model, to estimate the other correct label, using a first data sample comprising the estimated probability of the other correct label.
When considered individually, these added claim elements further elaborate on the abstract idea identified in the independent claims, because the dependent claim continues to recite the identified abstract idea: assessing labels, correcting them, classifying them and automating labeling. The elements in this dependent claim are comparable to performance of limitations expressing mathematical concepts like mathematical relationships, mathematical calculations. These fall under the Mathematical Concepts. i.e., mathematical relationships, mathematical formulas or equations, or mathematical calculations grouping of abstract ideas (see MPEP 2106.04(a)(2) I).
The dependent claim elements have the same relationship to the underlying abstract idea (assessing labels, correcting them, classifying them and automating labeling) as outlined in the independent claims analysis above. Thus, the dependent claim elements are not directed to any specific improvements of the independent claims and do not practically or significantly alter how the identified abstract idea would be performed. When considered as a whole, as an ordered combination, the dependent claim further elaborates on the previously identified abstract idea (assessing labels, correcting them, classifying them and automating labeling).
Therefore, dependent claim 7 (which is representative of dependent claims 20) is deemed ineligible.
Dependent claim 12 recites:
wherein the data sample comprises input data and the label corresponding to the input data, and the method further comprises:
generating a correct label corresponding to the incorrect label by applying the input data to a second model, as the label is determined as the incorrect label,
wherein the second model comprises a second neural network that is trained to estimate the correct label corresponding to the data sample comprising the incorrect label.
When considered individually, these added claim elements further elaborate on the abstract idea identified in the independent claims, because the dependent claim continues to recite the identified abstract idea: assessing labels, correcting them, classifying them and automating labeling. The elements in this dependent claim are comparable to performance of limitations expressing mathematical concepts like mathematical relationships, mathematical calculations. These fall under the Mathematical Concepts. i.e., mathematical relationships, mathematical formulas or equations, or mathematical calculations grouping of abstract ideas (see MPEP 2106.04(a)(2) I).
The dependent claim elements have the same relationship to the underlying abstract idea (assessing labels, correcting them, classifying them and automating labeling) as outlined in the independent claims analysis above. Thus, the dependent claim elements are not directed to any specific improvements of the independent claims and do not practically or significantly alter how the identified abstract idea would be performed. When considered as a whole, as an ordered combination, the dependent claim further elaborates on the previously identified abstract idea (assessing labels, correcting them, classifying them and automating labeling).
Therefore, dependent claim 12 is deemed ineligible.
Dependent claims 8-10, 13 recite:
wherein the data samples are mixed such that the first data sample comprises training data and the incorrect label corresponding to the training data and the second data sample comprises the training data and the correct label corresponding to the training data.
wherein the training data comprises another image data of a semiconductor obtained by an image sensor.
wherein the first model and the second model are each trained based on an expectation-maximization (EM) algorithm.
wherein the input data comprises another image data of a semiconductor obtained by the image sensor.
These further elements in the dependent claims do not perform any claimed method steps. They describe the nature, structure and/or content of other claim elements – the data samples; the training data; the first model; the second model; the input data – and as such, cannot change the nature of the identified abstract idea (assessing labels, correcting them, classifying them and automating labeling), from a judicial exception into eligible subject matter, because they do not represent significantly more (see MPEP 2106.07). The nature, form or structure of the other claim elements themselves do not practically or significantly alter how the identified abstract idea would be performed and do not provide more than a general link to a technological environment.
Therefore, dependent claims 8-10, 13 are deemed ineligible.
When the dependent claims are considered as a whole, as an ordered combination, the claim elements noted above appear to merely apply the abstract concept to a technical environment in a very general sense. The most significant elements, which form the abstract concept, are set forth in the independent claims. The fact that the computing devices and the dependent claims are facilitating the abstract concept is not enough to confer statutory subject matter eligibility, since their individual and combined significance do not transform the identified abstract concept at the core of the claimed invention into eligible subject matter. Therefore, it is concluded that the dependent claims of the instant application, considered individually, or as a as a whole, as an ordered combination, do not amount to significantly more (see MPEP 2106.07(a)II).
In sum, claims 1-20 are rejected under 35 USC 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION – The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-10, 14 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor regards as the invention.
Claims 1, 14 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor regards as the invention. The claim has been amended to recite “constructing data …” While the specification discloses at [0026] “constructing data,” it does not detail what constructing entails, therefore rendering the claim indefinite.
The remainder of the claims are rejected by virtue of dependency. The reference is provided for compact prosecution purpose.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the difference between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
i. Determining the scope and contents of the prior art.
ii. Ascertaining the differences between the prior art and the claims at issue.
iii. Resolving the level of ordinary skill in the pertinent art.
iv. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 6-17, 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Petit et al (“Computerized Medical Imaging and Graphics”), in view of McCourt (US 2020/0019884).
Regarding Claims 1, 14: Petit discloses: A processor-implemented method comprising:
receiving a training dataset comprising image data and labels corresponding to defect codes, the image data comprising a scanning electron microscopy (SEM) image of a semiconductor obtained by an image sensor; {see at least page 2 “… import of organ labels …”; page 3, “input images”; page 9, paragraph 4.4. external sources of images. The reference does not disclose the claim element “the image data comprising a scanning electron microscopy (SEM) image of a semiconductor obtained by an image sensor”. However, this difference is only found in the non-functional descriptive material and does not affect how the claimed invention functions (i.e., the descriptive material does not have any claim function in the claimed method; see MPEP 2111.05). Thus, this descriptive material will not distinguish the claimed invention from the prior art in terms of patentability}
classifying, based on a distribution of the confidences, data samples in the training dataset into a first data sample comprising an incorrect label and a second data sample comprising a correct label; {see at least paragr 3.2.2. page5 … discriminating correct from incorrect predictions …(based on the BRI (MPEP 2111), reads on classifying in incorrect and correct labels, which amounts to a distribution of the two complementary states)
training a first model to predict the confidences of labels for the data samples, including using a corrected data sample obtained by correcting the incorrect label based on a corresponding confidence detected by the first model and an estimated corrected label generated by a second model; {see at least sections 1, 3.2, 3.2.1, 3.2.2, Algorithm 2: training the confidence network based on the correct label estimation}
training the second model to estimate a correct other label corresponding to another incorrect label detected based on a corresponding confidence generated by the first model with respect to the other incorrect label; and {see at least sections 1, 3. 2, 3. 2 .1, 3.2.2, Algorithms l, 2: training the network to predict correct labels, relabeling)}
automatically correcting the other incorrect label with the estimated correct other label to generate corrected data samples; {see at least: relabeling in the next iteration.}
constructing, using the corrected data samples, data for training a semiconductor defect classification system. {see at least paragr 2.2, page3, “enlarge the training set with the union of …” (reads on constructing data for training). The claim element “for training a semiconductor defect classification system” consists entirely of language disclosing at most a reason to have performed earlier method steps (intended use or field of use), but does not affect the functions in a manipulative sense (see MPEP 2103 I C) and imparts neither structure nor functionality to the claimed method (see MPEP 2111.05, MPEP 2114 and authorities cited therein), so it is considered but given no patentable weight. The reference is provided for the purpose of compact prosecution.}
Petit does not disclose, however McCourt discloses:
determining a confidence comprising a first probability of each of the labels being correct and a second probability of each of the labels being incorrect; {see at least fig10, rc1040-rc1060, [0129]-[0132] correct probability, incorrect probability. In addition, it is pointed out that correct probability is the complement of incorrect probability, thus determining one of the two states automatically implies that the other state is implicitly determined as well}
It would have been obvious to one of ordinary skill in the art, at the time of filing, to modify Petit to include the elements of McCourt. One would have been motivated to do so, in order to provide the system with the right data. Furthermore, the Supreme Court has supported that combining well known prior art elements, in a well-known manner, to obtain predictable results is sufficient to determine an invention obvious over such combination (see KSR International Co. v. Teleflex Inc. (KSR), 550 U.S.,82 USPQ2d 1385 (2007) & MPEP 2143). In the instant case, Petit evidently discloses correcting labels and automating labeling. McCourt is merely relied upon to illustrate the functionality of a confidence level in labels in the same or similar context. Since both correcting labels and automating labeling, as well as confidence level in labels are implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Moreover, since the elements disclosed by Petit, as well as McCourt would function in the same manner in combination as they do in their separate embodiments, it is concluded that their resulting combination would be predictable. Accordingly, the claimed subject matter is obvious over Petit / McCourt.
Regarding Claims 2, 16: Petit, McCourt discloses the limitations of Claims 1, 15. Petit further discloses: wherein the respective trainings of the first and second models are iterative trainings that, based on the confidences of the labels in the training dataset, {see at least INERRANT (Iterative coNfidencE Relabeling of paRtial ANnoTations)}
iteratively trains the first model to detect incorrect labels and the second model to estimate the correct labels; the iterative training further comprises: {see at least page2, fig1 INERRANT (Iterative coNfidencE Relabeling of paRtial ANnoTations)}
training the first model by updating first parameters of the first model, to predict confidence, {see at least sections 1, 3.2, 3.2.1, 3.2.2, Algorithm 2: training the confidence network based on the correct label estimation}
using the corrected data samples obtained by correcting the incorrect labels in the first data sample. {see at least page 4, “… only leverages correct labels.”}
Regarding Claims 3: Petit, McCourt discloses the limitations of Claims 1, 15. Petit further discloses: wherein the classifying comprises:
sampling a second data sample comprising the correct labels based on a Bernoulli distribution. {see at least page 4, paragraph 3.2.1 “easy positive samples”. The reference does not disclose the term “based on a Bernoulli distribution”. However, this difference is only found in the non-functional descriptive material and does not affect how the claimed invention functions (i.e., the descriptive material does not have any claim function in the claimed method; see MPEP 2111.05). Thus, this descriptive material will not distinguish the claimed invention from the prior art in terms of patentability}
Regarding Claims 4, 17: Petit, McCourt discloses the limitations of Claims 1, 15. Petit further discloses: wherein the updating of the first parameters comprises:
updating the first parameters of the first model based on a maximum likelihood corresponding to the corrected data samples. {see at least page8, paragraph “table 7”, “…training on updated dataset.”}
Regarding Claims 6, 19: Petit, McCourt discloses the limitations of Claims 1, 15. Petit further discloses: wherein the training of the first model further comprises:
determining initial parameter values of the first model based on a calculated cross-entropy loss. {see at least page 4, “… the binary cross entropy to train our model …”. The reference does not disclose the term “based on a calculated cross-entropy loss”. However, this difference is only found in the non-functional descriptive material and does not affect how the claimed invention functions (i.e., the descriptive material does not have any claim function in the claimed method; see MPEP 2111.05). Thus, this descriptive material will not distinguish the claimed invention from the prior art in terms of patentability}
Regarding Claims 7, 20: Petit, McCourt discloses the limitations of Claims 1, 15. Petit further discloses: wherein the training of the second model comprises:
estimating a probability of the correct other label corresponding to the other incorrect label of the first data sample; {see at least page 3, paragraph 2.3 probability of the predicted class}
training the second model by updating second parameters of the second model, to estimate the other correct label, using a first data sample comprising the estimated probability of the other correct label. {see at least sections 1, 3. 2, 3. 2 .1, 3.2.2, Algorithms l, 2: training the network to predict correct labels, relabeling)}
Regarding Claims 8: Petit, McCourt discloses the limitations of Claims 1, 15. Petit further discloses:
wherein the data samples are mixed such that the first data sample comprises
training data and the incorrect label corresponding to the training data and the second data sample comprises the training data and the correct label corresponding to the training data. {see at least page2, fig1 INERRANT (Iterative coNfidencE Relabeling of partial ANnoTations)}
Regarding Claims 9: Petit, McCourt discloses the limitations of Claims 1, 15. Petit further discloses:
wherein the training data comprises another image data of a semiconductor obtained by the image sensor. {see at least page 9, paragraph 4.4. external sources of images. The reference does not disclose the term “image data of a semiconductor obtained by an image sensor”. However, this difference is only found in the non-functional descriptive material and does not affect how the claimed invention functions (i.e., the descriptive material does not have any claim function in the claimed method; see MPEP 2111.05). Thus, this descriptive material will not distinguish the claimed invention from the prior art in terms of patentability}
Regarding Claims 10: Petit, McCourt discloses the limitations of Claims 1, 15. Petit further discloses:
wherein the first model and the second model are each trained based on an expectation-maximization (EM) algorithm. {see at least sections 1, 3.2, 3.2.1, 3.2.2, Algorithm 2: training the confidence network based on the correct label estimation; sections 1, 3. 2, 3. 2 .1, 3.2.2, Algorithms l, 2: training the network to predict correct labels, relabeling. The reference does not disclose the term “based on an expectation-maximization (EM) algorithm”. However, this difference is only found in the non-functional descriptive material and does not affect how the claimed invention functions (i.e., the descriptive material does not have any claim function in the claimed method; see MPEP 2111.05). Thus, this descriptive material will not distinguish the claimed invention from the prior art in terms of patentability}
Regarding Claim 11: Petit first embodiment discloses: An automatic labeling method, comprising:
receiving a data sample comprising input data and a label corresponding to the input data, the input data comprising a scanning electron microscopy (SEM) image of a semiconductor obtained by an image sensor; {see at least page 2 “… import of organ labels …”; page 3, “input images”; page 9, paragraph 4.4. external sources of images (reads on data sample and label). The reference does not disclose the claim element “a scanning electron microscopy (SEM) image of a semiconductor obtained by an image sensor”. However, this difference is only found in the non-functional descriptive material and does not affect how the claimed invention functions (i.e., the descriptive material does not have any claim function in the claimed method; see MPEP 2111.05). Thus, this descriptive material will not distinguish the claimed invention from the prior art in terms of patentability}
detecting whether the label is an incorrect label based on the determined confidence, {see at least sections 1, 3.2, 3.2.l., 3.2.2, confidence estimation network.}
wherein the first model comprises a first neural network that is trained to detect the incorrect label comprised in the data sample based on the confidence of the label; and {see at least: [abstract] conventional neural network adapted to this context}
generating, in response to the label being detected as the incorrect label, a corrected data sample for constructing data for training a semiconductor defect classification system. {see at least paragr 3, page4, “… maximizing the number of correct label predictions …” (based on the BRI (MPEP 2111), reads on generating corrected data sample). The claim element “for constructing data for training a semiconductor defect classification system” consists entirely of language disclosing at most a reason to have performed earlier method steps (intended use or field of use), but does not affect the functions in a manipulative sense (see MPEP 2103 I C) and imparts neither structure nor functionality to the claimed method (see MPEP 2111.05, MPEP 2114 and authorities cited therein), so it is considered but given no patentable weight. The reference is provided for the purpose of compact prosecution.}
Petit does not disclose, however McCourt discloses:
determining a confidence of the label, comprising a first probability of the label being correct and a second probability of the label being incorrect, by applying the data sample to a first model; {see at least fig10, rc1040-rc1060, [0129]-[0132] correct probability, incorrect probability. In addition, it is pointed out that correct probability is the complement of incorrect probability, thus determining one of the two states automatically implies that the other state is implicitly determined as well}
It would have been obvious to one of ordinary skill in the art, at the time of filing, to modify Petit to include the elements of McCourt. One would have been motivated to do so, in order to provide the system with the right data. Furthermore, the Supreme Court has supported that combining well known prior art elements, in a well-known manner, to obtain predictable results is sufficient to determine an invention obvious over such combination (see KSR International Co. v. Teleflex Inc. (KSR), 550 U.S.,82 USPQ2d 1385 (2007) & MPEP 2143). In the instant case, Petit evidently discloses correcting labels and automating labeling. McCourt is merely relied upon to illustrate the functionality of a confidence level in labels in the same or similar context. Since both correcting labels and automating labeling, as well as confidence level in labels are implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Moreover, since the elements disclosed by Petit, as well as McCourt would function in the same manner in combination as they do in their separate embodiments, it is concluded that their resulting combination would be predictable. Accordingly, the claimed subject matter is obvious over Petit / McCourt.
Regarding Claims 12: Petit, McCourt discloses the limitations of Claims 1, 15. Petit further discloses:
wherein the data sample comprises input data and the label corresponding to the input data, and the method further comprises: {see at least page 2, paragraph 2.2, “… input images …”}
generating a correct label corresponding to the incorrect label by applying the input data to a second model, as the label is determined as the incorrect label, {see at least page 2, rc1, “… maximize the number of correct labels …”}
wherein the second model comprises a second neural network that is trained to estimate the correct label corresponding to the data sample comprising the incorrect label. {see at least sections 1, 3. 2, 3. 2 .1, 3.2.2, Algorithms l, 2: training the network to predict correct labels, relabeling)}
Regarding Claims 13: Petit, McCourt discloses the limitations of Claims 1, 15. Petit further discloses:
wherein the input data comprises another image data of a semiconductor obtained by the image sensor. {see at least page 2 “… import of organ labels …”; page 3, “input images”; page 9, paragraph 4.4. external sources of images. The reference does not disclose the term “image data of a semiconductor obtained by an image sensor”. However, this difference is only found in the non-functional descriptive material and does not affect how the claimed invention functions (i.e., the descriptive material does not have any claim function in the claimed method; see MPEP 2111.05). Thus, this descriptive material will not distinguish the claimed invention from the prior art in terms of patentability}
Regarding Claim 15: Petit discloses: An electronic device, comprising: a communication system; and a processor configured to, based on confidences of labels for data samples in a training dataset received by the communication system, {see at least page 2, fig2}
iteratively train a first model to detect incorrect labels in the training dataset and a second model to estimate correct labels corresponding to the incorrect labels, and {see at least page2, fig1 INERRANT (Iterative coNfidencE Relabeling of paRtial ANnoTations)}
generate a data sample in which an incorrect label is corrected using at least one of the first model or the second model, and {see at least page 4, paragraph 3.2.1 relabeling (reads on correcting labels)}
wherein the processor is further configured to:
classify, based on a distribution of the confidences, the data samples into a first data sample comprising an incorrect label and a second data sample comprising a correct label; {see at least paragr 3.2.2. page5 … discriminating correct from incorrect predictions …(based on the BRI (MPEP 2111), reads on classifying in incorrect and correct labels, which amounts to a distribution of the two complementary states)
train the first model to predict the confidences, including using the corrected data sample generated by correcting the incorrect label based on a corresponding confidence detected by the first model and an estimated corrected label generated by the second model {see at least sections 1, 3.2, 3.2.1, 3.2.2, Algorithm 2: training the confidence network based on the correct label estimation}
train the second model to estimate correct labels for the data samples, including estimating a corrected other label corresponding to another incorrect label detected based on a corresponding confidence generated by the first model with respect to the other incorrect and {see at least sections 1, 3. 2, 3. 2 .1, 3.2.2, Algorithms l, 2: training the network to predict correct labels, relabeling)}
automatically correct the other incorrect label with the estimated correct other label to generate corrected data samples used for constructing data for training a semiconductor defect classification system. {see at least: relabeling in the next iteration. The claim element “to generate a defect classification system robust against noise thereby improving performance in accuracy and increasing efficiency in terms of cost and time” consists entirely of language disclosing at most a reason to have performed earlier method steps (intended use or field of use), but does not affect the functions in a manipulative sense (see MPEP 2103 I C) and imparts neither structure nor functionality to the claimed method (see MPEP 2111.05, MPEP 2114 and authorities cited therein), so it is considered but given no patentable weight. The reference is provided for the purpose of compact prosecution.}
Petit does not disclose, however McCourt discloses:
determine a confidence comprising a first probability of each of the labels being correct and a second probability of each of the labels being incorrect; {see at least fig10, rc1040-rc1060, [0129]-[0132] correct probability, incorrect probability. In addition, it is pointed out that correct probability is the complement of incorrect probability, thus determining one of the two states automatically implies that the other state is implicitly determined as well}
It would have been obvious to one of ordinary skill in the art, at the time of filing, to modify Petit to include the elements of McCourt. One would have been motivated to do so, in order to provide the system with the right data. Furthermore, the Supreme Court has supported that combining well known prior art elements, in a well-known manner, to obtain predictable results is sufficient to determine an invention obvious over such combination (see KSR International Co. v. Teleflex Inc. (KSR), 550 U.S.,82 USPQ2d 1385 (2007) & MPEP 2143). In the instant case, Petit evidently discloses correcting labels and automating labeling. McCourt is merely relied upon to illustrate the functionality of a confidence level in labels in the same or similar context. Since both correcting labels and automating labeling, as well as confidence level in labels are implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Moreover, since the elements disclosed by Petit, as well as McCourt would function in the same manner in combination as they do in their separate embodiments, it is concluded that their resulting combination would be predictable. Accordingly, the claimed subject matter is obvious over Petit / McCourt.
Claims 5, 18 are rejected under 35 U.S.C. 103 as being unpatentable over Petit et al (“Computerized Medical Imaging and Graphics”), in view of McCourt (US 2020/0019884), in further view of Gross et al (US 2022/0064737).
Regarding Claims 5, 18: Petit, McCourt discloses the limitations of Claims 1, 15. Petit further discloses: wherein the training of the first model comprises:
training the first model by applying respective regularization penalties for the confidences to the updated first parameters. {see at least sections 1, 3.2, 3.2.1, 3.2.2, Algorithm 2: training the confidence network based on the correct label estimation}
Petit, McCourt does not disclose, however, Gross discloses:
determining a regularization penalty for the confidences; and {see at least [0240] regularization penalty}
It would have been obvious to one of ordinary skill in the art, at the time of filing, to modify Petit, McCourt to include the elements of Gross. One would have been motivated to do so, in order to promote a “first time right” strategy. Furthermore, the Supreme Court has supported that combining well known prior art elements, in a well-known manner, to obtain predictable results is sufficient to determine an invention obvious over such combination (see KSR International Co. v. Teleflex Inc. (KSR), 550 U.S.,82 USPQ2d 1385 (2007) & MPEP 2143). In the instant case, Petit, McCourt evidently discloses correcting labels and automating labeling. Gross is merely relied upon to illustrate the functionality of a regularization penalty in the same or similar context. Since both correcting labels and automating labeling, as well as regularization penalty are implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Moreover, since the elements disclosed by Petit, McCourt, as well as Gross would function in the same manner in combination as they do in their separate embodiments, it is concluded that their resulting combination would be predictable. Accordingly, the claimed subject matter is obvious over Petit, McCourt / Gross.
The prior art made of record and not relied upon which, however, is considered pertinent to applicant's disclosure:
US 20200117991 A1 Suzuki; Kanata et al. LEARNING APPARATUS, DETECTING APPARATUS, LEARNING METHOD, AND DETECTING METHOD A feature model, which calculates a feature value of an input image, is trained on a plurality of first images. First feature values corresponding one-to-one with the first images are calculated using the feature model, and feature distribution information representing a relationship between a plurality of classes and the first feature values is generated. When a detection model which determines, in an input image, each region with an object and a class to which the object belongs is trained on a plurality of second images, second feature values corresponding to regions determined within the second images by the detection model are calculated using the feature model, an evaluation value, which indicates class determination accuracy of the detection model, is modified using the feature distribution information and the second feature values, and the detection model is updated based on the modified evaluation value.
US 20220327808 A1 Bergen; Leon et al. Systems And Methods For Evaluating The Error Rate Of Human-Generated Data Systems, apparatuses, and methods for more efficiently and effectively determining the accuracy with which a human evaluates a set of data, as this may reduce the error in the assessment of a model's performance. This can be helpful in situations where human inputs are used to confirm the output of a machine learning generated classification and in situations where it is desired to evaluate the accuracy of data that may have been labeled or annotated by a human curator. This may assist in reducing the need for new validation/test data when evaluating a new model. The system and methods described can be used to evaluate the accuracy of a trained Machine Learning (ML) model, and as a result, allow a comparison between models based on different ML algorithms.
US 20210406644 A1 SALMAN; Nader et al. ACTIVE LEARNING FRAMEWORK FOR MACHINE-ASSISTED TASKS An active learning framework is provided that employs a plurality of machine learning components that operate over iterations of a training phase followed by an active learning phase. In each iteration of the training phase, the machine learning components are trained from a pool of labeled observations. In the active learning phase, the machine learning components are configured to generate metrics used to control sampling of unlabeled observations for labeling such that newly labeled observations are added to a pool of labeled observations for the next iteration of the training phase. The machine learning components can include an inspection (or primary) learning component that generates a predicted label and uncertainty score for an unlabeled observation, and at least one additional component that generates a quality metric related to the unlabeled observation or the predicted label. The uncertainty score and quality metric(s) can be combined for efficient sampling of observations for labeling.
US 20210245659 A1 Golov; Gil ARTIFICIAL INTELLIGENCE-BASED PERSISTENCE OF VEHICLE BLACK BOX DATA The disclosed embodiments are directed to improving the persistence of pre-accident data in vehicles. In one embodiment a method is disclosed comprising receiving events broadcast over a vehicle bus; classifying the events using a machine learning model, the classifying comprising indicating that a collision is imminent; and copying data from a cyclic buffer of a black box device into a long-term storage device in response to the classifying.
US 20220224659 A1 El Ghazzal; Sammy AUTOMATED MESSAGING REPLY-TO An automated messaging reply-to system can automatically select which message a potential reply message is replying to. The automated messaging reply-to system can obtain a message thread, a potential reply message, and a context. The automated messaging reply-to system can filter the message thread and generate model inputs based on the remaining messages, the potential reply message, and the context. The automated messaging reply-to system can apply the model input to a machine learning model, which can generate reply scores for the remaining messages. After generating reply scores, the automated messaging reply-to system can determine whether the remaining message with the highest reply score qualifies as an originating message being replied to. The automated messaging reply-to system can cause display of the potential reply message as a reply-to for the determined originating message.
US 20200311616 A1 Rajkumar; Nareshkumar et al. EVALUATING ROBOT LEARNING Methods, systems, and apparatus, including computer programs encoded on computer storage media for evaluating robot learning. In some implementations, one or more computers receive object classification examples from a plurality of robots. Each object classification example includes (i) an embedding that a robot generated using a machine learning model, and (ii) an object classification corresponding to the embedding. The object classification examples are evaluated based on a similarity of the received embeddings with respect to other embeddings. A subset of the object classification examples is selected based on the evaluation of the quality of the embeddings. The subset of the object classification examples is distributed to the robots in the plurality of robots.
US 20230315879 A1 JAVIDI; Bahram et al. SYSTEMS AND METHODS FOR PROTECTING MACHINE LEARNING MODELS AGAINST ADVERSARIAL ATTACKS Embodiments pertain to systems configured to and methods for analyzing a scene comprising one or more objects. The system may be configured to perform the following: obtaining a set of optically encrypted image data describing a scene, including applying an optical manipulation to light incoming to an image acquisition device, whereby the image acquisition device outputs the set of optically encrypted image data, and wherein the optical manipulation is based on an encryption key; providing the set of optically encrypted image data to a machine learning model trained in accordance with the encryption key; and receiving from the machine learning model a prediction related to the scene.
US 20220058347 A1 Singaraju; Gautam et al. TECHNIQUES FOR PROVIDING EXPLANATIONS FOR TEXT CLASSIFICATION A chatbot system is configured to execute code to perform determining, by the chatbot system, a classification result for an utterance and one or more anchors each anchor of the one or more anchors corresponding to one or more anchor words of the utterance. For each anchor of the one or more anchors, one or more synthetic utterances are generated, and one or more classification results for the one or more synthetic utterances are determined. A report is generated by the chatbot system comprising a representation of a particular anchor of the one or more anchors, the particular anchor corresponding to a highest confidence value among the one or more anchors. The one or more synthetic utterances may be used to generate a new training dataset for training a machine-learning model. The training dataset may be refined according to a threshold confidence values to filter out datasets for training.
US 20210174231 A1 SAWADA; Azusa et al. MODEL GENERATION DEVICE, MODEL GENERATION METHOD, AND NON-TRANSITORY RECODING MEDIUM Disclosed is a model generation device capable of mitigating the risk of overlooking a phenomenon of interest in machine learning. The model generation device determines whether or not a label of a first data is similar to a label of a second data. The model generation device assigns the label of the second data to the first data when determining that the label of the first data is similar to the label of the second data based on a degree of similarity between observation information representing a state where the first data is observed and observation information representing a state where the second data is observed. The model generation device calculates model representing a relevance between data information containing the first data and the second data and label information containing the assigned label and the label of the second data.
US 10305766 B1 Zhang; Ce et al. Coexistence-insensitive presence detection A system and method include processing logic receiving, from a wireless transceiver of a first device, first data indicative of channel state information (CSI) of a first communication link between the wireless transceiver and a wireless transmitter of a second device, the first device and the second device being located in a building. The logic pre-preprocesses the first data to generate input vectors composed of statistical parameter values derived from sets of discrete samples of the first data. The logic processes, through a long short-term memory (LSTM) layer of a neural network, the input vectors to generate multiple hidden state values of the neural network. The logic processes, through a set of additional layers of the neural network, respective hidden state values of the multiple hidden state values to determine that a human is present in the building.
US 12148417 B1 Cardella; Aidan Thomas et al. Label confidence scoring Devices and techniques are generally described for confidence score generation for label generation. In some examples, first data may be received from a first computing device. In various further examples, first label data classifying at least one aspect of the first data may be received. First metadata associated with how the first label data was generated may be received. In some cases, the first label data may be generated by a first user. In various examples, a first machine learning model may generate a first confidence score associated with the first label data based at least in part on the first data and second data related to label generation by the first person. In various examples, output data comprising the first confidence score may be sent to the first computing device.
US 11678011 B1 Fu; Sai-Wai et al. Mobile distributed security response The invention concerns a video feed monitoring app configured to be implemented with an interface and a processor. The interface may be configured to receive requests from an operator and present visual content to a display. The processor may be configured to process the requests and update the display. The video feed monitoring app may be configured to receive video streams from smart security devices, receive a priority signal, select a subset of the video streams in response to the priority signal and a user preference and arrange the subset of the video streams on the display. The priority signal may be generated externally based on events detected. The subset of the video streams may be updated based on the priority signal. Updating the subset of the video streams may comprise displaying the video streams that comprise events and removing the video streams that do not comprise events.
Response to Amendments/Arguments
Applicant’s submitted remarks and arguments have been fully considered.
Applicant disagrees with the Office Action conclusions and asserts that the presented claims fully comply with the requirements of 35 U.S.C. § 101 regrading judicial exceptions. Further, Applicant is of the opinion that the prior art fails to teach Applicant’s invention.
Examiner respectfully disagrees in both regards.
With respect to Applicant’s Remarks as to the claims being rejected under 35 USC § 101.
Applicant submits:
a. The pending claims are not directed to an abstract idea.
b. The identified abstract idea is integrated into a practical application.
c. The pending claims amount to significantly more.
Furthermore, Applicant asserts that the Office has failed to meet its burden to identify the abstract idea and to establish that the identified abstract idea is not integrated into a practical application and that the pending claims do not amount to significantly more.
Examiner responds – The arguments have been considered in light of Applicants’ amendments to the claims. The arguments ARE NOT PERSUASIVE. Therefore, the rejection is maintained.
The pending claims, as a whole, are directed to an abstract idea not integrated into a practical application. This is because (1) they do not effect improvements to the functioning of a computer, or to any other technology or technical field (see MPEP 2106.05 (a)); (2) they do not apply or use the abstract idea to effect a particular treatment or prophylaxis for a disease or a medical condition (see the Vanda memo); (3) they do not apply the abstract idea with, or by use of, a particular machine (see MPEP 2106.05 (b)); (4) they do not effect a transformation or reduction of a particular article to a different state or thing (see MPEP 2106.05 (c)); (5) they do not apply or use the abstract idea in some other meaningful way beyond generally linking the use of the identified abstract idea to a particular technological environment, such that the claim as a whole is more than a drafting effort designated to monopolize the exception (see MPEP 2106.05 (e) and the Vanda memo).
In addition, the pending claims do not amount to significantly more than the abstract idea itself.
As such, the pending claims, when considered as a whole, are directed to an abstract idea not integrated into a practical application and not amounting to significantly more.
More specific:
Applicant submits “As amended, the independent claims do not recite a judicial exception that can, as a practical matter, be performed in the human mind.”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
Based on the claim language (“automatically correcting the other incorrect label with the estimated correct other label to generate corrected data samples; constructing, using the corrected data samples, data for training a semiconductor defect classification system”) and in light of the specification (“training a first model to predict confidences of labels for data samples in a training dataset, including using a corrected data sample obtained by correcting an incorrect label based on a corresponding confidence detected by the first model and an estimated corrected label generated by a second model; training the second model to estimate correct labels for the data samples, including estimating a correct other label corresponding to another incorrect label detected based on a corresponding confidence generated by the first model with respect to the other incorrect label; and automatically correcting the other incorrect label with the estimated correct other label”), the claims of the instant application are directed to assessing labels, correcting them, classifying them and automating labeling, This is a combination that, under its broadest reasonable interpretation, covers performance of limitations expressing following rules or instructions. These fall under the Certain Methods of Organizing Human Activity, i.e., Managing Personal Behavior or Relationships, or Interactions Between People grouping of abstract ideas (see MPEP 2106.04(a)(2)). In addition, or alternatively, this is a combination that, under its broadest reasonable interpretation, covers reasonable performance of limitations expressing observation, judgement in the human mind. Nothing in the claim elements precludes the steps from being practically performed in the human mind. Therefore, it is concluded that the independent claims 1, 11, 14, 15 recite an abstract idea that corresponds to a judicial exception.
Thus, the rejection is proper and has been maintained.
Applicant submits “A human cannot, in the mind or with pen and paper, train a neural network, update model parameters across a training dataset, or model the distribution of model-generated confidences.”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
The eligibility analysis in the instant office action shows that the steps recited by the claims can be performed by a human, mentally and/or manually. In addition, the use of a physical aid would not negate the mental nature of this limitation (see MPEP 2106.04(a)(2) iii B)
Thus, the rejection is proper and has been maintained.
Applicant submits “Further, Applicants respectfully submit that the present claims are not directed to "managing personal behavior or relationships or interactions between people," or "mental process," for example.”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
See response immediately above.
Thus, the rejection is proper and has been maintained.
Applicant submits “Applicants respectfully submit that the claims,
considered as a whole without overqeneralization, are clearly not directed to an abstract idea. Based on the specific detail of the claims, Applicants respectfully submit that the claims are not directed to an abstract idea.”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
An examiner can describe an abstract idea at different levels of generality without affecting the patent-eligibility analysis. Cf Apple, Inc. v. Ameranth, Inc., 842 F.3d 1229, 1240--41 (Fed. Cir. 2016) ("An abstract idea can generally be described at different levels of abstraction .... The Board's slight revision of its abstract idea analysis does not impact the patentability analysis."). That is the case here. Regardless of the level of generality used to describe the abstract idea recited in claim 1, the result is the same – claim 1 recites an abstract idea. Cf Accenture Glob. Servs., GmbH v. Guidewire Software, Inc., 728 F.3d 1336, 1344 (Fed. Cir. 2013) ("Although not as broad as the district court's abstract idea of organizing data, it is nonetheless an abstract concept.").
Thus, the rejection is proper and has been maintained.
Applicant submits “The claimed subject matter provides a specific improvement to the technical field of training defect-classification models from imperfectly labeled semiconductor-inspection data.”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
MPEP 2106.04(d)(1) discloses:
An important consideration to evaluate when determining whether the claim as a whole integrates a judicial exception into a practical application is whether the claimed invention improves the functioning of a computer or other technology .... In short, first the specification should be evaluated to determine if the disclosure provides sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement. The specification need not explicitly set forth the improvement, but it must describe the invention such that the improvement would be apparent to one of ordinary skill in the art .... Second, if the specification sets forth an improvement in technology. the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement. (Emphasis added)
That is, the claimed invention may integrate the judicial exception into a practical application by demonstrating that it improves the relevant existing technology although it may not be an improvement over well-understood, routine, conventional activity. (Emphasis added)
Thus, the rejection is proper and has been maintained.
It follows from the above that there are no meaningful limitations in the claims that transform the judicial exception into a patent eligible application such that the claims amount to significantly more than the judicial exception itself. Therefore, the rejection under 35 U.S.C. § 101 is maintained.
With respect to Applicant’s Remarks as to the claims being rejected under 35 USC § 112(b).
The rejection is withdrawn, as a result of the amendments.
With respect to Applicant’s Remarks as to the claims being rejected under 35 USC § 103.
Applicant submits that the prior art of record does not disclose: “determining a confidence comprising a first probability of each of the labels being correct and a second probability of each of the labels being incorrect;” and “classifying, based on a distribution of the confidences, data samples in the training dataset into a first data sample comprising an incorrect label and a second data sample comprising a correct label”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
McCourt discloses:
determining a confidence comprising a first probability of each of the labels being correct and a second probability of each of the labels being incorrect; {see at least fig10, rc1040-rc1060, [0129]-[0132] correct probability, incorrect probability. In addition, it is pointed out that correct probability is the complement of incorrect probability, thus determining one of the two states automatically implies that the other state is implicitly determined as well}
Furthermore, Petit discloses:
classifying, based on a distribution of the confidences, data samples in the training dataset into a first data sample comprising an incorrect label and a second data sample comprising a correct label; {see at least paragr 3.2.2. page5 … discriminating correct from incorrect predictions …(reads classifying in incorrect and correct labels)
Therefore, Petit, McCourt discloses the amended claim limitations. Thus, the rejection is proper and has been maintained.
Applicant submits that the prior art of record does not disclose “Determining a confidence comprising both a probability of correctness and a probability of incorrectness (independent claims 1, 11, and 15)”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
MacCourt discloses:
determining a confidence comprising a first probability of each of the labels being correct and a second probability of each of the labels being incorrect; {see at least fig10, rc1040-rc1060, [0129]-[0132] correct probability, incorrect probability. In addition, it is pointed out that correct probability is the complement of incorrect probability, thus determining one of the two states automatically implies that the other state is implicitly determined as well}
Therefore, McCourt discloses the amended claim limitation. Thus, the rejection is proper and has been maintained.
Applicant submits that the prior art of record does not disclose “Classifying the data samples based on a distribution of the confidences (independent claims 1 and 15)”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
Petit discloses:
classifying, based on a distribution of the confidences, data samples in the training dataset into a first data sample comprising an incorrect label and a second data sample comprising a correct label; {see at least paragr 3.2.2. page5 … discriminating correct from incorrect predictions … (based on the BRI (MPEP 2111), reads on classifying in incorrect and correct labels, which amounts to a distribution of the two complementary states)
Thus, the rejection is proper and has been maintained.
Applicant submits that the prior art of record does not disclose: “Detecting and correcting an existing incorrect label. and the distinct roles of the first and second models {independent claims; see also claims 11-12).”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
Petit discloses:
…, including using a corrected data sample obtained by correcting the incorrect label based on a corresponding confidence detected by the first model and an estimated corrected label generated by a second model; {see at least sections 1, 3.2, 3.2.1, 3.2.2, Algorithm 2: training the confidence network based on the correct label estimation}
Thus, the rejection is proper and has been maintained.
Applicant submits “The SEM-image and defect-code limitations are functionally integral, not field-of-use recitations (all independent claims).”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
Although the specification is directed to SEM images, the specification does not disclose that the method will work exclusively for the semiconductor images.
Thus, the rejection is proper and has been maintained.
The other arguments presented by Applicant continually point back to the above arguments as being the basis for the arguments against the other 103 rejections, as the other arguments are presented only because those claims depend from the independent claims, and the main argument above is presented against the independent claims. Therefore, it is believed that all arguments put forth have been addressed by the points above.
Examiner has reviewed and considered all of Applicant’s remarks. The changes of the grounds for rejection, if any, have been necessitated by Applicant’s extensive amendments to the claims. Therefore, the rejection is maintained, necessitated by the extensive amendments and by the fact that the rejection of the claims under 35 USC § 101 has not been overcome.
Conclusion
THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Radu Andrei whose telephone number is 313.446.4948. The examiner can normally be reached on Monday – Friday 8:30am – 5pm EST. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Hayes can be reached at 571.272.6708. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http:/www.uspto.gov/interviewpractice.
As disclosed in MPEP 502.03, communications via Internet e-mail are at the discretion of the applicant. Without a written authorization by applicant in place, the USPTO will not respond via Internet e-mail to any Internet correspondence which contains information subject to the confidentiality requirement as set forth in 35 U.S.C. 122. A paper copy of such correspondence will be placed in the appropriate patent application. The following is a sample authorization form which may be used by applicant:
“Recognizing that Internet communications are not secure, I hereby authorize the USPTO to communicate with me concerning any subject matter of this application by electronic mail. I understand that a copy of these communications will be made of record in the application file.”
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center information webpage. Status information for unpublished applications is available to registered users through Patent Center information webpage only.
To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov.
Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (in USA or CANADA) or 571-272-1000.
Any response to this action should be mailed to:
Commissioner of Patents and Trademarks
P.O. Box 1450
Alexandria, VA 22313-1450
or faxed to 571-273-8300
/Radu Andrei/
Primary Examiner, AU 3697
