Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 10/23/2025 has been entered.
Remarks
This Office Action is responsive to Applicants' Amendment filed on September 24, 2025, in which claims 1 and 13-15 are amended. No claims have been newly added or cancelled. Claims 1-19 are currently pending.
Response to Arguments
Applicant’s Remarks received 09/24/2025 were addressed in the Advisory Action mailed 10/09/2025. The rejections of claims 1-19 under 35 U.S.C. 101 were stated to be withdrawn contingent on the amendments to the claims being filed, and as such the rejections of claims 1-19 under 35 U.S.C. 101 are withdrawn. However, Examiner stated that Applicant’s arguments against the rejections of claims 1-19 under 35 U.S.C 103 were not persuasive, and as such the rejections are maintained. Please review the response dated 10/09/2025.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Claims 1 and 13-15 have been interpreted under 35 U.S.C. 112(f) because they use either the generic placeholder “controlling apparatus” coupled with the functional language “configured to”, in claims 1, 13, and 15, or the generic placeholder “a data processing apparatus” coupled with the functional language “adapted to”, in claim 14, without reciting sufficient structure to achieve the function. Furthermore, the generic placeholder is not preceded by a structural modifier.
Since the claim limitation(s) invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, claim 1 and 13-15 have been interpreted to cover the corresponding structure described in the specification that achieves the claimed function, and equivalents thereof.
A review of the specification shows that the following appears to be the corresponding structure described in the specification for the 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph limitation:
For “controlling apparatus”: (Pg. 21) “The controlling apparatus comprises a sensor assembly adapted to acquire a data entity, the data entity being indicative of at least one property of a manufacturing of a respective product. The controlling apparatus further comprises a data processing apparatus adapted to classify the manufacturing of the product based on the data entity and a classification model. Furthermore, the controlling apparatus comprises a control interface adapted to output a control signal such that the at least one process parameter is adapted - in particular, changed - based on the classifying”.
For “a data processing apparatus”: (Pg. 27) “The data processing apparatus 108 is adapted to classify the manufacturing of the product based on the data entity and the classification model. So, in some modifications, the data processing apparatus 108 is adapted to capture an image of the current powder layer by the image capturing device such as a camera 102 and classifies the currently created powder layer with regard to the homogeneity of the powder layer by the classification model. Moreover, the data processing apparatus 108 is adapted to output by the control interface 106 a control signal such that the homogeneity of the current powder layer is changed on the classifying.”
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-6, 13, 14-17, are rejected under 35 U.S.C. 103 as being unpatentable over Mehr et al. (U.S. Patent Application Publication No. 2018/0341248), hereinafter Mehr, further in view of Gokalp (U.S. Patent Application Publication No. 2023/0376857), hereinafter Gokalp, further in view of Asar et al. (U.S. Patent Application Publication No. 2008/0133434), hereinafter Asar, further in view of Wang et al. “Cost-Effective Quality Assurance in Crowd Labeling”, hereinafter Wang.
Regarding claim 1,
Mehr teaches A computer-implemented method for training a classification model for controlling a manufacturing process ((Mehr Abstract) “Disclosed herein are machine learning-based methods and systems for automated object defect classification and adaptive, real-time control of additive manufacturing and/or welding processes”):
wherein products are manufactured according to at least one process parameter and wherein at least one property is indicative of the manufacturing of the products ((Mehr [0111]) “one or more process monitoring tools may be used to provide real-time data on process parameters or properties of the object being fabricated, both of which will be referred to herein as ‘process characterization data’”)
said method comprising: acquiring, by a sensor assembly of a controlling apparatus, ((Mehr [0125]) “The automated object defect classification methods will generally comprise: b) providing one or more sensors, wherein the one or more sensors provide real-time data for one or more object properties”) a set of data entities, each of the data entities being indicative of at least one property of a manufacturing of a respective product; ((Mehr [0145]) “The training data comprises a set of paired training examples, e.g., where each example comprises a set of defects detected for a given object and the resultant classification of the given object”, defects are a property of manufacturing a product)
receiving, by a data processing apparatus, the classification model from a data storage of the controlling apparatus configured to classify the manufacturing of the respective product and to control the manufacturing process; ((Mehr [0170]) “Some aspects of the methods and systems provided herein, such as the disclosed object defect classification or additive manufacturing process control algorithms, are implemented by way of machine (e.g., processor) executable code stored in an electronic storage location of the computer system, such as, for example, in the memory or electronic storage unit…In some cases, the code is retrieved from the storage unit and stored in the memory for ready access by the one or more processors”, the storage unit corresponds to a data storage of the controlling apparatus)
training the classification model using a training set, ((Mehr [0002]) “c) providing a predicted optimal set or sequence of one or more process control parameters for fabricating the object, wherein the predicted optimal set of one or more process control parameters are derived using a machine learning algorithm that has been trained using the training data set”)
wherein the classification model is an artificial intelligence model that classifies data entities into correct categories, ((Mehr [0007]) “In some embodiments, the object defects are detected as differences between object property data and a reference data set that are larger than a specified threshold, and are classified using a one-class support vector machine (SVM) or autoencoder algorithm”)
and wherein the training set comprises the data entities and the respective one or more labels; ((Mehr [0132]) “In some preferred embodiments, object defects may be detected and classified using an unsupervised one-class support vector machine (SVM), autoencoder, clustering, or nearest neighbor (e.g., kNN) machine learning algorithm and a training data set that comprises object property data for both defective and defect-free objects”)
Gokalp teaches the following further limitations that Mehr does not teach:
acquiring one or more labels for each of the data entities from an agent ((Gokalp [0045]) “the classification service may employ one or more label providers for a given classification problem, such as subject matter experts with respect to the problem domain, volunteers, or a group of individuals who have been identified via a web-based task marketplace (e.g., a web site at which individuals may register their interest in performing tasks such as labeling data items for a fee)”, label providers are agents)
training a labeling score model based on the data entities, the respective one or more labels acquired from the agent, a set of labeling metrics based on the acquiring from the agent ((Gokalp [0039]) “In at least some embodiments, at least two types of models may be trained iteratively: (a) a set of one or more models whose output with respect to a training set is used to select candidates for labeling feedback for subsequent training iterations (e.g., using an active learning algorithm which uses variance in predictions among the different models for a given data item)”, the model or set of models used to select candidates for labeling feedback corresponds to a labeling score model)
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr and Gokalp by taking the method for training a classification model to control a manufacturing process taught by Mehr, and combining it with the method for acquiring labels from agents taught by Gokalp, as obtaining labelled data is essential to the supervised training of machine learning classifiers, and more niche classification domains that may lack large and publicly-available datasets would benefit from the production of more labelled data, as models trained on larger amounts of data tend to be more accurate. Such a combination would be obvious.
Asar teaches the following further limitations that neither Mehr, nor Gokalp explicitly teaches:
and a classifier score obtained from a validation of the trained classification model using a validation set, ((Asar [0071]) “Validation dataset: Dataset used for validating the model during the learning phase and to estimate the prediction error for model selection”, prediction error of a model corresponds to a classifier score, a person of ordinary skill in the art understands a validation set to be used to validate the training of a model)
wherein the classifier score is a numerical measure of how well the trained classification model fits the validation set, ((Asar [0148]-[0149]) “We perform a single split and select a set of optimization parameters for training/validation. If this is a classification problem, then once training has been performed, we perform validation using multiple thresholds (assume T number of thresholds)…For each threshold value, we calculate validation error rate for that threshold as follows: errate=Sum(LF across all inputs in the validation set)/(Total number of element in the validation set)”, a validation error rate for a classification problem corresponds to a classifier score that is a numerical measure of how well a trained classification model fits a validation set)
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr, Gokalp, and Asar by taking the method for training a classification model to control a manufacturing process, including acquiring labels from agents, taught jointly by Mehr and Gokalp, and including the use of validation of the model on a validation dataset to determine the performance of the model, taught by Asar, as use of a validation dataset to validate machine learning models is a well-known technique within the art for getting early feedback on a model being trained to tune parameters, without exposing it to the test data set used to evaluate its ultimate performance. Such a combination would be obvious.
Wang teaches the following further limitation that neither Mehr, nor Gokalp, nor Asar teaches:
and wherein the labeling score model is an artificial intelligence model that generates and outputs a labeling score which is a numerical measure of an efficiency of a labeling process performed by the agent and of a quality of a label obtained from the agent ((Wang Pg. 6) “In this section, we describe several algorithms for inferring the true classes of objects and the quality of workers…Another advanced inference technique is EM, first proposed by Dawid and Skene (1979) in the context of medical diagnosis. The algorithm iterates until convergence, following two steps: (1) it estimates the true class for each object using the labels provided by a set of workers, accounting for the error rates of each worker; and (2) it estimates the error rates of each worker by comparing the submitted labels with estimated true class for each object…we propose a generative model of labels, abilities, and difficulties (GLAD) and use an EM approach to obtain the maximum likelihood estimates of the α(k) , β(o) , and t(o) for each worker (k) and each object (o)”, Wang Pg. 8, Algorithm 4 shows inference of an artificial intelligence model that generates scores for labels of objects and quality of workers that provide labels)
PNG
media_image1.png
433
777
media_image1.png
Greyscale
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr, Gokalp, Asar, and Wang by taking the method for training a classification model to control a manufacturing process, including acquiring labels from agents and validation on a validation set, taught jointly by Mehr, Gokalp, and Asar, and including a labelling score artificial intelligence model to create a score that evaluates the labelling efficiency of an agent and their label quality, taught by Wang, as Wang teaches: (Wang Pg. 19) “we introduce two novel metrics that can be used to objectively rank the performance of crowdsourced workers, both allowing employers to separate workers’ correctable errors from uncorrectable errors and incorporate unequal costs of different types of classification errors. In particular, the contributed value metric directly measures worker’s individual contribution in quality assurance through redundancy and provides a basis for employers to develop more fair and efficient compensation schemes”. Such a combination would be obvious.
Regarding claim 4,
Mehr, Gokalp, Asar, and Wang jointly teach The computer-implemented method of claim 1,
Gokalp further teaches:
wherein the set of labelling metrics comprises:
---a time span during which the one or more labels for the respective data entity have been acquired from the agent;
----a time span between acquiring one label and a further label from the agent;
----an amount of required energy;
----an effort for labeling a data entity;
----an importance score for a data entity;
----a count of labels for a data entity of the data entities;
----a count of labels for the set of data entities;
----a count of labels acquired from a group of agents;
----a count of labels acquired from the agent;
----a measure of the similarity between labels across the group;
----a measure of similarity of labels across multiple/different groups of agents; or
----an agent classification score ((Gokalp [0045]) “In at least some embodiments, as the training iterations proceed, the interactions with individual label providers may be analyzed, e.g., to determine which label providers are more proficient in identifying particular classes of data items, to determine the rate at which individual label providers are able to generate labels, and so on…one or more metrics pertaining to label submission by the label provider such as the rate at which labels are generated, a comparison of the labels with respect to predicted classes, and so on”, the rate of label generation by the label provider corresponds to a time span between acquiring one label and a further label from an agent)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Mehr, Gokalp, Asar, and Wang for the parent claim of claim 4, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 13,
Mehr teaches A method for controlling a manufacturing process ((Mehr Abstract) “Disclosed herein are machine learning-based methods and systems for automated object defect classification and adaptive, real-time control of additive manufacturing and/or welding processes”):
wherein products are manufactured according to at least one process parameter and wherein at least one property is indicative of the manufacturing of the products ((Mehr [0111]) “one or more process monitoring tools may be used to provide real-time data on process parameters or properties of the object being fabricated, both of which will be referred to herein as ‘process characterization data’”)
comprising: acquiring, by a sensor assembly of a controlling apparatus, a data entity, the data entity being indicative of at least one property of a manufacturing of a respective product ((Mehr [0125]) “The automated object defect classification methods will generally comprise: b) providing one or more sensors, wherein the one or more sensors provide real-time data for one or more object properties”)
receiving, by a data processing apparatus, a classification model from a data storage of the controlling apparatus configured to classify the manufacturing of the product and to control the manufacturing process; ((Mehr [0170]) “Some aspects of the methods and systems provided herein, such as the disclosed object defect classification or additive manufacturing process control algorithms, are implemented by way of machine (e.g., processor) executable code stored in an electronic storage location of the computer system, such as, for example, in the memory or electronic storage unit…In some cases, the code is retrieved from the storage unit and stored in the memory for ready access by the one or more processors”, the storage unit corresponds to a data storage of the controlling apparatus)
classifying the manufacturing of the product based on the data entity and a classification model ((Mehr [0125]) “The automated object defect classification methods will generally comprise:…c) providing a processor programmed to provide a classification of detected object defects using a machine learning algorithm that has been trained using the training data set of step (a), wherein the real-time data from the one or more sensors is provided as input to the machine learning algorithm and allows the classification of detected object defects to be adjusted in real-time”, the machine learning algorithm used for classification corresponds to a classification model)
and adapting the at least one process parameter based on the classifying ((Mehr [0031]) “In some embodiments, in-process inspection data (e.g., automated defect classification data) may be used by the machine learning algorithm to determine a set or sequence of process control parameter adjustments that will implement a corrective action, e.g., to adjust a layer dimension or thickness, so as to correct the defect when first detected”, process control parameter adjustments correspond to adapting at least one process parameter)
training, by a computer-implemented method, the classification model, ((Mehr [0144]) “the machine learning algorithm(s) employed in the disclosed automated defect classification and additive manufacturing process control methods may comprise a supervised learning algorithm”)
wherein the classification model is an artificial intelligence model that classifies data entities into correct categories, ((Mehr [0007]) “In some embodiments, the object defects are detected as differences between object property data and a reference data set that are larger than a specified threshold, and are classified using a one-class support vector machine (SVM) or autoencoder algorithm”)
wherein said training the classification model comprises: providing a set of data entities, each of the data entities being indicative of at least one property of a manufacturing of a respective product ((Mehr [0145]) “The training data comprises a set of paired training examples, e.g., where each example comprises a set of defects detected for a given object and the resultant classification of the given object”)
and training the classification model, using a training set that comprises the data entities and the respective one or more labels ((Mehr [0144]) “the machine learning algorithm(s) employed in the disclosed automated defect classification and additive manufacturing process control methods may comprise a supervised learning algorithm”, (Mehr [0145]) “Supervised learning algorithms: In the context of the present disclosure, supervised learning algorithms are algorithms that rely on the use of a set of labeled training data to infer the relationship between a set of one or more defects identified for a given object and a classification of the object…The training data comprises a set of paired training examples, e.g., where each example comprises a set of defects detected for a given object and the resultant classification of the given object”, the classification is the label)
and controlling the manufacturing process, ((Mehr [0001]) “Also disclosed are methods and systems for performing real-time adaptive control of free form deposition or joining processes, including additive manufacturing or welding processes”)
wherein the manufacturing process manufactures products, ((Mehr [0001]) “Additive manufacturing processes are fabrication techniques that allow one to produce functional complex parts layer by layer, without the use of molds or dies”)
wherein said controlling is based on a classification of the data entities by the classification model into correct categories, ((Mehr [0031]) “in-process inspection data (e.g., automated defect classification data) may be used by the machine learning algorithm to determine a set or sequence of process control parameter adjustments that will implement a corrective action, e.g., to adjust a layer dimension or thickness, so as to correct the defect when first detected”)
and wherein said controlling increases a yield of the products manufactured ((Mehr [0001]) “Also disclosed are methods and systems for performing real-time adaptive control of free form deposition or joining processes, including additive manufacturing or welding processes, to improve process yield, throughput, and quality”)
Gokalp teaches the following further limitations that Mehr does not teach:
acquiring one or more labels for each of the data entities from an agent ((Gokalp [0045]) “the classification service may employ one or more label providers for a given classification problem, such as subject matter experts with respect to the problem domain, volunteers, or a group of individuals who have been identified via a web-based task marketplace (e.g., a web site at which individuals may register their interest in performing tasks such as labeling data items for a fee)”, label providers are agents)
determining a set of labeling metrics based on the acquiring from the agent ((Gokalp [0045]) “In at least some embodiments, as the training iterations proceed, the interactions with individual label providers may be analyzed, e.g., to determine which label providers are more proficient in identifying particular classes of data items, to determine the rate at which individual label providers are able to generate labels, and so on…one or more metrics pertaining to label submission by the label provider such as the rate at which labels are generated, a comparison of the labels with respect to predicted classes, and so on”, the metrics pertaining to label submission correspond to labeling metrics)
validating the trained classification model ((Gokalp [0064]) “a final (with respect to the current training iteration) classifier may also be trained in a given iteration, e.g., using all the labeled training data available, and the results obtained from the final-with-respect-to-the-current-iteration classifier on a test set may be used to evaluate whether quality-related training completion criteria have been met”, evaluating the final classifier on a test set corresponds to validating it) based on predefined criteria and yielding a classifier score ((Gokalp [0055]) “with respect to a subset of training metrics and associated training status, a set of diagnosis tests may be defined to help determine when the training procedure has met its overall objectives and should therefore be terminated. In effect, a given diagnosis test may provide a binary indicator of whether a given metric’s status has met a particular threshold condition for publishing or finalizing the classifier being trained”, the binary indicator of if a metric meets the threshold for finalizing the classifier corresponds to a classifier score, the set of diagnosis tests corresponds to predefined criteria)
training a labeling score model based on the data entities, the respective one or more labels acquired from the agent, the set of labeling metrics based on the acquiring from the agent, and the classifier score [obtained from said validating the trained classification model using a validation set] ((Gokalp [0039]) “In at least some embodiments, at least two types of models may be trained iteratively: (a) a set of one or more models whose output with respect to a training set is used to select candidates for labeling feedback for subsequent training iterations (e.g., using an active learning algorithm which uses variance in predictions among the different models for a given data item)”, the model or set of models used to select candidates for labeling feedback corresponds to a labeling score model, Asar teaches obtaining a classifier score from validating a trained classification model using a validation set more explicitly)
determining a labeling score for the agent based on the labeling score model and the respective one or more labels and set of labeling metrics ((Gokalp [0132]) “In scenarios in which multiple label providers are used, individual ones of the label providers may have differing capabilities and responsiveness characteristics---e.g., some label providers may be faster or otherwise superior to others with respect to identifying data items of particular classes, and so on…a classifier training subsystem 2402 may comprise, among other components, a label provider skills/capabilities detector 2404 implemented using one or more computing devices. Such a detector may, for example, keep track of how quickly different label providers such as 2420A, 2420B or 2420C respond to label feedback requests, the extent to which the labels provided by the different label providers 2420 tend to agree with the class predictions generated at the training subsystem, and so on. Using such metrics, respective profiles of the different label providers may be generated in at least some embodiments”, the metric of how often the label provider’s labels match that of the training subsystem’s corresponds to a labeling score for an agent)
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr and Gokalp by taking the method for using a classification model to control a manufacturing process, including training the model, taught by Mehr, and combining it with the method for acquiring labels from agents taught by Gokalp, as obtaining labelled data is essential to the supervised training of machine learning classifiers, and more niche classification domains that may lack large and publicly-available datasets would benefit from the production of more labelled data, as models trained on larger amounts of data tend to be more accurate. Such a combination would be obvious.
Asar teaches the following further limitations that neither Mehr, nor Gokalp explicitly teaches:
and the classifier score obtained from said validation of the trained classification model using a validation set, ((Asar [0071]) “Validation dataset: Dataset used for validating the model during the learning phase and to estimate the prediction error for model selection”, prediction error of a model corresponds to a classifier score, a person of ordinary skill in the art understands a validation set to be used to validate the training of a model)
wherein the classifier score is a numerical measure of how well the trained classification model fits the validation set, ((Asar [0148]-[0149]) “We perform a single split and select a set of optimization parameters for training/validation. If this is a classification problem, then once training has been performed, we perform validation using multiple thresholds (assume T number of thresholds)…For each threshold value, we calculate validation error rate for that threshold as follows: errate=Sum(LF across all inputs in the validation set)/(Total number of element in the validation set)”, a validation error rate for a classification problem corresponds to a classifier score that is a numerical measure of how well a trained classification model fits a validation set)
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr, Gokalp, and Asar by taking the method for using a classification model to control a manufacturing process, including training the model and acquiring labels from agents, taught jointly by Mehr and Gokalp, and including the use of validation of the model on a validation dataset to determine the performance of the model, taught by Asar, as use of a validation dataset to validate machine learning models is a well-known technique within the art for getting early feedback on a model being trained to tune parameters, without exposing it to the test data set used to evaluate its ultimate performance. Such a combination would be obvious.
Wang teaches the following further limitation that neither Mehr, nor Gokalp, nor Asar teaches:
and wherein the labeling score model is an artificial intelligence model that generates and outputs a labeling score which is a numerical measure of an efficiency of a labeling process performed by the agent and of a quality of a label obtained from the agent ((Wang Pg. 6) “In this section, we describe several algorithms for inferring the true classes of objects and the quality of workers…Another advanced inference technique is EM, first proposed by Dawid and Skene (1979) in the context of medical diagnosis. The algorithm iterates until convergence, following two steps: (1) it estimates the true class for each object using the labels provided by a set of workers, accounting for the error rates of each worker; and (2) it estimates the error rates of each worker by comparing the submitted labels with estimated true class for each object…we propose a generative model of labels, abilities, and difficulties (GLAD) and use an EM approach to obtain the maximum likelihood estimates of the α(k) , β(o) , and t(o) for each worker (k) and each object (o)”, Wang Pg. 8, Algorithm 4 shows inference of an artificial intelligence model that generates scores for labels of objects and quality of workers that provide labels)
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr, Gokalp, Asar, and Wang by taking the method for using a classification model to control a manufacturing process, including training the model, acquiring labels from agents, and validation on a validation set, taught jointly by Mehr, Gokalp, and Asar, and including a labelling score artificial intelligence model to create a score that evaluates the labelling efficiency of an agent and their label quality, taught by Wang, as Wang teaches: (Wang Pg. 19) “we introduce two novel metrics that can be used to objectively rank the performance of crowdsourced workers, both allowing employers to separate workers’ correctable errors from uncorrectable errors and incorporate unequal costs of different types of classification errors. In particular, the contributed value metric directly measures worker’s individual contribution in quality assurance through redundancy and provides a basis for employers to develop more fair and efficient compensation schemes”. Such a combination would be obvious.
Regarding claim 14,
Mehr teaches A controlling apparatus for controlling a manufacturing process ((Mehr Abstract) “Disclosed herein are machine learning-based methods and systems for automated object defect classification and adaptive, real-time control of additive manufacturing and/or welding processes”):
wherein products are manufactured by a manufacturing system according to at least one process parameter and wherein at least one property is indicative of the manufacturing of the products, ((Mehr [0111]) “one or more process monitoring tools may be used to provide real-time data on process parameters or properties of the object being fabricated, both of which will be referred to herein as ‘process characterization data’”)
the controlling apparatus comprising: a sensor assembly adapted to acquire a data entity, the data entity being indicative of at least one property of a manufacturing of a respective product; ((Mehr [0125]) “The automated object defect classification methods will generally comprise: b) providing one or more sensors, wherein the one or more sensors provide real-time data for one or more object properties”, (Mehr [0177]) “One or more automated inspection tools, e.g., machine vision systems coupled with automated image processing algorithms, are used to monitor and measure feature dimensions, angles, surface finishes, and/or other properties of fabricated parts both in-process and post-build. Defects may be identified…and classified”, a machine vision system with image processing to classify defects corresponds to a data processing apparatus that captures and classifies images)
and a data processing apparatus adapted to classify the manufacturing of the product based on the data entity and a classification model; ((Mehr [0125]) “The automated object defect classification methods will generally comprise:…c) providing a processor programmed to provide a classification of detected object defects using a machine learning algorithm that has been trained using the training data set of step (a), wherein the real-time data from the one or more sensors is provided as input to the machine learning algorithm and allows the classification of detected object defects to be adjusted in real-time”)
wherein the data processing apparatus is further adapted to receive the classification model from a data storage of the controlling apparatus, onto which the classification model is stored, or from a distributed database ((Mehr [0170]) “Some aspects of the methods and systems provided herein, such as the disclosed object defect classification or additive manufacturing process control algorithms, are implemented by way of machine (e.g., processor) executable code stored in an electronic storage location of the computer system, such as, for example, in the memory or electronic storage unit…In some cases, the code is retrieved from the storage unit and stored in the memory for ready access by the one or more processors”, the storage unit corresponds to a data storage of the controlling apparatus)
said classification model being generated, trained, and validated by a computer-implemented method comprising: providing a set of data entities, each of the data entities being indicative of at least one property of a manufacturing of a respective product ((Mehr [0145]) “The training data comprises a set of paired training examples, e.g., where each example comprises a set of defects detected for a given object and the resultant classification of the given object”)
training the classification model using a training set, ((Mehr [0144]) “the machine learning algorithm(s) employed in the disclosed automated defect classification and additive manufacturing process control methods may comprise a supervised learning algorithm”, supervised learning algorithms use a training set)
wherein the classification model is an artificial intelligence model that classifies data entities into correct categories, ((Mehr [0007]) “In some embodiments, the object defects are detected as differences between object property data and a reference data set that are larger than a specified threshold, and are classified using a one-class support vector machine (SVM) or autoencoder algorithm”)
and wherein the training set comprises the data entities and the respective one or more labels; ((Mehr [0132]) “In some preferred embodiments, object defects may be detected and classified using an unsupervised one-class support vector machine (SVM), autoencoder, clustering, or nearest neighbor (e.g., kNN) machine learning algorithm and a training data set that comprises object property data for both defective and defect-free objects”)
Gokalp teaches the following further limitations that Mehr does not teach:
acquiring one or more labels for each of the data entities from an agent ((Gokalp [0045]) “the classification service may employ one or more label providers for a given classification problem, such as subject matter experts with respect to the problem domain, volunteers, or a group of individuals who have been identified via a web-based task marketplace (e.g., a web site at which individuals may register their interest in performing tasks such as labeling data items for a fee)”)
determining a set of labeling metrics based on the acquiring from the agent ((Gokalp [0045]) “In at least some embodiments, as the training iterations proceed, the interactions with individual label providers may be analyzed, e.g., to determine which label providers are more proficient in identifying particular classes of data items, to determine the rate at which individual label providers are able to generate labels, and so on…one or more metrics pertaining to label submission by the label provider such as the rate at which labels are generated, a comparison of the labels with respect to predicted classes, and so on”)
validating the trained classification model ((Gokalp [0064]) “a final (with respect to the current training iteration) classifier may also be trained in a given iteration, e.g., using all the labeled training data available, and the results obtained from the final-with-respect-to-the-current-iteration classifier on a test set may be used to evaluate whether quality-related training completion criteria have been met”) based on predefined criteria and yielding a classifier score ((Gokalp [0055]) “with respect to a subset of training metrics and associated training status, a set of diagnosis tests may be defined to help determine when the training procedure has met its overall objectives and should therefore be terminated. In effect, a given diagnosis test may provide a binary indicator of whether a given metric’s status has met a particular threshold condition for publishing or finalizing the classifier being trained”)
training a labeling score model based on the data entities, the respective one or more labels, the sets of labeling metrics based on the acquiring from the agent, ((Gokalp [0039]) “In at least some embodiments, at least two types of models may be trained iteratively: (a) a set of one or more models whose output with respect to a training set is used to select candidates for labeling feedback for subsequent training iterations (e.g., using an active learning algorithm which uses variance in predictions among the different models for a given data item)”)
and determining a labeling score for the agent based on the labeling score model and the respective one or more labels and set of labeling metrics ((Gokalp [0132]) “In scenarios in which multiple label providers are used, individual ones of the label providers may have differing capabilities and responsiveness characteristics---e.g., some label providers may be faster or otherwise superior to others with respect to identifying data items of particular classes, and so on…a classifier training subsystem 2402 may comprise, among other components, a label provider skills/capabilities detector 2404 implemented using one or more computing devices. Such a detector may, for example, keep track of how quickly different label providers such as 2420A, 2420B or 2420C respond to label feedback requests, the extent to which the labels provided by the different label providers 2420 tend to agree with the class predictions generated at the training subsystem, and so on. Using such metrics, respective profiles of the different label providers may be generated in at least some embodiments”, broadest reasonable interpretation of a labeling score includes a speed at which labeling requests are fulfilled or an extent to which labels provided are correct)
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr and Gokalp by taking the controlling apparatus for controlling a manufacturing process, wherein a classification model is a component, and wherein the model is trained, taught by Mehr, and combining it with the method for acquiring labels from agents taught by Gokalp, as obtaining labelled data is essential to the supervised training of machine learning classifiers, and more niche classification domains that may lack large and publicly-available datasets would benefit from the production of more labelled data, as models trained on larger amounts of data tend to be more accurate. Such a combination would be obvious.
Asar teaches the following further limitations that neither Mehr, nor Gokalp explicitly teaches:
and the classifier score obtained from said validating the trained classification model using a validation set, ((Asar [0071]) “Validation dataset: Dataset used for validating the model during the learning phase and to estimate the prediction error for model selection”, prediction error of a model corresponds to a classifier score, a person of ordinary skill in the art understands a validation set to be used to validate the training of a model)
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr, Gokalp, and Asar by taking the controlling apparatus for controlling a manufacturing process, wherein a classification model is a component, and including acquiring labels for the classification model from agents, taught jointly by Mehr and Gokalp, and including the use of validation of the model on a validation dataset to determine the performance of the model, taught by Asar, as use of a validation dataset to validate machine learning models is a well-known technique within the art for getting early feedback on a model being trained to tune parameters, without exposing it to the test data set used to evaluate its ultimate performance. Such a combination would be obvious.
Wang teaches the following further limitation that neither Mehr, nor Gokalp, nor Asar teaches:
wherein the labeling score model is an artificial intelligence model that generates and outputs a labeling score which is a numerical measure of an efficiency of a labeling process performed by the agent and of a quality of a label obtained from the agent ((Wang Pg. 6) “In this section, we describe several algorithms for inferring the true classes of objects and the quality of workers…Another advanced inference technique is EM, first proposed by Dawid and Skene (1979) in the context of medical diagnosis. The algorithm iterates until convergence, following two steps: (1) it estimates the true class for each object using the labels provided by a set of workers, accounting for the error rates of each worker; and (2) it estimates the error rates of each worker by comparing the submitted labels with estimated true class for each object…we propose a generative model of labels, abilities, and difficulties (GLAD) and use an EM approach to obtain the maximum likelihood estimates of the α(k) , β(o) , and t(o) for each worker (k) and each object (o)”, Wang Pg. 8, Algorithm 4 shows inference of an artificial intelligence model that generates scores for labels of objects and quality of workers that provide labels)
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr, Gokalp, Asar, and Wang by taking the controlling apparatus for controlling a manufacturing process, wherein a classification model is a component, and including acquiring labels for the classification model from agents and validation of the model on a validation set, taught jointly by Mehr, Gokalp, and Asar, and including a labelling score artificial intelligence model to create a score that evaluates the labelling efficiency of an agent and their label quality, taught by Wang, as Wang teaches: (Wang Pg. 19) “we introduce two novel metrics that can be used to objectively rank the performance of crowdsourced workers, both allowing employers to separate workers’ correctable errors from uncorrectable errors and incorporate unequal costs of different types of classification errors. In particular, the contributed value metric directly measures worker’s individual contribution in quality assurance through redundancy and provides a basis for employers to develop more fair and efficient compensation schemes”. Such a combination would be obvious.
Regarding claim 16,
Mehr, Gokalp, Asar, and Wang jointly teach The computer-implemented method of claim 1,
Wang further teaches:
said method further comprising: executing the labeling score model, wherein a labeling score for the agent is generated and outputted via said executing the labeling score model based on the respective one or more labels and set of labeling metrics ((Wang Pg. 15) “we leverage the advantage of simulated data to check the accuracy of worker quality estimation by calculating the Spearman coefficient between workers’ true quality values and the estimated quality values using different inference algorithms. The results show that both EM and GLAD achieve a fairly high level of accuracy and can be used as effective tools for evaluating worker quality”, using an inference algorithm on simulated data to check the accuracy of worker quality estimation using estimated quality values corresponds to executing a labeling score model that generates a labeling score for an agent)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Mehr, Gokalp, Asar, and Wang for the parent claim of claim 16, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 5,
Mehr, Gokalp, Asar, and Wang jointly teach The computer-implemented method of claim 16,
Gokalp further teaches:
said method being iteratively performed with a first and one or more further iterations, wherein in the further iterations the labeling score is determined based on the labeling score model of a respective previous iteration (Gokalp Fig. 26 shows labels are computed for data entities in multiple iterations (steps 2607 to 2619), and in further iterations the labeling scores are computed for data entities identified by the labeling score model of the prior iteration as requiring feedback (step 2619))
PNG
media_image2.png
828
543
media_image2.png
Greyscale
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Mehr, Gokalp, Asar, and Wang for the parent claim of claim 5, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 6,
Mehr, Gokalp, Asar, and Wang jointly teach The computer-implemented method of claim 5,
Gokalp further teaches:
wherein in the further iterations, the data entities of the first set or a further set are pre-filtered (Gokalp Fig. 26 teaches filtering of data entities as requiring feedback at step 2610) by the classification model, which has been trained in the respective previous iteration ((Gokalp [0071]) “the results obtained for the iteration-final model may be included in the criteria used at the search subsystem to rank data items for labeling feedback purposes---e.g., if a goal for a particular metric measured using the iteration-final model is more likely to be satisfied by obtaining a label for a data item D1 than by a data item D2, D1 may have a higher probability of being included in the search results generated at search subsystem 218”, the feedback is received at further iterations, the final classifier corresponds to the classification model)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Mehr, Gokalp, Asar, and Wang for the parent claim of claim 6, claim 5. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 17,
Mehr, Gokalp, Asar, and Wang jointly teach The computer-implemented method of claim 1,
Mehr further teaches:
controlling the manufacturing process, ((Mehr [0001]) “Also disclosed are methods and systems for performing real-time adaptive control of free form deposition or joining processes, including additive manufacturing or welding processes”)
wherein the manufacturing process manufactures products, ((Mehr [0001]) “Additive manufacturing processes are fabrication techniques that allow one to produce functional complex parts layer by layer, without the use of molds or dies”)
wherein said controlling is based on a classification of the data entities by the classification model into correct categories, ((Mehr [0031]) “in-process inspection data (e.g., automated defect classification data) may be used by the machine learning algorithm to determine a set or sequence of process control parameter adjustments that will implement a corrective action, e.g., to adjust a layer dimension or thickness, so as to correct the defect when first detected”)
and wherein said controlling increases a yield of the products manufactured ((Mehr [0001]) “Also disclosed are methods and systems for performing real-time adaptive control of free form deposition or joining processes, including additive manufacturing or welding processes, to improve process yield, throughput, and quality”)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Mehr, Gokalp, Asar, and Wang for the parent claim of claim 17, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Mehr, further in view of Gokalp, further in view of Asar, further in view of Wang, further in view of Hwang et al. (U.S. Patent Application Publication No. 2019/0283333), hereinafter Hwang.
Regarding claim 2,
Mehr, Gokalp, Asar, and Wang jointly teach The computer-implemented method of claim 1,
Mehr further teaches:
the manufacturing process is a process for additive manufacturing ((Mehr Abstract) “Disclosed herein are machine learning-based methods and systems for automated object defect classification and adaptive, real-time control of additive manufacturing and/or welding processes”)
Hwang teaches the following further limitations that neither Mehr, nor Gokalp, nor Asar, nor Wang teaches:
each of the data entities comprises an image of a powder layer ((Hwang [0045]) “One or more images on the target area would be captured using at least one image recording device at predetermined setting in sequence, wherein the captured image contains a coated powder layer”) of a powder bed for manufacturing a layer of the respective product ((Hwang [0004]) “The Powder-Bed-Fusion (PBF) technology widely is being adopted in the additive manufacturing industry. In PBF technology, raw material should be distributed uniformly and to spread evenly in layer-by-layer process to ensure density of built part as designed, and even when melted, it must have a uniform density to ensure quality without defects such as cracks, deformation or delamination”, an image of a powder layer from a Powder-Bed-Fusion process corresponds to an image of a powder layer of a powder bed)
and the at least one property is a homogeneity of powder of the respective powder layer ((Hwang [0045]) “Once the preprocessed image applied to a predictive model 200, then it identifies any defect of the applied powder layer from the extracted data-set (i.e., the preprocessed images), step 519, thereby determines any corrective actions to be followed, step 520. For example, one or more defect may be detected by calculating changes of contrast to ensure the applied powder layer is evenly distributed”, even distribution of powder corresponds to homogeneity of powder)
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr, Gokalp, Asar, Wang, and Hwang by taking the method for training a classification model to control an additive manufacturing process jointly taught by Mehr, Gokalp, Asar, and Wang, and combining it with the method for additive manufacturing with Powder-Bed-Fusion technology, including ensuring evenly distributed powder, taught by Hwang, as Powder-Bed-Fusion technology is well known within the art of additive manufacturing, and it can be simply substituted for the laser-metal wire deposition technology taught in a preferred embodiment of Mehr. Such a combination would be obvious.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Mehr, further in view of Gokalp, further in view of Asar, further in view of Wang, further in view of Buller et al. (U.S. Patent Application Publication No. 2017/0341183), hereinafter Buller.
Regarding claim 3,
Mehr, Gokalp, Asar, and Wang jointly teach The computer-implemented method of claim 1,
Buller teaches the following further limitation that neither Mehr, nor Gokalp, nor Asar, nor Wang teaches:
wherein each of the data entities is selected from the group consisting of: an image, a video, (Buller [0068]) “The optical detector can comprise a camera (e.g., stills and/or video)”, a material density, a sound recording, and a concentration of a chemical substance ((Buller [0158]) “Provided herein are systems, apparatuses, and methods for monitoring a manufacturing process. The manufacturing process can be a three-dimensional printing process. The manufacturing process can be an additive manufacturing process. The manufacturing process can be monitored in real time. In some cases, the manufacturing process can be monitored non-invasively such that the manufacturing process is undisturbed while one or more measurements are collected to monitor the manufacturing process. The manufacturing process can be monitored (e.g., adjusted, regulated, and/or directed) with a feedback loop. The adjustment may arise when an error (e.g. deviation, non-uniformity, adverse condition, and/or mistake) is detected, the error can be corrected”, (Buller [0178]) “The at least one sensor can be operatively coupled to a control system (e.g., computer control system). The sensor may comprise light sensor, acoustic sensor, vibration sensor, chemical sensor, electrical sensor, magnetic sensor, fluidity sensor, movement sensor, speed sensor, position sensor, pressure sensor, force sensor, density sensor, distance sensor, or proximity sensor. The sensor may include temperature sensor, weight sensor, material (e.g., powder) level sensor, metrology sensor, gas sensor, or humidity sensor”)
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr, Gokalp, Asar, Wang, and Buller by taking the method for training a classification model to control an additive manufacturing process jointly taught by Mehr, Gokalp, Asar, and Wang, and combining it with the method for additive manufacturing using sensors such as video and image cameras, density sensors, acoustic sensor, and chemical sensors to detect errors, taught by Buller, as the listed sensors provide further information that can be used to detect errors or defects, providing the predictable benefit of increased ability to detect and correct defects during manufacturing promptly. Such a combination would be obvious.
Claims 7-9, 11, and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Mehr, further in view of Gokalp, further in view of Asar, further in view of Wang, further in view of Anglin et al. (U.S. Patent Application Publication No. 2020/0218940), hereinafter Anglin.
Regarding claim 7,
Mehr and Gokalp jointly teach The computer-implemented method of claim 16,
Gokalp further teaches:
said method further comprising a labeling method for generating the training set ((Gokalp [0039]) “at least two types of models may be trained iteratively: (a) a set of one or more models whose output with respect to a training set is used to select candidates for labeling feedback for subsequent training iterations (e.g., using an active learning algorithm which uses variance in predictions among the different models for a given data item)”, a model or set of models for iteratively re-labelling a training set corresponds to a labeling method for generating the training set) and a validation method for training the classification model ((Gokalp [0064]) “a final (with respect to the current training iteration) classifier may also be trained in a given iteration, e.g., using all the labeled training data available, and the results obtained from the final-with-respect-to-the-current-iteration classifier on a test set may be used to evaluate whether quality-related training completion criteria have been met”, a method for evaluating the final classifier on a test set corresponds a validation method for training it)
wherein the labeling method at least comprises the acquiring of the labels ((Gokalp [0038]) “At least some class labels may be obtained asynchronously with respect to the start or end of the given training iteration in some embodiments-that is, individuals selected as label providers may submit respective batches of one or more labels at any convenient time”), and the executing of the labeling score model which yields the labeling score ((Gokalp [0132]) “In scenarios in which multiple label providers are used, individual ones of the label providers may have differing capabilities and responsiveness characteristics---e.g., some label providers may be faster or otherwise superior to others with respect to identifying data items of particular classes, and so on”, a measurement of the agent’s ability to label corresponds to a labeling score)
and further comprises: storing, for each of the data entities, the one or more labels and the set of labeling metrics in a [distributed] database; ((Gokalp [0134]) “the raw data and/or metadata for various machine learning tasks may be stored at storage servers 2525 (e.g., 2525A-2525D) of storage service 2523”, Gokalp does not teach a distributed database)
and wherein the validation method at least comprises the training of the classification model, the validation of the trained classification model ((Gokalp [0064]) “a final (with respect to the current training iteration) classifier may also be trained in a given iteration, e.g., using all the labeled training data available, and the results obtained from the final-with-respect-to-the-current-iteration classifier on a test set may be used to evaluate whether quality-related training completion criteria have been met”, the training and evaluation of the final classifier corresponds to the training and validation of the classification model) and the training of the labeling score model ((Gokalp [0039]) “In at least some embodiments, at least two types of models may be trained iteratively: (a) a set of one or more models whose output with respect to a training set is used to select candidates for labeling feedback for subsequent training iterations (e.g., using an active learning algorithm which uses variance in predictions among the different models for a given data item)”, the training of the model or set of models to select candidates for labeling feedback corresponds to the training for a labeling score model)
and further comprises: retrieving the one or more labels for each of the data entities from the [distributed] database ((Gokalp [0110]) “labels may be extracted or imported from a data store in some embodiments, e.g., using the ‘Import’ interface control shown in the session control section. Labels obtained during the session may be exported or saved (e.g., using the ‘Export’ interface element) to a data store, from where they may later be extracted”, Gokalp does not teach a distributed database)
and storing the trained classification model and the labeling score model in the [distributed] database ((Gokalp [0039]) “at least two types of models may be trained iteratively: (a) a set of one or more models whose output with respect to a training set is used to select candidates for labeling feedback for subsequent training iterations (e.g., using an active learning algorithm which uses variance in predictions among the different models for a given data item), and (b) a final (with respect to a current training iteration) model whose results are used to evaluate the overall progress towards the training objectives”, Gokalp Fig. 26 shows at step 2622 that the models may be stored, Gokalp does not teach a distributed database, the model or set of models to select candidates for labeling feedback corresponds to a labeling score model, the final model of a training iteration corresponds to a trained classification model)
Anglin teaches the following further limitations that neither Mehr, nor Gokalp, nor Asar, nor Wang teaches:
a distributed database ((Anglin [0027]) “As an overview, a blockchain is a distributed database that maintains a continuously growing list of data records”), (Anglin [0006]) “Changes to the machine learning model, training data or testing data made by at least one of the two or more entities are tracked and posted to a ledger of the blockchain infrastructure according to terms and specifications of a smart contract”, (Anglin [0007]) “Encrypted keys are generated to enable the two or more entities to utilize the blockchain infrastructure to exchange the tracked changes to the machine learning model, training data or testing data and to exchange an updated machine learning model”)
and retrieving the [labeling score] model from the distributed database ((Anglin [0007]) “Encrypted keys are generated to enable the two or more entities to utilize the blockchain infrastructure to…exchange an updated machine learning model”, Gokalp teaches a labeling score model)
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr, Gokalp, Asar, Wang, and Anglin by taking the method for training a classification model to control an additive manufacturing process, including labels, labeling metrics, a classification model, and a labeling score model, which are stored in a database, jointly taught by Mehr, Gokalp, Asar, and Wang, and combining it with the distributed database from which models are retrieved taught by Anglin, as distributed databases and data retrieval from a database are both well-known within the computing arts, yielding the predictable benefit of increased fault tolerance, as both centralized databases and data kept only locally are vulnerable to the singular system they rely on becoming inoperable. Such a combination would be obvious.
Regarding claim 8,
Mehr, Gokalp, Asar, Wang, and Anglin jointly teach The computer-implemented method of claim 7,
Gokalp further teaches:
wherein the agent is a first agent of a first group of agents ((Gokalp [0045]) “the classification service may employ one or more label providers for a given classification problem, such as subject matter experts with respect to the problem domain, volunteers, or a group of individuals who have been identified via a web-based task marketplace (e.g., a web site at which individuals may register their interest in performing tasks such as labeling data items for a fee)”, a group of individuals identified via a web-based task marketplace corresponds to a first group of agents)
wherein the labeling method further comprises: acquiring one or more labels for each of the data entities from a second agent ((Gokalp [0087]) In at least one embodiment, the classification service may generate class predictions for at least some data items for which labels have already been provided, e.g., in order to determine the extent to which the classifier differs in its conclusions from the label providers. FIG. 9 illustrates an example scenario in which a label provider may be requested, via an interactive interface, to reconsider whether a previously-provided label is appropriate for a labeling feedback candidate, a label provider that reconsider a label provided by another label provider corresponds to a second agent) of a second group of agents ((Gokalp [0088]) “Note that at least in one embodiment, the request to reconsider a previously-supplied label may be sent to a different individual/user than the source of the previously-supplied label---e.g., to one of a set of trusted individuals who are permitted to change previously-provided labels”, a set of trusted individuals who are permitted to change labels corresponds to a second group of agents)
storing, in a [distributed] database for each of the data entities and for each of the agents, the one or more labels and a set of labeling metrics ((Gokalp [0134]) “the raw data and/or metadata for various machine learning tasks may be stored at storage servers 2525 (e.g., 2525A-2525D) of storage service 2523”, Anglin teaches a distributed database) based on the acquiring from the second agent ((Gokalp [0045]) “In at least some embodiments, as the training iterations proceed, the interactions with individual label providers may be analyzed, e.g., to determine which label providers are more proficient in identifying particular classes of data items, to determine the rate at which individual label providers are able to generate labels, and so on…one or more metrics pertaining to label submission by the label provider such as the rate at which labels are generated, a comparison of the labels with respect to predicted classes, and so on”, the metrics pertaining to label submission correspond to labeling metrics)
executing the labeling score model, wherein a labeling score for each of the agents is generated and outputted via said executing the labeling score model based on the respective one or more labels and set of labeling metrics ((Gokalp [0132]) “In scenarios in which multiple label providers are used, individual ones of the label providers may have differing capabilities and responsiveness characteristics---e.g., some label providers may be faster or otherwise superior to others with respect to identifying data items of particular classes, and so on…a classifier training subsystem 2402 may comprise, among other components, a label provider skills/capabilities detector 2404 implemented using one or more computing devices. Such a detector may, for example, keep track of how quickly different label providers such as 2420A, 2420B or 2420C respond to label feedback requests, the extent to which the labels provided by the different label providers 2420 tend to agree with the class predictions generated at the training subsystem, and so on. Using such metrics, respective profiles of the different label providers may be generated in at least some embodiments”, the metric of how often the label provider’s labels match that of the training subsystem’s by the label provider skills/capabilities detector corresponds to a labeling score for an agent generated by a labeling score model)
and wherein the validation method further comprises: retrieving the one or more labels for each of the data entities and for each of the agents from the [distributed] database ((Gokalp [0110]) “labels may be extracted or imported from a data store in some embodiments, e.g., using the "Import" interface control shown in the session control section. Labels obtained during the session may be exported or saved (e.g., using the "Export" interface element) to a data store, from where they may later be extracted”, Anglin teaches a distributed database)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Mehr, Gokalp, Asar, Wang, and Anglin for the parent claim of claim 8, claim 7. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 9,
Mehr, Gokalp, Asar, Wang, and Anglin jointly teach The computer-implemented method of claim 8,
Gokalp further teaches:
wherein the labeling score for one agent of one of the groups of agents, the respective set of metrics and the one or more labels of one of the data entities acquired from the one agent are provided to the one agent or a further agent of the respective group ((Gokalp [0088]) “Furthermore, in the depicted embodiment, the classification service may have computed a high predicted score (e.g., 0.8) for class A membership for the item 914. A suggestion or request 920 for the user to reconsider the previously provided Class B label 922 may be included in the presented visualization data set in the depicted embodiment. In some embodiments, a prediction score which indicates that the current user-suggested label is incorrect may be indicated in the reconsideration request”, providing a labeling score from a label provider and the probability that it is correct to another label provider corresponds to providing the labeling score, the set of metrics, and the labels of one or more data entities from an agent to a further agent)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Mehr, Gokalp, Asar, Wang, and Anglin for the parent claim of claim 9, claim 8. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 11,
Mehr, Gokalp, Asar, Wang, and Anglin jointly teach The computer-implemented method of claim 8, wherein the labeling method further comprises:
Gokalp further teaches:
acquiring at least from the first agent, after determining the labeling score at least for the first and the second agent, an agent classification score with respect to the labels for one or more of the data entities acquired from the second agent ((Gokalp [0088]) “Furthermore, in the depicted embodiment, the classification service may have computed a high predicted score (e.g., 0.8) for class A membership for the item 914. A suggestion or request 920 for the user to reconsider the previously provided Class B label 922 may be included in the presented visualization data set in the depicted embodiment. In some embodiments, a prediction score which indicates that the current user-suggested label is incorrect may be indicated in the reconsideration request”, a classification service’s score of a label from a label provider, presented to a different label provider, corresponds to an agent classification score with respect to the acquired labels, acquired from the first agent) and an agent labeling score with respect to the labeling score and the set of labeling metrics of the second agent ((Gokalp [0132]) “In scenarios in which multiple label providers are used, individual ones of the label providers may have differing capabilities and responsiveness characteristics---e.g., some label providers may be faster or otherwise superior to others with respect to identifying data items of particular classes, and so on…a classifier training subsystem 2402 may comprise, among other components, a label provider skills/capabilities detector 2404 implemented using one or more computing devices. Such a detector may, for example, keep track of how quickly different label providers such as 2420A, 2420B or 2420C respond to label feedback requests, the extent to which the labels provided by the different label providers 2420 tend to agree with the class predictions generated at the training subsystem, and so on. Using such metrics, respective profiles of the different label providers may be generated in at least some embodiments”, a metric of the extent to which a label provider’s labels correspond with those of a training subsystem’s corresponds to an agent labeling score with respect to a labeling score and a set of labeling metrics)
and wherein: the training set for training the classification model further comprises the agent classification score ((Gokalp [0064]) “a final (with respect to the current training iteration) classifier may also be trained in a given iteration, e.g., using all the labeled training data available, and the results obtained from the final-with-respect-to-the-current-iteration classifier on a test set may be used to evaluate whether quality-related training completion criteria have been met”, Gokalp Fig. 26 shows at step 2619 that labeling feedback is incorporated into the training of the models at the next iteration, labeling feedback includes an agent classification score)
and the training of the labeling score model is further based on the agent classification score and the agent labeling score ((Gokalp [0039]) “In at least some embodiments, at least two types of models may be trained iteratively: (a) a set of one or more models whose output with respect to a training set is used to select candidates for labeling feedback for subsequent training iterations (e.g., using an active learning algorithm which uses variance in predictions among the different models for a given data item)”, Gokalp Fig. 26 shows at step 2619 that labelling feedback is incorporated into the training of the models at the next iteration labeling feedback includes an agent classification score and the agent labeling score)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Mehr, Gokalp, Asar, Wang, and Anglin for the parent claim of claim 11, claim 8. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 12,
Mehr, Gokalp, Asar, and Wang jointly teach The computer-implemented method of claim 1,
Anglin teaches the following further limitations that neither Mehr, nor Gokalp nor Asar, nor Wang teaches:
wherein, for each of the data entities, the one or more labels [and the set of labeling metrics of the first agent] are encrypted by a public key and are stored in encrypted form in a data storage ((Anglin [0063]) “When a data provider wants to sell a piece of training or testing data, the raw data is firstly formatted into a data entity 320 and data is then embedded into a privacy preserving signature vector. After that, referring now to FIGS. 2 and 3, access controller 204a, associated with the corresponding data source, generates an Advanced Encryption Standard (AES) key 314 and provides 318 the data encrypted 316 by the AES key 314 to data source selector 202. Data provider 302 then creates smart contract 246 in the blockchain ledger 248 as described above”, training and testing data includes labels, Gokalp teaches a set of labeling metrics of an agent)
wherein, [after acquiring labels during a current iteration of the labeling method] has been finished, the encrypted [set of labeling metrics] is retrieved from the data storage and is decrypted ((Anglin [0067]) “Once the transaction 310 is confirmed in the blockchain ledger 248, access controller 204a authenticates the data consumer 30 using consumer's private key information and provides encrypted data of interest to model engine 210. The data download begins once the payment is verified by the server and the data is decrypted using the AES key 314”, Gokalp teaches an iterative labeling method and labeling metrics)
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr, Gokalp, Asar, Wang, and Anglin by taking the method for training a classification model to control an additive manufacturing process, including labels and labeling metrics from a labeling method, jointly taught by Mehr, Gokalp, Asar, and Wang, and combining it with the data encryption and transmission process taught by Anglin, as data encryption for data transmitted to and from data storage is well-known within the computing arts for preventing unauthorized access to the data, yielding the predictable benefit of increased security. Such a combination would be obvious.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Mehr, further in view of Gokalp, further in view of Asar, further in view of Wang, further in view of Anglin, further in view of Horvitz et al. (U.S. Patent Application Publication No. 2014/0278657), hereinafter Horvitz.
Regarding claim 10,
Mehr, Gokalp, Asar, Wang, and Anglin jointly teach The computer-implemented method of claim 8,
Horvitz teaches the following further limitation that neither Mehr, nor Gokalp, nor Asar, nor Wang, nor Anglin teaches:
wherein the [labeling score for an agent of one of the groups of agents], the respective set of metrics and the one or more labels of one of the data entities acquired from this agent are provided to an agent of another group of the groups of agents depending on whether, during a current iteration of the labeling method, labels for the data entities may still be acquired from the other group ((Horvitz [0063]) “Step 206 represents determining the workers to hire. This may be based upon the task data, e.g., skill level, deadline (versus availability), budget and so forth may be factored into selection of the desired set of workers”, hiring workers based on availability corresponds to providing labels to a different group of agents if labels cannot be acquired from the other group, Horvitz Fig. 2 shows that step 206 where workers are hired in part based on availability can repeat multiple times for the same task, (Horvitz [0002]) “Crowdsourcing tasks are generally computer-based digital tasks, examples of which include…image labeling”, Gokalp teaches labeling scores for agents)
PNG
media_image3.png
772
394
media_image3.png
Greyscale
At the time of filing, one of ordinary skill in the art would have motivation to combine Mehr, Gokalp, Asar, Wang, Anglin, and Horvitz by taking the method for training a classification model to control an additive manufacturing process, including labeling scores for agents and groups of agents, jointly taught by Mehr, Gokalp, Asar, Wang, and Anglin, and combining it with the method for allocating tasks to workers/agents taught by Horvitz, as allocation of tasks to workers/agents based on availability yields the predictable benefit of allowing essential tasks to be completed even if the workers/agents of a previous task are no longer available. Such a combination would be obvious.
Claims 15, 18, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Gokalp, further in view of Asar, further in view of Wang, further in view of Anglin.
Regarding claim 15,
Gokalp teaches A computer-implemented method, said method comprising:
training a labeling score model based on data entities [acquired by a sensor assembly of a controlling apparatus], one or more labels acquired from an agent for each data entity, ((Gokalp [0039]) “In at least some embodiments, at least two types of models may be trained iteratively: (a) a set of one or more models whose output with respect to a training set is used to select candidates for labeling feedback for subsequent training iterations (e.g., using an active learning algorithm which uses variance in predictions among the different models for a given data item)”, a model or set of models for selecting candidates for labeling feedback corresponds to a labeling score model, a data item corresponds to a data entity, a prediction corresponds to a label, Gokalp does not teach a sensor assembly), at least one set of labeling metrics based on the acquiring from the agent (Gokalp [0045]) “In at least some embodiments, as the training iterations proceed, the interactions with individual label providers may be analyzed, e.g., to determine which label providers are more proficient in identifying particular classes of data items, to determine the rate at which individual label providers are able to generate labels, and so on…one or more metrics pertaining to label submission by the label provider such as the rate at which labels are generated, a comparison of the labels with respect to predicted classes, and so on”, the metrics pertaining to label submission correspond to labeling metrics)
wherein the classification model is an artificial intelligence model ((Gokalp [0035]) “Various embodiments of methods and apparatus for efficient training of machine learning models such as classifiers…are described”) that classifies data entities into correct categories, ((Gokalp [0038] “The given training iteration may, in some embodiments, also comprise generating, using one or more classifiers, classification predictions corresponding to a test set of data items”))
wherein the at least one set of labeling metrics is indicative of an acquiring of the label for the respective data entity ((Gokalp [0045]) “one or more metrics pertaining to label submission by the label provider such as the rate at which labels are generated, a comparison of the labels with respect to predicted classes, and so on”)
Asar teaches the following further limitation that Gokalp does not explicitly teach:
and a classifier score obtained from a validation of a trained classification model using a validation set, ((Asar [0071]) “Validation dataset: Dataset used for validating the model during the learning phase and to estimate the prediction error for model selection”, prediction error of a model corresponds to a classifier score, a person of ordinary skill in the art understands a validation set to be used to validate the training of a model)
wherein the classifier score is a numerical measure of how well the trained classification model fits the validation set, ((Asar [0148]-[0149]) “We perform a single split and select a set of optimization parameters for training/validation. If this is a classification problem, then once training has been performed, we perform validation using multiple thresholds (assume T number of thresholds)…For each threshold value, we calculate validation error rate for that threshold as follows: errate=Sum(LF across all inputs in the validation set)/(Total number of element in the validation set)”, a validation error rate for a classification problem corresponds to a classifier score that is a numerical measure of how well a trained classification model fits a validation set)
At the time of filing, one of ordinary skill in the art would have motivation to combine Gokalp, and Asar by taking the method for training a classification model, including acquiring labels from agents, taught by Gokalp, and including the use of validation of the model on a validation dataset to determine the performance of the model, taught by Asar, as use of a validation dataset to validate machine learning models is a well-known technique within the art for getting early feedback on a model being trained to tune parameters, without exposing it to the test data set used to evaluate its ultimate performance. Such a combination would be obvious.
Wang teaches the following further limitation that neither Gokalp, nor Asar teaches:
and wherein the labeling score model is an artificial intelligence model that generates and outputs a labeling score which is a numerical measure of an efficiency of a labeling process performed by the agent and of a quality of a label obtained from the agent ((Wang Pg. 6) “In this section, we describe several algorithms for inferring the true classes of objects and the quality of workers…Another advanced inference technique is EM, first proposed by Dawid and Skene (1979) in the context of medical diagnosis. The algorithm iterates until convergence, following two steps: (1) it estimates the true class for each object using the labels provided by a set of workers, accounting for the error rates of each worker; and (2) it estimates the error rates of each worker by comparing the submitted labels with estimated true class for each object…we propose a generative model of labels, abilities, and difficulties (GLAD) and use an EM approach to obtain the maximum likelihood estimates of the α(k) , β(o) , and t(o) for each worker (k) and each object (o)”, Wang Pg. 8, Algorithm 4 shows inference of an artificial intelligence model that generates scores for labels of objects and quality of workers that provide labels)
At the time of filing, one of ordinary skill in the art would have motivation to combine Gokalp, Asar, and Wang by taking the method for training a classification model, including acquiring labels from agents and validation on a validation set, taught jointly by Gokalp and Asar, and including a labelling score artificial intelligence model to create a score that evaluates the labelling efficiency of an agent and their label quality, taught by Wang, as Wang teaches: (Wang Pg. 19) “we introduce two novel metrics that can be used to objectively rank the performance of crowdsourced workers, both allowing employers to separate workers’ correctable errors from uncorrectable errors and incorporate unequal costs of different types of classification errors. In particular, the contributed value metric directly measures worker’s individual contribution in quality assurance through redundancy and provides a basis for employers to develop more fair and efficient compensation schemes”. Such a combination would be obvious.
Anglin teaches the following further limitations that neither Gokalp, nor Asar, nor Wang teaches:
storing, after said training, the [labeling score] model in a distributed database; ((Anglin [0027]) “As an overview, a blockchain is a distributed database that maintains a continuously growing list of data records”), (Anglin [0006]) “Changes to the machine learning model, training data or testing data made by at least one of the two or more entities are tracked and posted to a ledger of the blockchain infrastructure according to terms and specifications of a smart contract”, Gokalp teaches a labeling score model)
At the time of filing, one of ordinary skill in the art would have motivation to combine Gokalp, Asar, Wang, and Anglin by taking the method for training classification models, including acquiring labels from agents, and validation on a validation set, and determining a labeling score using an artificial intelligence model, taught jointly by Gokalp, Asar, and Wang, and combining it with the distributed database from which models are stored taught by Anglin, as distributed databases and data retrieval from a database are both well-known within the computing arts, yielding the predictable benefit of increased fault tolerance, as both centralized databases and data kept only locally are vulnerable to the singular system they rely on becoming inoperable. Such a combination would be obvious.
Mehr teaches the following further limitations that neither Gokalp, nor Asar, nor Wang, nor Anglin teaches:
…data entities acquired by a sensor assembly ((Mehr [0125]) “The automated object defect classification methods will generally comprise: b) providing one or more sensors, wherein the one or more sensors provide real-time data for one or more object properties”) of a controlling apparatus… ((Mehr [0022]) “FIG. 13 provides a schematic illustration of a distributed system comprising an additive manufacturing deposition apparatus, machine vision systems and/or other process monitoring tools, process simulation tools, postbuild inspection tools, and a processor for running a machine learning algorithm that utilizes data from the machine vision and/or process monitoring tools, the process simulation tools, the post-build inspection tools, or any combination thereof, to provide real-time adaptive control of the deposition process”)
wherein the classification model was received by a data processing apparatus from a data storage of the controlling apparatus configured to classify a manufacturing of a product and to control the manufacturing, ((Mehr [0170]) “Some aspects of the methods and systems provided herein, such as the disclosed object defect classification or additive manufacturing process control algorithms, are implemented by way of machine (e.g., processor) executable code stored in an electronic storage location of the computer system, such as, for example, in the memory or electronic storage unit…In some cases, the code is retrieved from the storage unit and stored in the memory for ready access by the one or more processors”, the storage unit corresponds to a data storage of the controlling apparatus)
At the time of filing, one of ordinary skill in the art would have motivation to combine Gokalp, Asar, Wang, Anglin, and Mehr by taking the method for training classification models, including acquiring labels from agents, and validation on a validation set, and determining a labeling score using an artificial intelligence model stored in a distributed database, taught jointly by Gokalp, Asar, Wang, and Anglin, and combining it with the method for controlling a manufacturing process with a machine learning model, taught by Mehr, as doing so would increase the quality of the machine learning models used by a manufacturing process by ensuring the labels provided to train the model in use in the manufacturing process are high quality. Such a combination would be obvious.
Regarding claim 18,
Gokalp, Asar, Wang, Anglin, and Mehr jointly teach The computer-implemented method of claim 15, said method further comprising:
Anglin further teaches:
retrieving a smart contract from the distributed database, ((Anglin [0065]) “Whenever a data access request is initiated by access controller 204a on a certain data entity, the access controller 204a first retrieves its associated smart contract 246 in blockchain ledger 248”, (Anglin [0027]) “As an overview, a blockchain is a distributed database that maintains a continuously growing list of data records”)
wherein the smart contract was previously generated based on the trained [labeling score] model and stored in the distribution database, ((Anglin [0027]) “As an overview, a blockchain is a distributed database that maintains a continuously growing list of data records”), (Anglin [0006]) “Changes to the machine learning model, training data or testing data made by at least one of the two or more entities are tracked and posted to a ledger of the blockchain infrastructure according to terms and specifications of a smart contract”, Gokalp teaches a labeling score model)
and wherein the smart contract is configured to generate a new [labeling score]; ((Anglin [0061]) “At least in some embodiments, model engine 210 may be configured to provide some kind of output 238. In one embodiment, output 238 may include results 240 provided by the model requested by model owner 234 and/or model consumer 236”, (Anglin [0059]) “As noted above, smart contract 246 governs functionality of one or more model engines 210”, a smart contract that governs the functionality of a model engine that provides output corresponds to a smart contract configured to generate output, Gokalp teaches a labeling score)
and executing the smart contract on a [labeling] system that stores the at least one label, the set of [labeling] metrics and the [labeling score] model, ((Anglin [0053]) “The disclosed techniques would allow model trainers 234 to study information within the private image collections without requiring access to the specific images. Such a feat can be achieved by, assuming the owner's permission is granted via smart contract 246, installing model engine 210 on each person's computer. The model engine 210 can receive local training data 104 in the form of original images along with other training information (e.g., annotations, classifications, scene descriptions, locations, time, settings, camera orientations, etc.)”, executing a smart contract to grant permission to install a model engine governed by the smart contract on a system with training data such as annotations (i.e. labels) and metrics corresponds to executing a smart contract on a system that stores labels, metrics, and a model, Gokalp teaches a labelling system and storage for labels, and labeling metrics, and a labeling score model)
said executing the smart contract comprising generating the new [labeling score] ((Anglin [0061]) “At least in some embodiments, model engine 210 may be configured to provide some kind of output 238. In one embodiment, output 238 may include results 240 provided by the model requested by model owner 234 and/or model consumer 236”, (Anglin [0059]) “As noted above, smart contract 246 governs functionality of one or more model engines 210”, a smart contract that governs the functionality of a model engine that provides output corresponds to a smart contract that when executed generates output, Gokalp teaches a labeling score)
At the time of filing, one of ordinary skill in the art would have motivation to combine Gokalp, Asar, Wang, Anglin, and Mehr by taking the method for training a classification models, including acquiring labels from agents, and validation on a validation set, and determining a labeling score using an artificial intelligence model stored in a distributed database, taught jointly by Gokalp, Asar, Wang, Anglin, and Mehr, and combining it with the method for governing usage of a machine learning model using a smart contract, as taught by Anglin, as Anglin teaches: (Anglin [0006]) “The smart contract enables two or more entities to collaboratively produce the machine learning model based on the training data using the blockchain infrastructure”. Such a combination would be obvious.
Regarding claim 19,
Gokalp, Asar, Wang, Anglin, and Mehr jointly teach The computer-implemented method of claim 15, said method further comprising:
Mehr further teaches:
controlling a manufacturing process, ((Mehr [0001]) “Also disclosed are methods and systems for performing real-time adaptive control of free form deposition or joining processes, including additive manufacturing or welding processes”)
wherein the manufacturing process manufactures products, ((Mehr [0001]) “Additive manufacturing processes are fabrication techniques that allow one to produce functional complex parts layer by layer, without the use of molds or dies”)
wherein said controlling is based on a classification of the data entities by the classification model into a correct category, ((Mehr [0031]) “in-process inspection data (e.g., automated defect classification data) may be used by the machine learning algorithm to determine a set or sequence of process control parameter adjustments that will implement a corrective action, e.g., to adjust a layer dimension or thickness, so as to correct the defect when first detected”)
and wherein said controlling increases a yield of the products manufactured ((Mehr [0001]) “Also disclosed are methods and systems for performing real-time adaptive control of free form deposition or joining processes, including additive manufacturing or welding processes, to improve process yield, throughput, and quality”)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Gokalp, Asar, Wang, Anglin, and Mehr for the parent claim of claim 19, claim 15. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Bruwer et al. (U.S. Patent Application Publication No. 2018/0162066) teaches analysis of a sound emitted during an additive manufacturing process in order to determine if an error has occurred.
Kumar et al. (U.S. Patent Application Publication No. 2020/0242395) teaches machine learning for image measurement in a manufacturing process.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VICTOR A NAULT whose telephone number is (703) 756-5745. The examiner can normally be reached M - F, 12 - 8.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at (571) 270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/V.A.N./Examiner, Art Unit 2124
/Kevin W Figueroa/Primary Examiner, Art Unit 2124
Prosecution Insights