DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 3/20/2026 has been entered. Claims 6 has been canceled, claims 1-5 and 7-21 remain pending in the application.
Claim Objections
Claim 13-21 are objected to because of the following informalities:
Claim 13 and 17 recite “the real images does not capture the anomaly associated with the particular industrial facility”, however claim 14 and 21 recite “one or more real images each capturing a corresponding anomaly”; independent and dependent claims conflict with each other.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1, 7, 9 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Cella U.S. Patent Application 20220187847 in view of Maschmeyer U.S. Patent Application 20240161258, and further in view of Mietzner U.S. Patent Application 20180094231.
Regarding claim 1, Cella discloses a method implemented by one or more processors, the method comprising:
for each of multiple text strings that each describe an anomaly and a corresponding industrial facility setting (paragraph [0487]: Supervised anomaly detection may include training on a data set wherein at least a portion of the training data has been labeled as “normal” and/or “abnormal.”; paragraph [0367]: a labeled data set where labels or tags indicate types of defects, favorable properties, or other characteristics, such that a machine learning system can learn, using the training data set, to identify the same characteristics):
training an anomaly detection machine learning (ML) model using the images and corresponding supervised labels for the images (paragraph [0367]: a labeled data set where labels or tags indicate types of defects, favorable properties, or other characteristics, such that a machine learning system can learn, using the training data set, to identify the same characteristics, which in turn can be used to automate the inspection process such that defects or favorable properties are automatically classified and detected in a set of video or still images; paragraph [0464]: The machine learning model 3000 may receive inputs of sensor data as training data, including event data 1034 and state data 1140 related to one or more of the value chain entities 652. The sensor data input to the machine learning model 3000 may be used to train the machine learning model 3000 to perform the analytics; paragraph [0487]: Supervised anomaly detection may include training on a data set wherein at least a portion of the training data has been labeled as “normal” and/or “abnormal.”); and
providing the trained anomaly detection ML model for use in anomaly detection within a particular industrial facility (paragraph [0487]: the machine learning model 3000 may be defined via anomaly detection, i.e., by identifying rare and/or outlier instances of one or more items, events and/or observations. The rare and/or outlier instances may be identified by the instances differing significantly from patterns and/or properties of a majority of the training data; paragraph [1510]: machine learning models may perform anomaly detection or outlier detection. For example, machine learning models can identify input data that does not conform to an expected pattern or other characteristic (e.g., as previously observed from previous input data). As examples, the anomaly detection can be used for fraud detection or system failure detection).
Cella discloses all the features with respect to claim 1 as outlined above. However, Cella fails to disclose performing multiple iterations of processing the text string, using a text-to-image model, to generate a corresponding synthetic image at each of the iterations, wherein the multiple text strings include a first text string that describes a first anomaly at a first location within a first industrial facility setting and that includes multiple first words and a second text string that describes a second anomaly at a second location within the first industrial facility setting and that includes multiple second words.
Maschmeyer discloses performing multiple iterations of processing the text string, using a text-to-image model, to generate a corresponding synthetic image at each of the iterations (paragraph [0071]: The image generation engine iteratively executes the customized generative model to obtain an output image satisfying certain defined criteria. The criteria may be defined by users of the image generative model; paragraph [0072]: In operation 304, the image generation engine obtains, via the customized generative model, an image generated based on an input; paragraph [0030]: the deep learning generative model may be configured to fine-tune a text-to-image diffusion model for training the customized generative model associated with the first product; paragraph [0040]: filtering the output of text-to-image models. The post-processing layer may include machine learning models that are trained to detect realism or anomalies in object composition).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella’s to use text-to-image model as taught by Maschmeyer, to generate image based on text input.
Cella as modified by Maschmeyer discloses all the features with respect to claim 1 as outlined above. However, Cella as modified by Maschmeyer fails to disclose the multiple text strings include a first text string that describes a first anomaly at a first location within a first industrial facility setting and that includes multiple first words and a second text string that describes a second anomaly at a second location within the first industrial facility setting and that includes multiple second words explicitly.
Mietzner discloses the multiple text strings include a first text string that describes a first anomaly at a first location within a first industrial facility setting and that includes multiple first words and a second text string that describes a second anomaly at a second location within the first industrial facility setting and that includes multiple second words (paragraph [0031]: four types of surface imperfections are commonly observed in biopharmaceutical manufacturing product contact surfaces—scratches, roughness, micropits, and pitting… a “pit” or “pitting” is a surface void having a measurable depth that is generally annular, circular, oval, or oblong in shape. As used herein, “scratch” means a surface void having a substantially linear shape with a measurable depth. As used herein, “vessel,” “reactor vessel,” and/or “processing equipment” means any device or system with at least one surface that comes in contact with process materials, including but not limited to tanks, pipes, filters, bioreactors, product hold vessels, WFI hold vessels, chromatography skids, ultrafiltration/diafiltration skids, filter housings, and any process contact surface that is cleaned via recirculating CIP (different locations within a first industrial facility setting); Mietzner’s teaching of using multiple text strings to define anomaly can be used in Cella and Maschmeyer’s device, such as to generate images for different anomalies to train anomaly detection model).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella and Maschmeyer’s to use multiple text strings as taught by Mietzner, to describe anomaly precisely.
Regarding claim 7, Cella as modified by Maschmeyer and Mietzner discloses the method of claim 1, wherein the multiple text strings include one text string describing one anomaly within one industrial facility setting and an additional text string describing the one anomaly within an additional industrial facility setting (Maschmeyer’s paragraph [0040]: detect structural anomalies in the subjects (e.g., counts of limbs (and fingers, toes, etc.), skeletal aligning, and the like); detect lighting anomalies on the subjects/scene depicted in the output images; detect anomalies in text or logos depicted in images; paragraph [0081]: if a detected anomaly in a generated sample relates to the number of fingers of a human subject, a corresponding modification text may comprise “with five fingers”. The text prompt may then be automatically modified to include this modification text. As another example, upon detecting a defect in light projections and/or shadows in a generated sample, a corresponding modification text such as “with correct shadow of [subject]” or “with consistent light and shadow conditions” may be included in the modified text prompt).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella’s to use text-to-image model as taught by Maschmeyer, to generate image based on text input; and combine Cella and Maschmeyer’s to use multiple text strings as taught by Mietzner, to describe anomaly precisely.
Regarding claim 9, Cella as modified by Maschmeyer and Mietzner discloses the method of claim 1, wherein providing the trained anomaly detection ML model for use in anomaly detection within the particular industrial facility comprises:
causing the trained anomaly detection ML model to be used in processing real images that are captured via a vision sensor that is within the particular industrial facility (Cella’s paragraph [0487]: the machine learning model 3000 may be defined via anomaly detection, i.e., by identifying rare and/or outlier instances of one or more items, events and/or observations. The rare and/or outlier instances may be identified by the instances differing significantly from patterns and/or properties of a majority of the training data; paragraph [1510]: machine learning models may perform anomaly detection or outlier detection. For example, machine learning models can identify input data that does not conform to an expected pattern or other characteristic (e.g., as previously observed from previous input data). As examples, the anomaly detection can be used for fraud detection or system failure detection).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella’s to use text-to-image model as taught by Maschmeyer, to generate image based on text input; and combine Cella and Maschmeyer’s to use multiple text strings as taught by Mietzner, to describe anomaly precisely.
Regarding claim 11, Cella as modified by Maschmeyer and Mietzner discloses the method of claim 9, wherein the vision sensor is carried by a mobile robot, and the real image is captured by the mobile robot at a designated location within the particular industrial facility (Cella’s paragraph [1596]: The chip(s) 9100 may be part of a mobile system (e.g., a robot), and/or may be part of a different device (e.g., a base station in communication with the robot) that receives inputs 9192 from the mobile system. A mobile system may include any system that is mobile and/or that has one or more mobile components as described herein; see fig. 109, physical orientation determination chip 9100 has image sensor capture circuit 9132).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella’s to use text-to-image model as taught by Maschmeyer, to generate image based on text input; and combine Cella and Maschmeyer’s to use multiple text strings as taught by Mietzner, to describe anomaly precisely.
Claim 2, 4-5, 8 are rejected under 35 U.S.C. 103 as being unpatentable over Cella U.S. Patent Application 20220187847 in view of Maschmeyer U.S. Patent Application 20240161258, in view of Mietzner U.S. Patent Application 20180094231, and further in view of Liu U.S. Patent Application 20210232932.
Regarding claim 2, Cella as modified by Maschmeyer and Mietzner discloses prior to providing the trained anomaly detection ML model for use in anomaly detection within the particular industrial facility: for each of multiple real images of the particular industrial facility: processing the real image and a prompt that describes the anomaly, to generate a corresponding translated synthetic image; and fine-tuning the trained anomaly detection ML model through further training of the anomaly detection ML model based on the translated synthetic images (Cella's paragraph [0464]: The machine learning model 3000 may receive inputs of sensor data as training data, including event data 1034 and state data 1140 related to one or more of the value chain entities 652. The sensor data input to the machine learning model 3000 may be used to train the machine learning model 3000 to perform the analytics; paragraph [0487]: Supervised anomaly detection may include training on a data set wherein at least a portion of the training data has been labeled as “normal” and/or “abnormal.”; Maschmeyer’s paragraph [0030]: the deep learning generative model may be configured to fine-tune a text-to-image diffusion model for training the customized generative model associated with the first product). However, Cella as modified by Maschmeyer and Mietzner fails to disclose using an image-to-image translation model.
Liu discloses using an image-to-image translation model (paragraph [0071]: the image-to-image translation model being configured for translating an inputted image from an image category to which the inputted image belongs into an image of another image category).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella, Maschmeyer and Mietzner’s to use image-to-image translation model as taught by Liu, to generate image base on other images.
Regarding claim 4, Cella as modified by Maschmeyer, Mietzner and Liu discloses the method of claim 1, further comprising:
prior to providing the trained anomaly detection ML model for use in anomaly detection within the particular industrial facility:
for each of multiple real images of the particular industrial facility: processing the real image and a prompt that describes the anomaly, using an image-to-image translation model, to generate a corresponding translated synthetic image (Cella's paragraph [0464]: The machine learning model 3000 may receive inputs of sensor data as training data, including event data 1034 and state data 1140 related to one or more of the value chain entities 652. The sensor data input to the machine learning model 3000 may be used to train the machine learning model 3000 to perform the analytics; paragraph [0487]: Supervised anomaly detection may include training on a data set wherein at least a portion of the training data has been labeled as “normal” and/or “abnormal.”; Liu’s paragraph [0071]: the image-to-image translation model being configured for translating an inputted image from an image category to which the inputted image belongs into an image of another image category); and
validating the trained anomaly detection ML model based on the translated synthetic images (Cella's paragraph [0479]: by using a set of testing data that is independent from the data set used for the creation and/or training of the machine learning model, such as to test the impact of various inputs to the accuracy of the model 3000; Maschmeyer’s paragraph [0030]: the deep learning generative model may be configured to fine-tune a text-to-image diffusion model for training the customized generative model associated with the first product; paragraph [0095]: Only the remaining reference images after application of the pre-processing filter may be used to train the customized generative model. Additionally, or alternatively, the pre-processing filter may inform the direct editing or modifying of reference images prior to the training; paragraph [0096]: The pre-processing filter may also be used to identify other parameters that are conducive to refining the customized generative model);
wherein providing the trained anomaly detection ML model for use in anomaly detection within the particular industrial facility is in response to determining the validating satisfies one or more conditions (Maschmeyer’s paragraph [0071]: The image generation engine iteratively executes the customized generative model to obtain an output image satisfying certain defined criteria. The criteria may be defined by users of the image generative model; paragraph [0040]: filtering the output of text-to-image models. The post-processing layer may include machine learning models that are trained to detect realism or anomalies in object composition).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella, Maschmeyer and Mietzner’s to use image-to-image translation model as taught by Liu, to generate image base on other images.
Regarding claim 5, Cella as modified by Maschmeyer, Mietzner and Liu discloses the method of claim 4, wherein validating the trained anomaly detection ML model based on the translated synthetic images comprises determining an accuracy measure of anomaly predictions made based on outputs, from the anomaly detection ML model, based on processing the translated synthetic images (Cella's paragraph [0479]: by using a set of testing data that is independent from the data set used for the creation and/or training of the machine learning model, such as to test the impact of various inputs to the accuracy of the model 3000); and
wherein determining the validating satisfies one or more conditions comprises determining that the accuracy measure satisfies a threshold accuracy measure (Maschmeyer’s paragraph [0071]: The image generation engine iteratively executes the customized generative model to obtain an output image satisfying certain defined criteria. The criteria may be defined by users of the image generative model).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella, Maschmeyer and Mietzner’s to use image-to-image translation model as taught by Liu, to generate image base on other images.
Regarding claim 8, Cella as modified by Maschmeyer, Mietzner and Liu discloses the method of claim 1, wherein the corresponding supervised labels for the generated synthetic images are automatically determined based at least on the multiple text strings describing the anomaly (Liu's paragraph [0071]: labeling the virtual image according to the image category to which the virtual image belongs, to generate a virtual image sample having a classification label; and performing supervised training on a second generative adversarial network using the virtual image sample; Maschmeyer’s paragraph [0081]: if a detected anomaly in a generated sample relates to the number of fingers of a human subject, a corresponding modification text may comprise “with five fingers”. The text prompt may then be automatically modified to include this modification text).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella, Maschmeyer and Mietzner’s to use image-to-image translation model as taught by Liu, to generate image base on other images.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Cella U.S. Patent Application 20220187847 in view of Maschmeyer U.S. Patent Application 20240161258, in view of Mietzner U.S. Patent Application 20180094231, in view of Liu U.S. Patent Application 20210232932, and further in view of Bhuiyan U.S. Patent Application 20200242763.
Regarding claim 3, Cella as modified by Maschmeyer, Mietzner and Liu discloses fine-tuning the trained anomaly detection ML model (Maschmeyer’s paragraph [0030]: the deep learning generative model may be configured to fine-tune a text-to-image diffusion model for training the customized generative model associated with the first product; Cella’s paragraph [1537]: The weights are adjusted during the training process and this adjustment of weights to determine the best set of weights that maximize the accuracy of the neural network is referred to as training). However, Cella as modified by Maschmeyer, Mietzner and Liu fails to disclose fine-tuning weights of one or more layers.
Bhuiyan discloses fine-tuning weights of one or more layers (paragraph [0124]: combining transfer learning (by using weights from a pre-trained network, trained on ImageNet) and fine-tuning these weights by allowing all the layers to continue learning through a slow learning rate (usually it is done in the last few layers)).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella, Maschmeyer, Mietzner and Liu’s to fine-tune weights of layers as taught by Bhuiyan, to shorten the training time.
Claim 10 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Cella U.S. Patent Application 20220187847 in view of Maschmeyer U.S. Patent Application 20240161258, in view of Mietzner U.S. Patent Application 20180094231, and further in view of Jiang U.S. Patent Application 20220323999.
Regarding claim 10, Cella as modified by Maschmeyer and Mietzner discloses using the trained anomaly detection ML model in processing the real images, wherein using the trained anomaly detection ML model in processing the real images comprises: processing a real image of the real images, using the trained anomaly detection ML model, to generate a model output that indicates whether any anomaly is present in one or more of the components (Cella's paragraph [0487]: the machine learning model 3000 may be defined via anomaly detection, i.e., by identifying rare and/or outlier instances of one or more items, events and/or observations. The rare and/or outlier instances may be identified by the instances differing significantly from patterns and/or properties of a majority of the training data; paragraph [1510]: machine learning models may perform anomaly detection or outlier detection. For example, machine learning models can identify input data that does not conform to an expected pattern or other characteristic (e.g., as previously observed from previous input data). As examples, the anomaly detection can be used for fraud detection or system failure detection). However, Cella as modified by Maschmeyer and Mietzner fails to disclose causing one or more remediating actions to be performed in response to the output indicating that an anomaly is present in one or more of the components.
Jiang discloses causing one or more remediating actions to be performed in response to the output indicating that an anomaly is present in one or more of the components (paragraph [0096]: At S1620, in response to a trigger instruction of the anomaly early-warning message, the anomaly positioning information and a detected target image having the anomaly are presented on a front-end page).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella, Maschmeyer and Mietzner’s to display warning message as taught by Jiang, to automatically monitor and detect the anomalies and improve production.
Regarding claim 12, Cella as modified by Maschmeyer, Mietzner and Jiang discloses the method of claim 10, wherein the one or more remediating actions include a warning message alerting detection of the anomaly in one or more of the components at the designated location (Jiang’s paragraph [0096]: At S1620, in response to a trigger instruction of the anomaly early-warning message, the anomaly positioning information and a detected target image having the anomaly are presented on a front-end page).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella, Maschmeyer and Mietzner’s to display warning message as taught by Jiang, to automatically monitor and detect the anomalies and improve production.
Claim 13 and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Cella U.S. Patent Application 20220187847 in view of Maschmeyer U.S. Patent Application 20240161258, and further in view of Harary U.S. Patent Application 20250005727.
Regarding claim 13, Cella discloses a method implemented by one or more processors, the method comprising:
for each of multiple real images of a particular industrial facility: processing the real image and a prompt that includes multiple words and that describes an anomaly associated with the particular industrial facility, to generate a corresponding model (paragraph [0367]: a labeled data set where labels or tags indicate types of defects, favorable properties, or other characteristics, such that a machine learning system can learn, using the training data set, to identify the same characteristics, which in turn can be used to automate the inspection process such that defects or favorable properties are automatically classified and detected in a set of video or still images; paragraph [0464]: The machine learning model 3000 may receive inputs of sensor data as training data, including event data 1034 and state data 1140 related to one or more of the value chain entities 652. The sensor data input to the machine learning model 3000 may be used to train the machine learning model 3000 to perform the analytics; paragraph [0487]: Supervised anomaly detection may include training on a data set wherein at least a portion of the training data has been labeled as “normal” and/or “abnormal.”).
Cella discloses all the features with respect to claim 13 as outlined above. However, Cella fails to disclose a prompt that includes multiple words explicitly, generating a corresponding translated synthetic image, wherein the image does not capture the anomaly associated with the particular object; fine-tuning a trained anomaly detection ML model through further training of the trained anomaly detection ML model based on the translated synthetic images and using an image-to-image translation model.
Maschmeyer discloses a prompt that includes multiple words (paragraph [0040]: detect structural anomalies in the subjects (e.g., counts of limbs (and fingers, toes, etc.), skeletal aligning, and the like); detect lighting anomalies on the subjects/scene depicted in the output images; detect anomalies in text or logos depicted in images; paragraph [0081]: if a detected anomaly in a generated sample relates to the number of fingers of a human subject, a corresponding modification text may comprise “with five fingers”. The text prompt may then be automatically modified to include this modification text. As another example, upon detecting a defect in light projections and/or shadows in a generated sample, a corresponding modification text such as “with correct shadow of [subject]” or “with consistent light and shadow conditions” may be included in the modified text prompt),
generating a corresponding translated synthetic image (paragraph [0071]: The image generation engine iteratively executes the customized generative model to obtain an output image satisfying certain defined criteria. The criteria may be defined by users of the image generative model; paragraph [0072]: In operation 304, the image generation engine obtains, via the customized generative model, an image generated based on an input);
fine-tuning a trained anomaly detection ML model through further training of the trained anomaly detection ML model based on the translated synthetic images (paragraph [0030]: the deep learning generative model may be configured to fine-tune a text-to-image diffusion model for training the customized generative model associated with the first product; paragraph [0040]: filtering the output of text-to-image models. The post-processing layer may include machine learning models that are trained to detect realism or anomalies in object composition).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella’s to fine-tune model as taught by Maschmeyer, to generate image based on text input.
Cella as modified by Maschmeyer discloses all the features with respect to claim 13 as outlined above. However, Cella as modified by Maschmeyer fails to disclose using an image-to-image translation model, wherein the image does not capture the anomaly associated with the particular object.
Harary discloses using an image-to-image translation model, wherein the image does not capture the anomaly associated with the particular object (paragraph [0081]: at block 705, where a system (e.g., the computing device 900 of FIG. 9) processes, using a trained model (e.g., 580 of FIG. 5), a plurality of normal images (e.g., 502 of FIG. 5) for an object to generate a set of normal features; paragraph [0082]: At block 710, the system processes, using the trained model, a plurality of negative text exemplars (e.g., 510 of FIG. 5) describing the object to generate a set of anomaly features; paragraph [0083]: At block 715, the system generates a plurality of anomaly images (e.g., 505 of FIG. 5) for the object based at least in part on the set of normal features (normal images)(e.g., 542 of FIG. 5) and the set of anomaly features).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella and Maschmeyer’s to use image-to-image translation model as taught by Harary, to maintain the trained model's accuracy.
Regarding claim 16, Cella as modified by Maschmeyer and Harary discloses the method of claim 13, wherein the trained anomaly detection ML model is previously trained based on a multiple synthetic images that respectively depict a realistic scene of a corresponding anomaly in an industrial facility setting, the multiple synthetic images generated using a text-to-image model based on one or more text string each describing an anomaly (Maschmeyer’s paragraph [0071]: The image generation engine iteratively executes the customized generative model to obtain an output image satisfying certain defined criteria. The criteria may be defined by users of the image generative model; paragraph [0072]: In operation 304, the image generation engine obtains, via the customized generative model, an image generated based on an input; paragraph [0030]: the deep learning generative model may be configured to fine-tune a text-to-image diffusion model for training the customized generative model associated with the first product; paragraph [0040]: filtering the output of text-to-image models. The post-processing layer may include machine learning models that are trained to detect realism or anomalies in object composition).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella’s to use text-to-image model as taught by Maschmeyer, to generate image based on text input; and combine Cella and Maschmeyer’s to use image-to-image translation model as taught by Harary, to maintain the trained model's accuracy.
Regarding claim 17, Cella discloses a method implemented by one or more processors, the method comprising:
for each of multiple real images of a particular industrial facility: processing the real image and a prompt that includes multiple words and that describes an anomaly associated with the particular industrial facility, to generate a corresponding model (paragraph [0367]: a labeled data set where labels or tags indicate types of defects, favorable properties, or other characteristics, such that a machine learning system can learn, using the training data set, to identify the same characteristics, which in turn can be used to automate the inspection process such that defects or favorable properties are automatically classified and detected in a set of video or still images; paragraph [0464]: The machine learning model 3000 may receive inputs of sensor data as training data, including event data 1034 and state data 1140 related to one or more of the value chain entities 652. The sensor data input to the machine learning model 3000 may be used to train the machine learning model 3000 to perform the analytics; paragraph [0487]: Supervised anomaly detection may include training on a data set wherein at least a portion of the training data has been labeled as “normal” and/or “abnormal.”);
validating a trained anomaly detection ML model based on the images (paragraph [0479]: by using a set of testing data that is independent from the data set used for the creation and/or training of the machine learning model, such as to test the impact of various inputs to the accuracy of the model 3000).
Cella discloses all the features with respect to claim 17 as outlined above. However, Cella fails to disclose a prompt that includes multiple words, generating a corresponding translated synthetic image, and using an image-to-image translation model, wherein the image does not capture the anomaly associated with the particular object.
Maschmeyer discloses a prompt that includes multiple words (paragraph [0040]: detect structural anomalies in the subjects (e.g., counts of limbs (and fingers, toes, etc.), skeletal aligning, and the like); detect lighting anomalies on the subjects/scene depicted in the output images; detect anomalies in text or logos depicted in images; paragraph [0081]: if a detected anomaly in a generated sample relates to the number of fingers of a human subject, a corresponding modification text may comprise “with five fingers”. The text prompt may then be automatically modified to include this modification text. As another example, upon detecting a defect in light projections and/or shadows in a generated sample, a corresponding modification text such as “with correct shadow of [subject]” or “with consistent light and shadow conditions” may be included in the modified text prompt),
generating a corresponding translated synthetic image (paragraph [0071]: The image generation engine iteratively executes the customized generative model to obtain an output image satisfying certain defined criteria. The criteria may be defined by users of the image generative model; paragraph [0072]: In operation 304, the image generation engine obtains, via the customized generative model, an image generated based on an input).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella’s to generate synthetic image as taught by Maschmeyer, to generate image based on text input.
Cella as modified by Maschmeyer discloses all the features with respect to claim 17 as outlined above. However, Cella as modified by Maschmeyer fails to disclose using an image-to-image translation model. wherein the image does not capture the anomaly associated with the particular object.
Harary discloses using an image-to-image translation model, wherein the image does not capture the anomaly associated with the particular object (paragraph [0081]: at block 705, where a system (e.g., the computing device 900 of FIG. 9) processes, using a trained model (e.g., 580 of FIG. 5), a plurality of normal images (e.g., 502 of FIG. 5) for an object to generate a set of normal features; paragraph [0082]: At block 710, the system processes, using the trained model, a plurality of negative text exemplars (e.g., 510 of FIG. 5) describing the object to generate a set of anomaly features; paragraph [0083]: At block 715, the system generates a plurality of anomaly images (e.g., 505 of FIG. 5) for the object based at least in part on the set of normal features (normal images)(e.g., 542 of FIG. 5) and the set of anomaly features).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella and Maschmeyer’s to use image-to-image translation model as taught by Harary, to maintain the trained model's accuracy.
Regarding claim 18, Cella as modified by Maschmeyer and Harary discloses the method of claim 17, further comprising:
determining whether the validating satisfies one or more conditions; and in response to determining the validating satisfies the one or more conditions, providing the trained anomaly detection ML model for use in anomaly detection within the particular industrial facility (Cella’s paragraph [0479]: by using a set of testing data that is independent from the data set used for the creation and/or training of the machine learning model, such as to test the impact of various inputs to the accuracy of the model 3000; Maschmeyer’s paragraph [0071]: The image generation engine iteratively executes the customized generative model to obtain an output image satisfying certain defined criteria. The criteria may be defined by users of the image generative model; paragraph [0040]: filtering the output of text-to-image models. The post-processing layer may include machine learning models that are trained to detect realism or anomalies in object composition).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella’s to use text-to-image model as taught by Maschmeyer, to generate image based on text input; and combine Cella and Maschmeyer’s to use image-to-image translation model as taught by Harary, to maintain the trained model's accuracy.
Regarding claim 19, Cella as modified by Maschmeyer and Harary discloses the method of claim 18, wherein validating the trained anomaly detection ML model based on the translated synthetic images comprises determining an accuracy measure of anomaly predictions made based on outputs, from the anomaly detection ML model, based on processing the translated synthetic images, and wherein the one or more conditions comprise a threshold accuracy measure (Cella’s paragraph [0479]: by using a set of testing data that is independent from the data set used for the creation and/or training of the machine learning model, such as to test the impact of various inputs to the accuracy of the model 3000; Maschmeyer’s paragraph [0071]: The image generation engine iteratively executes the customized generative model to obtain an output image satisfying certain defined criteria. The criteria may be defined by users of the image generative model; paragraph [0040]: filtering the output of text-to-image models. The post-processing layer may include machine learning models that are trained to detect realism or anomalies in object composition).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella’s to use text-to-image model as taught by Maschmeyer, to generate image based on text input; and combine Cella and Maschmeyer’s to use image-to-image translation model as taught by Harary, to maintain the trained model's accuracy.
Regarding claim 20, Cella as modified by Maschmeyer and Harary discloses the method of claim 17, wherein the trained anomaly detection ML model is previously trained based on multiple synthetic images that respectively depict a realistic scene of a corresponding anomaly in an industrial facility setting, wherein the multiple synthetic images are generated using a text-to-image model based on one or more text strings each describing an anomaly (Maschmeyer’s paragraph [0071]: The image generation engine iteratively executes the customized generative model to obtain an output image satisfying certain defined criteria. The criteria may be defined by users of the image generative model; paragraph [0072]: In operation 304, the image generation engine obtains, via the customized generative model, an image generated based on an input; paragraph [0040]: filtering the output of text-to-image models. The post-processing layer may include machine learning models that are trained to detect realism or anomalies in object composition).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella’s to use text-to-image model as taught by Maschmeyer, to generate image based on text input; and combine Cella and Maschmeyer’s to use image-to-image translation model as taught by Harary, to maintain the trained model's accuracy.
Claim 14-15 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Cella U.S. Patent Application 20220187847 in view of Maschmeyer U.S. Patent Application 20240161258, in view of Harary U.S. Patent Application 20250005727, and further in view of Jiang U.S. Patent Application 20220323999.
Regarding claim 14, Cella as modified by Maschmeyer and Harary discloses the trained anomaly detection ML model is previously trained based on one or more real images (Cella's paragraph [0464]: The machine learning model 3000 may receive inputs of sensor data as training data, including event data 1034 and state data 1140 related to one or more of the value chain entities 652. The sensor data input to the machine learning model 3000 may be used to train the machine learning model 3000 to perform the analytics). However, Cella as modified by Maschmeyer and Harary fails to disclose model is trained based on one or more real images each capturing a corresponding anomaly.
Jiang discloses model is trained based on one or more real images each capturing a corresponding anomaly (paragraph [0101]: after an anomalous image is detected, the image may be used to train and optimize the image recognition model, so as to further improve the accuracy of anomaly detection).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella, Maschmeyer and Harary’s to train model based on real anomaly image as taught by Jiang, to automatically monitor and detect the anomalies and improve production.
Regarding claim 15, Cella as modified by Maschmeyer, Harary and Jiang discloses the method of claim 14, wherein the corresponding anomaly is captured using a vision sensor within the particular industrial facility (Cella's paragraph [0487]: the machine learning model 3000 may be defined via anomaly detection, i.e., by identifying rare and/or outlier instances of one or more items, events and/or observations. The rare and/or outlier instances may be identified by the instances differing significantly from patterns and/or properties of a majority of the training data; paragraph [1510]: machine learning models may perform anomaly detection or outlier detection. For example, machine learning models can identify input data that does not conform to an expected pattern or other characteristic (e.g., as previously observed from previous input data). As examples, the anomaly detection can be used for fraud detection or system failure detection; paragraph [0464]: The machine learning model 3000 may receive inputs of sensor data as training data, including event data 1034 and state data 1140 related to one or more of the value chain entities 652. The sensor data input to the machine learning model 3000 may be used to train the machine learning model 3000 to perform the analytics).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella, Maschmeyer and Harary’s to train model based on real anomaly image as taught by Jiang, to automatically monitor and detect the anomalies and improve production.
Regarding claim 21, Cella as modified by Maschmeyer, Harary and Jiang discloses the method of claim 17, wherein the trained anomaly detection ML model is previously trained based on one or more real images each capturing a corresponding anomaly within an industrial facility setting (Cella's paragraph [0464]: The machine learning model 3000 may receive inputs of sensor data as training data, including event data 1034 and state data 1140 related to one or more of the value chain entities 652. The sensor data input to the machine learning model 3000 may be used to train the machine learning model 3000 to perform the analytics; Jiang’s paragraph [0101]: after an anomalous image is detected, the image may be used to train and optimize the image recognition model, so as to further improve the accuracy of anomaly detection).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Cella, Maschmeyer and Harary’s to train model based on real anomaly image as taught by Jiang, to automatically monitor and detect the anomalies and improve production.
Response to Arguments
Applicant's arguments filed 3/20/2026, page 9 - 10, with respect to the rejection(s) of claim(s) 1 under 103, have been fully considered but they are not persuasive. (FP 7.37)
Applicant's arguments on page 10 - 11, with respect to the rejection(s) of claim(s) 13 and 17 under 103, have been fully considered and are moot upon a new ground(s) of rejection made under 35 U.S.C. 103 as being unpatentable over Cella U.S. Patent Application 20220187847 in view of Maschmeyer U.S. Patent Application 20240161258, and further in view of Harary U.S. Patent Application 20250005727, as outlined above.
Applicant argues on page 9-10 about amended claim 1 that Mietzner, fail to teach or suggest that "the multiple text strings include a first text string that describes a first anomaly at a first location within a first industrial facility setting and a second text string that describes a second anomaly at a second location within the first industrial facility setting", as set forth in independent claim 1, as amended.
In reply, the rejection is based on Cella, Maschmeyer and Mietzner combined. Cella discloses industrial facility. Mietzner discloses the multiple text strings include a first text string that describes a first anomaly at a first location within a first industrial facility setting and that includes multiple first words and a second text string that describes a second anomaly at a second location within the first industrial facility setting and that includes multiple second words (paragraph [0031]: four types of surface imperfections are commonly observed in biopharmaceutical manufacturing product contact surfaces—scratches, roughness, micropits, and pitting… a “pit” or “pitting” is a surface void having a measurable depth that is generally annular, circular, oval, or oblong in shape. As used herein, “scratch” means a surface void having a substantially linear shape with a measurable depth. As used herein, “vessel,” “reactor vessel,” and/or “processing equipment” means any device or system with at least one surface that comes in contact with process materials, including but not limited to tanks, pipes, filters, bioreactors, product hold vessels, WFI hold vessels, chromatography skids, ultrafiltration/diafiltration skids, filter housings, and any process contact surface that is cleaned via recirculating CIP (different locations within a first industrial facility setting)). Mietzner’s teaching of using multiple text strings to define anomalies can be used in Cella and Maschmeyer’s device, to generate images for different anomalies such as post-commissioning pitting, scratching, or surface finish deviations at different locations, to train anomaly detection model for anomaly detection.
Applicant argues on page 10-11 regarding claim 13 and 17 about the image does not capture the anomaly associated with the particular object.
In reply, the rejection is based on Cella, Maschmeyer and Harary combined.
Harary discloses the image does not capture the anomaly associated with the particular object (paragraph [0081]: at block 705, where a system (e.g., the computing device 900 of FIG. 9) processes, using a trained model (e.g., 580 of FIG. 5), a plurality of normal images (e.g., 502 of FIG. 5) for an object to generate a set of normal features; paragraph [0082]: At block 710, the system processes, using the trained model, a plurality of negative text exemplars (e.g., 510 of FIG. 5) describing the object to generate a set of anomaly features; paragraph [0083]: At block 715, the system generates a plurality of anomaly images (e.g., 505 of FIG. 5) for the object based at least in part on the set of normal features (normal images)(e.g., 542 of FIG. 5) and the set of anomaly features).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Yi Yang whose telephone number is (571)272-9589. The examiner can normally be reached on Monday-Friday 9:00 AM-6:00 PM EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached on 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/YI YANG/
Primary Examiner, Art Unit 2616
Prosecution Insights