DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
This final office action is in response to the amendment filed June 30, 2026. Claims 1, 4-14, and 17-25 are pending in this application and have been considered below. Claims 2-3 and 15-16 are canceled by the applicant.
Applicant’s arguments with respect to claims 1, 4-14, 17-25 have been considered but are moot in view of new ground(s) of rejection because of the amendments.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 24-25 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
In claim 24, the five processes separated by commas does not contain an “and/or.” Therefore, the metes and bounds cannot be determined since it is unclear whether the specific process must include all of them or only one. Claim 25 is rejected by dependence.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 6-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Defard et al. , “PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization,” (hereinafter “Defard” or “PaDiM”) in view of JP 2021-047677 A to SCREEN Holdings Co., Ltd., (hereinafter "SCREEN") in view of Zavrtanik et al., “DRAEM – A discriminatively trained reconstruction embedding for surface anomaly detection” (hereinafter “DREAM”).
Claim 1.
Defard discloses a method of producing a non-transitory computer-readable storage medium storing a machine learning model (Defard: "PaDiM low memory consumption and its ease of use make it suitable for various applications, such as visual industrial control" (Section VI). The trained PaDiM model, including the pretrained CNN and learned Gaussian parameters, is necessarily stored on a computer-readable storage medium. This teaches producing a storage medium storing a machine learning model.), the machine learning model being used for anomaly detection to detect an anomaly in an object (Defard: "We present a new framework for Patch Distribution Modeling, PaDiM, to concurrently detect and localize anomalies in images in a one-class learning setting" (Abstract). This teaches anomaly detection in an object.), the machine learning model including an encoder configured to generate feature data for the object in response to captured image data obtained by capturing an image of the object being inputted (Defard: "we choose to avoid ponderous neural network optimization by only using a pretrained CNN to generate patch embedding vectors" (Section III-A). The pretrained CNN functions as an encoder that generates feature data (patch embedding vectors) from input image data. This teaches an encoder generating feature data from captured image data.), the encoder including a convolutional neural network, the method comprising: training the encoder using training image data (Defard: "PaDiM makes use of a pretrained convolutional neural network (CNN) for patch embedding" (Abstract). This teaches the encoder including a CNN.), the training image data being obtained by performing a specific image process (Defard: "During the training phase, each patch of the normal images is associated to its spatially corresponding activation vectors in the pretrained CNN activation maps" (Section III-A); "We resize the images from the MVTec AD to 256x256 and center crop them to 224x224" (Section IV-B). Defard teaches training using image data obtained by performing a specific image process (resize and crop) on original image data. However, Defard's original image data is captured image data of physical objects, not image data used to create the object.)
wherein the object includes a normal object and an anomalous object, the anomalous object containing the anomaly (Defard: "anomaly detection is a binary classification between the normal and the anomalous classes" (Section I). This teaches normal and anomalous objects.),
wherein the training image data includes first image data representing an image of the
normal object (Defard: "the training dataset contains only images from the normal class" (Section I). This teaches training data representing normal objects.) and second image data representing an image of the anomalous object (Defard: Section I, anomaly detection distinguishes normal and anomalous classes.),
wherein the training configures an image recognition model configured to generate output data indicating a recognition result of an image using data outputted from the encoder (Defard: "Neural network architectures like autoencoders (AE) [and] variational autoencoders (VAE) ... are trained to reconstruct normal training images only" (Section II). Defard implements "our own VAE as a reconstruction-based baseline implemented with a ResNet18 as encoder and a 8x8 convolutional latent variable" (Section IV-B). This teaches an image generation model with encoder and decoder.),
wherein the training is performed such that when the training image data is inputted into
the encoder, the output data identifies whether the training image data is the first image data or the second image data (Defard: "The anomaly map for the localization corresponds to the pixel-wise L2 error for reconstruction" (Section IV-B). The VAE is trained to reconstruct input images, meaning the decoder output reproduces the training image data. This teaches training performed such that decoder output reproduces training data.), and
wherein the specific image process includes a first image process and a second image
process,
Defard does not specifically teach that the original image data is "used to create the
object. " However, SCREEN in the same field of endeavor teaches using design data as input to train a deep learning model for defect inspection (SCREEN: The reference image generation device receives learning data combinations of substrate pattern design data and training images obtained by imaging patterns (claim 1). The "design data" is the pattern data used to fabricate the substrate (i.e., original image data used to create the object). This teaches original image data being used to create the object.). SCREEN further teaches performing image augmentation processes on the design data, including brightness and contrast modifications (SCREEN: The training dataset includes augmented versions with modified brightness or contrast of training images (claim 2). This teaches performing a specific image process on the original design data.) and rotation and inversion transformations (SCREEN: The
dataset includes rotation/inversion augmentations where both design data and training images are transformed identically (claim 3). This teaches image augmentation processes applied to the design data.).
Defard does not specifically teach “the first image process adjusting an image attribute that is different from a defect that should be identified as the anomaly, the second image process adding the defect to an image.” However, DRAEM in the same field of endeavor teaches the first image process adjusting an image attribute that is different from a defect that should be identified as the anomaly (DRAEM: “Image rotation in the range of (-45, 45) degrees is used as a data augmentation method on anomaly free images during training” (§ 4.1).), the second image process adding the defect to an image (DRAEM: “The anomalous regions are sampled from A according to Ma and placed on the anomaly free image to generate the anomalous image Ia.” (Fig. 4); “A noise image is generated by a Perlin noise generator” (§ 3.3). This shows the second image process adding the defect to an image.)
Therefore, it would have been obvious to one of ordinary skill in the art before the
effective filing date of the claimed invention to (1) substitute the design/template image data
of SCREEN for the captured training image data in Defard's anomaly detection framework to reduce the amount to captured image data required for anomaly detection (SCREEN ¶¶ 52, 57); and (2) generate anomalies training images by adding a defect to the normal image as taught by DRAEM, because DRAEM teaches discriminatively training on synthetically generated anomalies improves surface anomaly detection over one-class reconstruction methods (DRAEM Abstract, § 1). Combining SCREEN’s design data training and DRAEM’s synthetic defect generation with Defard’s anomaly classification applies known techniques to yield the predictable results of a model trained to classify normal versus anomalous images using design data derived normal images and synthetically derived anomalous images.
Claim 6.
Defard, SCREEN, and DRAEM discloses the method according to claim 1, wherein the anomaly detection includes: generating image data for feature extraction by executing a first adjustment process on the original image data, the image data for feature extraction representing an image of the normal object (Defard: “We resize the images from the MVTec AD to 256x256 and center crop them to 224x224" (Section IV-B). Normal images are preprocessed (first adjustment) before feature extraction. This teaches generating feature extraction image data by executing a first adjustment process.); generating feature data for the normal object by inputting the image data for feature extraction into the machine learning model trained in the training (Defard: “we first compute the set of patch embedding vectors at position (i,j), X_ij = {x^k_ij, k E [1,N]} from the N normal training images”(Section III-B). This teaches generating feature data by inputting preprocessed normal images into the trained model.); and detecting an anomaly in an inspection object using the feature data for the normal object and feature data for the inspection object (Defard: “we use the Mahalanobis distance M(xij) to give an anomaly score to the patch ... of a test image ... the distance between the test embedding xij and the learned distribution N(mu_ij, Sigma_ij)” (Section III-C).).
SCREEN further teaches that the original image data for feature extraction is the design data used to create the object (SCREEN: “reference image used for comparison with an image of the pattern in the manufacturing process of a substrate having a pattern on the surface, and images the design data of the pattern on the substrate and the pattern.” (claim 1).).
It would have been obvious to use the design data of SCREEN as the original image data in Defard's feature extraction process for the same reasons set forth in the rejection of claim 1.
Claim 7.
Defard, SCREEN, and DRAEM disclose wherein the first adjustment process adjusts an image attribute that is variable due to variations in factors other than a defect that should be identified as the anomaly (Defard: Resizing and center cropping adjust size and framing attributes, which vary due to imaging setup rather than defects (Section IV-B). This teaches adjusting a non-defect-related image attribute.), wherein the specific image process includes a second adjustment process adjusting the image attribute (Defard: For the VAE baseline, augmentation includes “random rotation (-2°, +2°), 292x292 resize, random crop to 282x282, and finally center crop to 256x256” (Section IV-B). This teaches a second adjustment process on training data.), and wherein a maximum adjustment amount of the image attribute in the second adjustment process is greater than a maximum adjustment amount of the image attribute in the first adjustment process (Defard: Training augmentation includes rotation and larger resize (292x292) compared to inference preprocessing (256x256 resize, 224x224 crop). The training process applies more aggressive transformations. This teaches a greater adjustment amount in the second process.).
SCREEN further teaches applying brightness/contrast modifications and rotation/inversion augmentations to design data for training (SCREEN: Claims 2-3. This teaches augmentation of the design-based original image data with greater adjustment range for training.).
Claim 8.
Defard, SCREEN, and DRAEM disclose wherein the object is a label affixed to a product. SCREEN teaches inspection of patterned substrates in a manufacturing context (SCREEN: Inspects substrates with pattern design data. This teaches inspecting manufactured objects with known designs.).
It would have been obvious to one of ordinary skill in the art before the effective filing date to apply the combined method of Defard and SCREEN to inspect labels affixed to products, since labels are a common manufactured object with readily available design data, and label inspection is a well-known application of visual anomaly detection in industrial quality control.
Claim 9.
Defard discloses a non-transitory computer-readable storage medium storing a set of computer-readable instructions for performing anomaly detection with a machine learning model, the anomaly detection detecting an anomaly in an object including a normal object and an anomalous object containing the anomaly (Defard: “anomaly detection is a binary classification between the normal and the anomalous classes” (Section I).), the machine learning model including an encoder configured to generate feature data for the object in response to captured image data obtained by capturing an image of the object being inputted, the encoder including a convolutional neural network (Defard: “PaDiM makes use of a pretrained convolutional neural network (CNN) for patch embedding” (Abstract); “a pretrained CNN to generate patch embedding vectors” (Section III-A).), the set of computer-readable instructions, when executed by a computer, causing the computer to perform:
after the encoder is trained, generating image data for feature extraction by executing a first adjustment process on original image data, (Defard: “We resize the images from the MVTec
AD to 256x256 and center crop them to 224x224” (Section IV-B), the image data for feature extraction representing an image of the normal object, wherein the first adjustment process adjusts an image attribute that is variable due to variations in factors other than a defect that should be identified as the anomaly (Defard: “We resize the images from the MVTec
AD to 256x256 and center crop them to 224x224” (Section IV-B) and wherein and
generating feature data for the normal object by inputting the image data for feature extraction into the machine learning model that has been trained (Defard: “PaDiM learns the Gaussian parameters … from the set of N training embedding vectors ... computed from N different
training images" (Fig. 2).); and
detecting an anomaly in an inspection object using the feature data for the normal object and feature data for the inspection object (Defard: “we use the Mahalanobis distance M(xij) to give an anomaly score to the patch ... of a test image ... the distance between the test embedding xij and the learned distribution N(mu_ij, Sigma_ij)” (Section III-C).).
Defard discloses all of the subject matter as described above except for specifically teaching “the original image data representing an image of the object and being used to create the object.” However, SCREEN in the same field of endeavor teaches the original image data representing an image of the object and being used to create the object (SCREEN: “reference image used for comparison with an image of the pattern in the manufacturing process of a substrate having a pattern on the surface, and images the design data of the pattern on the substrate and the pattern.” (claim 1). The “design data” is the pattern data used to fabricate the substrate, i.e ., original image data used to create the object. This teaches original image data being used to create the object.).
Defard discloses all of the subject matter as described above except for specifically teaching “an allowable adjustment amount of the image attribute is different from an allowable adjustment amount used in a second adjustment process for generate image data for training the encoder.” However, DRAEM in the same field of endeavor teaches an allowable adjustment amount of the image attribute is different from an allowable adjustment amount used in a second adjustment process for generate image data for training the encoder (DRAEM: “Image rotation in the range of (-45, 45) degrees is used as a data augmentation method on anomaly free images during training” (§ 4.1). This training augmentation is a different and greater adjustment amount than Deford's feature-extraction preprocessing, "resize ... to 256x256 and center crop ... to 224x224" (Deford, Section IV-B, no rotation). This shows an allowable adjustment amount used in training that is different from the allowable adjustment amount used for the original image data for feature extraction.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to (1) substitute the design/template image data of SCREEN for the captured training image data in Defard's anomaly detection framework to reduce the amount to captured image data required for anomaly detection (SCREEN ¶¶ 52, 57); and (2) augment the training images more, as taught by DRAEM, because DRAEM teaches the heavier training augmentation “to alleviate overfitting” (DRAEM § 4.1). Combining SCREEN’s design data training and DRAEM’s different training augmentation for training images with Defard’s anomaly classification applies known techniques to yield the predictable results of a more robust normal reference detection model.
Claim 10.
Defard, SCREEN, and DRAEM disclose the storage medium of claim 9. Claim 10 recites that the
encoder is trained using training image data obtained by executing a specific image process on
the original image data. This limitation is disclosed by Defard in view of SCREEN for the same
reasons as the corresponding training limitation in claim 1.
Claim 11.
Defard, SCREEN, and DRAEM renders claim(s) 11 obvious for the reasons discussed above for claim 7, mutatis mutandis.
Claim 12.
Defard, SCREEN, and DRAEM renders claim(s) 12 obvious for the reasons discussed above for claim 8, mutatis mutandis.
Claim 13, 24, and 25 is rejected under 35 U.S.C. 103 as being unpatentable over Defard in view of SCREEN in view of Roth et al., “Towards Total Recall in Industrial Anomaly Detection” (hereinafter “PatchCore”).
Claim 13.
Defard discloses a method of detecting an anomaly in an object with a machine learning model, the object including a normal object and an anomalous object containing the anomaly (Defard: “anomaly detection is a binary classification between the normal and the anomalous classes” (Section I).), the machine learning model including an encoder configured to generate feature data for the object in response to captured image data obtained by capturing an image of the object being inputted, the encoder including a convolutional neural network (Defard: "PaDiM makes use of a pretrained convolutional neural network (CNN) for patch embedding" (Abstract)), the method comprising:
training the encoder using training image data, the image data used for training being generated by executing a specific process on original image data (Defard: “We resize the images from the MVTec AD to 256x256 and center crop them to 224x224” (Section IV-B).), (Defard teaches training using image data obtained by performing a specific image process (resize and crop) on original image data (Section IV-B).);
(Defard’s Fig. 2 discloses normal feature data from N training embeddings.); and detecting an anomaly in an inspection object using the feature data for the normal object generated by the encoder and feature data for the inspection object (Defard: “we use the Mahalanobis distance M(xij) to give an anomaly score to the patch ... of a test image ... the distance between the test embedding xij and the learned distribution N(mu_ij, Sigma_ij)” (Section III-C).).
Defard discloses all of the subject matter as described above except for specifically teaching “the original image data representing an image of the object and being used to create the object.” However, SCREEN in the same field of endeavor teaches the original image data representing an image of the object and being used to create the object (SCREEN: “reference image used for comparison with an image of the pattern in the manufacturing process of a substrate having a pattern on the surface, and images the design data of the pattern on the substrate and the pattern.” (claim 1). The “design data” is the pattern data used to fabricate the substrate, i.e ., original image data used to create the object. This teaches original image data being used to create the object.).
Defard does not specifically teach “randomly selecting a subset of the image data used in training to input into the encoder for feature extraction; generating feature data for the normal object by inputting the randomly selected subset of image data for feature extraction into the encoder trained in the training.” However, PatchCore in the same field of endeavor teaches randomly selecting a subset of the image data used in training to input into the encoder for feature extraction; generating feature data for the normal object by inputting the randomly selected subset of image data for feature extraction into the encoder trained in the training (PatchCore: “coreset (top) vs. random subsampling” (Fig. 3); “Greedy coreset selection, random subsampling” (§ 4.4.2).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to (1) substitute the design/template image data of SCREEN for the captured training image data in Defard's anomaly detection framework to reduce the amount to captured image data required for anomaly detection (SCREEN ¶¶ 52, 57); and (2) randomly select a subset of the normal training data input for feature extraction as taught by PatchCore, to reduce inference memory and time (PatchCore Introduction, § 1, Fig. 2). Combining SCREEN's design-data training source and PatchCore's random subsampling with Defard's anomaly detection framework applies known techniques to yield the predictable result of an efficient normal reference detection model.
Claim 24.
Defard, SCREEN, and DRAEM discloses the method of claim 13, wherein the specific process includes a brightness correction process (SCREEN: “the learning image in which the brightness or contrast of the learning image of the one learning data is changed” (¶59).), a smoothing process, a noise adding process, a rotation process or shift process (DRAEM: “Image rotation in the range of (-45, 45) degrees is used as a data augmentation method on anomaly free images during training” (§ 4.1)).
Claim 25.
The method of claim 24, wherein the image data used for training comprises a set of normal image data (Defard: “the training dataset contains only images from the normal class” (§ I. Introduction) generated by the specific process (SCREEN: “the learning image in which the brightness or contrast of the learning image of the one learning data is changed” (¶59).; Defard preprocesses the images (§ IV-B).).
Claim Rejections - 35 USC § 103
Claim(s) 14 and 19-23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Defard in view of DREAM.
Claim 14.
Defard discloses a method of producing a non-transitory computer-readable storage medium storing a machine learning model, the machine learning model being used for anomaly detection detecting an anomaly in an object including a normal object and an anomalous object containing the anomaly (Defard: “We present a new framework for Patch Distribution Modeling, PaDiM, to concurrently detect and localize anomalies in images in a one-class learning setting" (Abstract).), the machine learning model including an encoder configured to generate feature data for the object in response to captured image data obtained by capturing an image of the object being inputted, the encoder including a convolutional neural network (Defard: “PaDiM makes use of a pretrained convolutional neural network (CNN) for patch embedding” (Abstract). “Neural network architectures like autoencoders (AE) [and] variational autoencoders (VAE) ... are trained to reconstruct normal training images only” (Section II).), the method comprising:
training the encoder using training image data, the training image data being obtained by performing a specific image process on original image data, the original image data being obtained by capturing an image of the object (Defard: “We resize the images from the MVTec
AD to 256x256 and center crop them to 224x224” (Section IV-B).),
wherein the object includes a normal object and an anomalous object, the anomalous object containing the anomaly, wherein the training image data includes first image data representing an image of the normal object (Defard: “the training dataset contains only images from the normal class” (Section I).) a(Defard: “Neural network architectures like autoencoders (AE) [and] variational autoencoders (VAE) ... are trained to reconstruct normal training images only” (Section II)., wherein the training is performed such that when the training image data is inputted into the encoder, the output data identifies whether the training image data is the first image data or the second image data (Defard: “anomaly detection is a binary classification between the normal and the anomalous classes” (Section I).), and
wherein the specific image process includes a first image process and a second image process, the first image process adjusting an image attribute that is different from a defect that should be identified as the anomaly (Defard Section IV-B),
wherein the anomaly detection includes: generating image data for feature extraction by executing a first adjustment process on the original image data, the image data for feature extraction representing an image of the normal object (Defard Section IV-B);
generating feature data for the normal object by inputting the image data for feature extraction into the machine learning model that has been trained (Defard: “PaDiM learns the Gaussian parameters … from the set of N training embedding vectors ... computed from N different
training images” (Fig. 2); “we first compute the set of patch embedding vectors … from the
normal training images N as shown on Figure 2.” (§ III-B)); and
detecting an anomaly in an inspection object using the feature data for the normal object and feature data for the inspection object (Defard: “we use the Mahalanobis distance M(xij) to give an anomaly score to the patch ... of a test image ... the distance between the test embedding xij and the learned distribution N(mu_ij, Sigma_ij)” (Section III-C).).
Defard does not specifically teach “second image data representing an image of the anomalous object” and “the second image process adding the defect to an image.” However, DRAEM in the same field of endeavor teaches the second image data representing an image of the anomalous object the second image process adding the defect to an image (DRAEM: “The anomalous regions are sampled from A according to Ma and placed on the anomaly free image I to generate the anomalous image Ia” (Fig. 4); “A noise image is generated by a Perlin noise generator [18] to capture a variety of anomaly shapes (Figure 4, P)” (§ 3.3). “Image rotation in the range of (-45, 45) degrees is used as a data augmentation method on anomaly free images during training” (§ 4.1).).
Therefore, it would have been obvious to one of ordinary skill in the art before the
effective filing date of the claimed invention to generate the anomalous (second) training
images by adding defects to the normal images as taught by DRAEM, because DRAEM
teaches that discriminative training on synthetically generated anomalies improves surface anomaly detection over one-class reconstruction methods (DRAEM, Abstract; § 1). Combining DRAEM's synthetic defect generation with Defard's anomaly-detection framework applies known techniques to yield the predictable result of a model trained to classify normal versus anomalous images.
Claim 19.
Defard and DRAEM teach wherein the first adjustment process adjusts an image attribute that is variable due to variations in factors other than a defect (Defard: Resizing and cropping adjust size/framing attributes variable due to imaging conditions (Section IV-B). This teaches adjusting a non-defect attribute.), that should be identified as the anomaly, wherein the specific image process includes a second adjustment process adjusting the image attribute (Defard: The same preprocessing is applied to training images.), and wherein a maximum adjustment amount of the image attribute in the second adjustment process is greater than a maximum adjustment amount of the image attribute in the first adjustment process (Defard: VAE baseline augmentation includes "random rotation (-2°, +2°), 292x292 resize, random crop to 282x282, and finally center crop to 256x256" (Section IV-B). Training applies more aggressive transformations than inference. This teaches a greater maximum adjustment amount in the second process.).
Claim 20.
Defard and DRAEM teach the method according to claim 14. It would have been obvious to one of ordinary skill in the art to apply the anomaly detection method of Defard to inspect labels affixed to products, since Defard explicitly targets industrial visual inspection applications (Section VI), and label inspection is a known and predictable use case within this field.
Claim 21.
Defard and DRAEM teach a non-transitory computer-readable storage medium storing a set of computer-readable instructions for performing anomaly detection with a machine learning model, the anomaly detection detecting an anomaly in an object including a normal object and an anomalous object containing the anomaly, the machine learning model including an encoder configured to generate feature data for the object in response to captured image data obtained by capturing an image of the object being inputted, the encoder including a convolutional neural network (Defard: "PaDiM makes use of a pretrained convolutional neural network (CNN) for patch embedding" (Abstract).), the set of computer-readable instructions, when executed by a computer (Defard: “GPU” (§ V-D).), causing the computer to perform: after the encoder is trained, generating image data for feature extraction by executing a first adjustment process on original image data, the original image data being obtained by capturing an image of the object, the image data for feature extraction representing an image of the normal object, wherein the first adjustment process adjusts an image attribute that is variable due to variations in factors other than a defect that should be identified as the anomaly (Defard: “We resize the images from the MVTec AD to 256x256 and center crop them to 224x224” (Section IV-B).) and wherein an
generating feature data for the normal object by inputting the image data for feature extraction into the machine learning model that has been trained (Defard: “PaDiM learns the Gaussian parameters … from the set of N training embedding vectors ... computed from N different
training images" (Fig. 2).); and
detecting an anomaly in an inspection object using the feature data for the normal object and feature data for the inspection object (Defard: “we use the Mahalanobis distance M(xij) to give an anomaly score to the patch ... of a test image ... the distance between the test embedding xij and the learned distribution N(mu_ij, Sigma_ij)” (Section III-C).).
Defard does not specifically teach " an allowable adjustment amount of the image attribute is different from an allowable adjustment amount used in a second adjustment process for generate image data for training the encoder." However, DRAEM in the same field of endeavor teaches an allowable adjustment amount of the image attribute is different from an allowable adjustment amount used in a second adjustment process for generate image data for training the encoder (DRAEM: “Image rotation in the range of (-45, 45) degrees is used as a data augmentation method on anomaly free images during training” (§ 4.1). This training\ augmentation is a different and greater adjustment amount than Deford's feature-extraction preprocessing, "resize ... to 256x256 and center crop ... to 224x224" (Deford, Section IV-B, no rotation). This shows an allowable adjustment amount used in training that is different from the allowable adjustment amount used for the original image data for feature extraction.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to augment the training images more than the feature-extraction images as taught by DRAEM, because DRAEM uses the heavier training augmentation “to alleviate overfitting” (DRAEM § 4.1 ), applying a known technique to Defard's framework to yield the predictable result of a more robust normal reference detection model.
Claim 23.
Defard and DRAEM teach wherein a maximum adjustment amount of the image attribute in the second adjustment process is greater than a maximum adjustment amount of the image attribute in the first adjustment process (DRAEM: “Image rotation in the range of (-45, 45) degrees is used as a data augmentation method on anomaly free images during training” (§ 4.1). This training augmentation is a different and greater adjustment amount than Deford's feature-extraction preprocessing, “resize ... to 256x256 and center crop ... to 224x224” (Deford, Section IV-B, no rotation). This shows an rotation adjustment of up to 45 degrees used in training (second) process that is greater than the 0 degree rotation adjustment amount used for the original (first) image data for feature extraction.).
Claim(s) 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Defard in view of PatchCore.
Claim 22.
Defard discloses a method of detecting an anomaly in an object with a machine learning model, the object including a normal object and an anomalous object containing the anomaly, the machine learning model including an encoder configured to generate feature data for the object in response to captured image data obtained by capturing an image of the object being inputted, the encoder including a convolutional neural network (Defard: “PaDiM makes use of a pretrained convolutional neural network (CNN) for patch embedding” (Abstract); “a pretrained CNN to generate patch embedding vectors” (Section III-A).), the method comprising:
training the encoder using training image data, the image data used for training being generated by executing a specific process on original image data, the original image data representing an image of the object and being used to create the object, the specific process adjusting an image attribute that is variable due to variations in factors other than a defect that should be identified as the anomaly (Defard: “We resize the images from the MVTec AD to 256x256 and center crop them to 224x224” (Section IV-B).);
(Defard: “PaDiM learns the Gaussian parameters … from the set of N training embedding vectors ... computed from N different training images” (Fig. 2).); and
detecting an anomaly in an inspection object using the feature data for the normal object generated by the encoder and feature data for the inspection object (Defard: “we use the Mahalanobis distance M(xij) to give an anomaly score to the patch ... of a test image ... the distance between the test embedding xij and the learned distribution N(mu_ij, Sigma_ij)” (Section III-C).).
Defard does not specifically teach “randomly selecting a subset of the image data used in training to input into the encoder for feature extraction; generating feature data for the normal object by inputting the randomly selected subset of image data for feature extraction into the encoder trained in the training.” However, PatchCore in the same field of endeavor teaches randomly selecting a subset of the nominal (normal training) data for feature-based detection (PatchCore: “coreset (top) vs. random subsampling” (Fig. 3); “Greedy coreset selection, random subsampling” (§ 4.4.2).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to randomly select a subset of the normal training data input for feature extraction as taught by PatchCore, to reduce inference memory and time (PatchCore Introduction, § 1, Fig. 2). Combining PatchCore's random subsampling with Defard's anomaly detection framework applies known techniques to yield the predictable result of an efficient normal reference detection model.
Allowable Subject Matter
Claims 4, 5, 17, and 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ross Varndell whose telephone number is (571)270-1922. The examiner can normally be reached M-F, 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/
interview practice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached at (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to
the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ross Varndell/
Primary Examiner, Art Unit 2674
Prosecution Insights