DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged that application is a National Stage application of PCT/JP2022/007813. Receipt is acknowledged that application claims priority to foreign application with application number JP2021-074653 dated 04/27/2021. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Response to Amendment
The amendment filed 12/12/2025 has been entered. Applicant’s amendments to the claims have overcome each and every objection previously set forth in the Non-Final Office Action mailed 09/16/2025. Claims 1-3 and 5-15 remain pending in the application, with claim 4 having been cancelled.
Response to Arguments
Regarding claim 1, on pg. 8 of the Remarks, filed 12/12/2025, Applicant argues that “Tandia does not suggest any optimization of pattern variations and does not provide any motivation to implement ‘larger than one time and smaller than four times the minimum size of an object included in the wiring layer of the semiconductor wafer.’” Examiner respectfully disagrees. In paragraph 33, cited below, Tandia describes a method for determining an optimal tile size and overlap. Tile size, as shown in Figure 3, determines 1) the amount/number of objects in the image versus what objects are cut off and 2) the ratio of object size to the rest of the image. Thus, Tandia suggests an optimization of pattern variation based on tile size. As demonstrated in the claim rejection below, Tandia’s disclosure, which includes application of the method to semiconductor wafers, states that tile size can range from as small as possible to the size of the image. Thus, Tandia provides motivation to implement the aforementioned claim limitation based on its explicit recitation and obviousness in combination with the invention of Lu. For similar reasons, Lu in view of Tandia also teaches independent claims 9 and 10.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1 and 9-10 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1 and 9 both recite “wherein a size of the plurality of images…is larger than…” It is unclear whether the “size” refers to the size of each individual image of the plurality of images, or the size of all the plurality of images together, such as the training image. Further, in claims 1, 9, and 10, it is unclear what characteristic the “size” of the image refers to (i.e., length of one side of the image).
For examination purposes, the “size” in claims 1, 9, and 10 will be interpreted to be the size of at least one side of each of the images, in accordance with paragraph 44 of the Specification (para 44: “the size of the sub-image to be clipped is assumed to be a square with one side having the minimum design dimension…For example, when one side is larger than the minimum design dimension, the number of pattern variations included in one side is larger than the value described above”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-7, and 9-15 are rejected under 35 U.S.C. 103 as being unpatentable over Lu et al. (U.S. Patent No. 2019/0287230 A1), hereinafter Lu, in view of Tandia et al. (U.S. Patent No. 2020/0410660 A1), hereinafter Tandia.
Regarding claim 1, Lu teaches a system configured to detect defects on a semiconductor wafer (Lu, abstract: "Defects on semiconductor wafers can be discovered using these approaches"; para 22: "The model is configured to receive an image of a wafer and determine presence of one or more anomalies in the image"), wherein the system is provided with one or more computer systems specifying the defects included in a received input image (Lu, para 83: "Computer subsystem 202 may be configured to perform any of the functions described herein using the output of the detector 207 and/or the electron beam images"; para 103: "The methods also may include any other step(s) that can be performed by the processor and/or computer subsystem(s) or system(s) described herein"), the one or more computer systems are provided with a training device including an autoencoder trained in advance (Lu, para 50: "the model includes a variational autoencoder") by inputting a plurality of images at different locations (Lu, see Figure 2 attached below wherein images of different locations are utilized to train the autoencoder; para 51: “The autoencoder can be trained on clean sites”), and the one or more computer systems divide images into a plurality of inspection sub-images (Lu, see Figure 5; para 71: “FIG. 5 illustrates input and reconstructed SEM patches with an autoencoder”), input the plurality of inspection sub-images to the autoencoder (Lu, Figure 5 and para 71; para 51: “The autoencoder can be trained on clean sites and inference can be run with test sites that may contain defects”; para 75: “Then the validation data, which contains some images with defects, was passed in”), and compare an output image output from the autoencoder with the input image (Lu, para 48: "a difference between reconstructed and original SEM images may be calculated at 104 to locate the anomaly patterns (e.g., defects)"; para 73: "The reconstruction error can be defined as the difference between the original input vector x and the reconstruction"; see also para 68).
PNG
media_image1.png
401
609
media_image1.png
Greyscale
However, Lu fails to explicitly teach wherein the autoencoder is trained by inputting a plurality of images at different locations included in a training image and wherein the input image is divided into the plurality of inspection sub-images (Emphasis added). Additionally, while Lu teaches an object included in a wiring layer of a semiconductor wafer (Lu, semiconductor wafer pattern, para 41: “Nominal patterns are collected at 101 to make a training set”), Lu fails to teach wherein a size of the plurality of images at the different locations included in the training image is larger than one time and smaller than four times a minimum dimension of an object included in a wiring layer of the semiconductor wafer in the plurality of images.
While Lu teaches that the autoencoder can be trained using patch images, Lu fails to teach explicitly wherein a plurality of the patches are from the same training image (Lu, para 64: "The semi-supervised model can be trained with same patch size images from the clean sites of the same SEM layer, then the detection output patches can be passed to this model"). However, Tandia teaches a similar system for detecting defects on semiconductor wafers (Tandia, abstract: “anomaly detection system for super-high-resolution images”; para 29: “For example, the tile size and overlap may be different for the content of the image (e.g., a mesh, semiconductor wafer, flat glass sheets) and the anomaly or defect that is being identified”), further disclosing wherein the machine learned model is trained by inputting a plurality of images at different locations included in a training image (Tandia, para 34: "splitting the images based on the currently selected tile size and step size from operation 410 and operation 415. The resulting sub-images from the split are then labeled according to the presence of an anomaly. At operation 425, the new data set is used to train and build a CNN model"). It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the different images from a training image, taught by Tandia, with the system of Lu in order to maximize the number of possible images for training from a given number of training images (Tandia, para 40: “Because the sub-images are overlap-split, all the features of original image may be captured in different patterns, but this results in an increase in the number of images in the dataset”). Doing so can improve the performance of the autoencoder by training with a larger dataset.
Additionally, Lu teaches that inspection sub-images are input to the autoencoder, but fails to teach explicitly wherein the sub-images are from the same input image. Tandia further discloses dividing the input image into a plurality of inspection sub-images (Tandia, steps 702-706 in Figure 7; para 49: "divide the image into tiles with a size and overlap between each tile"; para 50: "use a classifier model for each tile to identify anomaly presence in the tile"). It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the divided input image of Tandia with the system of Lu in order to detect defects on the edges of sub-images (Tandia, para 22: “if a defect were to present on the edge between two tiles then it may be undetected by the classifier as the portion appearing in each respective tile would not fall into the known defects of the CNN. Thus, a tile overlap is utilized to ensure a defect is not missed by being split along tile edges”). Utilizing overlapping sub-images from the same input image ensures that all image features are represented in the tested dataset (Tandia, para 21: “The methods and techniques use an ‘overlapped-step-by-step’ process to crop the original images and generate many sub-images (tiles) to ensure all the features are included in the new image model database and each tile is large enough to possibly contain all features representing the different possible characteristics (defects) to be classified”).
Lastly, Tandia teaches wherein a size of the plurality of images at the different locations included in the training image is larger than one time and smaller than four times a minimum dimension of an object included in a wiring layer of the semiconductor wafer in the plurality of images (Tandia, images can be any possible size, para 33: “At operation 410, a tile size is selected ranging from the smallest tile size possible to the size of the image. The smallest tile size possible may be the smallest size that captures an anomaly.”; para 29: “the tile size and overlap may be different for the content of the image (e.g., a mesh, semiconductor wafer, flat glass sheets) and the anomaly or defect that is being identified”; In a specific example, see Figure 3 below wherein the width of the tile image is greater than one time and smaller than four times the minimum dimension of the square objects in the image).
PNG
media_image2.png
442
482
media_image2.png
Greyscale
In Figure 3, Tandia demonstrates the dimensions of the object in the image compared to the size of the sub-image with an object on a mesh filter, while Lu teaches training images of semiconductor wafer patterns. Additionally, Tandia teaches that sub-images are generated ranging from the smallest size possible to the size of the image, which includes an image sized 1-4 times that of an object in the image. It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the object dimension relationship, as taught above by Tandia, with the semiconductor pattern in the system of Lu in order to ensure that the image data contains the entire semiconductor pattern defect, while also allowing for adequate classification of very small defects (Tandia, para 28: “Using tiling, with overlap, to divide the overall image into smaller sub-images for analysis may result in tile 310, tile 315, and tile 320. By using an overlap, the defect is fully captured in tile 315. There is a sufficient amount of the defect present in tile 315 for the CNN to recognize the defect and classify tile 315. This may result in appropriate classification of failing for the image with section 205”; see also para 20 and 23).
Regarding claim 2 (dependent on claim 1), Lu in view of Tandia teaches wherein the one or more computer systems are configured to divide the training image into a plurality of training sub-images (Taught by the combination with Tandia in claim 1. The images captured at different locations are sub-images of the training image) and train the autoencoder based on the plurality of divided training sub-images (Lu, sub-images are utilized to train the autoencoder, see Figure 2 wherein images of different locations are utilized to train the autoencoder).
Regarding claim 3 (dependent on claim 1), Lu in view of Tandia teaches wherein the one or more computer systems are configured to detect the defects included in the input image (Lu, para 22: "The model is configured to receive an image of a wafer and determine presence of one or more anomalies in the image") by training the autoencoder based on input of a training image (Lu, Figure 2 clean site images; para 51: “The autoencoder can be trained on clean sites”) and inputting the plurality of inspection sub-images to the autoencoder that is trained (Lu, Figure 5 and para 71; para 51: “The autoencoder can be trained on clean sites and inference can be run with test sites that may contain defects”; para 75: “Then the validation data, which contains some images with defects, was passed in”).
Regarding claim 5 (dependent on claim 1), Lu in view of Tandia teaches wherein the one or more computer systems are configured to divide the input image into the plurality of inspection sub-images while providing an overlapped region (Tandia, see Fig. 1, para 26: “The whole image 105 is then divided into tile 110 size images for analysis. To prevent an anomaly or defect from being missed by falling on the edge between two tiles, an overlap is also determined”; see also para 21).
Regarding claim 6 (dependent on claim 1), Lu in view of Tandia teaches wherein the one or more computer systems are configured to evaluate a degree of discrepancy of the input image and the output image (Lu, Figure 6, para 72-73: “FIG. 6 a graph of reconstruction errors. A threshold is used to distinguish anomaly from nominal…The reconstruction error can be defined as the difference between the original input vector x and the reconstruction”; the reconstruction error represents a discrepancy between the image inputted to the autoencoder and the reconstructed output).
Regarding claim 7 (dependent on claim 6), Lu in view of Tandia teaches wherein the one or more computer systems are configured to allow a display device (Tandia, para 53: “a display may be connected to the computer system performing the classification. When classification of an image is completed, the display may provide information about the classification of the image, such as with text or a graphical indicator”; see further combination with Lu described below) to display a frequency distribution of the degree of discrepancy (Taught by Lu, see claim 6 and the histogram of Figure 6) or a distribution on the semiconductor wafer.
Lu teaches a frequency distribution output from the computer system, but fails to explicitly teach a display device; while Tandia teaches displaying an output of the computer system on a display. It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the display of Tandia with the system of Lu in order to display the results of the image processing to a user for decision making regarding the imaged product (Tandia, para 53: “Based on the classification result, the sorting mechanism may move the product associated with the classified image to a particular bin. For example, a product that is classified as passing may be moved to bin for distribution or sales, while a product that is classified as failing may be moved to a bin of rejects for trash or recycling”).
Regarding claim 9, Lu teaches a non-transitory computer-readable medium storing program instructions executable on a computer system (Lu, para 92: “Program code or instructions for the processor 208 to implement various methods and functions may be stored in readable storage media, such as a memory in the electronic data storage unit 209 or other memory”) to perform a computer-implemented method of detecting defects on a semiconductor wafer (Lu, abstract: "Defects on semiconductor wafers can be discovered using these approaches"; para 22: "The model is configured to receive an image of a wafer and determine presence of one or more anomalies in the image").
The remaining claim limitations of claim 9 are met and rendered obvious by Lu in view of Tandia because the method steps of claim 9 are the same as claim 1.
Regarding claim 10, Lu teaches a system for processing image signals obtained based on irradiation of a semiconductor wafer with a beam (Lu, para 11: “The wafer inspection tool is configured to generate images of a wafer, and includes an electron beam source and a detector. The processor operates a model configured to find one or more anomalies in the image. The model is trained using semi-supervised machine learning based on only defect-free training images of semiconductor devices”), wherein the system includes one or more computer systems (Lu, para 14: “An image of a wafer is received at a processor. The processor operates a model configured to find one or more anomalies in the image”) computing difference information between first image data and second image data (Lu, reconstructed image data and original image data, para 48: “a difference between reconstructed and original SEM images may be calculated at 104 to locate the anomaly patterns (e.g., defects)”), and the one or more computer systems are configured to calculate a frequency for each degree of discrepancy between the first image data and the second image data (Lu, Figure 6, para 72-73: “FIG. 6 a graph of reconstruction errors. A threshold is used to distinguish anomaly from nominal…The reconstruction error can be defined as the difference between the original input vector x and the reconstruction”; the reconstruction error represents a discrepancy between the image inputted to the autoencoder and the reconstructed output).
While Lu teaches an object included in a wiring layer of a semiconductor wafer (Lu, semiconductor wafer pattern, para 41: “Nominal patterns are collected at 101 to make a training set”), Lu fails to explicitly teach wherein the first image data and the second image data are partial images that are cut into a size greater than one time and less than four times a minimum dimension of an object included in a wiring layer of the semiconductor wafer. However, Tandia teaches a similar system for detecting defects on semiconductor wafers (Tandia, abstract: “anomaly detection system for super-high-resolution images”; para 29: “For example, the tile size and overlap may be different for the content of the image (e.g., a mesh, semiconductor wafer, flat glass sheets) and the anomaly or defect that is being identified”), further disclosing wherein a first image data and second image data are partial images (Tandia, plurality of tile images) that are cut into a size greater than one time and less than four times a minimum dimension of an object included in a wiring layer of the semiconductor wafer (Tandia, images can be any possible size, para 33: “At operation 410, a tile size is selected ranging from the smallest tile size possible to the size of the image. The smallest tile size possible may be the smallest size that captures an anomaly.”; para 29: “the tile size and overlap may be different for the content of the image (e.g., a mesh, semiconductor wafer, flat glass sheets) and the anomaly or defect that is being identified”; In a specific example, see Figure 3 above wherein the width of the tile image is greater than one time and smaller than four times the minimum dimension of the square objects in the image).
In Figure 3, Tandia demonstrates the dimensions of the object in the image compared to the size of the sub-image with an object on a mesh filter, while Lu teaches training images of semiconductor wafer patterns. Additionally, Tandia teaches that sub-images are generated ranging from the smallest size possible to the size of the image, which includes an image sized 1-4 times that of an object in the image. It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the object dimension relationship, as taught above by Tandia, with the semiconductor pattern in the system of Lu in order to ensure that the image data contains the entire semiconductor pattern defect, while also allowing for adequate classification of very small defects (Tandia, para 28: “Using tiling, with overlap, to divide the overall image into smaller sub-images for analysis may result in tile 310, tile 315, and tile 320. By using an overlap, the defect is fully captured in tile 315. There is a sufficient amount of the defect present in tile 315 for the CNN to recognize the defect and classify tile 315. This may result in appropriate classification of failing for the image with section 205”; see also para 20 and 23).
Regarding claim 11 (dependent on claim 10), Lu in view of Tandia teaches wherein the one or more computer systems are configured to generate a histogram indicating the frequency for each degree of discrepancy for each pixel of the first image data and the second image data (Lu, Figure 6, para 72-72: “FIG. 6 a graph of reconstruction errors. A threshold is used to distinguish anomaly from nominal…The reconstruction error can be defined as the difference between the original input vector x and the reconstruction”; reconstruction error is calculated for each pixel, para 45: “The anomaly region can be identified by thresholding the pixel-level reconstruction error and/or probabilities”).
Regarding claim 12 (dependent on claim 11), Lu in view of Tandia teaches wherein the one or more computer systems are configured to evaluate a shape of the histogram (Lu, para 74: “FIG. 6 shows that reconstruction error can exhibit two modal distributions, which makes it possible for auto-thresholding to separate anomalies from nominal”; para 75: “from the error histogram separation, the 200 steps model may be sufficient to separate anomaly from nominal validation images, although longer training can be performed to get better reconstructed images”).
Regarding claim 13 (dependent on claim 11), Lu in view of Tandia teaches wherein the one or more computer systems are configured to allow a display device to display (Tandia, displaying results taught by Tandia, see note below regarding combination; para 53: “a display may be connected to the computer system performing the classification. When classification of an image is completed, the display may provide information about the classification of the image, such as with text or a graphical indicator”) different histograms (Histograms taught by Lu, see claim 6 and the histograms of Figure 6-7C) obtained from different semiconductor wafers manufactured at different manufacturing timings (Tandia, classified image for a product, para 53: “operation 710 to output the classification for the image to a graphical user interface…When classification of an image is completed, the display may provide information about the classification of the image, such as with text or a graphical indicator. The computer system execution the classification may be connected to a system which captures images of the products and includes a sorting mechanism”; images for different semiconductor wafers manufactured at different manufacturing timings are separately processed; see also next citation).
Lu teaches a histogram output from the computer system, but fails to explicitly teach a display device and histogram outputted for different semiconductor wafers; while Tandia teaches displaying the output of the computer system on a display and outputting classification results for each product manufactured. It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the display and processing operation (para 53 of Tandia) of Tandia with the system of Lu in order to display the results of the image processing to a user for decision making regarding the imaged product (Tandia, para 53: “Based on the classification result, the sorting mechanism may move the product associated with the classified image to a particular bin. For example, a product that is classified as passing may be moved to bin for distribution or sales, while a product that is classified as failing may be moved to a bin of rejects for trash or recycling”). Tandia teaches classification of each sub-image, and thus a person of ordinary skill in the art could combine the results of the classification with the histogram of Lu using known methods to represent the frequency of anomalies in an image.
Regarding claim 14 (dependent on claim 10), in the combination of Lu in view of Tandia (see combination rationale below), Lu teaches wherein the one or more of computer systems are provided with a training device including an autoencoder trained in advance (Lu, para 50: "the model includes a variational autoencoder") by inputting a plurality of images at different locations (Lu, see Figure 2 attached above wherein images of different locations are utilized to train the autoencoder; para 51: “The autoencoder can be trained on clean sites”), and wherein the one or more computer systems divide image data into a plurality of inspection sub-images (Lu, see Figure 5; para 71: “FIG. 5 illustrates input and reconstructed SEM patches with an autoencoder”), input the plurality of inspection sub-images to the autoencoder (Lu, Figure 5 and para 71; para 51: “The autoencoder can be trained on clean sites and inference can be run with test sites that may contain defects”; para 75: “Then the validation data, which contains some images with defects, was passed in”), and compare the first image data which is output from the autoencoder with the second image data (Lu, para 48: "a difference between reconstructed and original SEM images may be calculated at 104 to locate the anomaly patterns (e.g., defects)"; para 73: "The reconstruction error can be defined as the difference between the original input vector x and the reconstruction"; see also para 68).
However, Lu fails to explicitly teach wherein the autoencoder is trained by inputting a plurality of images at different locations included in a training image (Emphasis added). Further, Lu fails to explicitly teach wherein the second image is divided into the plurality of inspection sub-images (Emphasis added).
While Lu teaches that the autoencoder can be trained using patch images, Lu fails to teach explicitly wherein a plurality of the patches are from the same training image (Lu, para 64: "The semi-supervised model can be trained with same patch size images from the clean sites of the same SEM layer, then the detection output patches can be passed to this model"). However, Tandia teaches wherein the machine learned model is trained by inputting a plurality of images at different locations included in a training image (Tandia, para 34: "splitting the images based on the currently selected tile size and step size from operation 410 and operation 415. The resulting sub-images from the split are then labeled according to the presence of an anomaly. At operation 425, the new data set is used to train and build a CNN model"). It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the different images from a training image, taught by Tandia, with the system of Lu in order to maximize the number of possible images for training from a given number of training images (Tandia, para 40: “Because the sub-images are overlap-split, all the features of original image may be captured in different patterns, but this results in an increase in the number of images in the dataset”). Doing so can improve the performance of the autoencoder by training with a larger dataset.
Additionally, Lu teaches that validation sub-images are input to the autoencoder, but fails to teach explicitly wherein the sub-images are from the same input image. Tandia further discloses dividing the input image into a plurality of inspection sub-images (Tandia, steps 702-706 in Figure 7; para 49: "divide the image into tiles with a size and overlap between each tile"; para 50: "use a classifier model for each tile to identify anomaly presence in the tile"). It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the divided input image of Tandia with the system of Lu in order to detect defects on the edges of sub-images (Tandia, para 22: “if a defect were to present on the edge between two tiles then it may be undetected by the classifier as the portion appearing in each respective tile would not fall into the known defects of the CNN. Thus, a tile overlap is utilized to ensure a defect is not missed by being split along tile edges”). Utilizing overlapping sub-images from the same input image ensures that all image features are represented in the tested dataset (Tandia, para 21: “The methods and techniques use an ‘overlapped-step-by-step’ process to crop the original images and generate many sub-images (tiles) to ensure all the features are included in the new image model database and each tile is large enough to possibly contain all features representing the different possible characteristics (defects) to be classified”).
Regarding claim 15 (dependent on claim 10), Lu in view of Tandia teaches wherein the one or more computer systems are configured to evaluate the degree of discrepancy of the first image data and the second image data for each pixel (Lu, Figure 6, para 72-72: “FIG. 6 a graph of reconstruction errors. A threshold is used to distinguish anomaly from nominal…The reconstruction error can be defined as the difference between the original input vector x and the reconstruction”; reconstruction error is calculated for each pixel, para 45: “The anomaly region can be identified by thresholding the pixel-level reconstruction error and/or probabilities”).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Lu in view of Tandia, in further view of Enno (WO Patent No. 2018/235266 A1).
Regarding claim 8 (dependent on claim 6), Lu in view of Tandia teaches wherein the one or more computer systems are configured to divide the input image into the plurality of inspection sub-images while providing an overlapped region (Tandia, see Fig. 1, para 26: “The whole image 105 is then divided into tile 110 size images for analysis. To prevent an anomaly or defect from being missed by falling on the edge between two tiles, an overlap is also determined”; see also para 21 and combination in claim 1), evaluate the degree of discrepancy of the divided input image and the output image (Lu, Figure 6, para 72-73: “FIG. 6 a graph of reconstruction errors. A threshold is used to distinguish anomaly from nominal…The reconstruction error can be defined as the difference between the original input vector x and the reconstruction”; the reconstruction error represents a discrepancy between the image inputted to the autoencoder and the reconstructed output), and allow a display device to display identification information (Tandia, para 53: “a display may be connected to the computer system performing the classification. When classification of an image is completed, the display may provide information about the classification of the image, such as with text or a graphical indicator”; see further combination with Lu described below), but fails to teach wherein the identification information corresponds to the number of inspection sub-images with a degree of discrepancy being a predetermined value or more among the inspection sub-images constituting the overlapped region.
Lu teaches a histogram output from the computer system that displays degrees of discrepancy, but fails to explicitly teach a display device; while Tandia teaches displaying the output of the computer system on a display. It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the display of Tandia with the system of Lu in order to display the results of the image processing to a user for decision making regarding the imaged product (Tandia, para 53: “Based on the classification result, the sorting mechanism may move the product associated with the classified image to a particular bin. For example, a product that is classified as passing may be moved to bin for distribution or sales, while a product that is classified as failing may be moved to a bin of rejects for trash or recycling”).
Further, Enno teaches a device/method for detecting abnormalities in images (Enno, para 33: “The inspection device 10 acquires an image of the inspection object 100 taken by the camera 30 via the communication network N, and determines whether or not there is an abnormality in the inspection object 100 using a trained model.”). Enno discloses outputting identification information (Enno, results regarding classification, see also display unit in para 56) corresponding to the number of inspection sub-images with a degree of discrepancy being a predetermined value or more (Enno, para 51: “the classification unit 13 may calculate an evaluation value based on the partial image and compare the evaluation value with a threshold value to classify the partial image into one that contains an abnormality and one that does not contain an abnormality”) among the inspection sub-images constituting the overlapped region (para 8: “an image is divided into a plurality of first partial images and a plurality of second partial images different from the plurality of first partial images, and when it is determined that each of the plurality of first partial images and the plurality of second partial images contains an abnormality, the presence or absence of an abnormality in the object to be inspected is determined based on the overlap of those partial images”; the identification information corresponds to the number of sub-images, partial images, classified as containing an abnormality – the number being two partial images with the overlap, para 53: “The judgment unit 14 may judge whether or not there is an abnormality in the object to be inspected 100 based on the overlap of at least two types of partial images”).
It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the identification information, taught by Enno, with the system of Lu in order to inform the decision regarding whether an abnormality exists based on multiple processed images, increasing the probability of accurate results from the system (Enno, para 8: “the presence or absence of an abnormality in the object to be inspected is determined based on the overlap of those partial images, thereby reducing erroneous determinations and enabling the presence or absence of an abnormality in the object to be inspected to be detected accurately even when there are a variety of possible abnormalities”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Lee et al. (U.S. Patent No. 2020/0134810 A1) teaches a similar method/system.
Lyu (Introduced in Non-Final Office Action of 9/16/2025 - U.S. Patent No. 2021/0027101 A1) teaches training a machine learning model using sub-images (Para 43: "To train a deep learning model, the system may acquire a preliminary deep learning model, acquire at least one sample image, generate a plurality of sub-images based on the at least one sample image, train the preliminary deep learning model using the plurality of sub-images to obtain a trained deep learning model").
Matsubara et al. (Introduced in Non-Final Office Action of 9/16/2025 - Matsubara, T., Sato, K., Hama, K., Tachibana, R., & Uehara, K. (2020). Deep generative model using unregularized score for anomaly detection with heterogeneous complexity. IEEE Transactions on Cybernetics, 52(6), 5161-5173.) teaches training/testing with sub-images to detect anomalies in an image (Paragraph before section V on pg. 5: “During training, a training sample was randomly cropped to sizes of 96 × 96. In the test phase, a test sample was sequentially cropped to the same size with a stride of 16×16. If the anomaly score of at least one image patch exceeded a threshold, the test sample was considered an anomaly”).
Zhou et al. (Introduced in Non-Final Office Action of 9/16/2025 - CN Patent No. 109345538 A) teaches a method wherein the pixels of overlapping sub-images are classified (Para 16: “Input the test sample into the network, extract multiple continuous overlapping segments from each test image, average the multiple prediction results to obtain the classification probability of each pixel”).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMMA E DRYDEN whose telephone number is (571)272-1179. The examiner can normally be reached M-F 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANDREW BEE can be reached at (571) 270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EMMA E DRYDEN/Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677