Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
Claims 5 and 6 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 5 the phrase " according to a deviation situation” and “from an average pixel of the predicted image data” are unclear because the claim fails to specify what parameter is deviating, how the deviation is determined, what “average pixel” means, and from what population or region the average is derived. Accordingly, the metes and bounds of claim 5 are not clear.
Regarding claim 6 the phrase "exceptional color” and “accumulation situation of pixel values” are unclear because the claim fails to define what constitutes an exceptional color, how such color is determined to be different from the predetermined colors, what accumulation operation is performed on the pixel values, and how that accumulation is used to detect and classify the defect. Accordingly, the metes and bounds of claim 6 are not clear. See MPEP § 2173.05(d).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1-2, 13, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Banno et al. (US 20210166374 A1) in view of Namiki (US 20190012579 A1) and further in view of Rothmund et al. (US 20230022631 A1).
Regarding claim 1, Banno et al. teaches a defect detection method comprising steps of: applying, by a processing circuit, (see para [0050]; “The image data of the item 12 captured by the camera 31 is sent to the computer 32. The computer 32 inspects the item 12 for defects using the image data sent from the camera 31…. the computer 32 may be a microprocessor”), a predetermined color for each type of defect to a defective portion having a contour shape and a defective portion having a region shape that may be present in an inspection target object in an original image obtained by imaging of the inspection target object (see para [0076]; “along the contour of the defect Dg to outline a defect region DA…the channel addition section 61A adds the alpha channel Imα to the NG image data Imd. Then, the pixel value assignment section 61B assigns a second pixel value P2 as the pixel value that indicates a defect to a region Da (hereafter, also referred to as “defect region Da”) corresponding to the defect region DA in the alpha channel”, see also para [0012]; “a different second pixel value is assigned in accordance with the type of the defect” and see para [0095]; “assigned to the region corresponding to the defect region in the other channel of the image as the pixel value indicating a defect” Note: outlining a contour, defining a region (DA), and assigning specific pixel values to represent the defect type in an image channel); preparing one or more sets of first teaching data by the processing circuit using as the first teaching data, original image data obtained by imaging of the inspection target object and having a defect (see para [0064]; “the image data group IG includes a training data group TrD and a test data group TsD. The image data Im belonging to the training data group TrD serves as the original image data when the teaching data generation unit 52 generates teaching data for supervised learning. The training data group TrD contains OK image Im1 and NG image Imd (Im2 and Im3) shown in FIG. 3”, see also para [0074]; “The image data Im is the image data Imd of a defective item (also referred to as “NG image data Imd”) including a defect Dg”); wherein, in the step of preparing a learned machine learning model, tiling processing is performed on the original image data and the marked image
data in the first teaching data to obtain segmented images (see para [0006]; “In machine learning such as deep learning, a massive number of images need to be learned to determine a parameter of an identifier. For example, when an inspection is performed using an identifier, precision is improved if the image is divided .. In this case, a label needs to be assigned to each image segment”, see also para [0010]; “In the segmentation step, the image including the other channel is segmented into multiple image segments”, see also para [0014]; “segmented into the multiple image segments in which adjacent image segments are partially overlapped with each other”), and then learning of a
correspondence relationship between the original image data and the marked image
data is performed for each tiled segmented image (see para [0020]; “an identifier is generated by performing supervised or semi-supervised machine learning using labeled image data as teaching data, the labeled image data being generated by the method for generating labeled image data. In the determination step, multiple image segments are obtained by segmenting an image of an inspection subject into the number of segments that is the same as the number of segments of the image segments used for generating the identifier”),
in both types of the tiling processing, segmented images are created such that,
after tiling, both edge portions of adjacent segmented images overlap (see para [0100]; “the image Im is divided into M×N of the image segments DI having the same size and overlapped with the adjacent image segments DI by the overlap amount ΔL or 2.Math.ΔL”), and further, in a case where a marked image data as a segmented image in the first teaching data does not include a marking portion, learning of a correspondence relationship
between original image data as the segmented image and identity mapping data of the original image data is performed (see para [0106]; “step S27 correspond to a process for assigning a non-defect label to the image segment DI, in which the second pixel value P2 indicating a defect is not included in the other channel Therefore, the processes of steps S25 to S27 corresponds to one example of “label assignment step” Note; learning the correspondence between a normal original image and its corresponding normal mask (identity mapping, where input = output, mapping defect-free to defect-free) is fundamentally the same as assigning a "non-defect" label to an image segment. Therefore, steps S25–S27, which handle this logic, act as a label assignment step to identify that an image segment is healthy or non-defective). However, Bunno et al. does not teach and marked image data obtained by labeling with a predetermined color for each type of defect on the original image data in the step of applying a predetermined color, preparing one or more sets of second teaching data by the processing circuit using, as the second teaching data, original image data obtained by imaging of the inspection target object and having no defect, and identity mapping data having no difference from the original image data; and preparing a learned machine learning model by the processing circuit performing learning using the first teaching data and the second teaching data.
In the same field of endeavor, Namiki teaches and marked image data obtained by labeling with a predetermined color for each type of defect on the original image data in the step of applying a predetermined color (see para [0060]; “a predetermined color may be given so as to surround the defect 20 on the surface of the inspection target 12 by a predetermined color pen”). Accordingly, it would have been obvious to one ordinary skill in the art of the claimed invention before the effective filling date of the general use of a method for generating labeled image data of Bunno et al. in view of a machine learning device that creates training data to be used in machine learning of Namiki in order to distinguish between OK and NG on new input images of the inspection target (see para [0060]).
However, the combination of Bunno et al. Namiki as a whole does not teach preparing one or more sets of second teaching data by the processing circuit using, as the second teaching data, original image data obtained by imaging of the inspection target object and having no defect, and identity mapping data having no difference from the original image data; and preparing a learned machine learning model by the processing circuit performing learning using the first teaching data and the second teaching data.
In the same field of endeavor, Rothmund et al. teaches preparing one or more sets of second teaching data by the processing circuit using, as the second teaching data, original image data obtained by imaging of the inspection target object and having no defect (see para [0005]; “first neural network trained based on a first set of training images, and the first set of training images comprises a plurality of training images each showing a corresponding defect-free product…. one third set of training images, and the at least one third set of training images comprises a plurality of training images each showing a corresponding defective product”) and identity mapping data having no difference from the original image data (see para [0073]; “the at least one first neural network is trained to reconstruct the patches at the edge regions to be as similar as possible to the original image when training the same”); and preparing a learned machine learning model by the processing circuit performing learning using the first teaching data and the second teaching data (see para [0006]; “the first set of training images comprising a plurality of training images each showing a defect-free product; training at least a first neural network of an inpainting autoencoder based on the first set of training images….. creating at least one third set of training images, wherein the at least one third set of training images comprises a plurality of training images each showing a defective product, and further at least one defect is marked in each training image of the third set of training images; training at least one third neural network of a defect identifier based on the at least one third set of training images”), tiling processing is similarly performed on the original image data and the identity mapping data in the second teaching data to obtain segmented images, and then learning of a correspondence relationship between the original image data and the identity mapping data is performed for each tiled segmented image (see para [0005]; “wherein the autoencoder includes at least one first neural network trained based on a first set of training images, and the first set of training images includes a plurality of training images each showing a corresponding defect-free product”, see also para [0097]; “perform the reconstruction of the individual patches in parallel. The reconstructed patches can then be reassembled to form a reconstructed overall image”). Accordingly, it would have been obvious to one ordinary skill in the art of the claimed invention before the effective filling date of the general use of a method for generating labeled image data of Bunno et al. in view of a machine learning device that creates training data to be used in machine learning of Namiki and further in view of a method of analyzing a product includes performing an anomaly detection on a received image using an autoencoder of Rothmund et al. in order to generate anomaly-free equivalent of the received image with the same position or orientation (see para [0005]).
Regarding claim 2, the rejection of claim 1 is incorporated herein.
Namiki in the combination further teach wherein the machine learning model includes a convolutional neural network (CNN) or a fully convolutional neural network (FCNN) (see para [0054]; “the learning model constructed by a neural network including an input layer, output layer and intermediate layer can use an appropriate system. For example, CNN (Convolutional Neural Network) can also be applied”).
Regarding claim 13, the rejection of claim 1 is incorporated herein.
Namiki in the combination further teach a storage medium storing a computer program causing the processing circuit to execute the defect detection method (see para [0007]; “computer program product storing instructions embodied on a computer-readable medium or programmable circuitry, that, when executed by a processor or the programmable circuitry, cause the processor or the programmable circuitry to perform the method”).
Regarding claim 16, the rejection of claim 1 is incorporated herein.
Namiki in the combination further teach a defect detection system comprising a computer device and a storage device, wherein the computer device performs the defect detection method (see para [0007]; “computer program product storing instructions embodied on a computer-readable medium or programmable circuitry, that, when executed by a processor or the programmable circuitry, cause the processor or the programmable circuitry to perform the method”).
Claims 3, 14, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Banno et al., and Namiki et al. in view of Rothmund et al as applied in claim 1 above, and further in view of Kimura et al. (US 20200104990 A1) and Majurski et al. NPL “Exact Tile-Based Segmentation Inference for Images Larger than GPU Memory”.
Regarding claim 3, the rejection of claim 1 is incorporated herein.
Banno et al. in the combination further teach by inputting image data of the inspection target object to the machine learning model (see para [0028]; “The identifier is generated by performing supervised or semi-supervised machine learning using the labeled image data, which is generated by the labeled image data generation device, as teaching data…..The determination unit determines whether the inspection subject is defective based on an output result from the identifier when the image segments obtained by the image segment acquisition unit is input to the identifier”), the input image data having been subjected to the tiling processing (see para [0054]; “The image segment data is obtained by dividing the image of the item 12 captured by the camera 31”). However, the combination of Bunno et al., Rothmund et al and Namiki as a whole does not teach acquiring predicted image data by the processing circuit subtracting input image data from output image data.
In the same field of endeavor, Kimura et al. teaches acquiring predicted image data by the processing circuit subtracting input image data from output image data (see para [0063]; “The calculating section may subtract the reconstructed dataset from the original dataset to generate a differential dataset as the difference between the original and reconstructed datasets”). Accordingly, it would have been obvious to one ordinary skill in the art of the claimed invention before the effective filling date of the general use of a method for generating labeled image data of Bunno et al. in view of a machine learning device that creates training data to be used in machine learning of Namiki and further in view of a method of analyzing a product includes performing an anomaly detection on a received image using an autoencoder of Rothmund et al. and detecting an anomaly by using the weighted differential dataset of Kimura et al. in order to improve accuracy of anomaly detection (see para [0063]). However, the combination of Bunno et al., Rothmund et al, Namiki and Kimura et al. as a whole does not teach and performing reverse tiling processing, by the processing circuit, on the predicted image data, the reverse tiling processing performing combination by ignoring a portion of an outer half of an overlapping edge portion in each piece of the predicted image data before combination.
In the same field of endeavor, Majurski et al. teaches and performing reverse tiling processing, by the processing circuit, on the predicted image data, the reverse tiling processing performing combination by ignoring a portion of an outer half of an overlapping edge portion in each piece of the predicted image data before combination (see Abstract; “Our approach is to select a tile size that will fit into GPU memory with a halo border of half the network receptive field…..Next, stride across the image by that tile size without the halo. The input tile halos will overlap, while the output tiles join exactly at the seams”, see also page 5, 1st para; “The ZoR needs to be cropped out from the prediction so only those pixels with full context are used to build the final result”, Note; recombination of tiled outputs by excluding overlap-edge regions before stitching corresponds to “performing combination by ignoring a portion of an outer half of an overlapping edge portion in each piece of the predicted image data before combination”). Accordingly, it would have been obvious to one ordinary skill in the art of the claimed invention before the effective filling date of the general use of a method for generating labeled image data of Bunno et al. in view of a machine learning device that creates training data to be used in machine learning of Namiki and further in view of a method of analyzing a product includes performing an anomaly detection on a received image using an autoencoder of Rothmund et al. and detecting an anomaly by using the weighted differential dataset of Kimura et al. and further in view of exact tile-based segmentation inference for images larger than GPU memory of Majurski et al. in order to estimate the tiling parameter (see para [0063]).
Regarding claim 14, the rejection of claim 3 is incorporated herein.
Namiki in the combination further teach a storage medium storing a computer program causing the processing circuit to execute the defect detection method (see para [0007]; “computer program product storing instructions embodied on a computer-readable medium or programmable circuitry, that, when executed by a processor or the programmable circuitry, cause the processor or the programmable circuitry to perform the method”).
Regarding claim 17, the rejection of claim 3 is incorporated herein.
Namiki in the combination further teach a defect detection system comprising a computer device and a storage device, wherein the computer device performs the defect detection method (see para [0007]; “computer program product storing instructions embodied on a computer-readable medium or programmable circuitry, that, when executed by a processor or the programmable circuitry, cause the processor or the programmable circuitry to perform the method”).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Banno et al., Namiki and Rothmund et al. in view Kimura et al. and Majurski et al. of as applied in claims 1 and 3 above, and further in view of Hou et al. (US 20130223758 A1).
Regarding claim 4, the rejection of claim 3 is incorporated herein. The combination of Bunno et al., Rothmund et al, Namiki, Kimura et al. and Majurski et al. as a whole does not teach wherein in the step of performing the reverse tiling processing, hue, saturation, and brightness are adjusted based on a predetermined color for each type of defect between a plurality of pieces of the predicted image data before combination.
In the same field of endeavor, Hou et al. teaches wherein in the step of performing the reverse tiling processing, hue, saturation, and brightness are adjusted based on a predetermined color for each type of defect between a plurality of pieces of the predicted image data before combination (see para [0036]; “the categorization is undertaken by analyzing the brightness and saturation of pixels…. the brightness value (1 component in the HIS color model) and saturation value (S component in the HIS colour model)”, see also para [0044]; “The new HIS value of P0 is P0+DIFF/2, and the new HIS value of its corresponding pixel P1 is P1-DIFF/2”, and para [0057]; “The blended image portions are combined”). Accordingly, it would have been obvious to one ordinary skill in the art of the claimed invention before the effective filling date of the general use of a method for generating labeled image data of Bunno et al., Namiki and Rothmund et al. in view of detecting an anomaly by using the weighted differential dataset of Kimura et al. and exact tile-based segmentation inference for images larger than GPU memory of Majurski et al. and further in view of a method of blending stitched document image portions of Hou et al. in order to reduce visually perceivable seams (see para [0036]).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Banno et al., Namiki et al., and Rothmund et al in view of Kimura et al., Majurski et al. and Hou et al. as applied in claims 1, 3 and 4 above, and further in view of Matsumura (US 20060078191 A1).
Regarding claim 5, the rejection of claim 4 is incorporated herein. The combination of Bunno et al., Rothmund et al, Namiki, Kimura et al. and Majurski et al. as a whole does not teach further comprising a step of: detecting and classifying, by the processing circuit, a defect in the image data of the inspection target object according to a deviation situation, from an average pixel of the predicted image data, of a portion of the predetermined color included in the predicted image data.
In the same field of endeavor, Matsumura teach further comprising a step of: detecting and classifying, by the processing circuit, a defect in the image data of the inspection target object according to a deviation situation, from an average pixel of the predicted image data, of a portion of the predetermined color included in the predicted image data (see para [0050]; “an average value .mu.d and a standard deviation .sigma.d of values of pixels in the differential image……. where an absolute value of difference between the value Xd and the average value .mu.d is larger than a value obtained by multiplying the standard deviation .sigma.d by the defect candidate pixel threshold value (i.e., 3)”). Accordingly, it would have been obvious to one ordinary skill in the art of the claimed invention before the effective filling date of the general use of a method for generating labeled image data of Bunno et al., Namiki and Rothmund et al. in view of detecting an anomaly by using the weighted differential dataset of Kimura et al. and exact tile-based segmentation inference for images larger than GPU memory of Majurski et al. and further in view of a method of blending stitched document image portions of Hou et al. and apparatus and method for detecting defect on object of Matsumura in order to detect a defect on a substrate with high accuracy and high efficiency (see para [0050]).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Banno et al., Namiki et al., Rothmund et al in view of Kimura et al., Majurski et al. and Hou et al. as applied in claims 1, 3 and 4 above, and further in view of Hakuk (US 20170178322 A1).
Regarding claim 6, the rejection of claim 4 is incorporated herein. The combination of Bunno et al., Rothmund et al, Namiki, Kimura et al. and Majurski et al. as a whole does not teach further comprising a step of: in a case where the predicted image data includes a portion of an exceptional color that is not any of the predetermined colors, detecting and classifying, by the processing circuit, a defect that is not any of the types of defects according to an accumulation situation of pixel values for the portion of the exceptional color.
In the same field of endeavor, Hakuk teach further comprising a step of: in a case where the predicted image data includes a portion of an exceptional color that is not any of the predetermined colors (see para [0007]; “the processor may associate the pixel with a color histogram value from a color histogram database; determine, for each pixel, whether or not the color histogram value associated with the pixel exceeds a histogram value threshold; in which an associated color histogram value that does not exceed the histogram value threshold indicates an anomalous pixel”, see also para [0061]; “threshold values may be identified and/or chosen which distinguish normal colors from anomalous colors”), detecting and classifying, by the processing circuit, a defect that is not any of the types of defects according to an accumulation situation of pixel values for the portion of the exceptional color (see para [0007]; “identify one or more groups of adjacent anomalous pixels”, see also para [0076]; “a binary mask may include, or result in, an aggregated pixel size of the one or more identified groups or blobs of adjacent anomalous pixels, e.g., an aggregation, accumulation, sum, combination, unification, and/or collection etc., of, or related to, or calculated for, the pixel sizes of each of the one or more identified groups or blobs. In some embodiments, the aggregated pixel size may include a total pixel count of all the anomalous pixels which make up the identified blobs”). Accordingly, it would have been obvious to one ordinary skill in the art of the claimed invention before the effective filling date of the general use of a method for generating labeled image data of Bunno et al., Namiki and Rothmund et al. in view of detecting an anomaly by using the weighted differential dataset of Kimura et al. and exact tile-based segmentation inference for images larger than GPU memory of Majurski et al. and further in view of a method of blending stitched document image portions of Hou et al. and detecting an anomaly in an image from a set of images captured in vivo of Hakuk in order to avoid the triggering of many false alarms (see para [0007]).
Claims 7-10, 15, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Banno et al. and Kimura et al., in view of Majurski et al. and further in view of Hou et al.
Regarding claim 7, Banno et al. teaches a defect detection method see para [0050]; “The computer 32 inspects the item 12 for defects using the image data sent from the camera 31”) comprising steps of: preparing a learned machine learning model by a processing circuit performing learning using, as teaching data, original image data obtained by imaging of an inspection target object and marked image data (see para [0020]; “an identifier is generated by performing supervised or semi-supervised machine learning using labeled image data as teaching data, the labeled image data being generated by the method for generating labeled image data”, see also para [0055]; “the original images used to generate teaching data are not limited to the ones captured by the camera 31 of the inspection device 30 and may be images of the item 12 captured in advance at a different location by a camera that is separate from the inspection device 30”), wherein tiling processing is performed on the original image data and the marked image data to obtain segmented images, learning of a correspondence relationship between the original image data and the marked image data is performed for each tiled segmented image (see para [0020]; “an identifier is generated by performing supervised or semi-supervised machine learning using labeled image data as teaching data, the labeled image data being generated by the method for generating labeled image data. In the determination step, multiple image segments are obtained by segmenting an image of an inspection subject into the number of segments that is the same as the number of segments of the image segments used for generating the identifier”), and, in the tiling processing, segmented images are created such that, after tiling, both edge portions of adjacent segmented images overlap (see para [0014]; “the method for generating labeled image data, it is preferred that in the segmentation step, the image including the other channel be segmented into the multiple image segments in which adjacent image segments are partially overlapped with each other”); acquiring, by the processing circuit, labeled image data of an inspection target object output by the learned machine learning model by inputting image data of the inspection target object to the machine learning model (see para [0028]; “The determination unit determines whether the inspection subject is defective based on an output result from the identifier when the image segments obtained by the image segment acquisition unit is input to the identifier…”), the input image data having been subjected to the tiling processing (see para [0026]; “The segmentation unit divides the image including the other channel into multiple image segments”). However, Banno et al. does not teach in which a defective portion having a contour shape and a defective portion having a region shape that may be present in the inspection target object are labeled with a predetermined color for each type of defect, acquiring, by the processing circuit, predicted image data showing a detection performing reverse tiling processing, by the processing circuit, on the predicted image data, the reverse tiling processing performing combination by ignoring a portion of an outer half of an overlapping edge portion in each piece of the predicted image data before combination, wherein, in the step of performing the reverse tiling processing, hue, saturation, and brightness are adjusted based on a predetermined color for each type of defect between a plurality of pieces of the predicted image data before combination.
In the same field of endeavor, Namiki teaches in which a defective portion having a contour shape and a defective portion having a region shape that may be present in the inspection target object are labeled with a predetermined color for each type of defect (see para [0060]; “a predetermined color may be given so as to surround the defect 20 on the surface of the inspection target 12 by a predetermined color pen…a predetermined shape may be employed as the symbol 22. For example, it is possible to employ various shapes such as the graphic of a rectangle, or a star shape. The defect 20 may be surrounded by the graphic of a rectangle, or a star shape may be drawn on the site of the defect 20”). Accordingly, it would have been obvious to one ordinary skill in the art of the claimed invention before the effective filling date of the general use of a method for generating labeled image data of Bunno et al. in view of a machine learning device that creates training data to be used in machine learning of Namiki in order to distinguish between OK and NG on new input images of the inspection target (see para [0060]). However, the combination of Bunnno et al. and Namiki as a whole does not teach acquiring, by the processing circuit, predicted image data showing a detection performing reverse tiling processing, by the processing circuit, on the predicted image data, the reverse tiling processing performing combination by ignoring a portion of an outer half of an overlapping edge portion in each piece of the predicted image data before combination.
In the same field of endeavor, Majurski et al. teaches acquiring, by the processing circuit, predicted image data showing a detection performing reverse tiling processing, by the processing circuit, on the predicted image data, the reverse tiling processing performing combination by ignoring a portion of an outer half of an overlapping edge portion in each piece of the predicted image data before combination (see page 1, Abstract; “Our approach is to select a tile size that will fit into GPU memory with a halo border of half the network receptive field. Next, stride across the image by that tile size without the halo. The input tile halos will overlap, while the output tiles join exactly at the seams. Such an approach enables inference to be performed on whole slide microscopy images, such as those generated by a slide scanner”, see also page 5, 2nd para; “The ZoR needs to be cropped out from the prediction so only those pixels with full context are used to build the final result”). Accordingly, it would have been obvious to one ordinary skill in the art of the claimed invention before the effective filling date of the general use of a method for generating labeled image data of Bunno et al. in view of a machine learning device that creates training data to be used in machine learning of Namiki and exact tile-based segmentation inference for images larger than GPU memory of Majurski et al. in order to estimate the tiling parameters (see Abstract). However, the combination of Bunnno et al., Namiki and Majurski et al. as a whole does not teach wherein, in the step of performing the reverse tiling processing, hue, saturation, and brightness are adjusted based on a predetermined color for each type of defect between a plurality of pieces of the predicted image data before combination.
In the same field of endeavor, Hou et al. teaches wherein, in the step of performing the reverse tiling processing, hue, saturation, and brightness are adjusted based on a predetermined color for each type of defect between a plurality of pieces of the predicted image data before combination (see para [0036]; “the categorization is undertaken by analyzing the brightness and saturation of pixels…. the brightness value (1 component in the HIS color model) and saturation value (S component in the HIS colour model)”, see also para [0044]; “The new HIS value of P0 is P0+DIFF/2, and the new HIS value of its corresponding pixel P1 is P1-DIFF/2”, and para [0057]; “The blended image portions are combined”). Accordingly, it would have been obvious to one ordinary skill in the art of the claimed invention before the effective filling date of the general use of a method for generating labeled image data of Bunno et al. in view of a machine learning device that creates training data to be used in machine learning of Namiki and exact tile-based segmentation inference for images larger than GPU memory of Majurski et al. and further in view of a method of blending stitched document image portions of Hou et al. in order to reduce visually perceivable seams (see para [0036]).
Regarding claim 8, the rejection of claim 7 is incorporated herein.
Bunno et al. in the combination further teach further comprising a step of: classifying a defect in the image data of the inspection target object according to the predetermined color included in the predicted image data (see para [0012]; “a different second pixel value is assigned in accordance with the type of the defect. Further, it is preferred that in the label assignment step, a different defect label be assigned to the image segment including the second pixel value in accordance with the value of the second pixel value”).
Regarding claim 9, the rejection of claim 7 is incorporated herein.
Namiki in the combination further teach further comprising a step of: in a case where the predicted image data includes a portion of a color that is not any of the predetermined colors, classifying the portion of the color as a portion in which a defect that is not any of the types of defects is detected.
Regarding claim 10, the rejection of claim 7 is incorporated herein.
Namiki in the combination further teach wherein the machine learning model includes a convolutional neural network (CNN) or a fully convolutional neural network (FCNN) (see para [0054]; “CNN (Convolutional Neural Network) can also be applied”).
Regarding claim 15, the rejection of claim 7 is incorporated herein.
Namiki in the combination further teach a storage medium storing a computer program causing the processing circuit to execute the defect detection method (see para [0007]; “computer program product storing instructions embodied on a computer-readable medium or programmable circuitry, that, when executed by a processor or the programmable circuitry, cause the processor or the programmable circuitry to perform the method”).
Regarding claim 18, the rejection of claim 7 is incorporated herein.
Namiki in the combination further teach a defect detection system comprising a computer device and a storage device, wherein the computer device performs the defect detection method (see para [0007]; “computer program product storing instructions embodied on a computer-readable medium or programmable circuitry, that, when executed by a processor or the programmable circuitry, cause the processor or the programmable circuitry to perform the method”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WINTA GEBRESLASSIE whose telephone number is (571)272-3475. The examiner can normally be reached Monday-Friday9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at 571-270-5180. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WINTA GEBRESLASSIE/Examiner, Art Unit 2677
Prosecution Insights