DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 2, 5, and 8-10 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Roth et al. (NPL: Towards Total Recall in Industrial Anomaly Detection, hereafter referred as Roth).
Regarding Claim 1:
Roth teaches a method for detecting faulty manufactured items (Roth: 1. Introduction; PatchCore for industrial anomaly detection), the method comprises: obtaining an evaluated manufactured item (MI) image (Roth: 4.1. Experimental Details; MVTec AD contains 15 sub-datasets with a total of 5354 images, 1725 of which are in the test set; Magnetic Tile Defects (MTD) dataset as used in , which contains 925 defect-free and 392 anomalous magnetic tile images with varied illumination levels and image sizes); segmenting the image of the evaluated MI to provide different groups of evaluated MI patches, wherein evaluated MI patches of the different groups of evaluated MI patches differ from each other by one or more evaluated MI patch generation attribute (Roth: Fig. 2; images are classified as anomalies if at least one patch is anomalous, and pixel-level anomaly segmentation is generated by scoring each patch-feature); generating evaluated MI patches representations (EMIPRs) for the evaluated MI patches of the different groups of evaluated MI patches (Roth: 3. Method; use a memory bank M of patch-level features comprising intermediate or mid-level feature representations to make use of provided training context, avoiding features too generic or too heavily biased towards ImageNet classification; each patch-representation operates on a large enough receptive field size to account for meaningful anomalous context robust to local spatial variations); matching the EMIPRs to reference MI patches representations (RMIPRs) of reference MI patches, to provide comparison results (Roth: 3.3. Anomaly Detection with PatchCore; With the nominal patch-feature memory bank M, we estimate the image-level anomaly score s ∈ R for a test image xtest by the maximum distance score s∗ between test patch features in its patch collection P(xtest) = Ps,p(φj(xtest)) to each respective nearest neighbour m∗ in M); wherein the reference MI patches are selected from reference MI patches candidates, based on at least one of (a) a popularity of reference MI patch candidates representations or (b) feedback from a person (Roth: 3.2. Coreset-reduced patch-feature memory bank; Because PatchCore uses nearest neighbour computations (next Section), we use a minimax facility location coreset selection, see e.g., [48] and [49], to ensure approximately similar coverage of the M-coreset MCinpatch-level feature space as compared to the original memory bank M); wherein reference MI patches of the different groups of reference MI patches differ from each other by one or more reference MI patch attribute (Roth: Fig. 2; images are classified as anomalies if at least one patch is anomalous, and pixel-level anomaly segmentation is generated by scoring each patch-feature); and determining a state of the evaluated manufactured item based on the comparison results (Roth: 3.3. Anomaly Detection with PatchCore; To obtain s, we use scaling w on s∗ to account for the behaviour of neighbour patches: If memory bank features closest to anomaly candidate mtest,∗, m∗, are themselves far from neighbouring samples and thereby an already rare nominal occurrence, we increase the anomaly score).
In regards to Claim 2, Roth further teaches the method according to claim 1 wherein the reference MI patches are selected based on the popularity of the reference MI patch candidates representations, and without receiving feedback from the person (Roth: 3.2. Coreset-reduced patch-feature memory bank; Because PatchCore uses nearest neighbour computations (next Section), we use a minimax facility location coreset selection, see e.g., [48] and [49], to ensure approximately similar coverage of the M-coreset MCinpatch-level feature space as compared to the original memory bank M).
In regards to Claim 5, Roth further teaches the method according to claim 1, wherein the reference MI patch candidates are generated by: receiving test images of test Mis (Roth: 4.1. Experimental Details; MVTec AD contains 15 sub-datasets with a total of 5354 images, 1725 of which are in the test set; Magnetic Tile Defects (MTD) dataset as used in , which contains 925 defect-free and 392 anomalous magnetic tile images with varied illumination levels and image sizes); segmenting the test images of the test MIs to provide test MI patches (Roth: Fig. 2; images are classified as anomalies if at least one patch is anomalous, and pixel-level anomaly segmentation is generated by scoring each patch-feature); generating test MI patches representations (TM IPRs) that are reference MI patches candidates (Roth: 3. Method; use a memory bank M of patch-level features comprising intermediate or mid-level feature representations to make use of provided training context, avoiding features too generic or too heavily biased towards ImageNet classification; each patch-representation operates on a large enough receptive field size to account for meaningful anomalous context robust to local spatial variations); grouping the TIMPRs to provide TIMPRs groups (Roth: Fig. 2; images are classified as anomalies if at least one patch is anomalous, and pixel-level anomaly segmentation is generated by scoring each patch-feature); selecting, out of the TIMPRs groups, reference MI patches representations (RIMPRs) groups, wherein the selecting is based on the at least one of (a) the popularity of the reference MI patch candidates representations or (b) the feedback from the person (Roth: 3.2. Coreset-reduced patch-feature memory bank; Because PatchCore uses nearest neighbour computations (next Section), we use a minimax facility location coreset selection, see e.g., [48] and [49], to ensure approximately similar coverage of the M-coreset MCinpatch-level feature space as compared to the original memory bank M).
In regards to Claim 8, Roth further teaches the method according to claim 5 wherein the matching comprises matching the evaluated MI patch representations to the RIMPR groups (Roth: 3.3. Anomaly Detection with PatchCore; With the nominal patch-feature memory bank M, we estimate the image-level anomaly score s ∈ R for a test image xtest by the maximum distance score s∗ between test patch features in its patch collection P(xtest) = Ps,p(φj(xtest)) to each respective nearest neighbour m∗ in M).
In regards to Claim 9, Roth further teaches the method according to claim 1 wherein the determining of the state of the evaluated manufactured item comprising determining that the evaluated manufactured item is faulty when the comparison results indicate that there is at least a predefined number of evaluated MI patch representations that do match any of the reference MI patch representations (Roth: 3.2. Coreset-reduced patch-feature memory bank; Because PatchCore uses nearest neighbour computations (next Section), we use a minimax facility location coreset selection, see e.g., [48] and [49], to ensure approximately similar coverage of the M-coreset MCinpatch-level feature space as compared to the original memory bank M).
In regards to Claim 10, Roth further teaches the method according to claim 1 wherein the matching is executed regardless of a location of the reference MI patch (Roth: 3.1. Locally aware patch features; We then use φi,j(h,w) = φj(xi,h,w) ∈ Rc∗ to denote the c∗-dimensional feature slice at positions h ∈{1,...,h∗} and w ∈ {1,...,w∗}. Assuming the receptive field size of each φi,j to be larger than one, this effectively relates to image-patch feature representations. Ideally, each patch-representation operates on a large enough receptive field size to account for meaningful anomalous context robust to local spatial variations).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over Roth et al. (NPL: Towards Total Recall in Industrial Anomaly Detection, hereafter referred as Roth) in view of Balasubramanian et al. (U.S. Patent App. Pub No. 2022/0171995 A1, hereafter referred as Balasubramanian).
In regards to Claim 3, Roth fails to further teach the method according to claim 1 wherein the reference MI patches are selected based on the feedback from the person and regardless of the popularity of reference MI patch candidates representations.
Balasubramanian, like Roth, is directed to anomaly detection. Balasubramanian does teach wherein the reference MI patches are selected based on the feedback from the person and regardless of the popularity of reference MI patch candidates representations (Balasubramanian: Par. [0036]; feedback on the ML model's predictions (e.g. images which are incorrectly marked abnormal, or mark regions in the image where defects are not identified) may be used to improve the model accuracy, thereby learning from human expertise and getting better over time).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Roth to utilize the feedback technique, as taught by Balasubramanian, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. As taught by Balasubramanian, the proposed modification would allow the model to get better over time using human expertise (Balasubramanian: Par. [0036]).
In regards to Claim 4, Roth as modified by Balasubramanian further teaches the method according to claim 1 wherein the reference MI patches are selected from reference MI patches candidates, by (a) determining, based on the popularity of reference MI patch candidates representations, popular reference MI patch candidates representations of the reference MI patch candidates representations (Roth: 3.2. Coreset-reduced patch-feature memory bank; Because PatchCore uses nearest neighbour computations (next Section), we use a minimax facility location coreset selection, see e.g., [48] and [49], to ensure approximately similar coverage of the M-coreset MCinpatch-level feature space as compared to the original memory bank M), (b) providing to the person information regarding the popular reference MI patch candidates representations, (c) receiving feedback from the person regarding the popular reference MI patch candidates representations (Balasubramanian: Par. [0036]; feedback on the ML model's predictions (e.g. images which are incorrectly marked abnormal, or mark regions in the image where defects are not identified) may be used to improve the model accuracy, thereby learning from human expertise and getting better over time), and (d) selecting the reference MI patches based on the feedback.
Claims 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Roth et al. (NPL: Towards Total Recall in Industrial Anomaly Detection, hereafter referred as Roth) in view of Sohn et al. (U.S. Patent App. Pub No. 2023/0153980 A1, hereafter referred as Sohn).
In regards to Claim 6, Roth fails to further teaches the method according to claim 5, wherein the test images are unlabeled.
Sohn, like Roth, is directed to anomaly detection. Sohn does teach wherein the test images are unlabeled (Sohn: Par. [0041]; That is, each image 152 is not paired with any corresponding label indicating whether the image 152 includes an anomaly or a type of anomaly if one is present in the image 152. In these implementations, the anomaly detector 200 trains in an unsupervised fashion using the plurality of images 152 that are unlabeled.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Roth to utilize the unlabeled data, as taught by Sohn, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. As taught by Sohn, the proposed modification would solve how oftentimes a scarce amount of labeled anomalous data is available to train anomaly detection models (Sohn: Par. [0003]).
In regards to Claim 7, Roth as modified by Sohn further teaches the method according to claim 5 wherein the grouping comprises clustering the reference MI patch representations (Sohn: Par. [0024]; the anomaly clustering request 20 specifies K anomaly groups 302 for the anomaly detector 200 to use during classification).
Claims 11-12 and 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Roth et al. (NPL: Towards Total Recall in Industrial Anomaly Detection, hereafter referred as Roth) in view of Tsai et al. (NPL: Multi-Scale Patch-Based Representation Learning for Image Anomaly Detection and Segmentation, hereafter referred as Tsai).
In regards to Claim 11, Roth fails to further teach the method according to claim 1 wherein at least two of the different groups of MI patches cover an entirety of the evaluated manufactured item.
Tsai, like Roth, is directed to anomaly detection. Tsai does teach wherein at least two of the different groups of MI patches cover an entirety of the evaluated manufactured item (Tsai: 3.1. Training Stage; first select two patches of size 64 × 64 and use the same selection method for patches of size 32 × 32 and size 16 × 16 from the same image several times. The 3 different sizes of patches will then go through times. The 3 different sizes of patches will then go through the same work flow except that the encoders are with different architectures).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Roth to utilize the groups of different sized patches, as taught by Tsai, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. As taught by Tsai, the proposed modification would improve the self-supervised learning strategy by presenting better representation learning (Tsai: Abstract).
In regards to Claim 12, Roth as modified by Tsai further teaches the method according to claim 1 wherein the one or more evaluated MI patch generation attribute is an evaluated MI patch size (Tsai: 3.1. Training Stage; first select two patches of size 64 × 64 and use the same selection method for patches of size 32 × 32 and size 16 × 16 from the same image several times. The 3 different sizes of patches will then go through times. The 3 different sizes of patches will then go through the same work flow except that the encoders are with different architectures).
In regards to Claim 14, Roth as modified by Tsai further teaches the method according to claim 1 wherein the one or more evaluated MI patch generation attribute is an overlap evaluated MI patch between adjacent MI patch representations (Tsai: 3.4. Inference Stage; A test image will first be split into overlapped patches with patch size θ ∈ {64,32,16}; Next, we picture the anomaly map for each θ by making the above patch-wise calculated anomaly scores distributed to the pixels by averaging the scores overlapped on the same pixels).
In regards to Claim 15, Roth as modified by Tsai further teaches the method according to claim 1 wherein there at least three different groups of MI patch representations (Tsai: 3.1. Training Stage; first select two patches of size 64 × 64 and use the same selection method for patches of size 32 × 32 and size 16 × 16 from the same image several times. The 3 different sizes of patches will then go through times. The 3 different sizes of patches will then go through the same work flow except that the encoders are with different architectures).
In regards to Claim 16, Roth as modified by Tsai further teaches the method according to claim 1 wherein the evaluated MI patch representations comprise features of one or more layer of a neural network (Tsai: 3.2. Objective Functions; Our network is trained with four different objective func tions, which are SVDD loss, Cos loss, SSL loss, and Kmean loss.).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Roth et al. (NPL: Towards Total Recall in Industrial Anomaly Detection, hereafter referred as Roth) in view of Amzaleg et al. (U.S. Patent App. Pub No. 2014/0212021 A1, hereafter referred as Amzaleg).
In regards to Claim 13, Roth fails to further teach the method according to claim 1 wherein the one or more evaluated MI patch generation attribute is an evaluated MI patch shape.
Amzaleg, like Roth, is directed to anomaly detection. Amzaleg does teach wherein the one or more evaluated MI patch generation attribute is an evaluated MI patch shape (Amzaleg: Par. [0028]; for each of the multiple source patches, based on that source patch and a respective patch-similarity criterion, determining a similarity level with respect to each of a respective plurality of reference patches, each of which is associated with a reference image pixel; thereby for each of a plurality of reference image pixels, determining a plurality of similarity levels that are determined for Frespective reference image pixel).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Roth to utilize the groups of different shaped patches, as taught by Amzaleg, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. As taught by Amzaleg, the proposed modification would disregard the need for features with high precision and uniformity, which in turn necessitates careful process monitoring(Amzaleg: Par. [0002]).
Pertinent Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Hoshen et al. (U.S. Patent App. Pub No. 2023/0281959 A1) teaches deep learning-based anomaly detection in images.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RENAE BITOR whose telephone number is (703)756-5563. The examiner can normally be reached Monday to Friday: 8:00 - 5:30 but off the 1st Friday of the biweek.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, GREG MORSE can be reached on (571)272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RENAE A BITOR/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights