DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 04 May 2026 have been fully considered but they are not persuasive.
Applicant argues that Ye does not disclose “generat[e]ing a context image by applying an image degradation operation that reduces spatial visual detail of an input image to remove detailed information from the input image”.
In response, the terms “spatial visual detail” and “reducing spatial visual detail” are neither defined nor adequately disclosed in the instant specification. See the 112(a) and 112(b) rejections below.
Furthermore, Ye’s Attribute Erasing Module (AEM) applies an image degradation operation that reduces spatial visual detail of an input image to remove detailed information from the input image. See IIIA, figs. 1, 2 including Attribute Erasing Module (AEM) that erases or otherwise reduces spatial visual detail to remove detailed information from the input image via the “Graying” operation that “averages each pixel value along the channel dimension of the images”, pg. 121. This Graying operation is understood by the Examiner and Applicant to involve averaging the three-color channels (e.g. red, green and blue) into a single grayscale value.
Applicant argues that “the resulting grayscale image retains full spatial resolution, full sharpness, and all spatial structures including edges, textures and fine spatial features remain intact, as only chromatic information is removed.” No evidence is presented to support this contention.
In addition to the lack of guidance from the specification and unclear meaning of “spatial visual detail”, removing the color information by an averaging operation also removes or reduces “spatial visual detail.” Consider an input color image in having an edge feature with a color (e.g. a red edge or object) that only differs from the background by the color (e.g. a pure green background) and otherwise has the identical intensity (e.g. pixels in the edge have an RGB values of (10, 0, 0) while the background pixels have values of (0, 10, 0). Applying Ye’s graying algorithm, the resulting average value is the same (10/3) for the edge and background. As such the graying algorithm completely destroys the edge and otherwise reduces the spatial visual detail by removing detailed information (e.g. the red/green edge) from the input image.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-17 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
The instant specification fails to include an adequate written description for
“"generat[e]ing a context image by applying an image degradation operation that reduces spatial visual detail of an input image to remove detailed information from the input image” as recited in independent amended claims 1, 5 and 9.
First of all, the term “spatial visual detail” and its component parts of “spatial” and “visual detail” are not defined in the specification. Furthermore, “spatial visual detail” is not a standard term of art having a readily apparent or commonly understood plain meaning. Such a lack of clear definitions contributes to the 112(a) rejection.
Instead of defining these terms precisely, the instant specification speaks in vague terms of “removing detailed information” using a preprocessing scheme including mosaic processing, quantization processing, blurring processing, edge processing, and noise processing as per [0095] and [0110] of the instant specification (as published).
At least the mosaic and edge processing are wholly undefined and unexplained. A “mosaic” is commonly understood as a composite image formed by piecing together smaller images and appears to have no relation to reducing spatial visual detail. Edge processing is a broad category of processes including processes such as edge sharpening which does not appear to reduce “spatial visual detail”. Moreover, there are no specific examples or references to conventional edge processing or mosaic algorithms. Nor is there any explained relationship between any of these pre-processing algorithms and reducing “spatial visual detail”—as such, it is not clear what is meant by reducing spatial visual detail or how any of the disclosed algorithms achieve this undefined goal.
In summary, the complete lack of any detail of how such pre-processing operations are performed and how they achieve a reduction in the spatial visual details results in one of ordinary skill in the art not being apprised as to how to make and use the invention including "generat[e]ing a context image by applying an image degradation operation that reduces spatial visual detail of an input image to remove detailed information from the input image” as recited in independent claims 1, 5, and 9.
Furthermore, there is a complete the lack of any details regarding the specific preprocessing operations now recited in new claims 10, 16 and 17. Thus, one of ordinary skill in the art not being apprised as to how to make and use the invention as recited in those claims.
In regards to claim 11, the wherein the artificial neural network comprises a U-Net, such a U-Net implementation is not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor(s), at the time the application was filed, had possession of the claimed invention. No details are given regarding the U-Net and, instead, only passing reference to a U-net is made (e.g., specification, [0063]). There is no discussion of the size of the U-net, no discussion of which layers (or how many) are used, no discussion of any parameters or hyper parameters beyond passing references to an unspecified loss function in [0065]. There is no discussion of how much training data is needed, and thus there is not sufficient guidance on how that training data would be acquired.
There is also no discussion of how well these techniques work (such as any statistics on accuracy or other performance characteristics), and details on the effectiveness of a neural network model is expected from a reduction to practice.
New claim 14 is also not adequately described in the instant specification. Indeed, the specification, e.g. [0076] merely parrots the claim language of the image degradation operation includes generating at least one of an attention map, a depth map, or a segmentation map from the input image. Moreover, it is unclear what “additional artificial neural network” is used to perform these operations, how it is trained or how it is constructed.
Therefore, claims 1-17 (all claims) are rejected for lack of written description.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The phrase “"generat[e]ing a context image by applying an image degradation operation that reduces spatial visual detail of an input image to remove detailed information from the input image” as recited in independent claims 1, 5 and 9 is unclear and indefinite.
The term “spatial visual detail” and its component parts of “spatial” and “visual detail” are not defined in the specification and these are not standard terms of art having a readily apparent or commonly understood plain meaning. Instead, the instant specification speaks in term of “removing detailed information” using a preprocessing scheme including mosaic processing, quantization processing, blurring processing, edge processing, and noise processing as per [0095] and [0110] of the instant specification (as published) none of which has been described in adequate detail as noted above in the related 112(a) rejection. Such lack of disclosure also contributes the lack of clarity regarding the above-identified terms.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 5-7, 9, 10, 13, and 15-17 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ye {Ye, Fei, et al. "Attribute restoration framework for anomaly detection." IEEE Transactions on Multimedia 24 (2020): 116-127}.
Claim 1
In regards to claim 1, Ye discloses an anomaly detection apparatus comprising:
one or more processors; and memory storing instructions thereon, the instructions when executed by the one or more processors cause the one or more processors to {See section V Extended Experiments that implements and tests the disclosed methods using a processor and memory the numerical results of which are illustrated in Tables Vi and VII. See also pg. 121 Computational Cost discussing implementation details including GPU (processor) and memory}
generate a context image by applying an image degradation operation that reduces spatial visual detail of an input image to remove detailed information from the input image
{See IIIA, figs. 1, 2 copied below including Attribute Erasing Module (AEM) that erases/removed detailed information from an input image. See also IVA in which attribute erasing may include graying which erases color and averages pixel values to remove detailed information. It is also noted that this step is disclosed by the instant specification as being an optional step as per Figs. 3 and 6.};
PNG
media_image1.png
168
524
media_image1.png
Greyscale
approximate the context image to the input image through an artificial neural network so as to convert the context image into a restoration image
{See IIIB, fig. 2, Attribute Restoration Network (ARNet). See also abstract and Introduction}; and
determine whether the input image is abnormal based on a loss between the input image and the restoration image converted from
the context image
{See IIIC, figs. 1, 2 Anomaly Measurement based on loss/restoration error. See also Table 1}.
Claims 2 and 3
Ye discloses (claim 2) wherein the instruction further cause the one or more processors to train the artificial neural network by restoring detailed information removed from a normal training image and (claim 3) wherein the training unit is configured to update a parameter of the artificial neural network so that a loss between the training image and a restoration image obtained by approximating a context image from which the detailed information is removed from the training image to the training image falls within a configuration value
{initially, it is noted that these claims refer to the normal/conventional training of a restoration network such that the loss converges
See Abstract, Introduction, Section IIIA, IIIB including training dataset contains normal data and anomalous data is not used for training and training with normal training images to update parameters of the ARNet so that the loss function l2 converges. See also Section IIID and IV in which the training process is performed until convergence (loss within a configuration value)}.
Claims 5-7 and 9
The rejection of apparatus claims 1-3 and 1 above applies mutatis mutandis to the corresponding limitations of method claims 5-7 and computer readable medium claim 9 while noting that the rejection above cites to both device and method disclosures. For the computer readable storage medium storing program limitations of claim 9 see section IV including implementations using a GPU, memory, etc. for the ARNet clearly indicating a computer-implemented embodiment that may use a computer readable medium storing software for execution.
Claims 10, 16, and 17
In regards to claims 10, 16, and 17, Ye discloses wherein the image degradation operation comprises at least one of mosaic processing, quantization processing, blurring processing, edge processing, or noise processing {see the 112a and 112b rejection above. Ye’s Graying operation is considered edge processing. See mapping of claim 1 and response to arguments in which the Graying operation may eliminate or remove edges}.
Claim 13
In regards to claim 13, Ye discloses wherein the image degradation operation is performed without a separate artificial neural network {see above cites, in which the Graying operation is performed without a separate artificial network}.
Claim 15
In regards to claim 15, Ye discloses wherein the loss between the input image and the restoration image is smaller when the input image is a normal image than when the input image is an abnormal image {see IIIC Anomaly Measurement and IIIA Attribute Erasing Module employing a restoration loss wherein the loss between the input image and the restoration image is smaller when the input image is a normal image than when the input image is an abnormal image as claimed}.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Ye and Matsumoto (WO-2021161823-A1). A marked-up machine translation of Matsumoto has been provided with this office action, all cross-references are with respect to this translation and the mark-ups are hereby incorporated by reference to further demonstrate claim mapping.
Claims 4 and 8
In regards to claims 4 and 8, Ye discloses wherein the instructions cause the one or more processing to determine that the input image is abnormal responsive to the loss between the input image and the restoration image converted from the context image being
Matsumoto is a highly relevant and analogous reference that teaches all of the elements of claim 1 except for the disclosed-as-optional preprocessing step. See abstract, pgs. 2-5 including applying a restoration model 20 trained with normal data to determine an anomaly in the input image data on the basis of the restoration error.
Matsumoto also teaches wherein the determination unit is configured to determine that the input image is abnormal when the loss between the input image and the restoration image converted from the context image is equal to or greater than a threshold value {see pgs. 2-6, 10, fig. 5, Step S13, comparison unit 32 that determines presence/absence of an anomaly by comparing the restoration error L with a threshold value T.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Ye which already discloses a determination unit that is configured to determine that the input image is abnormal based on the loss between the input image and the restoration image converted from the context image such that this determination is responsive to loss between the input image and the restoration image being equal to or greater than a threshold value as taught by Matsumoto because Matsumoto motivates using a threshold on pg. 10, fig. 11 which is to detect abnormalities without omission using a properly chosen threshold, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claims 11, 12, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Ye and Hida (US 20210374928 A1).
Claim 11
In regards to claim 1, Ye discloses wherein the artificial neural network comprises
Hida is an analogous reference from the same field defect (abnormality) detection. See title, abstract, and cites below disclosing an encoder-decoder restoration artificial neural network.
Hida also teaches wherein the artificial neural network comprises a U-Net {See [0065]-[0066], Fig. 2 and 8 showing encoder/decoder architecture similar to Ye that may be implemented as a U-net}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Ye which already discloses a determination unit that is configured to determine that the input image is abnormal based on the loss between the input image and the restoration image converted from the context image including using an artificial neural network with an encoder/decoder architecture such that this encoder-decoder architecture comprises an U-net as taught by Hida because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 12
In regards to claim 12, Ye discloses wherein the instructions further cause the one or more processors to
Hida S105, [0094]-[0097] identifies defects/abnormalities and raises an alert in the form of highlighting identifies defects in the live image which alerts the operator of the production line as per [0122].
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Ye which already discloses a determination unit that is configured to determine that the input image is abnormal based on the loss between the input image and the restoration image converted from the context image including
determining an anomaly score such that the anomaly score is used to generate an alert when the input image is determined as abnormal taught by Hida because doing so alerts the operator of the production line as motivated by Hida, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 14
In regards to claim 14, Ye is not relied upon to disclose but Hida teaches wherein the image degradation operation includes generating at least one of an attention map, a depth map, or a segmentation map from the input image using an additional artificial neural network {see [0092]-[0097] including generating masks (segmentation maps) for highlighting defect locations using a neural network}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Ye which already discloses a determination unit that is configured to determine that the input image is abnormal based on the loss between the input image and the restoration image converted from the context image including using an artificial neural network with an encoder/decoder architecture such that wherein the image degradation operation includes generating at least one of an attention map, a depth map, or a segmentation map from the input image using an additional artificial neural network as taught by Hida because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
L. Zhu, Q. Zhang and W. Wang, "Residual Attention Dual Autoencoder for Anomaly Detection and Localization in Cigarette Packaging," 2020 Chinese Automation Congress (CAC), Shanghai, China, 2020, pp. 475-480, doi: 10.1109/CAC51589.2020.9327200 discloses anomaly detection and localization using a residual attention autoencoder, attention map generator and reconstruction neural network. See fig. 3 copied below and related disclosure
PNG
media_image2.png
286
488
media_image2.png
Greyscale
Yang, Jie, et al. "Visual anomaly detection for images: A survey." arXiv preprint arXiv:2109.13157 (2021) surveys a variety of anomaly detection prior art that employs image reconstruction to detect defects based on the reconstruction error. See section IIIC.
In regards to claims 10, 16, and 17 see also Castro {A. P. A. de Castro and J. D. S. da Silva, "Restoring images with a multiscale neural network based technique," 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), Hong Kong, China, 2008, pp. 3197-3203, doi: 10.1109/IJCNN.2008.4634251} teaching a neural net degraded images for training image restoration that uses a Gaussian low pass filter that blurs the edge (blurring processing) and/or performs edge processing with the low pass filter and also adds noise (noise processing) as another form of degradation operation.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael R Cammarata whose telephone number is (571)272-0113. The examiner can normally be reached M-Th 7am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ROBERT CAMMARATA/Primary Examiner, Art Unit 2667
Prosecution Insights