Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This communication is a non-Final office action in merits. Claims 1-6, as originally filed, are presently pending and have been elected and considered below.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. CN202211687353.4, filed on 12/27/2022.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on and 10/29/20245 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Drawings
The drawings are objected to under 37 CFR 1.83(a) because they fail to show details in functional blocks and naming/ illustrations in Figs 1-3, which assist or provide clear explanation of claimed invention as described in the specification. Any structural detail that is essential for a proper understanding of the disclosed invention should be shown in the drawing. MPEP § 608.02(d). Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or
nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 5 are rejected under 35 U.S.C. 103 as being unpatentable over CN 114677346 A, Liu et al. an IDS submission (hereinafter Liu) in view of KR 101449273, Hwang et al. (hereinafter Hwang).
As to claim 1, Liu discloses a method for detecting an abnormal defect on a steel surface based on semi-supervised contrastive learning (Abstract, semi-supervised image surface detection), comprising:
step (1), obtaining steel surface data in a real industrial scene, selecting images of machine tool area taken from different perspectives, taking data with a surface defect as an abnormal sample (page 2, par 10; page 4, pars 1-2, 5; page 5, par 3, page 7, par 6 from the bottom, selecting normal images as input, identifying or positioning abnormal region of an image), and taking data without the surface defect as a normal sample (page 4, pars 2-3, obtaining normal samples and comparing difference with abnormal samples);
step (2), generating an abnormal Perlin noise simulating the real industrial scene by simulating abnormality for the normal sample (claim 1, generating a two-dimensional Berlin noise for simulating abnormal samples), and combining with the normal sample βequally scaled to obtain a simulated abnormal sample image Ic:
Ic= Ir
O
Pt + β (Ir O Pt) + (1- β) (APt)
where Ir represents a compressed image equally scaled for a normal sample image I, A represents a texture pattern of a randomly simulated abnormality, and Pt represents a binary image of a random Berlin noise image P, β is a proportion coefficient, representing a fusion ratio between the normal sample image and a simulated abnormal defect, O represents a dot multiplication operation pixel by pixel between image matrices, and Pt represents a reverse value image of an abnormal binary image Pt (claim 1: (1)-(3), note the equation in (3) is equivalent to equation above);
step (3), constructing an abnormality reconstruction network based on an encoder-decoder structure for learning how to restore and reconstruct the abnormal sample into the normal sample ( page 4, pars 2-3; page 6, reconstruction network with encoder-decoder structure for abnormal samples reconstructuring), using the simulated abnormal sample of the step (2) as a network input ( simulated input ), obtaining a reconstructed recovery sample, calculating a loss relative to the normal sample (loss or objective function being calculated), and training the abnormality reconstruction network (abstract; pages 3-4; claim 3; page 8, last par); and
step (4), constructing a contrastive discriminative network based on semantic segmentation, inputting the abnormal sample of the step (1) into the trained abnormality reconstruction network to obtain a reconstructed recovery sample (page 2-3; page 4, par 2), combining a channel with a input abnormal sample as an input of the contrastive discriminative network, obtaining a difference between the reconstructed recovery sample and the input abnormal sample by contrastive learning (page 2, par 2; page 5, pars 4-6), and outputting a steel surface defect detection result (Fig 1; page 3, pars 5-6, obtaining abnormal region of the input image as output of the decoder).
Liu does not expressly disclose the metal with surface defect being steel and the surface defect comprises edge cracks and creases. Hwang, in the same or similar field of endeavor, further teaches the metal with surface defect being steel and the surface defect comprises edge cracks and creases (abstract; claim 1; page 3, par 3, defects of edge crease and crack of the steel surface).
Therefore, consider Liu and Hwang’s teachings as a whole, it would have been obvious to one of skill in the art before the filing date of invention to incorporate Hwang’s teachings in Liu’s method to detect steel surface detect including scenario of edge detects.
As to claim 5, Liu as modified discloses the method according to claim 1, wherein the texture pattern of the randomly simulated abnormality enhances a diversity of a simulated defect by randomized data augmentation (Liu: page 4, par 9; page 6, last par), and wherein three of the following are randomly selected and combined for usage: rotation (Liu: claim 3, random angular rotation), affine transformation, image brightness (Liu: claim 3, random adjustment of brightness), sharpness, equalization value, contrast and saturation (Liu: claim 3, random adjustment of saturation and hue).
Claims 2-4, 6 are rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Hwang and further in view of US 2020/0012657 A1, Walters et al. (hereinafter Walters).
As to claim 2, Liu as modified discloses the method according to claim 1, but does not expressly teach wherein a mask dilated convolution module is embedded in an encoder of the abnormality reconstruction network to expand a receptive field of the abnormality reconstruction network, and a transformer is used to replace a full connection integration operation of the mask dilated convolution module to achieve feature aggregation. Walters, in the same or similar field of endeavor, further teaches an encoder may be implemented with a mask dilated convolution (par 0041) or a transformer (pars 0040-0041, 0051-0051). Therefore, consider Liu as modified and Walters’s teachings as a whole, it would have been obvious to one of skill in the art before the filing date of invention to incorporate Walters’s teachings in Liu as modified’s method to select either a dilated convolution or a transformer in the encoder design depending upon the performance tradeoff.
As to claim 3, Liu as modified discloses the method according to claim 2, wherein for the contrastive discriminative network, a self-attention mechanism module is used to obtain channel and spatial self-attention input by the contrastive discriminative network, so as to optimize the steel surface defect detection result (Liu: claims 1-2, obtaining channel and extracting spatial attention map; Walters: pars 0041, 0066, 075, 0083).
As to claim 4, Liu as modified discloses the method according to claim 3, wherein for an input feature map of the contrastive discriminative network, attention is extracted in a channel-before-spatial manner, and the input feature map is multiplied with an original feature map to feed back after each attention extraction (Liu: claims 1-2).
As to claim 6, Liu as modified discloses the method according to claim 1, wherein the compressed image Ir and the abnormal binary image P overlap each other to obtain a simulated abnormal portion comprising original image information (Liu: abstract; claims 1-3), and the compressed image Ir and the reverse value image of the abnormal binary image Pr overlap each other to obtain an image region portion without the simulated abnormality (Liu: claims 1-3; Walters: pars 0020, 0048, 0052).
Examiner’s Note
Examiner has cited particular column, line number, paragraphs and/or figure(s) in the reference(s) as applied to the claims for the convenience of the Applicant. Although the specified citations are representative of the teachings of the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the reference(s) in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to QUN SHEN whose telephone number is (571)270-7927. The examiner can normally be reached on Mon-Fri 8:30-5:50 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QUN SHEN/
Primary Examiner, Art Unit 2662