Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1, 11 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over NPL1 (Attention-Mechanism-Containing Neural Networks for High-Resolution Remote Sensing Image Classification, Rudong Xu, Yiting Tao et al., MDPI, 2018, Pages 1-29) hereafter NPL1 in view of NPL2 (Transferable Attention for Domain Adaptation, Ximei Wang et al., AAAI, 2019, Pages 5345-5352) hereafter NPL2.
1. Regarding claim 1, NPL1 discloses a processor-implemented method (figs 1-2, pages 4-8 shows and discloses a method and page 14 discloses “all experiments were conducted on a computer” i.e a processor implemented method), comprising:
generating an attention mask based on a feature of an image (figs 1-2, pages 5-8 shows and discloses control attention mechanism performed on the image as seen in fig 1 and fig 2 shows the features, pages 7-8 shows and discloses “our control gate attention (page 7 section 2.2) performs pixel-to-pixel modeling on masks that are the same sizes as the original feature maps. The positions in the masks represent the weights or propriety values of the corresponding pixels on the original maps. The priorities of the pixels on the original feature maps can differ, indicating that each pixel may play a different role according to different classification objectives. These kinds of masks are more suitable for pixel-wise classification than global pooling. The control gate is also a feedforward fully convolutional neural network that maintains the size of the acquired mask. The generated mask indicates the calibrated importance of every position on the feature maps” meeting the limitations of generating an attention mask based on a feature of an image, examiner notes that the specifics of “generating” are not required by the current claim);
updating an image classification model by (figs 1-2 shows updating an image classification model in fig 1“
PNG
media_image1.png
851
772
media_image1.png
Greyscale
on the image based on the control gate attention (i.e attention mask) and feedback attention mechanism (i.e using the update “dotted arrow”) resulting in the final classification result); and
determining a class of the image using the updated image classification model (figs 1-2 shows the classification result based on the updated image classification model and page 10 discloses “The final classification result fused with the loss functions is used to update the network iteratively (i.e using the updated classification model) meeting the claim limitation). NPL1 discloses updating based on the feedback mechanism as seen in figs 1-2 and discloses “The generated mask indicates the calibrated importance of every position (i.e domain) on the feature maps. NPL1 however is silent and fails to disclose performing domain adaptation.
NPL2 discloses performing domain adaptation (fig 1, page 5348-5349 discloses performing domain adaptation in the image classification model). Before the effective filing date of the invention was made, NPL1 and NPL2 are combinable because they are from the same field of endeavor and are analogous art of image processing. The suggestion/motivation would be producing an enhanced image (page 5348 “By applying transferable local attention and transferable global attention modules, negative transfer for each region is alleviated and positive transfer for each image is enhanced.” and an accurate model (page 5349 results section)). Therefore, it would be obvious and within one of ordinary skill in the art to have recognized the advantages of NPL2 in the method of NPL1 to obtain the invention as specified in claim 1.
2. Claim 11 is a corresponding apparatus claim of claim 1. See the corresponding explanation of claim 1.
3. Regarding claim 12, NPL1 and NPL2 disclose the apparatus of claim 11. NPL2 discloses further wherein the Al model is trained based on images of a source domain, and the domain of the image is different from the source domain (fig 1, pages 5346-5347 discloses “In this paper, we focus on the unsupervised domain adaptation problem, which constitutes a labeled source domain Ds = {(xs i,ys i)}ns {xt j}nt i=1 and an unlabeled target domain Dt = j=1, where xi is an example and yi is the associated label. The goal of this paper is to build a deep network, which can be trained on the labeled source data and generalize well to the unlabeled target data.” meeting the above claim limitations).
Claims 2-3 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over NPL1, NPL2 and in further view of Badowski et al. (US20250046063) hereafter Badowski.
4. Regarding claim 2, NPL1 and NPL2 disclose the method of claim 1. NPL1 shows generating the attention mask. NPL1 and NPL2 however fail to disclose wherein the generating of the attention mask comprises: generating a spatial feature by embedding the image into a latent space; and generating the attention mask based on the spatial feature.
Badowski disclose generating a spatial feature by embedding the image into a latent space; and generating the attention mask based on the spatial feature (para 0122, FIG. 19A shows a schematic diagram of an encoder-decoder (or ‘autoencoder’) network that is configured to receive an input image of size (i×j) spatial elements×k channels, to encode the image into a feature map in a latent embedding space of size q×rx s, and to decode the feature map to produce a reconstructed image of size (i×j) spatial elements×k channels.) meeting the above claim limitations). Before the effective filing date of the invention was made, NPL1, NPL2 and Badowski are combinable because they are from the same field of endeavor and are analogous art of image processing. The suggestion/motivation would be a highly accurate method/system (para 0086). Therefore, it would be obvious and within one of ordinary skill in the art to have recognized the advantages of Badowski in the method of NPl1 and NPL2 to obtain the invention as specified in claim 2.
5. Regarding claim 3, NPL1, NPL2 and Budowski disclose the method of claim 2. NPL1 discloses and shows wherein the updating of the image classification model comprises: calculating a loss by masking, using the attention mask, a difference between the image and a reconstructed image generated updating the image classification model based on the loss
PNG
media_image1.png
851
772
media_image1.png
Greyscale
Figs 1-2 and page 10 shows and discloses calculating the loss based on the difference between the output results (i.e reconstructed images) generated at the output (i.e at the decoding) and the ground truth images (i.e the image) and discloses “The final classification result fused with the loss functions from previous modules is utilized to update the network iteratively” meeting the above claim limitations, NPL1 also discloses on page 20 encoder-decoder blocks meeting the claim limitations). As seen in NPL1, the loss and the features (page 18 discloses “The feedback attention mechanism returns higher-level features to the lower layers to re-weight the focus and re-update the feature (i.e decoding the spatial features are fed back in the feedback) learning, causing the network to re-learn the weights in an objected-oriented manner.”). NPL1 fails to recite in exact claim language based on a decoding of the spatial feature. Badowski discloses based on a decoding of the spatial feature (para 0122 discloses decode the feature map to produce the reconstructed image of size (ixj) spatial elements k channels meeting the limitations of a decoding of the spatial feature). NPL1, NPl2 and Badowski in combination would therefore meet the limitations as claimed in claim 3.
6. Claim 13 is a corresponding apparatus claim of claim 2. See the explanation of claim 2.
7. Claim 14 is a corresponding apparatus claim of claim 3. See the explanation of claim 3.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over NPL1, NPL2 and in further view of NPL3 (A New Backbone Network for Instance Segmentation: Application on a Semiconductor Process Inspection, JUNGHEE HAN et al., IEEE, 2020, Pages 218110-218121) hereafter NPL3.
8. Regarding claim 18, NPL1 discloses a defect inspection system (figs 1-2, pages 4-8 shows and discloses a method and page 14 discloses “all experiments were conducted on a computer” i.e a processor implemented system comprising one or more processors and memory storing instructions for execution of the instruction by the one or more processor) for a semiconductor manufacturing process, the system comprising:
one or more processors (figs 1-2, pages 4-8 shows and discloses a method and page 14 discloses “all experiments were conducted on a computer” i.e a processor implemented system comprising one or more processors and memory storing instructions for execution of the instruction by the one or more processor); and one or more memories storing an artificial intelligence (AI) model, trained based on (figs 1-2, pages 4-8 shows and discloses a method and page 14 discloses “all experiments were conducted on a computer” i.e a processor implemented system comprising one or more processors and one or more memories storing an artificial intelligence (AI) model, trained based on , and instructions that when executed by the one or more processors configures the one or more processors (figs 1-2, pages 4-8 shows and discloses a method and page 14 discloses “all experiments were conducted on a computer” i.e a processor implemented system comprising one or more processors and memory storing instructions for execution of the instruction by the one or more processor) to: generate an attention mask based on the input image acquired from an (figs 1-2,5, pages 5-8 shows and discloses control attention mechanism performed on the images as seen in fig 1 and fig 2 shows the features, pages 7-8 shows and discloses “our control gate attention (page 7 section 2.2) performs pixel-to-pixel modeling on masks that are the same sizes as the original feature maps. The positions in the masks represent the weights or propriety values of the corresponding pixels on the original maps. The priorities of the pixels on the original feature maps can differ, indicating that each pixel may play a different role according to different classification objectives. These kinds of masks are more suitable for pixel-wise classification than global pooling. The control gate is also a feedforward fully convolutional neural network that maintains the size of the acquired mask. The generated mask indicates the calibrated importance of every position on the feature maps” meeting the limitations of generating an attention mask based on a feature of an image, examiner notes that the specifics of “generating” are not required by the current claim);
update the Al model through domain adaptation using the attention mask updating an image classification model by (figs 1-2 shows updating an image classification model (i. AI model) in fig 1“
PNG
media_image1.png
851
772
media_image1.png
Greyscale
on the image based on the control gate attention (i.e attention mask) and feedback attention mechanism (i.e using the update “dotted arrow”) resulting in the final classification result); and
; and determine the class of the input image using the updated Al model determining a class of the image using the updated image classification model (figs 1-2 shows the classification result based on the updated image classification model and page 10 discloses “The final classification result fused with the loss functions is used to update the network iteratively (i.e using the updated classification model) meeting the claim limitation). NPL1 discloses updating based on the feedback mechanism as seen in figs 1-2 and discloses “The generated mask indicates the calibrated importance of every position (i.e domain) on the feature maps. NPL1 however is silent and fails to disclose performing domain adaptation, source images of a source domain and inspection equipment in an in-fabrication.
NPL2 discloses performing domain adaptation (fig 1, page 5348-5349 discloses performing domain adaptation in the image classification model, and also shows source image(s) in fig 1 meeting the limitations of source images of a source domain). Before the effective filing date of the invention was made, NPL1 and NPL2 are combinable because they are from the same field of endeavor and are analogous art of image processing. The suggestion/motivation would be producing an enhanced image (page 5348 “By applying transferable local attention and transferable global attention modules, negative transfer for each region is alleviated and positive transfer for each image is enhanced.” and an accurate model (page 5349 results section)). NPL1 and NPL2, shows and discloses the images of the environment respectively. NPL1 and NPL2, however, are silent and fail to disclose inspection equipment in an in-fabrication.
NPL3 discloses inspection equipment in an in-fabrication (figs 1 and 4-7 shows and discloses inspection equipment in an in-fabrication). Before the effective filing date of the invention was made, NPL1, NPL2 and NPL3 are combinable because they are from the same field of endeavor and are analogous art of image processing. The suggestion/motivation would be a system/architecture which extracts an enhanced feature map of the semiconductor (page 218113 section III). Therefore, it would be obvious and within one of ordinary skill in the art to have recognized the advantages of NPL3 in the system of NPL1 and NPL2 to obtain the invention as specified in claim 18.
Claims 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over NPL1, NPL2, NPL3 and in further view of Badowski.
9. Regarding claim 19, NPL1, NPL2 and NPL3, disclose the defect inspection system of claim 18. NPL1 discloses wherein, for the updating of the Al model, the instructions are configured to cause the one or more processors to: reconstruct the input image through a decoding of features
Badowski disclose reconstruct the input image through decoding of a spatial feature (para 0122, FIG. 19A shows a schematic diagram of an encoder-decoder (or ‘autoencoder’) network that is configured to receive an input image of size (i×j) spatial elements×k channels, to encode the image into a feature map in a latent embedding space of size q×rx s, and to decode the feature map to produce a reconstructed image of size (i×j) spatial elements×k channels.) meeting the above claim limitations). Before the effective filing date of the invention was made, NPL1, NPL2, NPL3 and Badowski are combinable because they are from the same field of endeavor and are analogous art of image processing. The suggestion/motivation would be a highly accurate method/system (para 0086). Therefore, it would be obvious and within one of ordinary skill in the art to have recognized the advantages of Badowski in the method of NPL1, NPL2 and NPL3 to obtain the invention as specified in claim 19.
10. Regarding claim 20, NPL1, NPL2, NPL3 and Badowski disclose the defect inspection system of claim 19. NPL1 discloses wherein, for the updating of the Al model, the instructions are configured to cause the one or more processors to: reconstruct the input image through a decoding of features (see fig 1 and fig 2); and update the Al model through a calculation of a loss using the input image, the reconstructed image, and the attention mask (figs 1-2, 4 and pages 8-10 shows and discloses update the Al model through a calculation of a loss using through a masking, using the attention mask, of a difference between the input image and the reconstructed image meeting the limitations of wherein, for the calculating of the loss, the instructions are configured to cause the one or more processors to calculate the loss through a masking, using the attention mask, of a difference between the input image and the reconstructed image).
Examiner's Note: Examiner has cited figures, and paragraphs in the references as applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested for the applicant, in preparing the responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner. Examiner has also cited references in PTO892 but not relied on, which are relevant and pertinent to the applicant’s disclosure, and may also be reading (anticipatory/obvious) on the claims and claimed limitations. Applicant is advised to consider the references in preparing the response/amendments in-order to expedite the prosecution.
Allowable Subject Matter
Claims 4-10 and 15-17 are objected to as being dependent upon a rejected base claim but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAYESH PATEL whose telephone number is (571)270-1227. The examiner can normally be reached IFW Mon-FRI.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at 571-270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAYESH A PATEL/Primary Examiner, Art Unit 2677
/JAYESH PATEL/
Primary Examiner
Art Unit 2677
Prosecution Insights