Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This communication is filed in response to the action filed on 03/09/2026.
Claims 1-20 are currently pending.
Response to Arguments
Applicant’s arguments filed on 03/09/2026 on pages 6-10, under REMARKS with respect to 35
U.S.C. 102 and 35 U.S.C. 103 have been fully considered but they are not persuasive. Regarding claim 1
applicants on page 6 states that:
PNG
media_image1.png
398
770
media_image1.png
Greyscale
The examiner respectfully disagrees. The examiner would like to point to the primary reference and particularly the paragraphs [0080], [0086-0087] and figure 4. The applicant claims that the is no defect free inspection image, however it is seen that in figure 4 the design image 400 as stated in paragraph [0086] is “the simulated image includes an augmented defect image of a synthetic defect… a
"synthetic" defect is created in a design for a specimen through the intentional manipulation of an otherwise defect free portion of the design” this clearly shows that design image with added defect 400 began without a defect, had defect attribute 402 added to it and then via the GNN generated a synthetic image having the synthetic defect applied as generated patch image 406. The primary reference further stating at paragraph [0080] that ”generating a simulated image for a specimen by inputting a portion of design data for the specimen into the GAN” wherein the input portion of design data used for generating a synthetic/simulated image is the defect attribute 402. Finally, the primary reference at paragraph [0087] states that ” input design image 400 (conditional image) with added defect 402 into trained generator network 404. The added or synthetic defect may be created in the design data portion shown in design image 400 as described further herein, which may be performed by the one or more computer subsystems or another system or method. The trained generator network may output generated patch image 406 showing defect 408. In this manner, the synthetic defect is added defect 402 in the portion of design data shown in design image 400, which is input to the GAN by the one or more computer subsystems to thereby generate simulated image 406”, which clearly show that a defect free input image is simulated into a defect comprising synthetic image generated from a defect free image using a GNN. Please see full rejection to the claims below.
PNG
media_image2.png
975
944
media_image2.png
Greyscale
The examiner respectfully disagrees. The examiner would like to point out the instant application specification at paragraphs [0029] and [0054], refer to the inspection image as an image with languages including the phrases “such as” and “can be” and this can be interpreted to mean that the inspection image can be images such as SEM images but can also be interpreted as substantially similar to the design data of BRAUER and the inspection image can be an SEM image or it can be design data image of Brauer since the language isn’t explicitly limiting to only SEM images and design data would read on the claimed “inspection image”. Please see full rejection to the claim below.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-6, and 11-20 are rejected under 35 § U.S.C. 102(a)(1) as being anticipated by US 2021/0272273 A1 to BRAUER (hereinafter “BRAUER”).
As per claim 1, BRAUER discloses an apparatus for generating a synthetic defect image (a system and method of image processing using a computing system comprising a generative adversarial neural network GAN to generate synthetic images comprising specified defects of a specimen wherein the specimen is a defect free semiconductor wafer image; abstract; figs 1-4; paragraphs [0026], [0030], [0080], [0086-0087]), comprising: a memory storing a set of instructions (the computing system which carries out the method comprises a memory storing instructions and programs related to the method; abstract; figs 1-4; paragraph [0046]); and at least one processor configured to execute the set of instructions to cause the apparatus to perform (the system further comprises a computer processing component to execute the stored instructions and programs to perform said method; abstract; figs 1-4; paragraph [0046]): acquiring a machine learning-based generator model (the system includes/acquires a machine learning model which is adapted to be trained using the synthetically generated images from the GAN; abstract; figs 1-4; paragraphs [0080], [0086-0087]; claims 4-9); providing a defect-free inspection image and a defect attribute combination as inputs to the generator model (via input the system is provided a defect free specimen image of the semiconductor wafer and the die image which is an image of a wafer having no defects and acts as the template/reference to which the GAN applies synthetic simulated defects; abstract; figs 1-4; paragraphs [0080], [0086], [0092]; claims 3-5); and generating by the generator model, based on the defect-free inspection image, a predicted synthetic defect image with a predicted defect that accords with the defect attribute combination (using the specimen image and its die area the GAN is adapted to generate synthetic defects of various types and produce predicted synthetic defect images of the specimen in order to use the defect images to train the machine learning classification model to identify the predicted/applied defects; abstract; figs 1-4; paragraphs [0080], [0086-0087]; claims 5-9).
As per claim 2, BRAUER discloses the apparatus of claim 1, wherein the defect attribute combination comprises at least one of a defect type, a defect size, a defect location, or defect strength (the defect of interest may be any known real or actual defect seen or observed in semiconductor wafers and may comprise a variety of defects and may be narrowed to a single defect type; abstract; figs 1-4; paragraphs [0024], [0080], [0086-0087], [0090-0091]).
As per claim 3, BRAUER discloses the apparatus of claim 1, wherein the defect attribute combination consists of only a single defect attribute (the defect of interest would be set to a single defect; paragraphs [0024], [0101]).
As per claim 4, BRAUER discloses the apparatus of claim 1, wherein the at least one processor is configured to execute the set of instructions to cause the apparatus to further perform: encoding the defect attribute combination into a condition vector before providing the defect attribute combination to the generator model (the encoder generates feature layer 512 by encoding the input 504 input into block 506 and 508 and includes defect of interest data and is encoded as a latent space vector 301 which is all input into GAN 302; figs 3 and 5; paragraphs [0024], [0071-0072], [0074-0077]).
As per claim 5, BRAUER discloses the apparatus of claim 1, wherein the generator model is a conditional generative adversarial network model (the generative model utilizes a conditional GAN “cGAN”; figs 1-4; paragraphs [0060], [0066], [0080], [0086-0087]).
As per claim 6, BRAUER discloses the apparatus of claim 1, wherein the defect-free inspection image is a scanning electron microscope (SEM) image of a wafer (images are captured using a scanning electron microscope, and are of a specimen which is a semiconductor wafer; fig 1; paragraphs [0011-0012], [0023-0027], [0090]).
As per claim 11, BRAUER discloses a non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a computing device to cause the computing device to perform a method for generating a synthetic defect image (a system and method of image processing using a computing system comprising a generative adversarial neural network GAN to generate synthetic images comprising specified defects of a specimen wherein the specimen is a defect free semiconductor wafer image, the computing system which carries out the method comprises a memory storing instructions and programs and a computing processor component to execute said instructions/programs related to the method; abstract; figs 1-4; paragraphs [0026], [0030], [0046], [0080], [0086-0087]), the method comprising: acquiring a machine learning-based generator model (the system includes/acquires a machine learning model which is adapted to be trained using the synthetically generated images from the GAN; abstract; figs 1-4; paragraphs [0080], [0086-0087]; claims 4-9); providing a defect-free inspection image and a defect attribute combination as inputs to the generator model (via input the system provides a defect free specimen image of the semiconductor wafer and the die image which is an image of a wafer having no defects and acts as the template/reference to which the GAN applies synthetic simulated defects; abstract; figs 1-4; paragraphs [0080], [0086-0087], [0092]; claims 3-5); and generating by the generator model, based on the defect-free inspection image, a predicted synthetic defect image with a predicted defect that accords with the defect attribute combination (using the specimen image and its die area the GAN is adapted to generate synthetic defects of various types and produce predicted synthetic defect images of the specimen in order to use the defect images to train the machine learning classification model to identify the predicted/applied defects; abstract; figs 1-4; paragraphs [0080], [0086-0087]; claims 5-9).
As per claim 12, BRAUER discloses the computer readable medium of claim 11, wherein the defect attribute combination comprises at least one of a defect type, a defect size, a defect location, or defect strength (the defect of interest may be any known real or actual defect seen or observed in semiconductor wafers and may comprise a variety of defects and may be narrowed to a single defect type; abstract; figs 1-4; paragraphs [0024], [0080], [0086], [0090-0091]).
As per claim 13, BRAUER discloses the computer readable medium of claim 11, wherein the defect attribute combination consists of only a single defect attribute (the defect of interest would be set to a single defect; paragraphs [0024], [0101]).
As per claim 14, BRAUER discloses the computer readable medium of claim 11, wherein the set of instructions that is executable by at least one processor of the computing device cause the computing device to further perform: encoding the defect attribute combination into a condition vector before providing the defect attribute combination to the generator model (the encoder generates feature layer 512 by encoding the input 504 input into block 506 and 508 and includes defect of interest data and is encoded as a latent space vector 301 which is all input into GAN 302; figs 3 and 5; paragraphs [0024], [0071-0072], [0074-0077]).
As per claim 15, BRAUER discloses the computer readable medium of claim 11, wherein the generator model is a conditional generative adversarial network model (the generative model utilizes a conditional GAN “cGAN”; figs 1-4; paragraphs [0060], [0066], [0080], [0086]).
As per claim 16, BRAUER discloses a method for generating a synthetic defect image (a system and method of image processing using a computing system comprising a generative adversarial neural network GAN to generate synthetic images comprising specified defects of a specimen wherein the specimen is a defect free semiconductor wafer image; abstract; figs 1-4; paragraphs [0026], [0030], [0080]), comprising: acquiring a machine learning-based generator model (the system includes/acquires a machine learning model which is adapted to be trained using the synthetically generated images from the GAN; abstract; figs 1-4; paragraph [0080]; claims 4-9); providing a defect-free inspection image and a defect attribute combination as inputs to the generator model (via input the system is provided a defect free specimen image of the semiconductor wafer and the die image which is an image of a wafer having no defects and acts as the template/reference to which the GAN applies synthetic simulated defects; abstract; figs 1-4; paragraphs [0080], [0086], [0092]; claims 3-5); and generating by the generator model, based on the defect-free inspection image, a predicted synthetic defect image with a predicted defect that accords with the defect attribute combination (using the specimen image and its die area the GAN is adapted to generate synthetic defects of various types and produce predicted synthetic defect images of the specimen in order to use the defect images to train the machine learning classification model to identify the predicted/applied defects; abstract; figs 1-4; paragraphs [0080], [0086]; claims 5-9).
As per claim 17, BRAUER discloses the method of claim 16, wherein the defect attribute combination comprises at least one of a defect type, a defect size, a defect location, or defect strength (the defect of interest may be any known real or actual defect seen or observed in semiconductor wafers and may comprise a variety of defects and may be narrowed to a single defect type; abstract; figs 1-4; paragraphs [0024], [0080], [0086], [0090-0091]).
As per claim 18, BRAUER discloses the method of claim 16, wherein the defect attribute combination consists of only a single defect attribute (the defect of interest would be set to a single defect; paragraphs [0024], [0101]).
As per claim 19, BRAUER discloses the method of claim 16, further comprising: encoding the defect attribute combination into a condition vector before providing the defect attribute combination to the generator model (the encoder generates feature layer 512 by encoding the input 504 input into block 506 and 508 and includes defect of interest data and is encoded as a latent space vector 301 which is all input into GAN 302; figs 3 and 5; paragraphs [0024], [0071-0072], [0074-0077]).
As per claim 20, BRAUER discloses the method of claim 16, wherein the generator model is a conditional generative adversarial network model (the generative model utilizes a conditional GAN “cGAN”; figs 1-4; paragraphs [0060], [0066], [0080], [0086]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claim 7 is rejected under 35 § U.S.C. 103 as being obvious over US 2021/0272273 A1 to BRAUER (hereinafter “BRAUER”) in view of Conditional Generative Adversarial Nets to MIRZA et al. ( hereinafter “MIRZA”).
As per claim 7, BRAUER discloses the apparatus of claim 1, wherein, in acquiring the machine learning- based generator model (the machine learning model is trained using a GAN/cGAN and is trained based on stored instructions stored to the memory component; figs 1-4; paragraphs [0060], [0066], [0080], [0086]), the at least one processor is configured to execute the set of instructions to cause the apparatus to further perform pretraining the machine learning based-generator model (the machine learning model is trained using the generated synthetic defect images with various defect types as defect of interest; figs 1-4; paragraphs [0060], [0066], [0080], [0086], [0090-0092]), and wherein pretraining the machine learning based-generator model comprises: acquiring a first training defect-free inspection image and a first training defect attribute combination (a first specimen image is captured of a defect free die and semiconductor wafer, having no defects, using a cGAN defects are generated and synthetic images comprising those defects are generated and used to train the model to identify those defects; figs 1-4; paragraphs [0024], [0080], [0086], [0092], [0101]); generating, by the generator model, based on the first training defect-free inspection image, a first predicted synthetic defect image with a first predicted defect that accords with the first training defect attribute combination (using the cGAN the defects of interest are applied to the specimen and are applied according to the decided defects of interest in the die are of interest to be identified by the model so the model may identify real world defects based on training derived from the synthetic wafer defects; paragraphs [0024], [0080], [0086], [0092], [0101]). BRAUER fails to disclose and evaluating, by a machine learning-based discriminator model, whether the first predicted synthetic defect image is classified as a real inspection image under a condition of the first training defect attribute combination.
MIRZA discloses and evaluating, by a machine learning-based discriminator model, whether the first predicted synthetic defect image is classified as a real inspection image under a condition of the first training defect attribute combination (and estimating the probability (evaluating) that a sample image of a wafer came from the real training data rather than the generated synthetic image data based on recognition of the applied type of wafer defect; section 3.1, paragraphs 1-3).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to modify BRAUER to have and evaluating, by a machine learning-based discriminator model, whether the first predicted synthetic defect image is classified as a real inspection image under a condition of the first training defect attribute combination of MIRZA reference. The Suggestion/motivation for doing so would have been to provide a single scalar representing the probability that X came from training data rather than real/collected data as suggested by MIRZA section 3.1, paragraph 3. Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine MIRZA with BRAUER to obtain the invention as specified in claim 7.
Claims 8-10 are rejected under 35 § U.S.C. 103 as being obvious over US 2021/0272273 A1 to BRAUER (hereinafter “BRAUER”) in view of Conditional Generative Adversarial Nets to MIRZA et al. (hereinafter “MIRZA”) in view of US 2021/0343001 A1 to GRAMA et al. (hereinafter “GRAMA”).
As per claim 8, BRAUER in view of MIRZA discloses the apparatus of claim 7. Modified BRAUER fails to disclose wherein, in pretraining the machine learning based- generator model, the at least one processor is configured to execute the set of instructions to cause the apparatus to further perform training the discriminator model, and wherein training the discriminator model comprises: acquiring a first training defect-containing inspection image associated with the first training defect attribute combination; and evaluating, by the discriminator model, whether the first defect-containing inspection image is classified as a real inspection image under a condition of the first training defect attribute combination.
GRAMA discloses wherein, in pretraining the machine learning based- generator model (the machine learning based classification model is trained to identify defects in semiconductor wafer images; fig 1; paragraphs [0072-0076]), the at least one processor is configured to execute the set of instructions to cause the apparatus to further perform training the discriminator model (the computer system comprising one or more computer processors to execute the programs in order to train said model; paragraphs [0058], [0072-0076]), and wherein training the discriminator model comprises: acquiring a first training defect-containing inspection image associated with the first training defect attribute combination (the computing system is adapted to perform the method of acquiring simulated inspection images and include simulated test images with simulated defects and/or other variations such as LER and corresponding simulated reference images which are, defect-free specimen images); paragraphs [0077]); and evaluating, by the discriminator model, whether the first defect-containing inspection image is classified as a real inspection image under a condition of the first training defect attribute combination (and using the trained model to classify the images as a defect classification model to perform defect classification on real inspection images; paragraphs [0072-0077], [0082], [0126], [0129]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to further modify BRAUER to have evaluating, by the discriminator model, whether the first defect-containing inspection image is classified as a real inspection image under a condition of the first training defect attribute combination of GRAMA reference. The Suggestion/motivation for doing so would have been to provide a trained model to detect and classify defects of interest in real non-simulated captured test images as suggested by GRAMA at paragraph [0126]. Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine GRAMA with modified BRAUER to obtain the invention as specified in claim 8.
As per claim 9, BRAUER in view of MIRZA discloses the apparatus of claim 7. Modified BRAUER fails to disclose wherein, in pretraining the machine learning-based generator model, the at least one processor is configured to execute the set of instructions to cause the apparatus to further perform training the machine learning-based generator model with a plurality of training defect-free inspection images and a plurality of training defect attribute combinations associated with plurality of training defect-containing inspection images.
GRAMA discloses wherein, in pretraining the machine learning-based generator model (the machine learning based classification model is trained to identify defects in semiconductor wafer images; fig 1; paragraphs [0072-0076]), the at least one processor is configured to execute the set of instructions to cause the apparatus to further perform training the machine learning-based generator model with a plurality of training defect-free inspection images and a plurality of training defect attribute combinations associated with plurality of training defect-containing inspection images (a defect free image is acquired and synthetic defects of interest are applied to the image to produce synthetic defect training images to train the model to recognize a defect of interest of a plurality of defect types that are trained for the model to recognize wherein the defect types may differ; paragraphs [0072-0077], [0082], [0106], [0126], [0129]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to further modify BRAUER to have training the machine learning-based generator model with a plurality of training defect-free inspection images and a plurality of training defect attribute combinations associated with plurality of training defect-containing inspection images of GRAMA reference. The Suggestion/motivation for doing so would have been to provide a trained model to detect and classify defects of interest in real non-simulated captured test images as suggested by GRAMA at paragraph [0126]. Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine GRAMA with modified BRAUER to obtain the invention as specified in claim 9.
As per claim 10, BRAUER in view of MIRZA discloses the apparatus of claim 9. Modified BRAUER fails to disclose wherein the defect attribute combination is one from the plurality of training defect attribute combinations.
GRAMA discloses wherein the defect attribute combination is one from the plurality of training defect attribute combinations (simulated higher resolution images used in the pre-training phase may be higher resolution images that may include the simulated defects and other variations like LER; abstract; paragraphs [0034-0035], [0043], [0056], [0077], [0104]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to further modify BRAUER to have wherein the defect attribute combination is one from the plurality of training defect attribute combinations of GRAMA reference. The Suggestion/motivation for doing so would have been to provide a variety of defects and defect types to train a model that can identify defects of interest and not identify unnecessary defects as suggested by paragraphs [0033-0034], [0077]. Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine GRAMA with modified BRAUER to obtain the invention as specified in claim 10.
Conclusion
Examiner's Note: Examiner has cited figures, and paragraphs in the references as applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested for the applicant, in preparing the responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner. Examiner has also cited references in PTO892 but not relied on, which are relevant and pertinent to the applicant’s disclosure, and may also be reading (anticipatory/obvious) on the claims and claimed limitations. Applicant is advised to consider the references in preparing the response/amendments in-order to expedite the prosecution.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DEVIN JACOB DHOOGE whose telephone number is (571) 270-0999. The examiner can normally be reached 7:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached on (571) 270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800- 786-9199 (IN USA OR CANADA) or 571-272-1000.
/Devin Dhooge/
USPTO Patent Examiner
Art Unit 2677
/Jonathan S Lee/Primary Examiner, Art Unit 2677
Prosecution Insights