Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 8 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 8 currently recites “…wherein the process comprises determining information for the specimen based on the encoded representation in combination with output generated for the specimen by the tool during the process and without the information for the design or other information generated from the design.”. The claim is interpreted as listing two ways in which information can be determined about the specimen – however it is unclear how information can be determined both based on the encoded representation and without the information for the design (which is used to generate the encoded representation).
Claim Rejections - 35 USC § 102
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 8-9, 12, 14, and 16-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhang and Bhaskar (US 2017/0345140; hereinafter “Zhang”).
Regarding Claim 1, Zhang discloses a system configured for determining information for a specimen, comprising:
a computer system: and one or more components executed by the computer system (see Fig. 1), wherein the one or more components comprise a deep learning model configured for transforming information for a design for a specimen input to the deep learning model by the computer system into an encoded representation of the design comprising encoded design attributes as a function of position in the design (Fig. 2, [0052-0053], [0060], [0070], [0077-0080], Zhang discloses neural deep learning model which takes input of an image of a specimen, generates images patches based on the image input, and inputs the set of image patches into an image encoder. The Examiner notes that the image encoder “encodes” information from the input image into a more compact representation (i.e., an encoded representation). Additionally, “design attributes” is broadly interpreted as any form of data/information related to the design of the input image, which the Examiner notes is obtained through the convolutional layers of the encoder. Furthermore, the encoding performed by Zhang can be directly associated with an input image patch, and hence is considered to be a “function of position” (i.e., an encoded representation output from the encoder can be directly associated with an input image patch).); and
wherein the computer system is configured for storing the encoded representation for use1 in a process performed on the specimen by a tool ([0080-0081], Zhang discloses utilizing the encoded representation of the input image, and using that as input into a decoder in order to generate a simulated image (i.e., the “process” is the process of generating a simulated image).).
Claims 19 and 20 are the non-transitory computer-readable medium and method claims, respectively, corresponding to claim 1, and are similarly rejected (see Figs. 1-2).
Regarding Claim 8, Zhang discloses the system of claim 1, wherein the process comprises determining information for the specimen based on the encoded representation in combination with output generated for the specimen by the tool during the process and without the information for the design or other information generated from the design (Fig. 2, Zhang discloses using an input image of a specimen, which is then goes through an encoder to generate an encoded representation of the image. The encoded representation is then used to generate a simulated image, from which additional processes such as defect classification can occur. The Examiner notes that Zhang’s disclosure does not require additional external information regarding the design of the specimen).
.
Regarding Claim 9, Zhang discloses the system of claim 1, wherein the process comprises determining information for the specimen based on the encoded representation in combination with output generated for the specimen by the tool during the process, and wherein determining the information for the specimen is performed during the process (Fig. 2, Zhang discloses using an input image of a specimen, which is then goes through an encoder to generate an encoded representation of the image. The encoded representation is then used to generate a simulated image, from which additional processes such as defect classification can occur.).
Regarding Claim 12, Zhang discloses the system of claim 1, wherein the encoded representation is configured as a design attribute map comprising the encoded design attributes as the function of the position in the design (Fig. 2, [0052-0053], [0060], [0070], [0077-0080], Zhang discloses neural deep learning model which takes input of an image of a specimen, generates images patches based on the image input, and inputs the set of image patches into an image encoder. The Examiner notes that the image encoder “encodes” information from the input image into a more compact representation (i.e., an encoded representation). The Examiner notes that “design attributes” is broadly interpreted as any form of data/information related to the design of the input image, which the Examiner notes is obtained through the convolutional layers of the encoder. Furthermore, the encoding performed by Zhang can be directly associated with an input image patch, and hence is considered to be a “function of position” (i.e., an encoded representation output from the encoder can be directly associated with an input image patch).).
Regarding Claim 14, Zhang discloses the system of claim 1, wherein the deep learning model is further configured for learning how to perform said transforming by supervised learning ([0105], Zhang discloses that training of the deep learning model can be performed as disclosed in US patent application 15/176,139 (which corresponds to US 2017/0148226), where the Examiner notes in [0089] of the ‘139 applicate recites that the model can be trained using a supervised process).
Regarding Claim 16, Zhang discloses the system of claim 1, wherein the tool comprises a light-based inspection tool (see Fig. 1, [0031]).
Regarding Claim 17, Zhang discloses the system of claim 1, wherein the tool comprises an electron beam-based inspection tool (see [0049]).
Regarding Claim 18, Zhang discloses the system of claim 1, wherein the tool comprises a defect review tool (see [0058]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 6, and 21 are rejected as being unpatentable over Zhang in view of Christian (“Object detection with Deep Learning (R-CNN Model Family)”, Publication Year: 2022, https://medium.com/@albertcstn/object-detection-with-deep-learning-r-cnn-model-family-7126ab0fa2a2; hereinafter “Christian”) .
Regarding Claim 2, Zhang discloses the system of claim 1, wherein the process comprises determining information for the specimen (Fig. 2, [0113-0115], Zhang discloses decoding image patches (i.e., encoded design attributes), and consequently performing defect classification based on the decoded output.).
Zhang does not explicitly disclose determining information for the specimen (italicized for context) by identifying positions in the encoded representation corresponding to positions on the specimen at which output is generated by the tool during the process, decoding the encoded design attributes (italicized for context) at the identified positions.
PNG
media_image1.png
496
695
media_image1.png
Greyscale
Christian discloses determining information for the specimen (italicized for context) by identifying positions in the encoded representation corresponding to positions on the specimen at which output is generated by the tool during the process, decoding the encoded design attributes (italicized for context) at the identified positions (Christian discloses utilizing regions of interest to identify portions of interest in a feature map (i.e., an encoded representation) in order to select those specific portions of interest for further processing.).
Zhang and Christian are considered to be analogous to the claimed invention as they are in the same field of utilizing deep learning methods to process and identify regions of interest in an image. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang such that the defects of interest identified by Zhang are utilized as regions of interest and selected from the encoded representation (as disclosed by Christian), such that the selected regions of interest are then decoded to determine decoded information as disclosed by Zhang. The motivation for this combination being able to reduce the amount of potentially extraneous information which has to be processed by the system, improving the efficiency of the system.
Regarding Claim 6, Zhang discloses the system of claim 1, wherein the process comprises detecting defects on the specimen and classifying the defects (Fig. 2, [0113-0115], Zhang discloses decoding image patches (i.e., encoded design attributes), and consequently performing defect classification based on the decoded output.).
Zhang does not explicitly disclose classifying the defects (italicized for context) by identifying positions in the encoded representation corresponding to positions of the defects, decoding the encoded design attributes (italicized for context) at the identified positions.
Christian discloses classifying the defects (italicized for context) by identifying positions in the encoded representation corresponding to positions of the defects, decoding the encoded design attributes (italicized for context) at the identified positions (Christian discloses utilizing regions of interest to identify portions of interest in a feature map (i.e., an encoded representation) in order to select those specific portions of interest for further processing.).
Zhang and Christian are considered to be analogous to the claimed invention as they are in the same field of utilizing deep learning methods to process and identify regions of interest in an image. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang such that the defects of interest identified by Zhang are utilized as regions of interest and selected from the encoded representation (as disclosed by Christian), such that the selected regions of interest are then decoded to determine decoded information as disclosed by Zhang. The motivation for this combination being able to reduce the amount of potentially extraneous information which has to be processed by the system, improving the efficiency of the system.
Regarding Claim 21, Zhang discloses a system configured for determining information for a specimen, comprising: a computer system configured for (see Fig. 1):
(Fig. 2, Zhang discloses obtaining images of a semiconductor chip using an imaging tool, and consequently encoding the image.), wherein the encoded representation is generated by a deep learning model configured for transforming information for the design for the specimen into the encoded representation, and wherein the encoded representation comprises encoded design attributes as a function of position in the design (Fig. 2, [0052-0053], [0060], [0070], [0077-0080], Zhang discloses neural deep learning model which takes input of an image of a specimen, generates images patches based on the image input, and inputs the set of image patches into an image encoder. The Examiner notes that the image encoder “encodes” information from the input image into a more compact representation (i.e., an encoded representation). Additionally, “design attributes” is broadly interpreted as any form of data/information related to the design of the input image, which the Examiner notes is obtained through the convolutional layers of the encoder. Furthermore, the encoding performed by Zhang can be directly associated with an input image patch, and hence is considered to be a “function of position” (i.e., an encoded representation output from the encoder can be directly associated with an input image patch).);
determining information for the specimen based on the decoded design attributes (Fig. 2, [0113-0115], Zhang discloses decoding image patches (i.e., encoded design attributes), and consequently performing defect classification based on the decoded output.).
Zhang does not explicitly disclose identifying positions in an encoded representation of a design for a specimen (italicized for context) corresponding to positions on the specimen at which output is generated by a tool during a process performed on the specimen by the tool (italicized for context).
Christian discloses identifying positions in an encoded representation of a design for a specimen (italicized for context) corresponding to positions on the specimen (Christian discloses utilizing regions of interest to identify portions of interest in a feature map (i.e., an encoded representation) in order to select those specific portions of interest for further processing.) at which output is generated by a tool during a process performed on the specimen by the tool (italicized for context).
Zhang and Christian are considered to be analogous to the claimed invention as they are in the same field of utilizing deep learning methods to process and identify regions of interest in an image. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang such that the defects of interest identified by Zhang are utilized as regions of interest and selected from the encoded representation (as disclosed by Christian), such that the selected regions of interest are then decoded to determine decoded information as disclosed by Zhang. The motivation for this combination being able to reduce the amount of potentially extraneous information which has to be processed by the system, improving the efficiency of the system.
Claims 3 and 7 are rejected as being unpatentable over Zhang in view of Christian in view of Kwon et al. (US 2019/0188840; hereinafter “Kwon”).
Regarding Claim 3, Zhang in view of Christian teaches the system of claim 2, wherein the information is determined based on the decoded design attributes (see claim 2)
Zhang in view of Christian does not explicitly teach wherein the information is determined based on the decoded design attributes (italicized for context) in combination with the output generated by the tool during the process.
Kwon discloses wherein the information is determined based on the decoded design attributes (italicized for context) in combination with the output generated by the tool during the process ([0060-0061], Kwon discloses utilizing meta information (i.e., defect attributes) associated with a characteristic of the a specimen, the device used to make the specimen, and/or information associated with an imaging device used to generate an image of a specimen, in order to classify a defect on a specimen.).
Zhang, Christian, and Kwon are considered to be analogous to the claimed invention as they are in the same field of utilizing deep learning methods to process and identify regions of interest in an image. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang in view of Christian such that the information based on decoded design attributes, as taught by Zhang in view of Christian, also utilizes additional meta information to aide in the classification of the defect, as disclosed by Kwon. The motivation for this combination being the ability to improve the accuracy of the information obtained by the system by utilizing additional information.
Regarding Claim 7, Zhang in view of Christian teaches the system of claim 6, wherein classifying the defects is further performed based on the decoded design attributes (see claim 6)
Zhang in view of Christian do not explicitly teach wherein classifying the defects is further performed based on the decoded design attributes (italicized for context) in combination with one or more defect attributes determined from output generated by the tool during the process.
Kwon discloses wherein classifying the defects is further performed based on the decoded design attributes (italicized for context) in combination with one or more defect attributes determined from output generated by the tool during the process ([0060-0061], Kwon discloses utilizing meta information (i.e., defect attributes) associated with a characteristic of the a specimen, the device used to make the specimen, and/or information associated with an imaging device used to generate an image of a specimen, in order to classify a defect on a specimen.).
Zhang, Christian, and Kwon are considered to be analogous to the claimed invention as they are in the same field of utilizing deep learning methods to process and identify regions of interest in an image. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang in view of Christian such that the classification of defects, as taught by Zhang in view of Christian, also utilizes additional meta information to aide in the classification of the defect, as disclosed by Kwon. The motivation for this combination being the ability to improve the accuracy of the defect classification process by utilizing additional information.
Claim 4 is rejected as being unpatentable over Zhang in view of Huang et al. (US 2021/0125325; hereinafter “Huang”) in view of Christian.
Regarding Claim 4, Zhang discloses the system of claim 1, wherein the process comprises detecting defects on the specimen ([0118], Zhang discloses detecting defects and identifying the defects as a real defect or nuisance defect.) (Fig. 2, [0113-0115], Zhang discloses decoding image patches (i.e., encoded design attributes), and consequently performing defect classification based on the decoded output to determine if the defect is real or is a nuisance defect.),
Zhang does not explicitly disclose detecting defects on the specimen (italicized for context) and filtering nuisances from the defects by identifying positions in the encoded representation corresponding to positions of the defects, decoding the encoded design attributes at the identified positions, identifying the nuisances based on the decoded design attributes, and removing the nuisances from results for the defects.
Huang discloses detecting defects on the specimen (italicized for context) and filtering nuisances from the defects ([0054], Huang discloses identifying and filtering our nuisance defects.) ([0054], Huang discloses filtering out defect candidates based on whether they are nuisance defects or not.).
Zhang and Huang are considered to be analogous to the claimed invention as they are in the same field of utilizing deep learning methods to process and identify regions of interest in an image. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang such that the nuisance defects identified by Zhang are filtered out based on the methods disclosed by Huang. The motivation for this combination being the ability to reduce the effects of identifying false defects by removing them from the processing pipeline.
Zhang in view of Huang does not explicitly disclose identifying positions in the encoded representation corresponding to positions of the defects, decoding the encoded design attributes at the identified positions.
Christian discloses identifying positions in the encoded representation corresponding to positions of the defects, decoding the encoded design attributes at the identified positions (Christian discloses utilizing regions of interest to identify portions of interest in a feature map (i.e., an encoded representation) in order to select those specific portions of interest for further processing.).
Zhang, Huang, and Christian are considered to be analogous to the claimed invention as they are in the same field of utilizing deep learning methods to process and identify regions of interest in an image. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang in view of Huang such that the defects of interest identified by Zhang in view of Huang are utilized as regions of interest and selected from the encoded representation (as disclosed by Christian), such that the selected regions of interest are then decoded to determine decoded information as disclosed by Zhang in view of Huang. The motivation for this combination being able to reduce the amount of potentially extraneous information which has to be processed by the system, improving the efficiency of the system.
Claims 5 is rejected as being unpatentable over Zhang in view of Huang in view of Christian in view of Kwon.
Regarding Claim 5, Zhang in view of Huang in view of Christian teaches the system of claim 4, wherein identifying the nuisances is further performed based on the decoded design attributes (see claim 4)
Zhang in view of Christian does not explicitly teach wherein identifying the nuisances is further performed based on the decoded design attributes (italicized for context) in combination with one or more defect attributes determined from output generated by the tool during the process.
Kwon discloses wherein identifying the nuisances is further performed based on the decoded design attributes (italicized for context) in combination with one or more defect attributes determined from output generated by the tool during the process ([0060-0061], Kwon discloses utilizing meta information (i.e., defect attributes) associated with a characteristic of the a specimen, the device used to make the specimen, and/or information associated with an imaging device used to generate an image of a specimen, in order to classify a defect on a specimen.).
Zhang, Huang, Christian, and Kwon are considered to be analogous to the claimed invention as they are in the same field of utilizing deep learning methods to process and identify regions of interest in an image. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang in view of Huang in view of Christian such that the information based on decoded design attributes, as taught by Zhang in view of Huang in view of Christian, also utilizes additional meta information to aide in the classification of the defect, as disclosed by Kwon. The motivation for this combination being the ability to improve the accuracy of the information obtained by the system by utilizing additional information.
Claims 10-11 are rejected as being unpatentable over Zhang in view of Novacam (“Inspecting Edge Breaks / Edge Radius with Novacam EdgeInspect System”, https://www.thesempregroup.com/wp-content/uploads/2020/07/ApplicationNote_Inspecting_Edge_Breaks_Edge_Radius_rev1.1.pdf, Publicly Available on 11/4/2024; hereinafter “Novacam”).
Regarding Claim 10, Zhang discloses the system of claim 1, (Fig. 2, Zhang discloses using an input image of a specimen, which is then goes through an encoder to generate an encoded representation of the image. The encoded representation is then used to generate a simulated image, from which additional processes such as defect classification can occur.).
Zhang does not explicitly disclose wherein the process comprises detecting defects on the specimen at a rate of more than 100,000 defects per second.
Novacam discloses wherein the process comprises detecting defects on the specimen at a rate of more than 100,000 defects per second (How It Works, Novacam discloses an EdgeInspect system which can analyze a specimen at a rate of 100,000 points or higher.).
Zhang and Novacam are considered to be analogous to the claimed invention as they are in the same field of using tools and imaging devices to image and further analyze physical objects (i.e., a specimen). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang such that the defect detection was performed at the specific rate disclosed by the disclosure in Novacam. The motivation for this combination being the ability to image and obtain data at a sufficient rate, improving the efficiency of the system.
Regarding Claim 11, Zhang in view of Novacam teaches the system of claim 10, wherein determining the information for said each of the detected defects during the process is further performed by a computer system included in the tool (see the computer subsystem(s) presented in Fig. 1, Zhang).
Claims 13 is rejected as being unpatentable over Zhang in view of Horikoshi (JP 2007168129; hereinafter “Horikoshi”).
Regarding Claim 13, Zhang discloses the system of claim 1.
Zhang does not explicitly disclose wherein the encoded representation is configured as a raster image for at least a portion of an entire device in the design, and wherein each pixel in the raster image comprises the encoded design attributes at the position in the design corresponding to said each pixel.
Horikoshi discloses wherein the encoded representation is configured as a raster image for at least a portion of an entire device in the design, and wherein each pixel in the raster image comprises the encoded design attributes at the position in the design corresponding to said each pixel ([0064], Horikoshi discloses generating a raster image where an attribute is associated with each pixel, and consequently performing encoding and decoding on the raster image.).
Zhang and Horikoshi are considered to be analogous to the claimed invention as they are in the same field of embedding information into images and feature maps. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang such that the encoded representation disclosed by Zhang is in the form of a raster image with per-pixel embeddings as disclosed by Horikoshi. The motivation for this combination being the ability to map attributes and other information regarding the image on a per-pixel basis, allowing for mapping between the embedded image/representation and the input image.
Claims 15 is rejected as being unpatentable over Zhang in view of Kim et al. (US 2023/0029163; hereinafter “Kim”).
Regarding Claim 15, Zhang discloses the system of claim 1.
Zhang does not explicitly disclose wherein the computer system is further configured for training the deep learning model by supervised learning with labeled training data.
Kim discloses wherein the computer system is further configured for training the deep learning model by supervised learning with labeled training data ([0150], Kim discloses performing training of a deep learning model using supervised learning with a labeled dataset.).
Zhang and Kim are considered to be analogous to the claimed invention as they are in the same field of utilizing deep learning methods to process and identify regions of interest in an image. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang such that the training of the deep learning model disclosed by Zhang is based on the supervised learning method disclosed by Kim. The motivation for this combination being the ability to utilize a specific training dataset which contains labels that can help to improve model accuracy.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PROMOTTO TAJRIAN ISLAM whose telephone number is (703)756-5584. The examiner can normally be reached Monday - Friday 8:30 am - 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PROMOTTO TAJRIAN ISLAM/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
1 The Examiner notes that the language “for use” indicates a purpose/intended use, which is not given patentable weight (see MPEP 2103(I)(C)). Given the claim its broadest reasonable interpretation, the system of claim 1 is only limited by the storage of the encoded representation, and claim does not necessarily require that the stored encoded representation is used “in a process performed on the specimen by a tool”. Additionally, note how the process claimed in claim 1 is the basis for several dependent claims. For clarity of the record, the Examiner will provide prior art mapping to the presented limitations.
Prosecution Insights