DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sherman et al (US 20240095903 A1) in view of Woodard et al (US 20220028086 A1).
Regarding claim 1, Sherman discloses a computer-implemented method for data augmentation in electron microscope imaging ([0043] machine learning (ML) technologies can be used to assist the defect examination process so as to provide accurate and efficient solutions for automating specific examination applications and promoting higher yield), the method comprising:
receiving, using a processor, an input image captured by an electron microscope ([0049] examination tools 120 can be implemented as machines of various types, such as optical machines, electron beam machines (e.g., Scanning Electron Microscope (SEM), Atomic Force Microscopy (AFM), or Transmission Electron Microscope (TEM), etc.), and so on.);
processing, using a processor, the input image to generate an augmented image dataset, the processing ([0050] inspection tool configured to scan a specimen (e.g., an entire wafer, an entire die, or portions thereof) to capture inspection images (typically, at a relatively high-speed and/or low-resolution) for detection of potential defects (i.e., defect candidates)) comprising:
generating a first transformed image by converting an interior region of an object region of interest in the input image to a single color ([0086] contextual region is used for the purpose of searching for similar candidate areas in a target image. The term “target image” used herein refers to any image to be augmented with a defective feature so as to create a synthetic defective image);
generating a second transformed image by converting a region other than the object region of interest in the input image to a single color ([0094] SIFT searches, independently in each image, difference of Gaussians (DoG) over scale and space for local extrema, which can be used as key points); and
outputting the augmented image dataset to train a machine learning model ([0113] training set can be used (506) to train a machine learning (ML) model (such as e.g., the ML module 110 in the PMC 102).).
Woodard discloses generating a third transformed image by modifying pixel positions exclusively within the interior region of the object region of interest in the input image ([0070] transform-based image processing can be provided to extract textures from various layers of an integrated circuit. Furthermore, according to various embodiments, the various layers can be segmented based on the textures)
Sherman and Woodard are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify defect examination process of Sherman to include generating a third transformed image by modifying pixel positions exclusively within the interior region of the object region of interest in the input image as described by Woodard.
The motivation for doing so would have been for accelerated segmentation for reverse engineering of integrated circuits by or segmenting a Scanning Electron Microscopy (SEM) image of an integrated circuit (IC) (Woodard, [0004-0005]).
Therefore, it would have been obvious to combine Sherman and Woodard to obtain the invention as specified in claim 1.
Regarding claim 2, Sherman discloses wherein, in at least one of generating the first transformed image or generating the second transformed image, the single color is selected from black, white, or gray ([0094] a local descriptor is calculated as a histogram of image gradients around a key point to characterize the local appearance of the key point).
Regarding claim 3, Sherman discloses wherein the single color is black ([0094] a local descriptor is calculated as a histogram of image gradients around a key point to characterize the local appearance of the key point).
Regarding claim 4, Sherman discloses wherein generating the third transformed image is performed not to change a contour shape of the object region of interest ([0080] second region can be defined in accordance with a predetermined parameter indicative of the distance between the contours of the first region and the second region.).
Regarding claim 5, Sherman discloses wherein generating the third transformed image is performed not to change a size of the object region of interest ([0050] During inspection, the wafer can move at a step size relative to the detector of the inspection tool (or the wafer and the tool can move in opposite directions relative to each other) during the exposure, and the wafer can be scanned step-by-step along swaths of the wafer by the inspection tool, where the inspection tool images a part/portion (within a swath) of the specimen at a time).
Regarding claim 6, Sherman discloses wherein at least one of generating the first transformed image, generating the second transformed image, or generating the third transformed image includes utilizing positional information of the object region of interest obtained from a correct image or reference image ([0104] the reference image can be captured by the examination tool from one or more reference areas of an inspection area (such as, e.g., one or more neighboring dies of an inspection die in D2D inspection) which are known to be defect-free.).
Regarding claim 7, Sherman discloses wherein the input image is a transmission electron microscope (TEM) image or a scanning electron microscope (SEM) image ([0037] scanning electron microscopes (SEM), atomic force microscopes (AFM), optical inspection tools, etc.).
Regarding claim 8, Sherman discloses a computer-implemented method of deriving object regions of interest from images captured by an electron microscope ([0043] machine learning (ML) technologies can be used to assist the defect examination process so as to provide accurate and efficient solutions for automating specific examination applications and promoting higher yield), the method comprising:
receiving an augmented image dataset generated using the data augmentation method of claim 1;
training a machine learning model using the augmented image dataset ([0059] training module 108 can be configured to associate a defect label with the first region or part thereof in the augmented target image, include the augmented target image associated with the defect label in a training set, and use the training set to train the ML model); and
deriving an object region of interest using the trained machine learning model ([0058] image processing module can be configured to specify a second region in the original image containing the first region, giving rise to a contextual region (of the first region or of the defective feature) between the first region and the second region.).
Regarding claim 9, Sherman discloses an electronic device for augmenting image data for machine learning on images captured by an electron microscope ([0043] machine learning (ML) technologies can be used to assist the defect examination process so as to provide accurate and efficient solutions for automating specific examination applications and promoting higher yield), the device comprising:
a processor ([0055] system 101 includes a processor and memory circuitry (PMC)); and
a memory storing one or more instructions that, when executed by the processor ([0055] system 101 includes a processor and memory circuitry (PMC)), cause the processor to:
receive an input image captured by the electron microscope ([0049] examination tools 120 can be implemented as machines of various types, such as optical machines, electron beam machines (e.g., Scanning Electron Microscope (SEM), Atomic Force Microscopy (AFM), or Transmission Electron Microscope (TEM), etc.), and so on.); and
generate an augmented image dataset by applying data augmentation techniques to the input image ([0050] inspection tool configured to scan a specimen (e.g., an entire wafer, an entire die, or portions thereof) to capture inspection images (typically, at a relatively high-speed and/or low-resolution) for detection of potential defects (i.e., defect candidates)),
wherein generating the augmented image dataset includes performing at least one of:
generating a first transformed image by converting an interior region of an object region of interest in the input image to a single color ([0086] contextual region is used for the purpose of searching for similar candidate areas in a target image. The term “target image” used herein refers to any image to be augmented with a defective feature so as to create a synthetic defective image);
generating a second transformed image by converting a region other than the object region of interest in the input image to a single color ([0094] SIFT searches, independently in each image, difference of Gaussians (DoG) over scale and space for local extrema, which can be used as key points); or
Woodard discloses generating a third transformed image by modifying pixel positions exclusively within the interior region of the object region of interest in the input image ([0070] transform-based image processing can be provided to extract textures from various layers of an integrated circuit. Furthermore, according to various embodiments, the various layers can be segmented based on the textures)
Sherman and Woodard are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify defect examination process of Sherman to include generating generating a third transformed image by modifying pixel positions exclusively within the interior region of the object region of interest in the input image as described by Woodard.
The motivation for doing so would have been for accelerated segmentation for reverse engineering of integrated circuits by or segmenting a Scanning Electron Microscopy (SEM) image of an integrated circuit (IC) (Woodard, [0004-0005]).
Therefore, it would have been obvious to combine Sherman and Woodard to obtain the invention as specified in claim 9.
Regarding claim 10, Sherman discloses wherein, in at least one of generating the first transformed image or generating the transformed image, the single color is selected from black, white, or gray ([0094] a local descriptor is calculated as a histogram of image gradients around a key point to characterize the local appearance of the key point).
Regarding claim 11, Sherman discloses wherein the single color is black ([0094] a local descriptor is calculated as a histogram of image gradients around a key point to characterize the local appearance of the key point).
Regarding claim 12, Sherman discloses wherein generating the third transformed image is performed not to change a contour shape of the object region of interest ([0080] second region can be defined in accordance with a predetermined parameter indicative of the distance between the contours of the first region and the second region.).
Regarding claim 13, Sherman discloses wherein generating the third transformed image is performed not to change a size of the object region of interest ([0050] During inspection, the wafer can move at a step size relative to the detector of the inspection tool (or the wafer and the tool can move in opposite directions relative to each other) during the exposure, and the wafer can be scanned step-by-step along swaths of the wafer by the inspection tool, where the inspection tool images a part/portion (within a swath) of the specimen at a time).
Regarding claim 14, Sherman discloses wherein at least one of generating the first transformed image, generating the second transformed image, or generating the third transformed image includes utilizing positional information of the object region of interest obtained from a correct image or reference image ([0104] the reference image can be captured by the examination tool from one or more reference areas of an inspection area (such as, e.g., one or more neighboring dies of an inspection die in D2D inspection) which are known to be defect-free.).
Regarding claim 15, Sherman discloses wherein the input image is a transmission electron microscope (TEM) image or a scanning electron microscope (SEM) image ([0037] scanning electron microscopes (SEM), atomic force microscopes (AFM), optical inspection tools, etc.).
Regarding claim 16, Sherman discloses an electronic device for deriving object regions of interest from images captured by an electron microscope using machine learning, the device comprising: the electronic device of claim 9 ([0043] machine learning (ML) technologies can be used to assist the defect examination process so as to provide accurate and efficient solutions for automating specific examination applications and promoting higher yield).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHIVANG I PATEL whose telephone number is (571)272-8964. The examiner can normally be reached on M-F 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached on (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHIVANG I PATEL/Primary Examiner, Art Unit 2615
Prosecution Insights