DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged that application claims priority to foreign application with application number GERMANY 102023121983.9 dated 08/16/2023. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Drawings
The drawings are objected to because Fig 1, Fig 4, and Fig 7-10 do not clearly convey the subject matter of the specification due to the boxes being labeled with only reference numbers and no description. For ease of understanding the Examiner suggests adding descriptions to the numbered boxes for clarity of scope and readability. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“an imaging device configured to” in claim 24, in the specification “an imaging device” is defined as “ imaging device 70 for obtaining one or more imaging datasets of the object 72 comprising integrated circuit patterns can, for example, comprise a charged particle beam device, for example, a Helium ion microscope, a cross-beam device including FIB and SEM, an atomic force microscope or any charged particle imaging device, or an aerial image acquisition system” in ¶0116.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
The claims in this application are given their broadest reasonable interpretation using the
plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification.
Under MPEP 2143.03, "All words in a claim must be considered in judging the patentability of that claim against the prior art." In re Wilson, 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 1970). As a general matter, the grammar and ordinary meaning of terms as understood by one having ordinary skill in the art used in a claim will dictate whether, and to what extent, the language limits the claim scope. Language that suggests or makes a feature or step optional but does not require that feature or step does not limit the scope of a claim under the broadest reasonable claim interpretation. In addition, when a claim requires selection of an element from a list of alternatives, the prior art teaches the element if one of the alternatives is taught by the prior art. See, e.g., Fresenius USA, Inc. v. Baxter Int’l, Inc., 582 F.3d 1288, 1298, 92 USPQ2d 1163, 1171 (Fed. Cir. 2009).
Claim 3 recite “or” then listing “comprises a pixel- wise map or a voxel-wise map.” Since “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required. Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15 and 18-24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. When reviewing independent claims 1, 22, 23 and 24 and based upon consideration of all of the relevant factors with respect to the claim as a whole, claims 1-15 and 18-24 are held to claim an abstract idea without reciting elements that amount to significantly more than the abstract idea and is/are therefore rejected as ineligible subject matter under 35 U.S.C. 101. The Examiner will analyze Claim 1 and similar rationale applies to independent Claim 24. Claims 22 and 23 will be analyzed separately, see below. The rationale, under MPEP § 2106, for this finding is explained below:
The claimed invention (1) must be directed to one of the four statutory categories, and (2) must not be wholly directed to subject matter encompassing a judicially recognized exception, as defined below. The following two step analysis is used to evaluate these criteria.
Step 1: Is the claim directed to one of the four patent-eligible subject matter categories: process, machine, manufacture, or composition of matter?
When examining the claim under 35 U.S.C. 101, the Examiner interprets that the claims 1-21 are directed to a process since the claim is directed to a method.
Step 2a, Prong 1: Does the claim wholly embrace a judicially recognized exception, which includes laws of nature, physical phenomena, and abstract ideas, or is it a particular practical application of a judicial exception?
The Examiner interprets that the judicial exception applies since Claim 1 limitation of
jointly processing three datasets is directed to an abstract idea. The claim is related to mathematical relationships including jointly processing three image datasets to detect defects. Under Recentive Analytics, training a neural network on domain-specific data is an abstract idea. Recentive Analytics, Inc. v. Fox Corp., No. 23-2437, 134 F.4th 1205 (Fed. Cir. 2025). If the claim recites a judicial exception (i.e., an abstract idea enumerated in MPEP § 2106.04(a), a law of nature, or a natural phenomenon), the claim requires further analysis in Prong Two.
Step 2a, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
The Examiner interprets that Claim 1 limitation does not provide additional elements or combination of additional elements to a practical application since the claims are not adding insignificant extra-solution activity to the judicial exception. Since the claim is generally processing domain specific data. Specifically, the analysis method does not integrate a judicial exception into practical application. See Genetic Techs. v. Merial LLC, 818 F.3d 1369, 1376, 118 USPQ2d 1541, 1546 (Fed. Cir. 2016) (eligibility "cannot be furnished by the unpatentable law of nature (or natural phenomenon or abstract idea) itself."). For a claim reciting a judicial exception to be eligible, the additional elements (if any) in the claim must "transform the nature of the claim" into a patent-eligible application of the judicial exception, Alice Corp., 573 U.S. at 217, 110 USPQ2d at 1981, either at Prong Two or in Step 2B. The Examiner interprets that Claim 1 limitation does not provide additional elements or combination of additional elements to a practical application since the abstract idea is not integrated into a practical application because the additional elements fail to provide a technical improvement. Limiting the training to “defect indicators” is a non-qualifying field-of-use limitation. Data gathering (obtaining three imaging datasets) and storage (memory storing instructions) are routine activities that do not add a practical application. Reciting an “machine learning model” in further dependent claims functionally does not reflect a specific technical improvement to the device. Specifically, the statement of using machine learning does not integrate an improvement to a technology or to computer functionality. See Genetic Techs. v. Merial LLC, 818 F.3d 1369, 1376, 118 USPQ2d 1541, 1546 (Fed. Cir. 2016) (eligibility "cannot be furnished by the unpatentable law of nature (or natural phenomenon or abstract idea) itself."). For a claim reciting a judicial exception to be eligible, the additional elements (if any) in the claim must "transform the nature of the claim" into a patent-eligible application of the judicial exception, For a claim reciting a judicial exception to be eligible, the additional elements (if any) in the claim must "transform the nature of the claim" into a patent-eligible application of the judicial exception, Alice Corp., 573 U.S. at 217, 110 USPQ2d at 1981, either at Prong Two or in Step 2B. If there are no additional elements in the claim, then it cannot be eligible. In such a case, after making the appropriate rejection, it is a best practice for the examiner to recommend an amendment, if possible, that would resolve eligibility of the claim.
Step 2b: If a judicial exception into a practical application is not recited in the claim, the Examiner must interpret if the claim recites additional elements that amount to significantly more than the judicial exception.
The Examiner interprets that the Claims do not amount to significantly more since the Claims state using processing with a high level of generality and the claims lack an inventive concept because the elements, considered individually and as an order combination, are well-understood, routine, and conventional (WURC) in the field. Further, the specification acknowledges that multiple image data sets and joint processing are known in the art. See, simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, e.g., a claim to an abstract idea requiring no more than a generic computer to perform generic computer functions that are well-understood, routine and conventional activities previously known to the industry, as discussed in Alice Corp., 573 U.S. at 225, 110 USPQ2d at 1984.
Claims 1-15 and 18-21 depending on the independent claim/s include all the limitation of the independent claim. The Examiner finds that Claim 2, 7, and 14 bring in the variable of defect indicators, which is mere data gathering activity. This is seen as an abstract idea related to a mathematical process. The claim describes mere data gathering. See MPEP 2106.05(g). The Examiner finds that Claim 3 “Generating a pixel-wise or voxel-wise map" which is a mathematical operation, which are further abstract mathematical concepts. This is seen as an abstract idea related to a mathematical process. The claim describes the further manipulation of data. See MPEP 2106.05(g). The Examiner finds that Claim 4 brings in the variable of defect descriptions, which is mere data gathering activity. This is seen as an abstract idea related to a mathematical process. The claim describes mere data gathering. See MPEP 2106.05(g). The Examiner finds that Claim 5 brings in the variable of probabilistic defect indicators, which is mere data gathering activity. This is seen as an abstract idea related to a mathematical process. The claim describes mere data gathering. See MPEP 2106.05(g). The Examiner finds that Claim 6 brings in the variable of unique identifiers, which is mere data gathering activity. This is seen as an abstract idea related to a mathematical process. The claim describes mere data gathering. See MPEP 2106.05(g). The Examiner finds that Claim 8 compares the defect indicators for all datasets, which is mere data manipulation activity. This is seen as an abstract idea related to a mathematical process. The claim describes mere data manipulation. See MPEP 2106.05(g). The Examiner finds that Claim 9 and 18 bring in the variable of registered, which is mere data gathering activity. This is seen as an abstract idea related to a mathematical process. The claim describes mere data gathering. See MPEP 2106.05(g). The Examiner finds that Claim 10 and 20 “Processing the likelihood of a defect" which is a mathematical operation, which are further abstract mathematical concepts. This is seen as an abstract idea related to a mathematical process. The claim describes the further manipulation of data. See MPEP 2106.05(g). The Examiner finds that Claim 11 inputting datasets into a general machine learning model which is a mathematical operation, which are further abstract mathematical concepts. This is seen as an abstract idea related to a mathematical process. The claim describes the further manipulation of data. See MPEP 2106.05(g). The Examiner finds that Claim 12 inputting three datasets into a general machine learning model which is a mathematical operation, which are further abstract mathematical concepts. This is seen as an abstract idea related to a mathematical process. The claim describes the further manipulation of data. See MPEP 2106.05(g). The Examiner finds that Claim 13 merging three inputs which is a mathematical operation, which are further abstract mathematical concepts. This is seen as an abstract idea related to a mathematical process. The claim describes the further manipulation of data. See MPEP 2106.05(g). The Examiner finds that Claim 15 outputting three outputs which is a mathematical operation, which are further abstract mathematical concepts. This is seen as an abstract idea related to a mathematical process. The claim describes the further manipulation of data. See MPEP 2106.05(g). The Examiner finds that Claim 19 brings in the specific data requirements for all datasets, which is mere data gathering activity. This is seen as an abstract idea related to a mathematical process. The claim describes mere data gathering. See MPEP 2106.05(g). The Examiner finds that Claim 21 applying the datasets into a general machine learning model which is a mathematical operation, which are further abstract mathematical concepts. This is seen as an abstract idea related to a mathematical process. The claim describes the further manipulation of data. See MPEP 2106.05(g).
Thus, Claims 1-15 and 18-21 recite the same abstract idea and therefore are not drawn to the eligible subject matter as they are directed to the abstract idea without significantly more.22. Claim 22 is rejected under 35 U.S.C. 101 because the claims appear to be directed to a software embodiment and not to hardware embodiment, where a machine claim is directed towards a system, apparatus, or arrangement. The claim appears to be directed towards a software embodiment. Paragraphs 0139-0140 of the Published Specification describes the elements of the system being implemented as software (computer program) alone actualizing the embodiments of the invention. The claimed limitations are capable of being performed as software as described in the above paragraphs, alone since no hardware component is being claimed. Software, alone, are not physical components and thus are not statutory since software do not define any structural and functional interrelationships between the computer programs and other claimed elements of a computer, which permit the computer' s program functionality to be realized. Hence, the stated functions comprise software and is thus not directed to a hardware embodiment. Data structures not claimed as embodied in computer readable media are descriptive material per se and are not statutory because they are not capable of causing functional change in the computer. See e.g., Warmerdam, 33 F.3d at 1361, 31, USPQ2d at 1760 (claim to a data structure per se held non-statutory). Such claimed data structures do not define any structural and functional interrelationships between data and other claimed aspects of the invention, which permit the data structure' s functionality to be realized. In contrast, a claimed computer readable medium encoded with a data structure defines structural and functional interrelationships between the data structure and the computer software and hardware components which permit the data structure' s functionality to be realized, and is thus statutory.
For claim 23 the examiner recognizes that applicants may have claims directed to computer readable media that cover signals per se, which the USPTO must reject under 35 U.S.C. § 101 as covering both non-statutory subject matter and statutory subject matter. In an effort to assist the patent community in overcoming a rejection or potential rejection under 35 U.S.C. § 101 in this situation, the USPTO suggests the following approach. A claim drawn to such a computer readable medium that covers both transitory and non-transitory embodiments may be amended to narrow the claim to cover only statutory embodiments to avoid a rejection under 35 U.S.C. § 101 by adding the limitation "non-transitory" to the claim. Cf. Animals - Patentability, 1077 Off. Gaz. Pat. Office 24 (April 21, 1987) (suggesting that applicants add the limitation "non-human" to a claim covering a multi-cellular organism to avoid a rejection under 35 U.S.C. § 101). Such an amendment would typically not raise the issue of new matter, even when the specification is silent because the broadest reasonable interpretation relies on the ordinary and customary meaning that includes signals per se. The limited situations in which such an amendment could raise issues of new matter occur, for example, when the specification does not support a non-transitory embodiment because a signal per se is the only viable embodiment such that the amended claim is impermissibly broadened beyond the supporting disclosure. See, e.g., Gentry Gallery, Inc. v. Berkline Corp., 134 F.3d 1473(Fed. Cir. 1998). Claim 23 is rejected under U.S.C 101 because the claimed invention is directed to non-statutory subject matter as follows. Claim 23 defines a “computer-readable medium” embodying functional descriptive material. However, the claim does not define a non-transitory computer-readable medium or memory and is thus non-statutory for that reason (i.e., “examination the pending claims must be interpreted as broadly as their terms reasonably allow). The broadest reasonable interpretation of a claim drawn to a computer readable medium (also called machine readable medium and other such variations) typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent. See MPEP 2111.01.
For the analogous independent claim 24 to the independent claim 1, the analogous limitations can be analyzed in the same way as above for the claim 1 hence rejected under 101. Moreover, the claim of 24 further recites a different statutory category of “system” which is a limitation that the examiner interprets that the claims is related to a machine since the claim is directed to a an imaging device, one or more processing devices, and one or more machine-readable hardware storage devices, and is consistent with the abstract ideas of jointly processing multiple image datasets. Under Recentive Analytics, training a neural network on domain-specific data is an abstract idea. Recentive Analytics, Inc. v. Fox Corp., No. 23-2437, 134 F.4th 1205 (Fed. Cir. 2025). This limitation of system just further implement the abstract ideas to be performed by generic computer or software/hardware components of additional elements of the different types of analyzation methods. Therefore, the Examiner interprets that the claims are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(B) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1, 10, and 19-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. The Examiner strongly suggested that appropriate corrections be made to clarify the claim scope.
With respect to Claim 1, the claim recites the following, each of which renders the claim indefinite:
“ predominately the same” on line 5 (unclear how this affects the scope of the claim), “predominately” can be considered relative terminology as the amount of “the same” cannot be quantified and therefore it is unclear how similar the images of the circuit board are in the scope of capturing the image, furthermore, there is no grounding in the specification for these terms and the claim language itself does not adequately define these terms;
Claim(s) 2-24 depend either directly or indirectly from the rejection of Claim 1, therefore they are also rejected. Appropriate correction is required.
With respect to Claim 10, the claim recites the following, each of which renders the claim indefinite:
“ the likelihood” on line 2 (unclear antecedent basis, there is no definition or mention of the likelihood of defects in claim 1, its parent claim).
“ the layout” on line 2 (unclear antecedent basis there is no definition or mention of the layout of the circuit patterns in claim 1, its parent claim).
With respect to Claim 19, the claim recites the following, each of which renders the claim indefinite:
“ the group” on line 2 (unclear antecedent basis, there is no definition or mention of the group of images in claim 1, its parent claim).
With respect to Claim 20, the claim recites the following, each of which renders the claim indefinite:
“ the likelihood” on line 2 (unclear antecedent basis, there is no definition or mention of the likelihood of defects in claim 1, its parent claim).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 4-9, 11-14, 17-24 are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Zhang et al (US Patent Publication 2023/0118839 A1, hereafter referred to as Zhang).
Regarding Claim 1, Zhang teaches a computer implemented method for defect detection (Zhang ¶0010, ¶0012, ¶0014 discloses a computer implemented method for defect detection on a specimen) comprising:
- obtaining a first imaging dataset (Zhang Fig 2, 200, ¶0070, ¶0071 discloses a first set of images being input into the model) of a portion of an object comprising integrated circuit patterns (Zhang ¶0150, ¶0153, discloses the specimen being imaged a wafer on which patterned features has been formed using multiple lithography exposure steps, which examiner understand to mean printed circuity board);
- obtaining at least a second imaging dataset and a third imaging dataset (Zhang Fig 2, 202, 204, ¶0071 discloses a second and third set of images being input into the model) comprising predominantly the same (Zhang ¶0013, ¶0014, ¶0070 discloses the images being of the same location and the same field of view) integrated circuit patterns as the portion of the object (Zhang ¶0150, ¶0153, discloses the specimen being imaged a wafer on which patterned features has been formed using multiple lithography exposure steps, which examiner understand to mean printed circuity board); and
- jointly processing at least the first imaging dataset, the second imaging dataset and the third imaging dataset (Zhang ¶0085-¶0087, and ¶0091 discloses joining the latent spaces for processing and ¶0132 discloses jointly training the LLRI generator and the defect detection component with one or more training images and pixel-level ground truth information for the one or more training images) to detect defects (Zhang ¶0085-¶0087, and ¶0091 discloses combining the image sets into a joint latent space for defect detection).
Regarding Claim 2, Zhang teaches the method of claim 1, wherein the processing comprises estimating one or more defect indicators (Zhang ¶0069 discloses the defect indicators being determined based on a measure of the similarity), and wherein defects are detected using the one or more defect indicators (Zhang ¶0069-¶0070 discloses using the defect indicator to determine if the pixel in the image location is defective or non-defective).
Regarding Claim 4, Zhang teaches the method of claim 2, wherein at least one defect indicator (Zhang ¶0069 discloses the defect indicators being determined based on a measure of the similarity) comprises a list of defect descriptions (Zhang ¶0153 discloses the list of defect types being user-specified defects, hot spots, or weak patterns).
Regarding Claim 5, Zhang teaches the method of claim 2, wherein at least one defect indicator (Zhang ¶0069 discloses the defect indicators being determined based on a measure of the similarity) is a probabilistic defect indicator (Zhang ¶0125 discloses a probabilistic model for determining the defect).
Regarding Claim 6, Zhang teaches the method of claim 2, wherein at least one defect indicator (Zhang ¶0069 discloses the defect indicators being determined based on a measure of the similarity) comprises a defect (Zhang ¶0069-¶0070 discloses using the defect indicator to determine if the pixel in the image location is defective or non-defective) and a unique identifier of an imaging dataset that contains the defect (Zhang ¶0070 discloses labeling each pixel location in the image that is input into the machine learning model defective or non-defective).
Regarding Claim 7, Zhang teaches the method of claim 2, wherein a defect indicator (Zhang ¶0069 discloses the defect indicators being determined based on a measure of the similarity) is obtained for at least each of the first imaging dataset, the second imaging dataset and the third imaging dataset (Zhang Fig 2, 200, 202, 204, ¶0071 discloses a first, a second and third set of images being input).
Regarding Claim 8, Zhang teaches the method of claim 7, wherein defects are detected by comparing at least the defect indicators (Zhang ¶0144, ¶0069 discloses comparing the difference between the images to a threshold to determine the defect) for the first imaging dataset, the second imaging dataset and the third imaging dataset (Zhang Fig 2, 200, 202, 204, ¶0071 discloses a first, a second and third set of images being input)
Regarding Claim 9, Zhang teaches the method of claim 1, wherein at least the first imaging dataset, the second imaging dataset and the third imaging dataset (Zhang Fig 2, 200, 202, 204, ¶0071 discloses a first, a second and third set of images being input into the model) are registered (Zhang ¶0069 discloses the images being transformed into a latent space, in the instant specification registered is defined as transforming the image datasets in ¶0052 therefore Zhang teaches on this definition as defined in the instant specification).
Regarding Claim 11, Zhang teaches the method of claim 1, wherein at least the first imaging dataset, the second imaging dataset and the third imaging dataset are input datasets of a machine learning model (Zhang Fig 2, 200, 202, and 204 and ¶0055 disclose the imaging datasets being input into the machine learning model).
Regarding Claim 12, Zhang teaches the method of claim 11, wherein the machine learning model comprises a neural network ( Zhang ¶0069 discloses the model being a neural network) with at least three input paths (Zhang Fig 2 , 200, 202, 204 disclose the images with three input paths) wherein the first imaging dataset is processed along a first input path (Zhang Fig 2 206 discloses the images being input to a first CNN input), the second imaging dataset is processed along a second input path (Zhang Fig 2 210 discloses the images being input to a second CNN input) and the third imaging dataset is processed along a third input path (Zhang Fig 2 212 discloses the images being input to a third CNN input).
Regarding Claim 13, Zhang teaches the method of claim 12, wherein the at least three input paths (Zhang Fig 2 , 200, 202, 204 disclose the images with three input paths) merge in a single merging layer (Zhang Fig 1 and Fig 3 disclose the merging of the layers to output one final layer, the defect detection 318 and ¶0089 disclose combining multiple channels).
Regarding Claim 14, Zhang teaches the method of claim 12, wherein the neural network ( Zhang ¶0069 discloses the model being a neural network) generates a defect indicator (Zhang ¶0069 discloses the defect indicators being determined based on a measure of the similarity, Fig 3, 310 discloses the layer in the neural network that outputs these indicators) for at least each of the first imaging dataset, the second imaging dataset and the third imaging dataset (Zhang Fig 2, 200, 202, and 204 and ¶0055 disclose the imaging datasets being input into the machine learning model and outputting the defect detection in Fig 3, 318).
Regarding Claim 17, Zhang teaches the method of claim 11, wherein the machine learning model (Zhang Fig 2, 200, 202, and 204 and ¶0055 disclose the imaging datasets being input into the machine learning model) is configured to process the input paths asynchronously (Zhang ¶0055, ¶0067 discloses the latent space being independently determined for the different portions of the test image such that those distance can be used for detecting defects in each of the different portions of the test image, the examiner in interpreting "independently determined" to mean that they are determined at different times are not tied together).
Regarding Claim 18, Zhang teaches the method of claim 11, wherein the first imaging dataset, the second imaging dataset and the third imaging dataset are registered (Zhang ¶0069 discloses the images being transformed into a latent space, in the instant specification registered is defined as transforming the image datasets in ¶0052 therefore Zhang teaches on this definition as defined in the instant specification) using a machine learning model (Zhang Fig 2, 200, 202, and 204 and ¶0055 disclose the imaging datasets being input into the machine learning model).
Regarding Claim 19, Zhang teaches the method of claim 1, wherein the first imaging dataset, the second imaging dataset and the third imaging dataset (Zhang Fig 2, 200, 202, and 204 and ¶0055 disclose the imaging datasets being input into the machine learning model) are from the group comprising acquired imaging datasets (Zhang ¶0026 and Fig 1 disclose generating optical images by directing light to or scanning light over the specimen and detecting light from the specimen), design datasets (Zhang ¶0059 discloses the reference image is acquired from a database containing design data for the specimen), simulated datasets (Zhang ¶0076 discloses the image may include any known, simulated, measured, estimated, calculated, etc. parameters involved in the imaging process).
Regarding Claim 20, Zhang teaches the method of claim 1, wherein at least one imaging dataset (Zhang Fig 2, 200, 202, and 204 and ¶0055 disclose the imaging datasets being input into the machine learning model) comprises additional information on the likelihood of a defect being present within the imaging dataset (Zhang ¶0061 discloses including parameters that indicate which areas are to be inspected based on different parameters).
Regarding Claim 21, Zhang teaches a computer implemented method (Zhang ¶0010, ¶0012, ¶0014 discloses a computer implemented method for defect detection on a specimen) for training a machine learning model for defect detection (Zhang ¶0092-¶0094 discloses using images to train a machine learning model for detecting defects) of claim 11, the method comprising:
- providing training images of objects comprising integrated circuit patterns (Zhang ¶0094, ¶0312, discloses training images consisting of images from wafers), the training images comprising at least triplets of first imaging datasets, second imaging datasets and third imaging datasets (Zhang Fig 2, 200, 202, 204, ¶0071 discloses a first, a second, and third set of images being input into the model) including annotated defects (Zhang ¶0094, ¶0133-¶0135 discloses the training images having defect classes and the ground truth images being labeled to train the network); and
- training the machine learning model (Zhang ¶0092-¶0094 discloses using images to train a machine learning model for detecting defects) using the provided training images (Zhang ¶0094, ¶0312, discloses training images consisting of images from wafers) by minimizing a loss function configured for defect detection (Zhang ¶0080-¶0081, ¶00146, discloses optimizing a loss function for the machine learning model used for defect detection).
Regarding Claim 22, Zhang teaches a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method (Zhang ¶0182 discloses a non-transitory computer-readable medium storing program instructions executable on one or more computer systems for performing a computer-implemented method for detecting defects on a specimen and/or generating a reference image for a specimen) of claim 1.
Regarding Claim 23, Zhang teaches a computer-readable medium (Zhang ¶0041, ¶0050 disclose computer readable storage mediums), on which a computer program executable by a computing device is stored (Zhang ¶0040 discloses one or more processors, which executes instructions from a memory medium) , the computer program comprising code for executing the method (Zhang ¶0182 discloses a non-transitory computer-readable medium storing program instructions executable on one or more computer systems for performing a computer-implemented method for detecting defects on a specimen and/or generating a reference image for a specimen) of claim 1.
Regarding Claim 24, Zhang teaches a system for defect detection (Zhang ¶0001 discloses systems for learnable defect detection for semiconductor applications) comprises:
- an imaging device configured to provide one or more imaging dataset (Zhang ¶0024 discloses an imaging system that consists of an energy source and detector or an optical imaging systems to provide images of the wafer) of a portion of an object comprising integrated circuit patterns (Zhang ¶0150, ¶0153, discloses the specimen being imaged a wafer on which patterned features has been formed using multiple lithography exposure steps, which examiner understand to mean printed circuity board);
- one or more processing devices (Zhang ¶0040 discloses one or more processors); and
- one or more machine-readable hardware storage devices (Zhang ¶0041, ¶0050 disclose computer readable storage mediums) comprising instructions that are executable by the one or more processing devices (Zhang ¶0040 discloses one or more processors, which executes instructions from a memory medium) to apply the method for defect detection (Zhang ¶0010, ¶0012, ¶0014 discloses a computer implemented method for defect detection on a specimen) according to claim 1.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3, 10, and 15-16 are rejected under 35 U.S.C. 103 as unpatentable over Zhang et al (US Patent Publication 2023/0118839 A1, hereafter referred to as Zhang) in view of Prunella et al (M. Prunella et al., "Deep Learning for Automatic Vision-Based Recognition of Industrial Surface Defects: A Survey," in IEEE Access, vol. 11, pp. 43370-43423, 2023, hereafter referred to as Prunella).
Regarding Claim 3, Zhang teaches the method of claim 2, wherein at least one defect indicator (Zhang ¶0069 discloses the defect indicators being determined based on a measure of the similarity).
Zhang does not explicitly disclose comprises a pixel- wise map or a voxel-wise map.
Prunella is in the same field of image analysis using machine learning for defect detection. Further, Prunella teaches comprises a pixel- wise map or a voxel-wise map (Prunella Pg 25 Col 1 ¶01 discloses a pixel wise probability for normal texture and defective parts being mapped).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang by incorporating the specific visual representation of the pixels, the network outputting a probability of the defect and the specific network architecture of the network comprising three outputs and skip connection between layers as taught by Prunella to make an invention that can automatically communicate and predict the multiple types of defects in an image more efficiently using skip connections in the network structure; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need meet the growing demands in defect representation power enrichment, especially by transferring pre-trained layers to an optimized network and by explaining the network decisions to suggest trustworthy retention or rejection of the products being evaluated. (Prunella, Abstract).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 10, Zhang teaches the method of claim 1, wherein the processing (Zhang ¶0085-¶0087, and ¶0091 discloses joining the latent spaces for processing) from properties of the layout of the integrated circuit patterns (Zhang ¶0135, ¶0153 discloses that one of the characteristics of the pattern is a weak pattern which are more susceptible to defects) in the portion of the object (Zhang ¶0150, ¶0153, discloses the specimen being imaged a wafer on which patterned features has been formed using multiple lithography exposure steps, which examiner understand to mean printed circuity board).
Zhang does not explicitly disclose comprises evaluating the likelihood of defects.
Prunella is in the same field of image analysis using machine learning for defect detection. Further, Prunella teaches comprises evaluating the likelihood of defects (Prunella Pg 18 Col 1 ¶03 and Pg 19 Col 1 ¶01 and Pg 25 Col 1 ¶02 discloses a network that outputs the probability of the defect).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang by incorporating the specific visual representation of the pixels, the network outputting a probability of the defect and the specific network architecture of the network comprising three outputs and skip connection between layers as taught by Prunella to make an invention that can automatically communicate and predict the multiple types of defects in an image more efficiently using skip connections in the network structure; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need meet the growing demands in defect representation power enrichment, especially by transferring pre-trained layers to an optimized network and by explaining the network decisions to suggest trustworthy retention or rejection of the products being evaluated. (Prunella, Abstract).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 15, Zhang teaches the method of claim 14, wherein the neural network ( Zhang ¶0069 discloses the model being a neural network), wherein each defect indicator (Zhang ¶0069-¶0070 discloses using the defect indicator to determine if the pixel in the image location is defective or non-defective).
Zhang does not explicitly disclose comprises at least three output paths, is generated by a different output path.
Prunella is in the same field of image analysis using machine learning for defect detection. Further, Prunella teaches comprises at least three output paths (Prunella Pg 29 Col 2, ¶02 discloses a network with three different outputs), is generated by a different output path (Prunella Pg 29 Col 2, ¶02 discloses the network learns to output three different information regarding the defect: the center, the size, and the class).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang by incorporating the specific visual representation of the pixels, the network outputting a probability of the defect and the specific network architecture of the network comprising three outputs and skip connection between layers as taught by Prunella to make an invention that can automatically communicate and predict the multiple types of defects in an image more efficiently using skip connections in the network structure; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need meet the growing demands in defect representation power enrichment, especially by transferring pre-trained layers to an optimized network and by explaining the network decisions to suggest trustworthy retention or rejection of the products being evaluated. (Prunella, Abstract).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 16, Zhang in view of Prunella teaches the method of claim 15, wherein the neural network ( Zhang ¶0069 discloses the model being a neural network) comprises skip connections each configured to connect a layer of an input path with a layer of an output path (Prunella Pg 20 Col 1 ¶02 and Pg 14 Col 2 ¶02 discloses skip connections connecting layers) See Claim 15 for rationale, its parent claim.
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
J. Richter and D. Streitferdt, "Deep Learning Based Fault Correction in 3D Measurements of Printed Circuit Boards," 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), Vancouver, BC, Canada, 2018, pp. 227-232, doi: 10.1109/IEMCON.2018.8614932. discloses a method for using deep learning in determining defects in printed circuit boards.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL ROBERTS whose telephone number is (571)272-6413. The examiner can normally be reached Monday- Friday 7:30am- 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on (313) 446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL L ROBERTS/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights