DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation 112(f)
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
As to claims 1-7, the “detection unit” is considered to read on a computer with a processor for operating the detection process (Specification as filed: [0050]; PGPUB: [0146-0148]).
As to claims 1-7, the “reporting unit” is considered to read on a computer with a processor for operating the reporting process (Specification as filed: [0050]; PGPUB: [0146-0148]).
As to claims 1-7, the “storage unit” is considered to read on a computer with a storage device thereon (Specification as filed: [0050]; PGPUB: [0146-0148]).
As to claims 1-7, the “changing unit” is considered to read on a computer with a processor for operating the changing process (Specification as filed: [0050]; PGPUB: [0146-0148]).
As to claims 1-7, the “retraining unit” is considered to read on a computer with a processor for operating the retraining process (Specification as filed: [0050]; PGPUB: [0146-0148]).
As to claims 1-7, the “input unit” is considered to read on a computer with an input device (Specification as filed: [0050]; PGPUB: [0146-0148]).
As to claims 1-7, the “presentation unit” is considered to read on a computer with a display (Specification as filed: [0050]; PGPUB: [0146-0148]).
As to claims 1-7, the “collection unit” is considered to read on a computer with a processor for operating the collection process (Specification as filed: [0050]; PGPUB: [0146-0148]).
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 2, and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Ghanta et al., US 2019/0377984 A1 (Ghanta) and further in view of Mori, US 2022/0252857 A1 (Mori).
Regarding claim 1, Ghanta teaches an incompatibility detection device (machine learning (ML) management apparatus 104) (Fig. 1; [0038]) comprising:
a learning model (machine learning model) ([0038]) configured to perform and to be inputted with or to output an image (wherein the training data set is input; wherein the input can be images) ([0038-0039] and [0109-0110]);
an incompatibility detection unit configured to detect whether the learning model is incompatible (learning that the machine learning model used to analyze inference data set is not suitable for the inference training data) ([0038]);
an incompatibility reporting unit configured to report detected incompatibility (wherein when the ML apparatus detects that the machine learning model isn’t suitable, it can provide recommendations for generating a more accurate machine learning model, and/or the like) ([0038]) (an action module can also send an alert, message, notification, or the like) ([0115]); and
a storage unit configured to store (storage medium) ([0021-0023]), as a model-compatible region (the machine learning model being compatible if the suitability score satisfies a suitability threshold or doesn’t satisfy an unsuitability threshold) ([0038]), a distribution of evaluation values of training images used in a training stage of the learning model, in association with the learning model (the suitability score indicates how suitable, e.g., how applicable or accurate a machine learning model trained using the training data set is for the inference data set, the degree with which the inference data set deviates from the training data set, and/or the like) ([0080-0081]),
wherein the incompatibility detection unit determines that the learning model is incompatible when an evaluation value of the image is not within the model-compatible region (a suitability score that satisfies (is equal to or greater than) a suitability threshold indicates that the training data set and the machine learning model trained with the training data set are suitable, accurate, or the like for the inference data set; otherwise, the training data set may be unsuitable for the inference data set) ([0038] and [0086]).
Ghanta teaches that the models can be able to deal with high dimensional data such as images, and generate accurate predictive performance ([0109]). However, Ghanta does not explicitly teach to perform “an image conversion”.
Mori teaches an image processing system includes a microscope system that acquires an input image to be input to an image processing device, and the image processing device including a circuitry (Abstract); wherein the circuitry selects a learned model from a plurality of learned models that have learned image conversion that converts the input image into an output image having an image quality higher than an image quality of the input image, and performs the image conversion using the selected learned model (Abstract and [0030]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ghanta to include using the learned model of machine learning for image conversion since it can create an output image having higher image quality than the input image, and improvements in image quality such as noise reduction, resolution improvement, and aberration correction are achieved (Mori; [0030]).
Regarding claim 2, Ghanta teaches further comprising: a use model changing unit (part of the ML management apparatus) ([0038]), wherein when the incompatibility detection unit detects incompatibility (if the suitability score satisfies an unsuitability threshold; indicating the machine learning model isn’t suitable) ([0038]), the use model changing unit searches the storage unit for another learning model corresponding to the model-compatible region that is compatible with the evaluation value of the image (wherein the ML management apparatus may change the machine learning model) ([0038]) (swap the current machine learning model with a different machine learning model) ([0113] and [0115]), and uses the other learning model for processing of the image (using the new machine learning model) ([0112-0113]) (wherein the dataset can be images) ([0109-0110]).
Ghanta teaches that the models can be able to deal with high dimensional data such as images, and generate accurate predictive performance ([0109]). However, Ghanta does not explicitly teach to perform “an image conversion”.
Mori teaches an image processing system includes a microscope system that acquires an input image to be input to an image processing device, and the image processing device including a circuitry (Abstract); wherein the circuitry selects a learned model from a plurality of learned models that have learned image conversion that converts the input image into an output image having an image quality higher than an image quality of the input image, and performs the image conversion using the selected learned model (Abstract and [0030]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ghanta to include using the learned model of machine learning for image conversion since it can create an output image having higher image quality than the input image, and improvements in image quality such as noise reduction, resolution improvement, and aberration correction are achieved (Mori; [0030]).
Regarding claim 6, Ghanta teaches an incompatibility detection method (method for detecting suitability of machine learning models for datasets) ([0003]), wherein
an incompatibility detection device (machine learning (ML) management apparatus 104) (Fig. 1; [0038]) includes
a learning model (machine learning model) ([0038]) configured to perform and to be inputted with or to output an image (wherein the training data set is input; wherein the input can be images) ([0038-0039] and [0109-0110]);
an incompatibility detection unit configured to detect whether the learning model is incompatible (learning that the machine learning model used to analyze inference data set is not suitable for the inference training data) ([0038]);
an incompatibility reporting unit configured to report detected incompatibility (wherein when the ML apparatus detects that the machine learning model isn’t suitable, it can provide recommendations for generating a more accurate machine learning model, and/or the like) ([0038]) (an action module can also send an alert, message, notification, or the like) ([0115]); and
a storage unit configured to store (storage medium) ([0021-0023]), as a model-compatible region (the machine learning model being compatible if the suitability score satisfies a suitability threshold or doesn’t satisfy an unsuitability threshold) ([0038]), a distribution of evaluation values of training images used in a training stage of the learning model, in association with the learning model (the suitability score indicates how suitable, e.g., how applicable or accurate a machine learning model trained using the training data set is for the inference data set, the degree with which the inference data set deviates from the training data set, and/or the like) ([0080-0081]),
wherein the incompatibility detection unit determines that the learning model is incompatible when an evaluation value of the image is not within the model-compatible region (a suitability score that satisfies (is equal to or greater than) a suitability threshold indicates that the training data set and the machine learning model trained with the training data set are suitable, accurate, or the like for the inference data set; otherwise, the training data set may be unsuitable for the inference data set) ([0038] and [0086]).
Ghanta teaches that the models can be able to deal with high dimensional data such as images, and generate accurate predictive performance ([0109]). However, Ghanta does not explicitly teach to perform “an image conversion”.
Mori teaches an image processing system includes a microscope system that acquires an input image to be input to an image processing device, and the image processing device including a circuitry (Abstract); wherein the circuitry selects a learned model from a plurality of learned models that have learned image conversion that converts the input image into an output image having an image quality higher than an image quality of the input image, and performs the image conversion using the selected learned model (Abstract and [0030]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ghanta to include using the learned model of machine learning for image conversion since it can create an output image having higher image quality than the input image, and improvements in image quality such as noise reduction, resolution improvement, and aberration correction are achieved (Mori; [0030]).
Claim(s) 3-5 are rejected under 35 U.S.C. 103 as being unpatentable over Ghanta et al., US 2019/0377984 A1 (Ghanta), Mori, US 2022/0252857 A1 (Mori), and further in view of Ueta et al., US 2021/0264210 A1 (Ueta).
Regarding claim 3, Ghanta teaches further comprising: an existing model retraining unit (wherein the ML management apparatus also includes the ability to retrain the machine learning model) ([0038] and [0114-0115]), wherein when the incompatibility detection unit detects incompatibility (if the suitability score satisfies an unsuitability threshold; indicating the machine learning model isn’t suitable) ([0038]), the existing model retraining unit the model-compatible region of the learning model by retraining (retrain the machine learning model with different training data; for increasing the suitability score) ([0114-0115]), based on an additional training image (using a different training data set) ([0114]), the learning model which is incompatible (retraining the machine learning model that is unsuitable) ([0038] and [0114-0115]). Mori teaches an image processing system includes a microscope system that acquires an input image to be input to an image processing device, and the image processing device including a circuitry (Abstract); and wherein the circuitry selects a learned model from a plurality of learned models that have learned image conversion that converts the input image into an output image (Abstract and [0030]).
However, neither explicitly teaches “expands the model-compatible region of the learning model by retraining, based on an additional training image”.
Ueta teaches a collection of training data for image recognition, in order to support a reduction in collection of improper images which are not suitable as training data (Abstract); and wherein it expands the model-compatible region of the learning model by retraining, based on an additional training image (realizing if the training data is suitable as training data; and if not capturing a new image as a training image and thus expanding, based on the additional training image, the accuracy of the learning model) (Abstract and [0076]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of prior arts to include using an additional training image since it increases the accuracy of the model (Ueta; [0052] and [0076]).
Regarding claim 4, Ueta teaches further comprising: a countermeasure method presentation unit (notification operation which causes a display device to display a notification screen) ([0022]); and a retraining data input unit (re-training input into the model) ([0076]), wherein the countermeasure method presentation unit presents (notification operation which causes a display device to display a notification screen) ([0022]) an incompatibility countermeasure method (prompting the image capturing person to reshoot a new image) ([0022]) including an imaging method of the additional training image (wherein the notification prompts the user to acquire the new image with different shooting conditions) ([0076]), and the retraining data input unit receives an input of the additional training image captured by the presented imaging method (re-training the model based on the newly acquired image) ([0076]).
Regarding claim 5, Ueta teaches further comprising: a retraining data collection unit (re-training the model) ([0076]), wherein the retraining data collection unit (re-training the model) ([0076]) receives an input of the additional training image (re-training the model based on the newly acquired image) ([0076]) captured by operating an imaging device (the image capturing device can include a plurality of cameras) ([0049]) according to an operation procedure set in advance (according to the procedure of capturing an image) ([0076]).
Claim(s) 7 is rejected under 35 U.S.C. 103 as being unpatentable over Ghanta et al., US 2019/0377984 A1 (Ghanta), Mori, US 2022/0252857 A1 (Mori), and further in view of Grama et al., US 12,051,183 B2 (Grama).
Regarding claim 7, Ghanta teaches that the models can be able to deal with high dimensional data such as images, and generate accurate predictive performance ([0109]); wherein detecting the degree with which the inference data set deviates from the training data set ([0080]); and a distribution of evaluation values of training images used in a training stage of the learning model, in association with the learning model (the suitability score indicates how suitable, e.g., how applicable or accurate a machine learning model trained using the training data set is for the inference data set, the degree with which the inference data set deviates from the training data set, and/or the like) ([0080-0081]). Mori teaches wherein the image is an image (input image) ([0030]) and the evaluation value (image quality factor) ([0042-0045]) of the image is a an aberration (aberration correction) ([0030] and [0044]), or a noise removal amount obtained from the image (improvements in image quality such as noise reduction) ([0030] and [0042]).
However, neither explicitly teaches wherein the image is “a circuit pattern formed on a semiconductor wafer”.
Grama teaches training a machine learning (ML) model for generation higher resolution images (Abstract); wherein the image is of a circuit pattern (patterned features) (col. 5, lines 37-41) formed on a semiconductor wafer (formed in the area of a specimen) (col. 5, lines 37-41) (wherein the specimen can be a wafer know in the semiconductor arts) (col. 6, lines 50-51); wherein one or more computer subsystems may determine if one or more pre-trained ML model parameter(s) should be adjusted based on the comparison (col. 20, lines 58-60); and wherein if it is determined that one or more pre-trained ML model parameter(s) should be adjusted, the one or more computer subsystems may determine adjusted pre-trained ML model parameters, which may be determined based on any differences between the training outputs and the simulated outputs (col. 20, lines 60-66).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of prior arts to include using the machine learning on a semiconductor wafer since it increases the modality of the invention while also generating higher resolution images which enables improved sensitivity and helps classify defect types accurately (Grama; col. 6, lines 45-47).
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL J VANCHY JR whose telephone number is (571)270-1193. The examiner can normally be reached Monday - Friday 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL J VANCHY JR/Primary Examiner, Art Unit 2666 Michael.Vanchy@uspto.gov
Prosecution Insights