DETAILED ACTION
Response to Arguments
Applicant's arguments filed with respect to claims 1, 2, 4-10, and 12-16 have been fully considered but are moot in view of the new ground(s) of rejection. The rejections are necessitated due to claim amendments.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
In this case, the preamble states both “an estimated captured image” and “an actual captured image of the physical sample”. However, the input step subsequently recites “an image capturing condition of the captured image” and “pixel values of each pixel of the captured image”, without specifying which previously stated image is referenced by “the captured image” in the preamble.
It is also unclear whether the image capturing condition must correspond to the acquisition of the actual captured image or to the conditions assumed in estimating the captured image. The estimation step also does not clearly identify whether the probability density function represents attainable pixel values of the actual captured image under the mentioned process conditions or pixel values associated with the estimated captured image. These alternatives affect the required relationship between the process information and the calculated captured image statistics. Applicant should clarify the image referenced in each limitation and its relationship to the process information and probability density function.
Claim 9 is also rejected for the same reasons as claim 1 above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Aaron (“conditional Image Generation with PixelCNN Decoders”) in view of Bhaskar et al. (Pub. US 2017/0200265).
Regarding Claims 1 and 9, Aaron (“conditional Image Generation with PixelCNN Decoders”) teaches an image processing method for acquiring data of an estimated captured image obtained from reference data of a physical sample by using a system including an input acceptance unit, an estimation unit, and an output unit, the data being used when comparing the estimated captured image and an actual captured image of the sample, [Abstract] the image processing method comprising: an input step of accepting, by the input acceptance unit, input of the reference data, process information of the sample, and trained model data (it’s clear that trained PixelCNN has weights) [Abstract “The model can be conditioned on any vector, including descriptive labels or tags, or latent embeddings created by other networks [trained model data with weights]”; Section 2 paragraph 1 and 2]; an estimation step of calculating, by the estimation unit, captured image statistics which represent a probabilistic distribution of values that are attainable by data of the captured image by using the reference data, the process information, and the model data [Section 2 para. 1-3; 2.3, and 3.3]; and an output step of outputting, by the output unit, the captured image statistics, [Section 2 para. 1-3] wherein the estimated captured image is able to be generated from the captured image statistics [section “We show that a single Conditional PixelCNN model can be used to generate images from diverse classes such as dogs, lawn mowers and coral reefs, by simply conditioning on a one-hot encoding of the class”; Section 2 para. 4 “During sampling the predictions are sequential: every time a pixel is predicted, it is 2 fed back into the network to predict the next pixel”. Abstract “When conditioned on an embedding produced by a convolutional network given a single image of an unseen face, it generates a variety of new portraits of the same person with different facial expressions, poses and lighting conditions”].
However, Aaron doesn’t explicitly teach an input step of accepting, by the input acceptance unit, input of the reference data, process information including at least one of a manufacturing condition of the physical sample, and trained model data; an estimation step of calculating, by the estimation unit, captured image statistics as a probability density function which represent a probabilistic distribution of pixel values of each pixel of the captured image as values that are attainable by data of the captured image by using the reference data, the process information, and the model data; and the process information is combined with a feature of the captured image.
Bhaskar teaches an input step of accepting, by the input acceptance unit, input of the reference data (design data), process information (process parameters) including at least one of a manufacturing condition of the physical sample (parameter of a process performed on the specimen), and trained model data (trained model) [Para. 89, 94, 144-150]; an estimation step of calculating, by the estimation unit, captured image statistics (desired representation) as a probability density function (parametric probability density function) which represent a probabilistic distribution of pixel values of each pixel of the captured image (pixel value at (x, y) location) as values that are attainable by data of the captured image by using the reference data (design information), the process information (process parameters), and the model data (trained model) [Para. 89, 124, 127 (generic probability density function), 84 (predicting a representation), 89 (pixel value under process parameters), 144-150 (trained model prediction)]; and the process information (process conditions/parameters) is combined with (any combination thereof) a feature of the captured image (feature images/data) [Para. 97, 99, and 104].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Aaron’s condition PixelCNN by incorporating Bhaskar’s design data, process parameter, feature data, and trained model conditioning so that the per pixel image distribution reflects a specimen’s manufacturing state. This modification improves Aaron by generating specimen specific estimated images that account for actual process variation while retaining explicit pixel level probabilistic output.
Claims 2 and 10, are rejected under 35 U.S.C. 103 as being unpatentable over Aaron (“conditional Image Generation with PixelCNN Decoders”) in view of Bhaskar et al. (Pub. US 2017/0200265) further in view of Grove et al. (Patent No. US 10,163,061).
Regarding claims 2 and 10, Aaron in view of Bhashkar doesn’t explicitly teach the claim limitation.
However, Grove wherein the system further includes a machine learning unit and a storage unit, the image processing method further comprises a learning necessity (retraining) determination step of determining, by the machine learning unit, necessity of learning for the model data [Abstract Col. 2, lines 48-51, Col. 6 lines 1-10, 27-35], in a case where it is determined in the learning necessity determination step that the learning is necessary, input of a training dataset including the reference data, the process information, and the captured image for the learning is accepted, the captured image statistics and the data of the captured image of the training dataset are compared with each other, and the model data is updated based on a result of the comparison, [Col. 6 lines 5-17] in a case where it is determined in the learning necessity determination step that the learning is unnecessary, the storage unit stores, as the model data, a parameter used when the estimation unit calculates the captured image statistics [Abstract Col. 2, lines 48-51, Col. 6 lines 1-10, 27-35].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify to Aaron in view of Bhashkar to teach the claim limitation, feature as taught by Grove; because the modification enables the system improve the performance quality of the machine learning.
Claims 4, 6, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Aaron (“conditional Image Generation with PixelCNN Decoders”) in view of Bhaskar et al. (Pub. US 2017/0200265) further in view of Mitsui (Pub. No. US 2006/0261268).
Regarding claim 4, Aaron in view of Bhashkar doesn’t explicitly teach the claim limitation.
Mitsui teaches a step of evaluating an influence of the process information on the sample by using the captured image statistics [Para. 11, 18, and 22].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify to Aaron in view of Bhashkar to teach the claim limitation, feature as taught by Mitsui; because the modification enables the system to determine the location in the process where manufacturing defect happens.
Regarding claims 6 and 14, Aaron in view of Bhashkar doesn’t explicitly teach the claim limitation.
However, Mitsui teaches wherein the sample is a semiconductor circuit [Para. 11, 18, and 22].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify to Aaron in view of Bhashkar to teach the claim limitation, feature as taught by Mitsui; because the modification enables the system to determine the location in the process where manufacturing defect happens.
Claims 5 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Aaron (“conditional Image Generation with PixelCNN Decoders”) in view of Bhaskar et al. (Pub. US 2017/0200265) further in view of WINDMARK et al. (Pub. No. US 2018/0293460).
Regarding claims 5 and 13, Aaron in view of Bhashkar doesn’t explicitly teach the claim limitation.
However, WINDMARK teaches wherein the captured image statistics include a mean image and a standard deviation image [Para. 44 and 82].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify to Aaron in view of Bhashkar to teach the claim limitation, feature as taught by WINDMARK; because the modification enables the system to determine the location in the process where manufacturing defect happens.
Claims 7 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Aaron (“conditional Image Generation with PixelCNN Decoders”) in view of Bhaskar et al. (Pub. US 2017/0200265) further in view of Companion et al. (Patent No. US 6,330,354).
Regarding claims 7 and 15, Aaron in view of Bhashkar doesn’t explicitly teach the claim limitation.
However, Companion teaches the system further including a template image creation unit and a pattern matching processing unit, and the pattern inspection method comprising: accepting, by the input acceptance unit, input of the data of the captured image [Abstract, brief summary]; creating, by the template image creation unit, a template image from the captured image statistics [Abstract, Summary]; performing, by the pattern matching processing unit, pattern matching between the template image and the captured image (difference image) [Abstract, Col. 5 lines 1-49]; and outputting, by the output unit, a result of the pattern matching [Col. 8 lines 55-64, Col. 9 lines 40-45].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify to Aaron in view of Bhashkar to teach the claim limitation, feature as taught by Mitsui; because the modification enables the system to determine the location in the process where manufacturing defect happens.
Claims 8 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Aaron (“conditional Image Generation with PixelCNN Decoders”) in view of Bhaskar et al. (Pub. US 2017/0200265) further in view of Grove et al. (Patent No. US 10,163,061) further in view of Companion et al. (Patent No. US 6,330,354).
Regarding claims 8 and 16, Aaron in view of Bhashkar in view of Grove doesn’t explicitly teach the claim limitation.
However, Companion teaches the system further including a template image creation unit and a pattern matching processing unit, and the pattern inspection method comprising: accepting, by the input acceptance unit, input of the data of the captured image ) [Abstract, Col. 5 lines 1-49]; creating, by the template image creation unit, a template image from the captured image statistics; performing, by the pattern matching processing unit, pattern matching between the template image and the captured image [Abstract, brief summary] ; and outputting, by the output unit, a result of the pattern matching [Col. 8 lines 55-64, Col. 9 lines 40-45].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify to Aaron in view of Bhashkar to teach the claim limitation, feature as taught by Mitsui; because the modification enables the system to determine the location in the process where manufacturing defects happens.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Aaron (“conditional Image Generation with PixelCNN Decoders”) in view of Bhaskar et al. (Pub. US 2017/0200265) further in view of Black et al. (Patent No. US 9,785,752).
Regarding claim 12, Aaron doesn’t explicitly teach the claim limitation.
However, Black teaches wherein an influence of the process information on the sample is evaluated by using the captured image statistics [fig. 9, 10 and related description].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify to Aaron to teach the claim limitation, feature as taught by Black; because the modification provides a method for evaluating tissue image analysis feature distribution functions with the intent to stratify patient cohorts into two or more distinct categories of interest.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOLOMON G BEZUAYEHU whose telephone number is (571)270-7452. The examiner can normally be reached on Monday-Friday 10 AM-8 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 888-786-0101 (IN USA OR CANADA) or 571-272-4000.
/SOLOMON G BEZUAYEHU/
Primary Examiner, Art Unit 2666
