DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/05/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 8 and 17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
It is somewhat unclear as the examiner interprets the claimed invention to take the captured image and the text prompt as input to the LLM simultaneously. Therefore, under that circumstance it would be impossible to also use the text prompt as a trigger for the capture of the image.
Clarification is requested.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-6, 10-15 and 20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Alshehri, Abdulelah S., et al. Multimodal Deep Learning for Scientific Imaging Interpretation. September 2023, https://doi.org/10.48550/arxiv.2309.12460 referred to as ALSHEHRI hereinafter.
Regarding Claim 1, BASEREF shows a system, comprising:
a processor that executes computer-executable components stored in a non-transitory computer-readable memory (Pages 9-10 describe implementation via software and trained/executed on computing hardware.), wherein the computer-executable components comprise:
an access component that accesses an image captured by a charged-particle microscope, wherein the image depicts a specimen (Introduction, pages 5-6, 9-10);
and a localization component that localizes one or more outlying instantiations of a structure of interest of the specimen (Page 19, Defect and Anomaly Detection talks about a particular focus on an area of interest [localization].), based on feeding both the image and a text prompt to a large language model (The Abstract describes text + image integration into an LLM for analysis.).
Regarding claim 2, BASEREF shows the limitations as per Claim 1 above, wherein the text prompt comprises one or more sentences or sentence fragments that semantically define or describe the one or more outlying instantiations (Pages 6-9, Table 2).
Regarding claim 3, BASEREF shows the limitations as per Claim 2 above, wherein the one or more sentences or sentence fragments semantically define or describe the one or more outlying instantiations in terms of their sizes with respect to other instantiations of the structure of interest (Page 8, Table 2 discusses prompts for size and anomalies [respect to other instantiations].).
Regarding claim 4, BASEREF shows the limitations as per Claim 2 above, wherein the one or more sentences or sentence fragments semantically define or describe the one or more outlying instantiations in terms of their shapes with respect to other instantiations of the structure of interest (Page 8, Table 2).
Regarding claim 5, BASEREF shows the limitations as per Claim 2 above, wherein the one or more sentences or sentence fragments semantically define or describe the one or more outlying instantiations in terms of their distances from other instantiations of the structure of interest (Page 8, Table 2 wherein queries into boundaries, homo- vs. hetero-geneity and arrangements all expressly speak to distances with respect to other similar items of interest.).
Regarding claim 6, BASEREF shows the limitations as per Claim 2 above, wherein the one or more sentences or sentence fragments semantically define or describe the one or more outlying instantiations in terms of their arrangements with respect to other instantiations of the structure of interest (Page 8, Table 2 discusses arrangements.).
Regarding Claim 10, BASEREF shows a computer-implemented method, comprising:
accessing, by a device operatively coupled to a processor, an image captured by a charged-particle microscope, wherein the image depicts a specimen (Pages 5-6, Data Extraction and Generation section discusses this feature.);
and localizing, by the device, one or more outlying instantiations of a structure of interest of the specimen, based on feeding both the image and a text prompt to a large language model (The Abstract describes text + image integration into an LLM for analysis. Page 19, Defect and Anomaly Detection talks about a particular focus on an area of interest [localization].).
Regarding claim 11, BASEREF shows the limitations as per Claim 10 above, wherein the text prompt comprises one or more sentences or sentence fragments that semantically define or describe the one or more outlying instantiations (Page 8, Table 2 represents this.).
Regarding claim 12, BASEREF shows the limitations as per Claim 11 above, wherein the one or more sentences or sentence fragments semantically define or describe the one or more outlying instantiations in terms of their sizes with respect to other instantiations of the structure of interest (Page 8, Table 2 represents this.).
Regarding claim 13, BASEREF shows the limitations as per Claim 11 above, wherein the one or more sentences or sentence fragments semantically define or describe the one or more outlying instantiations in terms of their shapes with respect to other instantiations of the structure of interest (Page 8, Table 2 represents this.).
Regarding claim 14, BASEREF shows the limitations as per Claim 11 above, wherein the one or more sentences or sentence fragments semantically define or describe the one or more outlying instantiations in terms of their distances from other instantiations of the structure of interest (Page 8, Table 2 wherein queries into boundaries, homo- vs. hetero-geneity and arragements all expressly speak to distances with respect to other similar items of interest.).
Regarding claim 15, BASEREF shows the limitations as per Claim 11 above, wherein the one or more sentences or sentence fragments semantically define or describe the one or more outlying instantiations in terms of their arrangements with respect to other instantiations of the structure of interest (Page 8, Table 2 discusses arrangements.).
Regarding claim 20, BASEREF shows the limitations as per Claim 19 above, wherein the outlying characteristic comprises a size, a shape, a separation distance or a layout arrangement associated with instantiations of the structure of interest (Page 8, Table 2 describes all of the above.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 19 is rejected under 35 U.S.C. 103 as being unpatentable over ALSHEHRI in view of Shen, Mingren, et al. "Multi defect detection and analysis of electron microscopy images with deep learning." Computational Materials Science, vol. 199, Aug. 2021. https://doi.org/10.1016/j.commatsci.2021.110576 referred to as SHEN hereinafter.
Regarding Claim 19, BASEREF shows a computer program product for facilitating text-conditioned anomaly detection for charged-particle microscopy images (Pages 9-10 describe implementation via software and trained/executed on computing hardware.), the computer program product comprising a non-transitory computer-readable memory having program instructions embodied therewith (Pages 9-10 describe implementation via software and trained/executed on computing hardware.), the program instructions executable by a processor to cause the processor to:
access a charged-particle microscope and a text prompt, wherein the charged-particle microscope is loaded with a specimen (Pages 5-6 and 9-10 discuss accessing a specimen and text prompts.), and wherein the text prompt describes an outlying characteristic that might be exhibited by various instantiations of a structure of interest of the specimen (Page 19, Defect and Anomaly Detection talks about a particular focus on an area of interest [localization].); and
cause, in response to receipt of the text prompt, the charged-particle microscope to scan the specimen, thereby yielding a scanned image of the specimen (Pages 8-10, wherein said prompts inherently require a scanning of the specimen in order to respond and analyze the localized ROI.);
However, BASEREF fails to but SHEN does specifically show generating one or more bounding boxes that circumscribe one or more instantiations of the structure of interest that exhibit the outlying characteristic, based on executing a large language model on both the scanned image and the text prompt (Page 14, FIG. 3).
Both ALSHEHRI and SHEN are analogous to that of the claimed invention in that they fall into the same field of endeavor.
Therefore, it would have been obvious to one possessing ordinary skill in the art before the effective filing date of the claimed invention to modify ALSHEHRI in the spirit of SHEN because in some cases, even with the most robust of systems available, manual checking of very sensitive anomalies must still be done by humans and bounding boxes allow for this to be more easily accomplished (Introduction, Pages 4-5).
Allowable Subject Matter
Claims 7, 9, 16 and 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
As allowable subject matter has been indicated, applicant's reply must either comply with all formal requirements or specifically traverse each requirement not complied with. See 37 CFR 1.111(b) and MPEP § 707.07(a).
Claims 8 and 17 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See Notice of References Cited (PTO-892) for additional references cited but not relied upon in this action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUSTIN W. RIDER whose telephone number is (571)270-1068. The examiner can normally be reached Monday-Friday, 7.00 am - 4.30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jamie J Atala can be reached at (571) 272-7384. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JUSTIN W. RIDER
Primary Patent Examiner
Art Unit 2486
/Justin W Rider/Primary Patent Examiner, Art Unit 2486
Prosecution Insights