DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
1- This office action is a response to an application filed on 5/27/2025, in which claims 1-20 are currently pending. The Application is a Continuation of 17493957, filed 10/05/2021, now U.S. Patent # 12332420, and claims foreign priority to 10 2020 126 522.0, filed 10/09/2020.
Information Disclosure Statement
2- The submitted information disclosure statement(s) (IDS) is(are) in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement(s) is(are) being considered by the examiner.
Specification
3- The specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which application may become aware in the specification.
Drawings
4- The drawings were received on 5/27/2025. These drawings are acceptable.
Claim Rejections - 35 USC § 112
5- The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
6- Claim 13 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
As to claim 13, which reads “… wherein a movement of the sample stage occurs…”, the underlined clauses appear to present antecedence issues.
Double Patenting
7- The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
8- Claim 1-3, 8-9, 18-19 are rejected on the ground of non-statutory obviousness-type double patenting as being unpatentable over claims 1, 3, 8 and 19 of Amthor et al. (U.S. Patent N. 12332420). Although the claims at issue are not identical, they are not patentably distinct from each other because the aforementioned claims of the instant application merely inconsequentially rephrase the scope of Wang1, reorder its claims and or limitations within the claims.
This is a non-provisional non-statutory double patenting rejection.
Claims of Application 19219210
Claim of Patent 12332420
A microscopy system comprising: a microscope with an overview camera for capturing at least one overview image of a sample carrier including at least one sample receptacle for receiving at least one sample fluid, the at least one sample fluid including a sample to be analyzed; and a computing device configured to instruct capturing a plurality of overview images after time intervals; wherein the computing device is configured to determine at least one sample image area of the at least one sample receptacle with the at least one sample fluid within each of the overview images; wherein the computing device comprises an evaluation program into which each determined sample image area is input and which is configured to determine a fluid state of the sample fluid in the at least one sample receptacle based on the sample image area, the fluid state being a sample receptacle fill level or another fluid state; wherein the computing device is configured for: determining a progression of the sample receptacle fill level or the another fluid state in the sample receptacle over time, or calculating a prediction regarding when the sample receptacle fill level will fall below a minimum fill level.
A computer-implemented method for evaluating overview images, comprising: receiving at least one overview image of a sample carrier including at least one sample receptacle for receiving at least one sample fluid, the at least one sample fluid including a sample to be analyzed; determining at least one sample image area of the at least one sample receptacle with the at least one sample fluid within the at least one overview image; and inputting each determined sample image area into an evaluation program, wherein the evaluation program determines a fluid state of the sample fluid based on the sample image area.
8. The method according to claim 3, further comprising: capturing a plurality of overview images after time intervals; determining the at least one sample image area of the at least one sample fluid in each of the overview images; and determining a progression of the fill level or of another fluid state over time, or calculating a prediction regarding when a fill level will fall below a minimum fill level.
1. A microscopy system comprising a microscope with an overview camera for capturing at least one overview image of a sample carrier comprising at least one sample receptacle for receiving at least one sample fluid; and a computing device configured to determine at least one sample image area of the at least one sample fluid within the at least one overview image; wherein the computing device comprises an evaluation program into which each determined sample image area is entered and which is configured to determine a fill level of the sample fluid based on the sample image area; wherein a maximum positioning speed of the sample stage is set as a function of the determined fill level in the at least one sample receptacle, wherein, in the case of sample receptacles with different fill levels, the maximum positioning speed is set as a function of a highest of the different fill levels.
3. A computer-implemented method for evaluating overview images, comprising an overview camera of a microscope capturing at least one overview image of a sample carrier designed to receive at least one sample fluid; a computing device determining at least one sample image area of the at least one sample fluid within the at least one overview image; and entering each determined sample image area into an evaluation program, wherein the evaluation program determines a fluid state of the sample fluid based on the sample image area; wherein the microscope comprises an upright microscope stand and an immersion objective, wherein the evaluation program determines a fill level of at least one sample receptacle of the sample carrier as the fluid state, and wherein a warning is output for the purposes of preventing potential spillage due to an immersion of the objective in the sample receptacle or due to a sample stage movement when the fill level exceeds a maximum value.
8. The method according to claim 3, wherein a plurality of overview images are captured after time intervals; wherein the at least one sample image area of the at least one sample fluid is respectively determined in these overview images; and wherein a progression of the fill level or of another fluid state over time is determined by means of the evaluation program, or wherein a prediction regarding when a fill level will fall below a minimum fill level is calculated by means of the evaluation program.
2. wherein the computing device is configured for: determining a respective sample receptacle fill level of the at least one sample fluid from each determined sample image area, using an evaluation model learned from training data, wherein the training data comprises annotated training images which show sample receptacles filled with sample fluid to different levels and for which sample receptacle fill levels are respectively specified as annotations; and outputting a warning when the determined sample receptacle fill level exceeds a maximum value or falls below a minimum value.
9. further comprising: determining a respective sample receptacle fill level of the at least one sample fluid from each determined sample image area, using an evaluation model learned from training data, wherein the training data comprises annotated training images which show sample receptacles filled with sample fluid to different levels and for which sample receptacle fill levels are respectively specified as annotations; and outputting a warning when the determined sample receptacle fill level exceeds a maximum value or falls below a minimum value.
18. wherein the microscope comprises an objective, wherein the sample carrier comprises a plurality of sample receptacles for receiving the at least one sample fluid, wherein the evaluation program determines a fill level of at least one of the sample receptacles as the fluid state, and outputting a warning for the purposes of preventing potential spillage due to the objective being brought into the sample receptacle if the fill level exceeds a maximum value.
Claim 3 here above or
19. A computer-implemented method for evaluating overview images, comprising: capturing at least one overview image of a sample carrier having at least one sample receptacle configured for receiving at least one sample fluid; determining at least one sample image area of the at least one sample fluid within the at least one overview image; and determining a respective sample receptacle fill level of the at least one sample fluid from each determined sample image area, using an evaluation model learned from training data, wherein the training data comprises annotated training images which show sample receptacles filled with sample fluid to different levels and for which sample receptacle fill levels are respectively specified as annotations; and outputting a warning for the purposes of preventing potential spillage when the determined sample receptacle fill level exceeds a maximum value.
19. wherein the sample carrier comprises at least one sample receptacle for receiving the at least one sample fluid, wherein a maximum positioning speed of the sample stage is set as a function of a determined fill level in the at least one sample receptacle, wherein, in the case of sample receptacles with different fill levels, the maximum positioning speed is set as a function of a highest of the different fill levels, and wherein a magnitude of a cross-section of the at least one sample receptacle is determined from the overview image and wherein the maximum positioning speed is also set as a function of the determined magnitude of the cross-section.
16. The method according to claim 3, wherein the sample carrier comprises at least one sample receptacle for receiving the at least one sample fluid, wherein a maximum positioning speed of the sample stage is set as a function of a determined fill level in the at least one sample receptacle, wherein, in the case of sample receptacles with different fill levels, the maximum positioning speed is set as a function of a highest of the different fill levels, and wherein a magnitude of a cross-section of the at least one sample receptacle is determined from the overview image and wherein the maximum positioning speed is also set as a function of the determined magnitude of the cross-section.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status (MPEP 706.02(m)).
9- The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
In addition, the functional recitation in the claims (e.g. "configured to" or "adapted to" or the like) that does not limit a claim limitation to a particular structure does not limit the scope of the claim. It has been held that the recitation that an element is "adapted to", "configured to", "designed to", or "operable to" perform a function is not a positive limitation but only requires the ability to so perform and may not constitute a limitation in a patentable sense. In re Hutchinson, 69 USPQ 139. (See MPEP 2111.04); see also In In re Giannelli, 739 F.3d 1375, 1378, 109 USPQ2d 1333, 1336 (Fed. Cir. 2014).
Also, it should be noted that it has been held that a recitation with respect to the manner in which a claimed device is intended to be employed does not differentiate the claimed device from a prior art apparatus satisfying the claimed structural limitations Ex-parte Masham 2 USPQ2d 1647 1987).
The claimed system in the instant application is capable of performing the claimed functionality, as is the prior art used in the present office action. The Examiner notes that where the patent office has reason to believe that a functional limitation asserted to be critical for establishing novelty in the claimed subject matter may, in fact, be an inherent characteristic of the prior art, it possesses the authority to require the applicant to prove that the subject matter shown to be in the prior art does not possess the characteristic relied on. In re Swinehart and sfiligoj, 169 USPQ 226 (C.C.P.A. 1971).
10- Claims 1, 3-8, 11-17 and 20 are rejected under AIA 35 U.S.C. 102(a)(1) as being anticipated by Haase et al. (DE 102017111718, cited by Applicants)
As to amended claims 1, 3 and 20, Haase teaches a microscopy system, its method of use and its NTCRM (Abstract, Figs. 1-12), comprising: a microscope with an overview camera for capturing at least one overview image of a sample carrier including at least one sample receptacle for receiving at least one sample fluid, the at least one sample fluid including a sample to be analyzed (Figs. 1-2, 10-12; ¶ 2-3, 11, 50-55 for ex; a microscope camera 2/4 acquiring overview image areas 14/15 or under 21 or 25 of a sample carrier 1/20, 22 or 24 with wells 23 containing a sample fluid); and a computing device (5/6/7) configured to instruct capturing a plurality of overview images after time intervals (Abstract, ¶ 18-22, 58); wherein the computing device is configured to determine at least one sample image area of the at least one sample receptacle with the at least one sample fluid within each of the overview images (¶ 2-3, 76-77; image areas 14/15 or under 21 or 25 of a sample carrier 1/20, 22 or 24 with wells 23 containing a sample fluid); wherein the computing device comprises an evaluation program into which each determined sample image area is input and which is configured to determine a fluid state of the sample fluid in the at least one sample receptacle based on the sample image area, the fluid state being a sample receptacle fill level or another fluid state (¶ 58 a state of the sample is determined such as presence of sample, its contamination, bubble content in addition to possible QR annotations. ¶ 37, 58, 82-83; liquid volume, position and shape in the wells, i.e. fill level is considered); wherein the computing device is configured for: determining a progression of the sample receptacle fill level or the another fluid state in the sample receptacle over time, or calculating a prediction regarding when the sample receptacle fill level will fall below a minimum fill level (¶ 19 in combination with ¶ 37, 58, 82-83).
(Claim 4) wherein the sample carrier comprises a plurality of sample receptacles for receiving a respective sample fluid; wherein a plurality of sample image areas depicting the sample receptacles are determined within the same overview image (Figs. 1-2, 10-12; ¶ 2-3, 11, 50-55 for ex; a microscope camera 2/4 acquiring overview image areas 14/15 or under 21 or 25 of a sample carrier 1/20, 22 or 24 with wells 23 containing a sample fluid); wherein a fluid state of the sample fluid is respectively determined for each of the sample image areas (¶ 58 a state of the sample is determined such as presence of sample, its contamination, bubble content in addition to possible QR annotations. ¶ 37, 58, 82-83; liquid volume, position and shape in the wells, i.e. fill level is considered); wherein the computing device is configured for: determining a progression of the sample receptacle fill level or the another fluid state in the sample receptacle over time, or calculating a prediction regarding when the sample receptacle fill level will fall below a minimum fill level (¶ 19 in combination with ¶ 37, 58, 82-83).
(Claims 5-7, 16) wherein the evaluation program comprises an evaluation model learned using training data, wherein the training data comprises annotated training images, which are overview images or sample image areas of overview images for which a fluid state is respectively specified in the form of an annotation; (Claim 6) wherein the determined fluid state indicates a fill level relating to the associated sample fluid; and wherein the training data of the evaluation model shows sample receptacles filled to different levels and fill levels are respectively indicated in the form of annotations; (Claim 7) wherein the evaluation program comprises an evaluation model for anomaly detection learned through unsupervised learning using training data; wherein the evaluation model infers an anomalous fluid state as a function of a similarity of the input sample image area with the training data; (Claim 16) wherein a localization model learned using training data determines the at least one sample image area in the overview image; wherein all sample image areas determined by the localization model are entered into the evaluation program (CNN deep machine learning is used ¶ 37-46, 58, 78, 82-83, with annotation being considered and/or position and displacement, or anomalies are being detected such as bubble presence; ¶ 58. In ¶ 33-36, 76-77 for ex ; the positions of the fluid sample or solids in the fluid sample are calculated and/or calibrated with respect to the carrier position in a calibration step).
(Claim 8) further comprising: capturing a plurality of overview images after time intervals; determining the at least one sample image area of the at least one sample fluid in each of the overview images (see rejection of claim 3); and determining a progression of the fill level or of another fluid state over time, or calculating a prediction regarding when a fill level will fall below a minimum fill level (¶ 19 in combination with ¶ 37, 58, 82-83).
(Claim 10) wherein a plurality of overview images are captured by differently positioned overview cameras or a plurality of overview images are captured between movements of a sample stage, wherein a sample image area relating to the sample fluid is respectively determined in the plurality of overview images, and wherein the plurality of sample image areas relating to the sample fluid from different overview images are collectively fed to the evaluation program in order to determine the fluid state (¶ 33-36; moving a stage with the calculations taught in the rejection of claims 5-6).
(Claim 11) wherein the evaluation program for determining the fluid state takes into account at least one of the following features:- a color intensity in the sample image area (¶ 17, 20-24; different colors or polarizations is used to perform the measurement of the fluid state, in association with for the contamination detection in ¶ 58);- how image content in the sample image area appears distorted or altered in size or position due to a lens effect caused by a fluid surface of the sample fluid;- whether drops are discernible on a side wall or cover of an associated sample receptacle in the sample image area;- whether a color or color distribution within the sample image area deviates from an expected sample color or sample color distribution, based on which a contamination of the sample can be inferred (¶ 17, 20-24; different colors or polarizations is used to perform the measurement of the fluid state, in association with for the contamination detection in ¶ 58. ¶ 7; contaminants to be detected); - whether there are crystals in the sample image area, which the evaluation program uses as an indicator of a desiccation of the sample fluid;- fluid surfaces or reflections on a fluid surface in the sample image area (¶ 35, 81);- a lens effect of a surface of the sample fluid, wherein the evaluation program assesses a sample image area on the basis of how distorted or altered in size a background visible through the sample fluid appears or how large a sample receptacle visible through the sample fluid appears.
(Claim 12) wherein each of the at least one sample image area shows a sample receptacle, wherein the evaluation program infers a fill level or whether there is any sample fluid at all in the sample receptacle as a function of detected drops or condensate in the sample image area which shows the sample receptacle (¶ 37, 58, 82-83; liquid volume, position and shape in the wells, i.e. fill level is considered).
(Claim 13) wherein a movement of the sample stage occurs in conjunction with a capture of a plurality of overview images, in which the at least one sample image area is respectively determined (¶ 2-3, 33-36, 76-77 for ex. in addition to the above indications), and wherein the evaluation program takes into account changes in appearance or lens effects in the sample image area brought about by the movement of the sample stage in order to determine the fluid state; or wherein the evaluation program takes into account a movement of the sample or an inertia of the sample in surrounding fluid in view of a movement of the sample stage in order to determine the fluid state (¶ 76-77; using the Homography techniques, see definition in Conclusion, fluid surface changes, due to movements of the sample carrier/fluid, are measured).
(Claims 14-15) wherein a plurality of overview images are captured with different lighting and a difference between corresponding sample image areas in the plurality of overview images is assessed by the evaluation program in order to determine the fluid state; (Claim 15) wherein a plurality of overview images are captured in succession and the evaluation program determines, via a color change in the sample image area, a contamination as the fluid state (¶ 17, 20-24; different colors or polarizations is used to perform the measurement of the fluid state, in association with for the contamination detection in ¶ 58).
(Claim 17) wherein the sample carrier comprises a plurality of sample receptacles for receiving the at least one sample fluid; determining which of the sample receptacles are positioned for and subjected to further analysis as a function of the determined fluid state (Figs. 10-12, the receptacles under coverslips are considered; ¶ 76).
Allowable Subject Matter
11- Claims 2, 9, 18-19 would be allowable if rewritten to overcome the rejection(s) under the double patenting rationale as set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is an examiner's statement of reasons for the possibly allowable matter:
As to apparatus and method claims 2, 9, 18-19, the prior art of record, taken either alone or in combination, fails to disclose or render obvious:
(Claim 2) wherein the computing device is configured for: determining a respective sample receptacle fill level of the at least one sample fluid from each determined sample image area, using an evaluation model learned from training data, wherein the training data comprises annotated training images which show sample receptacles filled with sample fluid to different levels and for which sample receptacle fill levels are respectively specified as annotations; and outputting a warning when the determined sample receptacle fill level exceeds a maximum value or falls below a minimum value.
(claim 9) further comprising: determining a respective sample receptacle fill level of the at least one sample fluid from each determined sample image area, using an evaluation model learned from training data, wherein the training data comprises annotated training images which show sample receptacles filled with sample fluid to different levels and for which sample receptacle fill levels are respectively specified as annotations; and outputting a warning when the determined sample receptacle fill level exceeds a maximum value or falls below a minimum value.
(Claim 18) wherein the microscope comprises an objective, wherein the sample carrier comprises a plurality of sample receptacles for receiving the at least one sample fluid, wherein the evaluation program determines a fill level of at least one of the sample receptacles as the fluid state, and outputting a warning for the purposes of preventing potential spillage due to the objective being brought into the sample receptacle if the fill level exceeds a maximum value.
(Claim 19) wherein the sample carrier comprises at least one sample receptacle for receiving the at least one sample fluid, wherein a maximum positioning speed of the sample stage is set as a function of a determined fill level in the at least one sample receptacle, wherein, in the case of sample receptacles with different fill levels, the maximum positioning speed is set as a function of a highest of the different fill levels, and wherein a magnitude of a cross-section of the at least one sample receptacle is determined from the overview image and wherein the maximum positioning speed is also set as a function of the determined magnitude of the cross-section.
in combination with the other limitations.
The closest prior art found that pertains to the invention, with emphasis added, is Haase, Chen and Watanabe. However, the prior art fail to teach, suggest or render obvious the entire invention as claimed.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled "Comments on Statement of Reasons for Allowance."
Relevant Prior art
US 20210160434, US 20220409078 appear relevant to the use of alerts or warnings when special conditions are met in an optical measurements of samples.
US 20050090961, US 20140210980 appear to teach most the elements of the claimed apparatus and its method of use but fail the claimed invention as a whole
Conclusion
PNG
media_image1.png
964
966
media_image1.png
Greyscale
Downloaded from :
https://www.nasa.gov/earth-science-at-ames/what-we-do/las-laboratory-for-advance-sensing/las-fluid-lensing-seeing-through-waves/#:~:text=The%20algorithm%20introduces%20a%20fluid,a%20resolution%20less%20than%2025cm. (which appears relevant to the independent claims)
The examiner has pointed out particular references contained in the prior art of record in the body of this action for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. Applicant should consider the entire prior art as applicable as to the limitations of the claims. It is respectfully requested from the applicant, in preparing the response, to consider fully the entire references as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMED K AMARA whose telephone number is (571)272-7847. The examiner can normally be reached on Monday-Friday: 9:00-17:00
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tarifur Chowdhury can be reached on (571-272-2287. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Mohamed K AMARA/
Primary Examiner, Art Unit 2877
