DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 6, 8 and 15 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by US 12,135,273 B2 to Blandin et al (hereinafter ‘Blandin’).
Regarding claim 1, Blandin discloses, an imaging system for automated sample analysis (Fig. 1), the system comprising: an imaging device configured to detect optical signals encoded with information associated with a sample, the imaging device comprising: an emitter comprising a light source configured to illuminate the sample (Column 4, lines 59-62, wherein a light source 11 is configured to emit a light wave 12, referred to as the incident light wave, that propagates in the direction of a sample 10, along a propagation axis Z); a sensor configured to detect the optical signals encoded with information associated with the sample (column 6, lines 22-27, wherein the image sensor 16 is able to form an image I.0 of the sample 10 in the detection plane P.0. In the example shown, the image sensor 16 is an image sensor comprising a matrix array of CCD or CMOS pixels. The detection plane P.0 preferably lies perpendicular to the propagation axis Z of the incident light wave 12); and a sample holder disposed between the light source and the sensor, the sample holder configured to hold the sample and allow light to pass through from the light source to the sensor (column 5, lines 33-34 and Fig. 1, wherein the sample 10 is, in this example, contained in a fluidic chamber 15, as being located between the light source 11 and the sensor 16); a processor operatively coupled with the sensor of imaging device and configured to receive data indicative of the detected optical signals (column 6, lines 59-60, wherein a processor 20, a microprocessor for example, is able to process each image I.0 acquired by the image sensor 16.) and generate a plurality of images of the sample based at least in part on the detected optical signals (column 8, lines 35-39, wherein FIGS. 3B and 3C, as generated images, show an observation image I.10 corresponding to the modulus and phase of one portion of a complex image A.10 reconstructed, in the plane of the sample, on the basis of the hologram shown in FIG. 3A, respectively); and a machine learning model configured to receive the plurality of images of the sample as input from the processor and determine an output indicative of one or more attributes of the sample based at least in part on the received plurality of images of the sample (column 10, lines 53-59, wherein the characterization may consist in a classification of the particle of interest 10.p,i among predetermined particle classes. The classification may assume a training phase has been carried out using one or more training samples containing known particles. The characterization may make use of deep-learning techniques).
Regarding claim 6, Blandin discloses wherein: the machine learning model is a deep learning model configured to determine the one or more attributes of the sample (column 10, lines 58-59, wherein the characterization may make use of deep-learning techniques)..
Regarding claim 8, Blandin discloses wherein: at least two of the imaging device, processor, and machine learning model are integrated on the same device (column 6, lines 59-64, wherein a processor 20, a microprocessor for example, is able to process each image I.0 acquired by the image sensor 16. In particular, the processor is a microprocessor connected to a programmable memory 22 in which is stored a sequence of instructions for performing the image-processing and computing operations described in this description.).
Regarding claim 15, please refer to the corresponding system claim 1 above for further teachings.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2, 3, 7, 9-11 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Blandin in view of US 11,347,046 B2 to Moore et al (hereinafter ‘Moore’).
Regarding claim 2, Blandin does not specifically disclose wherein: the machine learning model is trained at least in part using training images having a higher resolution than a resolution of the received plurality of images. Moore discloses receiving high or low resolution images by the ML (column 31, lines 39-45, wherein referring back to FIG. 1, in some embodiments, low resolution images, high or low resolution QPI's, as receiving high or low resolution images, and/or information obtained during post processing (e.g., feature extraction) may be evaluated for the desired criteria using machine learning algorithms, deep learning algorithms, or a combination thereof for computer-assisted evaluation (see block S154).). Blandin and Moore are combinable because they both disclose microscopy imaging. It would have been obvious to one of ordinary skill in the art to combine the receiving high or low resolution images by the ML, of Moore’s system with Blandin’s so super-resolution imaging of a sample is achieved without limitation of the quality of optical lenses (column 16, lines 34-36).
Regarding claim 3, in the combination of Blandin and Moore, Blandin further discloses wherein: the one or more attributes comprise at least one of: a presence of an inclusion in the sample, a size of an inclusion in the sample, a shape of an inclusion in the sample, a movement of an inclusion in the sample, and a number of an inclusion in the sample (column 5, line 20-26, wherein by to characterize, what is notably meant is: to determine the nature of a particle, i.e. to classify this particle among one or more predetermined classes; to determine the state of a particle, among one or more predetermined states; to estimate a size of a particle, or its shape, or its volume or any other geometric parameter;).
Regarding claim 7, Blandin does not specifically disclose the machine learning model is a first machine learning model configured to determine at least a first attribute of the one or more attributes of the sample; the system further comprises a second machine learning model configured to determine at least a second attribute of the one or more attributes of the sample; and the output includes a first determination from the first machine learning model and a second determination from the second machine learning model. Moore discloses one or more attributes of the sample being determined by one or more machine learning models (column 14, lines 19-28, and Fig. 1, wherein as shown in post-processing at block S152 and artificial intelligence, machine learning, or deep learning assessment at block S154 in FIG. 1, analysis of one or more images may include feature identification using one or more post-processing methods, one or more machine learning models, and/or deep learning models. Features that may be identified in one or more images of one or more samples may include, but not be limited to: cell count, nucleus, edges, groupings, clump size, spatial information, or a combination thereof). Blandin and Moore are combinable because they both disclose microscopy imaging. It would have been obvious to one of ordinary skill in the art to combine the one or more attributes of the sample being determined by one or more machine learning models, of Moore’s system with Blandin’s so that to evaluate or test data and images to inform a user of a desired sample characteristic (column 32, lines 20-23).
Regarding claim 9, Blandin does not specifically disclose wherein: the light source is a first light source; the emitter comprises a plurality of light sources arranged in an array; and the emitter is configured to vary at least one illumination condition of the plurality of light sources. Moore discloses the light source is a first light source; the emitter comprises a plurality of light sources arranged in an array; and the emitter is configured to vary at least one illumination condition of the plurality of light sources (column 12, lines 61-65, and column 15, lines 65 through column 16, line 9, wherein an array of light sources or diffracted/reflected light may be used for illuminating the sample from different incident angles and acquiring corresponding low-resolution images using a monochromatic camera, and wherein after autofocusing, a plurality of images of a sample may be obtained under a variety of predetermined lighting conditions for further evaluation/analysis. In some embodiments, images include frames of a video. In some embodiments, the total number (N) of the plurality of different illumination conditions is between 2 to 10, between 5 to 50, between 10 to 100, between 50 to 1000, or more than 1000). Blandin and Moore are combinable because they both disclose microscopy imaging. It would have been obvious to one of ordinary skill in the art to combine the light source is a first light source; the emitter comprises a plurality of light sources arranged in an array; and the emitter is configured to vary at least one illumination condition of the plurality of light sources, of Moore’s system with Blandin’s so that to obtain computationally derived super-resolution, quantitative phase images for microscopic evaluation (column 16, lines 11-13).
Regarding claim 10, in the combination of Blandin and Moore, Moore further discloses wherein: the optical signals detected by the imaging device are a first set of optical signals associated with a first illumination condition of the plurality of light sources; and the imaging device is further configured to detect a second set of optical signals associated with a second illumination condition of the plurality of light sources (column 15, line 65 through column 16, lines 5, wherein a plurality of images of a sample may be obtained under a variety of predetermined lighting conditions for further evaluation/analysis. In some embodiments, images include frames of a video. In some embodiments, the total number (N) of the plurality of different illumination conditions is between 2 to 10, between 5 to 50, between 10 to 100, between 50 to 1000, or more than 1000).
Regarding claim 11, in the combination of Blandin and Moore, Moore further discloses wherein: generating the plurality of images comprises generating a plurality of high-resolution images based at least in part on the first and second sets of optical signals (column 16, lines 10-14, wherein the plurality of images captured under multiple lighting conditions at block S144 in FIG. 1 may be used to obtain computationally derived super-resolution, quantitative phase images for microscopic evaluation).
Regarding claim 18, Blandin does not specifically disclose wherein determining the output indicative of one or more attributes of the sample using the machine learning model comprises: generating, using a first portion of the machine learning model, data indicative of the one or more attributes of the sample based on the plurality of images of the sample; and generating, using a second portion of the machine learning model, new data indicative of the one or more attributes of the sample based at least in part on the data generated by the first portion. Moore discloses generating, using a first portion of the machine learning model, data indicative of the one or more attributes of the sample based on the plurality of images of the sample; and generating, using a second portion of the machine learning model, new data indicative of the one or more attributes of the sample based at least in part on the data generated by the first portion (column 27, lines 55-65, wherein a machine learning or deep learning model, inherently as including the first and second portion of the model, (e.g., trained to detect adequacy based on a predetermined threshold) may be used to select one or more regions of interest based on a presence of one or more clusters, as the data indicative of the one or more attributes generated by the first position, and assess the adequacy of the biological sample, as the second data corresponding to the selected first data generated by the second portion. In other embodiments, a machine learning or deep learning model may be used to select one or more regions of interest based on a presence of one or more clusters, and computer vision may be used to classify the one or more regions of interest based on the adequacy in each region of interest). Blandin and Moore are combinable because they both disclose microscopy imaging. It would have been obvious to one of ordinary skill in the art to combine the generating, using a first and second portion of the machine learning model, the data indicative of the one or more attributes of the sample and new data indicative of the one or more attributes of the sample based at least in part on the data generated by the first portion, of Moore’s system with Blandin’s so that to classify the one or more regions of interest based on the adequacy in each region of interest (column 27, lines 63-65).
Regarding claim 19, Blandin does not specifically disclose wherein detecting the optical signals comprises: illuminating the sample with two or more illumination conditions of the imaging device; and detecting optical signals associated with each illumination condition of the two or more illumination conditions. Moore discloses illuminating the sample with two or more illumination conditions of the imaging device; and detecting optical signals associated with each illumination condition of the two or more illumination conditions (column 15, lines 65 through column 16, line 9, wherein after autofocusing, a plurality of images of a sample may be obtained under a variety of predetermined lighting conditions for further evaluation/analysis. In some embodiments, images include frames of a video. In some embodiments, the total number (N) of the plurality of different illumination conditions is between 2 to 10, between 5 to 50, between 10 to 100, between 50 to 1000, or more than 1000. In some embodiments, one or more light sources and/or one or more system operations may be controlled using an embedded computer system directly or through a Field Programmable Gate Array (FPGA) or other digital logic device). Blandin and Moore are combinable because they both disclose microscopy imaging. It would have been obvious to one of ordinary skill in the art to combine the illuminating the sample with two or more illumination conditions of the imaging device; and detecting optical signals associated with each illumination condition of the two or more illumination conditions, of Moore’s system with Blandin’s so that to so that to obtain computationally derived super-resolution, quantitative phase images for microscopic evaluation (column 16, lines 11-13).
Regarding claim 20, in the combination of Blandin and Moore, Moore further discloses wherein generating the plurality of images of the sample comprises: matching first features of the detected optical signals associated with a first illumination condition with second features of the detected optical signals associated with a second illumination condition; and generating the plurality of images of the sample based on the matched first features and second features (column 40, lines 15-19, wherein in any of the preceding embodiments, the method further includes selecting one or more of: a pattern of illumination, a frequency of illumination, a wavelength of illumination, or a combination thereof of the illumination source, as selecting a first and the second illumination conditions for generating images, based on one or more features of the biological sample, as the matching first and second features based on the combination of illumination).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Blandin in view of WO 2021144800 A1 to Cassuto et al (hereinafter ‘Cassuto’).
Regarding claim 12, Blandin does not specifically disclose wherein each of at least a subset of the plurality of images is associated with a respective depth of the sperm sample. Cassuto discloses each of at least a subset of the plurality of images is associated with a respective depth of the sperm sample (Para [0050], wherein the imagining device 106 comprises two or more separate devices configured to collect image data in 2D and/or 3D. For example, in some embodiments, the imaging device 106 is configured to obtain one or more of image data and depth image data of a semen sample). Blandin and Cassuto are combinable because they both disclose microscopy imaging. It would have been obvious to one of ordinary skill in the art to combine the each of at least a subset of the plurality of images is associated with a respective depth of the sperm sample, of Cassuto’s system with Blandin’s I order to provide for the location of the spermatozoon candidates (Para [0035]).
Claim 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Blandin in view of US 11,708,563 B2 to Wagner et al (hereinafter ‘Wagner’).
Regarding claim 13, Blandin does not specifically disclose wherein the emitter is configured to operate in a pulse mode and operation of the emitter is configured to be synchronized with operation of the sensor. Wagner discloses the emitter is configured to operate in a pulse mode and operation of the emitter is configured to be synchronized with operation of the sensor (column 41, lines 16-20, wherein central pulsed laser is run at a consistent pulse rate, the photodetector 1852 may be used to acquire the laser pulse signal timing and synchronize the optical scanning subsystem to the laser pulse timing). Blandin and Wagner are combinable because they both disclose microscopy imaging. It would have been obvious to one of ordinary skill in the art to combine the emitter is configured to operate in a pulse mode and operation of the emitter is configured to be synchronized with operation of the sensor, of Wagner’s system with Blandin’s so that alleviate the need for a separate electronic synchronization system and wiring (column 41, lines 21-22).
Regarding claim 14, in the combination of Blandin and Wagner, Wagner further discloses wherein the operation of the emitter is synchronized with the operation of the sensor by receiving, with the emitter, one or more signals from the sensor (column 35, lines 44-58, wherein after the light passes through the slit aperture 1410, a collimating lens 1412 re-collimates the spatially filtered light before it reaches dispersive element 1414. The dispersive element 1414 may be a reflective diffraction grating, transmissive diffraction grating, prism, or other component that disperses the wavelength components of the optical signal in a continuous manner. The wavelength-separated (and thus angle-separated) light is then incident on focusing lens 1416 that focuses the light onto detector array 1418. The detector array 1418 may be configured to detect individual wavelength and components, may be composed of individual detectors or elements within a larger detector array (for example a 2D detector array). A processor 1420 receives electrical signals from the detector array 1418 and produces a signal 1422 representative of the sample.)
Allowable Subject Matter
Claims 4, 5, 16 and 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter: the prior art or the prior art of record specifically, Blandin and Moore, does not disclose:
. . . . wherein: the machine learning model is an adversarial neural network comprising: a first portion, trained using the training images having the higher resolution, configured to generate data indicative of the one or more attributes of the sample based on the received plurality of images of the sample; and a second portion configured to generate new data indicative of the one or more attributes of the sample based at least in part on the data generated by the first portion; and the output is determined based at least on the data generated by the first portion and the new data generated by the second portion, of claim 4 combined with other features and elements of the claim;
. . . . wherein: the machine learning model is an adversarial neural network comprising: a first portion, trained using the training images having the higher resolution, configured to extract image features based on the received plurality of images of the sample; a second portion configured to generate data indicative of the one or more attributes of the sample based on the features received from the first portion; and a third portion configured to generate new data indicative of the one or more attributes of the sample based at least in part on the data generated by the first portion; and the output is determined based at least on the data generated by the second portion and the new data generated by the third portion, of claim 5 combined with other features and elements of the claim;
the prior art or the prior art of record specifically, Blandin and US 10,871,745 B2 to Ozcan et al, does not disclose:
Regarding claim 16, Blandin does not specifically disclose wherein generating the plurality of images of the sample comprises: determining first amplitude values and first phase values for the detected optical signals at a first plane of the sensor; determining second amplitude values and second phase values for the detected optical signals at a second plane of the sample based on the first amplitude values and first phase values; and updating the first amplitude values and first phase values at the first plane based on the second amplitude values and second phase values, of claim 16 combined with other features and elements of the claim;
Claim 17 depends from an allowable base claim and is thus allowable itself.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHERVIN K NAKHJAVAN whose telephone number is (571)272-5731. The examiner can normally be reached Monday-Friday 9:00-12:00 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sue Lefkowitz can be reached at (571)272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHERVIN K NAKHJAVAN/ Primary Examiner, Art Unit 2672
Prosecution Insights