DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 12-13 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by D1.1
With regard to claim 1, D1 teach receiving an unstained inference image, the inference image comprising a microscopic image capturing sperm cells (see abstract, fig. 1: sperm cell sample image captured by microscope); generating a virtual-stained image of sperm from the inference image using a trained generator machine learning model, the generator machine learning model taking the inference image as input, the generator machine learning model trained using a set of training images comprising microscopic images of sperm cells and a set of ground-truth images showing staining that identifies the sperm cells in the training images, the generator machine learning model trained by propagating determined losses between generated virtual-stained images and corresponding ground-truth images (see abstract, figs. 1, 2, 4, § 2 ¶ 2, § 2.1 ¶ 1: virtual staining using generative adversarial network trained using ground truth images); and outputting the generated virtual-stained image of sperm (see figs, 1, 2, 4, abstract, § 2.1 ¶ 1: outputting virtually stained image).
With regard to claim 2, D1 teach method of claim 1, wherein the staining that identifies the sperm cells in the ground-truth images comprises staining of only the sperm cells (see § 2 ¶ 2, § 2.1 ¶ 1, figs. 1: ground truth images comprise chemically stained sperm cells).
With regard to claims 12-13, see discussion of claims 1-2, respectively.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3-4, 7-8, 10-11, 14-15, 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over D1.
With regard to claim 3, D1 teach method of claim 1, the staining that identifies the sperm cells in the ground-truth images comprises at least see § 2.3 ¶¶ 1-2: acridine orange staining agent). D1 fails to explicitly teach using at least two different types of stains. However, Examiner takes Official Notice to the fact that it is extremely well known in the art that there are a plurality of different staining agents that can be used to stain sperm cells and one skilled in the art would have been motivated to incorporate other staining agents, yielding predictable results. The motivation would have been to enhance diagnosis or evaluation by revealing different features of the cells. Moreover, the generative adversarial network disclosed in D1 may be trained using training data from a different staining agent.
With regard to claim 4, D1 teach the method of claim 1, but fail to explicitly wherein the generator machine learning model is trained using a first set of training images comprising microscopic images of sperm cells collected at a first time-period post-coitus and a first set of ground-truth images showing staining that identifies the sperm cells in the first set of training images, and wherein the generator machine learning model is further trained using a second set of training images comprising microscopic images of sperm cells collected at a later time-period post-coitus and a second set of ground-truth images showing staining that identifies the sperm cells in the second set of training images. However, Examiner takes Official Notice to the fact that it is extremely well known that the structure of sperm cells changes over time and also experience DNA degradation. Because of these changes, it is clear that the staining of cells would also necessarily change. It would have been obvious to train the generative adversarial network disclosed in D1 using ground truth training data of chemically stained cells at different time intervals using the exact same training method described in D1, yielding predictable results. The motivation would have been to determine the age or time elapsed since collection of the sperm sample.
With regard to claim 14, see discussion of claim 3.
With regard to claim 15, see discussion of claim 4.
With regard to claim 7, D1 teach method of claim 1, but fail to explicitly teach further comprising determining a time interval between sperm deposition and sample collection of the sperm cells captured in the microscopic image using a second machine learning model, the second machine learning model takes as input the unstained inference image, one or more generated virtual-stained images of sperm, and the quantity of sperm cells in the one or more generated virtual-stained images, the second machine learning model trained using samples collected at known intervals post-coitus. However, Examiner takes Official Notice to the fact that it is extremely well known that the structure of sperm cells changes over time and also experience DNA degradation. Because of these changes, it is clear that the staining of cells would also necessarily change. It would have been obvious to train the generative adversarial network disclosed in D1 using ground truth training data of chemically stained cells at different time intervals using the exact same training method described in D1, yielding predictable results. The motivation would have been to determine the age or time elapsed since collection of the sperm sample.
With regard to claim 18, see discussion of claim 7.
With regard to claim 8, D1 teach method of claim 1, wherein the generator machine learning model generates
With regard to claim 10, D1 fails to explicitly teach method of claim 1, wherein the generated virtual-stained image of sperm comprises one or more of virtual HY-LITER fluorescent staining, virtual DAPI (4′,6-diamidino-2-phenylindole) fluorescent staining, virtual haematoxylin and eosin staining, and a virtual picroindigocarmine staining. However, Examiner takes Official Notice to the fact that these staining agents are extremely well known in the art before the effective filing date and one skilled in the art would have been motivated to incorporate known teachings into the configuration of D1 yielding predictable and enhanced results. The motivation for using different staining agents would have been to reveal different features of interest.
With regard to claim 11, D1 fails to explicitly teach method of claim 1, further comprising performing pre-processing on the unstained inference image, the pre-processing comprising dividing the inference image into tiles and providing each of the tiles as input to the generator machine learning model. However, Examiner takes Official Notice to the fact that using tiles as input to a generative machine learning model is extremely well known in the art before the effective filing date and one skilled in the art would have been motivated to incorporate known teachings into the configuration of D1 yielding predictable and enhanced results. The motivation for using tiles would have been to improve memory efficiency and computational efficiency.
With regard to claim 19, see discussion of claim 8.
Claims 5-6, 9, 16-17, 20 are rejected under 35 U.S.C. 103 as being unpatentable over D1 and further in view of D2.2
With regard to claim 5, D1 teach method of claim 1, further comprising determining locations of sperm cells in the generated virtual-stained image of sperm (see fig. 1, § 2.1 ¶ 1-2: detection and extraction of sperm cells in the image), but fail to explicitly teach by applying an intensity threshold across locations of the generated virtual-stained image of sperm. However, D2 teach the missing feature (see § 2.1: intensity thresholding). One skilled in the art would have found it obvious to combine the teachings to arrive at the claimed invention. In particular, it would have been obvious to substitute the extraction of cells as disclosed in D1 with the method described in D2, yielding predictable and enhanced results. The motivation for using simple thresholding such as Otsu method is that it is computationally fast and simple.
With regard to claim 6, D2 teach method of claim 5, further comprising determining a quantity of sperm cells in the generated virtual-stained image by determining contours around the locations having an intensity greater than the threshold, wherein each continuous region can be counted as a single sperm cell (see § 2.1: thresholding to detect and segment continuous cell region). D2 does not explicitly teach counting the cells, but one skilled in the art would have found it obvious to count the number of segmented regions. The motivation for combining the references is the same as stated above.
With regard to claim 9, D2 teach method of claim 8, further comprising determining locations of sperm cells in the generated virtual-stained image of sperm by applying an intensity threshold across locations of the generated virtual-stained image of sperm, wherein the locations of sperm cells being where two or more of the stains are above the intensity threshold (see § 2.1: thresholding to detect and segment continuous cell region).
With regard to claim 16, see discussion of claim 5.
With regard to claim 17, see discussion of claim 6.
With regard to claim 20, see discussion of claim 9.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AVINASH YENTRAPATI whose telephone number is (571)270-7982. The examiner can normally be reached on 8AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached on (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AVINASH YENTRAPATI/Primary Examiner, Art Unit 2672
1 Ben-Yehuda, Keren, et al. "Simultaneous morphology, motility, and fragmentation analysis of live individual sperm cells for male fertility evaluation." Advanced Intelligent Systems 4.4 (2022): 2100200.
2 Shaker, Fariba, S. Amirhassan Monadjemi, and Ahmad Reza Naghsh-Nilchi. "Automatic detection and segmentation of sperm head, acrosome and nucleus in microscopic images of human semen smears." Computer methods and programs in biomedicine 132 (2016): 11-20.
Prosecution Insights