DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the response to this Office Action, the Examiner respectfully requests that support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line numbers in the specification and/or drawing figure(s). This will assist the Examiner in prosecuting this application.
Election/Restrictions
Applicant's election without traverse of Species II: Figure 4 in the reply filed on 11/10/2025 is acknowledged.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-19 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2022/0198657 A1 to Hall et al. (hereinafter "Hall") in view of U.S. Patent 10,510,143 B1 to Zhou et al. (hereinafter "Zhou").
Regarding Claim 1, Hall teaches a computer-implemented method for predicting viability of an embryo (Claim 1; Para. 82-99 of Hall; method for computationally generating an Artificial Intelligence (AI) model configured to estimate an embryo viability score from an image), the method comprising: receiving a single image over a real-time communication link with an image capturing device (Claim 1; Para. 82-99. 258 of Hall; Artificial Intelligence (AI) model 100 configured to estimate an embryo viability score from a single image of an embryo… Images may be captured using a conventional optical microscope fitted with a camera or image sensor… CPU may comprise an Input/Output Interface, an Arithmetic and Logic Unit (ALU) and a Control Unit and Program Counter element which is in communication with input and output devices through the Input/Output Interface); cropping the single image to a boundary of the embryo via a first convolutional neural network (Claim 1; Para. 82-99 of Hall; images are then pre-processed 102, with the pre-processing including segmenting the image to identify a Zona Pellucida region of the image. The segmentation may also include identification of the IntraZonal Cavity (IZC) which is surrounded by the Zona Pellucida region. Pre-processing an image may also involve one or more (or all) of object detection, alpha channel removal, padding, cropping/localising); generating a viability score for the embryo (Para. 82-99 of Hall; Artificial Intelligence (AI) model 100 configured to estimate an embryo viability score from a single image of an embryo).
Hall does not explicitly disclose generating a viability score for the embryo by classifying the cropped single image via at least a second convolutional neural network.
However, Zhou teaches generating a viability score for an embryo by classifying an image via at least a second convolutional neural network (Figs. 11-12; Col. 29-30 of Zhou; the machine learning module can generate a viability predication, with can include a prediction of euploidy or aneuploidy, a prediction of the likelihood of the embryo implanting, or the like… classifiers can be arranged into multiple levels, and in some embodiments, each of these multiple levels can include one or several classifiers. In some embodiments, a refining algorithm can be applied to the output of the last image classifier to further refine the classification of the image… the outputted viability prediction can include the outputting of one or several categories into which one or several imaged embryos have been placed, the outputting of a ranking of one or several embryos relative to each other, which ranking can identify the embryos that are most likely to be viable or least likely to be viable).
Therefore, at the time when the invention was filed, it would have been obvious to a person of ordinary skill in the art to include generating a viability score for the embryo by classifying the cropped single image via at least a second convolutional neural network using the teachings of Zhou in order to modify the method taught by Hall. The motivation to combine these analogous arts would have been for generating an embryo mask applied to a series of time-lapse images of human embryos generated with an imaging system (Col. 2 of Zhou).
Regarding Claim 2, the combination of Hall and Zhou teaches that the single image is not part of a time series of images (Para. 82-99 of Hall; Artificial Intelligence (AI) model 100 configured to estimate an embryo viability score from a single image of an embryo).
Regarding Claim 3, the combination of Hall and Zhou teaches that generating the viability score for the embryo is performed in response to determining that the single image depicts an embryo (Para. 82-99 of Hall; cloud based computation system 1 configured to computationally generate and use an Artificial Intelligence (AI) model 100 configured to estimate an embryo viability score from a single image of an embryo).
Regarding Claim 4, the combination of Hall and Zhou teaches that in response to determining that the single image does not depict an embryo, providing an alert to a user of the image capturing device (Para. 82-99 of Hall; Images may also undergo quality assessment, to allow rejection of clearly poor images and allow capture of replacement images… training servers 37 manage the training process. This may include may dividing the images in to training, validation, and blind validation sets… cloud-based delivery platform 30 system then allows users 10 to drag and drop images directly onto the web application 34, which prepares the image and passes the image to the trained/validated AI model 100 to obtain an embryo viability score which is immediately returned in a report… Modifying the method taught by the combination of Hall and Zhou to include determining that the single image does not depict an embryo, providing an alert to a user of the image capturing device would require routine skill for a person having ordinary skill in the art at the time when the invention was effectively filed and can be accomplished without any undue experimentation. In addition, according to MPEP §2144, it is proper to take into account not only specific teachings of the reference but also the inferences which one skilled in the art would reasonably be expected to draw therefrom. In re Preda, 401 F.2d 825, 826, 159 USPQ 342, 344 (CCPA 1968)).
Regarding Claim 5, the combination of Hall and Zhou teaches determining a probability that the embryo is a single blastocyst (Fig. 3, Col. 11 of Zhou; software in the selected channel and/or the computer 125 analyzes the captured images and measures predictive parameters to provide a prediction of which embryos will reach blastocyst).
Regarding Claim 6, the combination of Hall and Zhou teaches that the real-time communication link is provided by an application executed on a computing device communicably coupled to the image capturing device (Para. 91, 268 of Hall; Images may be captured using a conventional optical microscope fitted with a camera or image sensor… CPU may comprise an Input/Output Interface, an Arithmetic and Logic Unit (ALU) and a Control Unit and Program Counter element which is in communication with input and output devices through the Input/Output Interface).
Regarding Claim 7, the combination of Hall and Zhou teaches that the application causes a display on the computing device to display a capture button (Fig. 1; Cols. 10-11 of Zhou; touch-screen panels 130-140 may be configured to enable users to control the operation of the imaging systems 110-120 with an easy-to-use graphical user interface (“GUI”). In one embodiment, multiple imaging systems, e.g., the systems 110-120, may be controlled from a single touch-screen panel, and multiple touch-screen panels may be controlled from a single computer, e.g., the computer 125).
Regarding Claim 8, the combination of Hall and Zhou teaches that in response to a user selecting the capture button, the image capturing device captures the first single image of the embryo (Fig. 1; Cols. 10-11 of Zhou; touch-screen panels 130-140 may be configured to enable users to control the operation of the imaging systems 110-120 with an easy-to-use graphical user interface (“GUI”). In one embodiment, multiple imaging systems, e.g., the systems 110-120, may be controlled from a single touch-screen panel, and multiple touch-screen panels may be controlled from a single computer, e.g., the computer 125… Para. 82 of Hall; plurality of images and associated metadata is received (or obtained) from one or more data sources 101. Each image is captured during a pre-determined time window).
Regarding Claim 9, the combination of Hall and Zhou teaches that the viability score represents a likelihood of the embryo reaching clinical pregnancy (Para. 82-99 of Hall; input 10 comprises data such as the images of the embryo and pregnancy outcome information (e.g. heart beat detected at first ultrasound scan post IVF, live birth or not, or successful implantation) which can be used to generate a viability classification).
Regarding Claim 10, the combination of Hall and Zhou teaches that the viability score represents a likelihood of the embryo reaching live birth (Para. 82-99, 198, 233, Table 7-8 of Hall; Monash IVF provided the ensemble model with approximately 10,000 embryo images and related pregnancy and live birth data for each image. Additional data provided included patient age, BMI, whether the embryo was implanted fresh or was frozen prior, and any fertility related medical conditions).
Regarding Claim 11, the combination of Hall and Zhou teaches that the likelihood of the embryo reaching clinical pregnancy is associated with an outcome of a fetal cardiac activity (Abstract, Claim 8; Para. 82-99 of Hall; pregnancy outcome data, such as detection (or not) of a heartbeat in the first ultrasound scan after implantation… ground-truth pregnancy outcome measurement is whether a fetal heartbeat is detected… Para. 198, 233, Table 8 of Hall; Monash IVF provided the ensemble model with approximately 10,000 embryo images and related pregnancy and live birth data for each image. Additional data provided included patient age, BMI, whether the embryo was implanted fresh or was frozen prior, and any fertility related medical conditions).
Regarding Claim 12, the combination of Hall and Zhou teaches that the viability score is based at least in part on data associated with a patient (Para. 82-99, 198, 233, Table 8 of Hall; Monash IVF provided the ensemble model with approximately 10,000 embryo images and related pregnancy and live birth data for each image. Additional data provided included patient age, BMI, whether the embryo was implanted fresh or was frozen prior, and any fertility related medical conditions).
Regarding Claim 13, the combination of Hall and Zhou teaches that the data includes at least one of age, body mass index, day of image capture, and donor status (Para. 82-99, 198, 233, Table 8 of Hall; Monash IVF provided the ensemble model with approximately 10,000 embryo images and related pregnancy and live birth data for each image. Additional data provided included patient age, BMI, whether the embryo was implanted fresh or was frozen prior, and any fertility related medical conditions. Data for some of the images contained the embryologist's score for the viability of the embryo).
Regarding Claim 14, the combination of Hall and Zhou teaches storing the viability score in a database (Para. 82-99, 198 of Hall; Monash IVF provided the ensemble model with approximately 10,000 embryo images and related pregnancy and live birth data for each image. Additional data provided included patient age, BMI, whether the embryo was implanted fresh or was frozen prior, and any fertility related medical conditions. Data for some of the images contained the embryologist's score for the viability of the embryo).
Regarding Claim 15, the combination of Hall and Zhou teaches communicating the viability score to at least one of a patient and a clinician (Figs. 11-12; Col. 29-30 of Zhou; viability prediction can be provided to the user via, for example, the dashboard that can be accessible to the user via the input/output module 1018).
Regarding Claim 16, the combination of Hall and Zhou teaches predicting, via a fourth convolutional neural network, whether the embryo is euploid or aneuploid (Figs. 11-12; Col. 29-30 of Zhou; the machine learning module can generate a viability predication, with can include a prediction of euploidy or aneuploidy, a prediction of the likelihood of the embryo implanting, or the like… classifiers can be arranged into multiple levels, and in some embodiments, each of these multiple levels can include one or several classifiers. In some embodiments, a refining algorithm can be applied to the output of the last image classifier to further refine the classification of the image).
Regarding Claim 17, the combination of Hall and Zhou teaches that predicting whether the embryo is euploid or aneuploid depends at least in part on data associated with a subject (Figs. 11-12; Col. 30, ln.15-26 of Zhou; viability prediction can be based on the inputted image features and one or several other parameters such as, for example, age of the human source of the egg at the time of egg harvesting).
Regarding Claim 18, the combination of Hall and Zhou teaches that the data is at least one of age and day of biopsy (Figs. 11-12; Col. 30, ln.15-26 of Zhou; viability prediction can be based on the inputted image features and one or several other parameters such as, for example, age of the human source of the egg at the time of egg harvesting).
Regarding Claim 19, the combination of Hall and Zhou teaches generating a ploidy outcome based on whether the embryo is euploid or aneuploid; and updating at least the fourth convolutional neural network based at least in part on the ploidy outcome and the data (Figs. 11-12; Col. 29-30 of Zhou; the machine learning module can generate a viability predication, with can include a prediction of euploidy or aneuploidy, a prediction of the likelihood of the embryo implanting, or the like… classifiers can be arranged into multiple levels, and in some embodiments, each of these multiple levels can include one or several classifiers. In some embodiments, a refining algorithm can be applied to the output of the last image classifier to further refine the classification of the image).
Regarding Claim 23, the combination of Hall and Zhou teaches receiving a plurality of single images, each single image depicting a respective embryo of a plurality of embryos, generating a viability score for each embryo by classifying each single image via the second convolutional neural network, and ranking the plurality of embryos based on the viability scores for the plurality of embryos (Figs. 11-12; Col. 29-30 of Zhou; the machine learning module can generate a viability predication, with can include a prediction of euploidy or aneuploidy, a prediction of the likelihood of the embryo implanting, or the like… classifiers can be arranged into multiple levels, and in some embodiments, each of these multiple levels can include one or several classifiers. In some embodiments, a refining algorithm can be applied to the output of the last image classifier to further refine the classification of the image… the outputted viability prediction can include the outputting of one or several categories into which one or several imaged embryos have been placed, the outputting of a ranking of one or several embryos relative to each other, which ranking can identify the embryos that are most likely to be viable or least likely to be viable).
Claims 20-22 are rejected under 35 U.S.C. 103 as being unpatentable over Hall in view of Zhou, and further in view of PCT Publication WO 2020/031185 A2 to Cnaani et al. (hereinafter "Cnaani", included in IDS provided by Applicant).
Regarding Claim 20, the combination of Hall and Zhou does not explicitly disclose that the embryo is to undergo at least one of biopsy and freezing, and wherein the method further comprises receiving the single image of the embryo prior to biopsy or freezing, and determining viability of the embryo prior to at least one of biopsy and freezing.
However, Cnaani teaches that an embryo is to undergo at least one of biopsy and freezing, and determining viability of the embryo prior to at least one of biopsy and freezing (Fig. 18; Claim 19; Pg. 14, ln. 28-32; Pg. 44, ln. 30-32 of Cnaani; scoring algorithm, is used to calculate a success probability of frozen embryos to survive after thawing… scoring algorithm is used to calculate a survival rate of a cryopreserved embryo to undergo freezing and thawing… machine learning algorithm, used for scoring oocytes and/or in-vitro grown embryos).
Therefore, at the time when the invention was filed, it would have been obvious to a person of ordinary skill in the art to include that the embryo is to undergo at least one of biopsy and freezing, and wherein the method further comprises receiving the single image of the embryo prior to biopsy or freezing, and determining viability of the embryo prior to at least one of biopsy and freezing using the teachings of Cnaani in order to modify the method taught by the combination of Hall and Zhou. The motivation to combine these analogous arts would have been to calculate a survival rate of a cryopreserved embryo to undergo freezing and thawing (Pg. 14, ln. 28-32of Cnaani).
Regarding Claim 21, the combination of Hall and Zhou does not explicitly disclose that the embryo has been frozen and thawed, and wherein the method further comprises receiving the single image of the embryo post-thaw, and determining viability of the embryo post-thaw via the second convolutional neural network.
However, Cnaani teaches that an embryo has been frozen and thawed and determining viability of the embryo post-thaw via a neural network (Fig. 18; Claim 19; Pg. 14, ln. 28-32; Pg. 44, ln. 30-32 of Cnaani; scoring algorithm, is used to calculate a success probability of frozen embryos to survive after thawing… scoring algorithm is used to calculate a survival rate of a cryopreserved embryo to undergo freezing and thawing… machine learning algorithm, used for scoring oocytes and/or in-vitro grown embryos).
Therefore, at the time when the invention was filed, it would have been obvious to a person of ordinary skill in the art to include that the embryo has been frozen and thawed, and wherein the method further comprises receiving the single image of the embryo post-thaw, and determining viability of the embryo post-thaw via the second convolutional neural network using the teachings of Cnaani in order to modify the method taught by the combination of Hall and Zhou. The motivation to combine these analogous arts would have been to calculate a survival rate of a cryopreserved embryo to undergo freezing and thawing (Pg. 14, ln. 28-32of Cnaani).
Regarding Claim 22, the combination of Hall, Zhou, and Cnaani teaches that determining viability of the embryo post-thaw comprises classifying the single image into either a first class indicating that the embryo has survived post-thaw, or a second class indicating that the embryo has not survived post-thaw (Fig. 18; Claim 19; Pg. 14, ln. 28-32; Pg. 44, ln. 30-32 of Cnaani; scoring algorithm, is used to calculate a success probability of frozen embryos to survive after thawing… scoring algorithm is used to calculate a survival rate of a cryopreserved embryo to undergo freezing and thawing… machine learning algorithm, used for scoring oocytes and/or in-vitro grown embryos).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABHISHEK SARMA whose telephone number is (571)272-9887. The examiner can normally be reached on Mon - Fri 8:00-5:00.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amr Awad can be reached on 571-272-7764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABHISHEK SARMA/
Primary Examiner, Art Unit 2621