DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of claims 1-13 in the reply filed on 12/04/2025 is acknowledged.
Claims 14-20 cancelled.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
Claims 1, 3-5, 7-13 are rejected under 35 U.S.C. 102(a1) as being anticipated by Levy et al. (Levy) (US 11,869,188 B1).
Regarding claim 1, Levy discloses an ultrasound system (e.g., an exemplary system depicted in FIG. 1, the system may include a conventional ultrasound system 10, paragraph 6) comprising:
an ultrasound scanner configured to transmit ultrasound at a patient anatomy and generate ultrasound data as part of an ultrasound examination (e.g., in a fetal ultrasound examination conducted in accordance with the principles of the present invention, following review of the real-time ultrasound motion video clips generated by the ultrasound scanner 10 as displayed on display computer 20, the clinician then may review the analysis results generated and returned by the interpretative component residing on server computer 30 , paragraph 18);
an ultrasound machine configured to generate, based on the ultrasound data, an ultrasound image (e.g., In this manner, the clinician may review the contents of display 50 of FIG. 2, review the selected raw image data corresponding to each standard guideline view (by clicking on the checkboxes in column 52), and review the detailed machine learning analysis of that selected image frame by clicking on the labels in column 53, paragraph 18); and
a processor system (e.g., FIG. 3, exemplary flowchart 60 for the interpretative component of the analysis software is described. At step 61, motion video clips are received from ultrasound system 10 or display computer 20, paragraph 19) implemented to:
determine features selected from the group consisting of regional features, patient features (e.g., ne the interpretive component has identified and selected an image frame from an uploaded motion video clip as representative of the 3VT view, the machine learning feature will analyze the selected image frame for features identified in Table 3 as being visible in the 3VT view: aorta greater than pulmonary artery, associated with coarctation of the aorta and conotruncal lesions; right aortic arch, associated with conotruncal lesions; abnormal vessel alignment, associated with transposition of the great arteries; and additional visible vessel, associated with anomalous pulmonary venous connection, paragraph 11), clinician features (e.g., , paragraph 12), and ultrasound examination features (e.g., Server system 30 includes the interpretive component of the inventive system, including machine learning algorithms for analyzing the motion video clips received from display compute 20 to compare the ultrasound video clips to a set of the preferred image templates that correspond to the fetal ultrasound examination guidelines, paragraph 10); and
generate, based on the ultrasound image and the features, a patient recommendation (e.g., The selected image frames and analytical results then are transmitted back to display computer 20 for presentation to, and consideration by, the clinician. As clinicians often have multiple patients, the clinician may be sent or may otherwise be tasked with reviewing results from several patients. To facilitate efficient review by the clinician and/or expert, the system may automatically organize the results with the most relevant information, such as detected morphological abnormalities, appearing first or otherwise most prominently. Additionally, or alternatively, the results may be organized by patient in order of severity, paragraph 12).
Regarding claim 3, Levy discloses wherein at least one of the features is unavailable during the ultrasound examination (e.g., The absence of a checkbox in column 52 indicates that the interpretative component was unable to locate an image in the motion video clips suitable for analysis by the machine learning feature. Clicking on the empty checkbox, for example, for RVOT in FIG. 2, may be configured to display a prompt to the clinician to rescan a portion of the patient's abdomen to acquire a new motion video clip containing the desired view, which then may be sent to server computer 30 for supplemental analysis, paragraph 16).
Regarding claim 4, Levy discloses wherein the processor system is implemented to generate, based on the ultrasound image and the features, an audit report that compares the ultrasound examination and an additional ultrasound examination (e.g., If the interpretative component adjudges that a corresponding frame is available in the received motion video clip, the process moves to step 66, where the selected image frames, and optionally non-selected image frames, are analyzed by another machine learning algorithm to detect the presence of an abnormality associated with that standard view. For example, if the selected image frame corresponds to the 4C standard view template, the algorithm will analyze the selected frame for the presence of any of the defects listed in Table 3 for that standard view. If a defect is detected in the selected image frame, the algorithm may look at adjacent frames of the video clip to confirm the presence of the same defect, paragraph 20).
Regarding claim 5, Levy discloses wherein the patient recommendation includes at least one of an additional examination, a clinical trial, and a medication (e.g., In accordance with another aspect of the invention, the analysis results returned to the user interface component may be displayed and further annotated by the clinician to include additional graphical indicia or textual remarks. The resulting analysis results and annotations may be stored for later referral to an expert to develop a plan for further diagnosis or treatment, paragraph 24).
Regarding claim 7, Levy discloses further comprising a display device and a synchronizer circuit, wherein the ultrasound machine is implemented to generate a video clip that includes the ultrasound image (e.g., In a preferred embodiment, the inventive system employs two components: a user interface component that provides a clinician tools to analyze and review fetal ultrasound images and ultrasound motion video clips, paragraph 22), wherein the processor system is implemented to obtain an additional video clip of ultrasound images, wherein the synchronizer circuit is implemented to synchronize the video clip and the additional video clip and the display device is implemented to simultaneously and synchronously display the video clip and the additional video clip (e.g., and a machine learning interpretative component that receives ultrasound motion video clips and images from a conventional fetal ultrasound screening system, identifies images within the motion video clips that correspond to fetal ultrasound screening guidelines. The interpretative component also analyzes the identified images to detect and identify the presence of morphological abnormalities, and provides that information to the user interface component to highlight such abnormalities for the clinician's review, paragraph 22).
Regarding claim 8, Levy discloses further comprising a display device, wherein the processor system is implemented to:
obtain an additional ultrasound image generated during a previous ultrasound examination; and generate a first segmentation of the patient anatomy from the ultrasound image and a second segmentation of the patient anatomy from the additional ultrasound image, wherein the display device is implemented to display the first segmentation and the second segmentation (e.g., The present invention is directed to systems and methods for conducting fetal ultrasound examinations that aids in the detection of critical heart defects during a second semester ultrasound exam. The inventive systems and methods help trained and qualified physicians to interpret ultrasound recording motion video clips by identifying standard views appearing within motion video clips. In addition, the systems and methods of the present invention may assist in detecting and identifying morphological abnormalities that might be indicative of critical CHDs. For example, Table 3 provides an exemplary correspondence between representative CHDs, the views in which those CHDs usually appear, and the morphological abnormalities that typically can be identified in those views, paragraph 20).
Regarding claim 9, Levy discloses further comprising a display device implemented to display a user interface configured to: display a value measured from the patient anatomy in the ultrasound image; receive a user selection of the value; and display, responsive to the user selection, the ultrasound image (e.g., The interpretative component preferably resides on server computer 30, receives ultrasound motion video clips and images from ultrasound system 10 or display computer 20, and uses machine learning algorithms to identify images within the motion video clips that correspond to fetal ultrasound screening guidelines. The interpretative component also analyses the identified images as well as any non-identified images (e.g., corresponding to non-standard or non-recommended views) to detect and identify the presence of morphological abnormalities, and provides that information to the user interface component to highlight such abnormalities for the clinician's review, paragraph 7).
Regarding claim 10, Levy discloses wherein the features include the regional features, and the regional features include at least one of a population size, a population density, a pollution level, a number of airports, a zip code, and a regional news event (e.g., In addition, a fetal ultrasound screening examination typically generates thousands of image frames spanning multiple structures per single video “sweep,” so the diagnostic frames of interest for CHD may be only a handful and thus are easily missed. Moreover, the prevalence of CHD in the population (˜0.8-1%) is low enough that non-experts see it only rarely and may discount or overlook abnormal images. Together, these factors make CHD detection one of the most difficult diagnostic challenges in ultrasound, with a dramatic impact on post-natal outcomes and quality of life, paragraph 15).
Regarding claim 11, Levy discloses wherein the features include the patient features, and the patient features include at least one of a patient age, a patient weight, a symptom description, patient genomic data, and a patient medical history (e.g., FIG. 1, the system may include a conventional ultrasound system 10, display computer 20, and server system 30 that communicate with each other via wide area network 40, illustratively, the Internet. In a preferred embodiment, the systems and methods are embodied in a computer assisted diagnostic aid for use in two-dimensional fetal ultrasound exams, such as usually performed during the second trimester of pregnancy. Machine learning algorithms are employed to assist users with the identification and interpretation of standard views in fetal cardiac ultrasound motion video clips, paragraph 6).
Regarding claim 12, Levy discloses wherein the features include the clinician features, and the clinician features include at least one of a clinician age, an education level, a number of ultrasound examinations performed, and an amount of time since a vacation (e.g., The images, video clips, analysis, results, annotations, and/or report may be shared with or otherwise made available to an expert or clinician (e.g., upon referral to an expert or clinician). Each type of morphological abnormality may be associated with an expert or clinician and their contact information. If a morphological abnormality is detected at step 66, an expert or clinician corresponding to the morphological abnormality may optionally be recommended, paragraph 23).
Regarding claim 13, Levy discloses wherein the features include the ultrasound examination features, and the ultrasound examination features include at least one of cleaning data for the ultrasound system, calibration data for the ultrasound system, radio frequency measurements obtained during the ultrasound examination, indicators of equipment proximate to the ultrasound system during the ultrasound examination, weather during the ultrasound examination, a time of day of the ultrasound examination, and a day of week of the ultrasound examination (e.g., systems and methods for conducting fetal ultrasound examinations that aids in the detection of critical heart defects during a second semester ultrasound exam. The inventive systems and methods help trained and qualified physicians to interpret ultrasound recording motion video clips by identifying standard views appearing within motion video clips. In addition, the systems and methods of the present invention may assist in detecting and identifying morphological abnormalities that might be indicative of critical CHDs. For example, Table 3 provides an exemplary correspondence between representative CHDs, the views in which those CHDs usually appear, and the morphological abnormalities that typically can be identified in those views, paragraph 20).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over by Levy et al. (Levy) (US 11,869,188 B1) as applied to claim 1 above, and further in view of Voznyuk et al. (Voznyuk) (EP 3964136 A1).
Regarding claim 2, Levy does not specifically disclose wherein the processor system implements: a neural network to generate, based on the ultrasound image, an inference for the patient anatomy; and an additional neural network to generate the patient recommendation based on the inference.
Voznyuk discloses wherein the processor system implements: a neural network to generate, based on the ultrasound image, an inference for the patient anatomy; and an additional neural network to generate the patient recommendation based on the inference (e.g., European Patent Application Publication EP 3964136 to Voznyuk et al. describes a machine learning system that analyzes ultrasound images generated during an examination, uses a first convolutional neural network (CNN) to compare acquired images to views required by those guidelines, and a second CNN to analyze the images to identify potential abnormalities (see Levy reference, paragraph 13).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention to have modified Levy to include wherein the processor system implements: a neural network to generate, based on the ultrasound image, an inference for the patient anatomy; and an additional neural network to generate the patient recommendation based on the inference as taught by Voznyuk. It would have been obvious to one of ordinary skill in the art at the time of the invention to have modified Levy by the teaching of Voznyuk to use for particular application.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over by Levy et al. (Levy) (US 11,869,188 B1) as applied to claim 1 above, and further in view of Ciofolo-Veit et al. (Ciofolo) (US 2021/0345987 A1).
Regarding claim 6, Levy does not specifically disclose further comprising a display device, wherein the processor system is implemented to obtain an additional ultrasound image generated during an additional ultrasound examination, wherein the display device is implemented to simultaneously display the ultrasound image and the additional ultrasound image.
Ciofolo discloses further comprising a display device, wherein the processor system is implemented to obtain an additional ultrasound image generated during an additional ultrasound examination, wherein the display device is implemented to simultaneously display the ultrasound image and the additional ultrasound image (e.g., an ultrasound imaging system that uses machine learning algorithms to analyze acquired images to detect anomalous features, and if an anomalous feature is detected, uses machine learning algorithms to determine and display other previously-acquired ultrasound images that provide complementary views of the potential anomalous feature to permit improved diagnosis, paragraph 14, background, Levy).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention to have modified Levy to include further comprising a display device, wherein the processor system is implemented to obtain an additional ultrasound image generated during an additional ultrasound examination, wherein the display device is implemented to simultaneously display the ultrasound image and the additional ultrasound image as taught by Ciofolo. It would have been obvious to one of ordinary skill in the art at the time of the invention to have modified Levy by the teaching of Ciofolo to obtain improved diagnosis.
ConclusionAny inquiry concerning this communication or earlier communications from the examiner should be directed to QUANG N VO whose telephone number is (571)270-1121. The examiner can normally be reached Monday-Friday, 7AM-4PM, EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Abderrahim Merouan can be reached at (571)270-5254. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QUANG N VO/Primary Examiner, Art Unit 2683