DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is in response to Applicant’s remarks, filed on 6/25/2025. The amendments to claim(s) 1 and 16-17 have been entered. Claim(s) 6 and 10 is/are cancelled by Applicant and therefore withdrawn from further consideration pursuant to 37 CFR 1.142(b). Corresponding rejections to claim 10 in the prior office action have been withdrawn as moot in view of the applicant’s cancellation. Accordingly, claim(s) 1-5, 7-9 and 11-19 remain pending for examination on the merits.
Response to Arguments
Applicant’s arguments, see p. 9-11, with respect to the rejection of claim(s) 1-5, 7-9 and 11-19 have been fully considered. Regarding the rejection(s) under 35 U.S.C. § 112, Examiner respectfully disagrees with the remarks and does not find Applicant’s arguments persuasive. The 35 U.S.C. § 112(b) rejections from the prior office action have been withdrawn and in view of Applicant’s amendments, and new 35 U.S.C. § 112(b) rejections are issued. Applicant's arguments regarding the rejections under 35 U.S.C. § 102 rejections in the prior office action have been fully considered but they are not persuasive. New grounds of rejection are made in view of the following: new amendments provided by Applicant and attached remarks; updated search and review of pertinent, eligible prior art; and/or different interpretation of the previously applied references. Regarding the rejection of claim(s) 1-5, 7-9 and 11-19 under 35 U.S.C. § 102, Applicant provides the following:
Claim Rejections - 35 U.S.C. § 102 Claims 1-5 and 7-19 were rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as allegedly being anticipated by Voigt et al. (US2015/0366532 Al).
Claim 1 as amended further clarifies the inputs and outputs of the learning model. Claim 1 as amended features data for B-mode image generation which does not include blood flow information is input into the machine-learning model to acquire one frame of data including estimated blood flow information.
Voigt discloses that a model for reconstructing anatomical structure of a cardiac valve is provided and that B-mode data and Color Doppler data are used to adjust the model for B-mode data to determine the position of the regurgitant orifice and/or jet.
The process performed by claim 1 as amended differs from a process performed by the model of the Voigt reference.
Examiner respectfully disagrees with Applicant. First, the Applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references. Applicant's arguments do not comply with 37 CFR 1.111(c) because they do not clearly point out the patentable novelty which he or she thinks the claims present in view of the state of the art disclosed by the references cited or the objections made. Further, they do not show how the amendments avoid such references or objections.
Furthermore, Applicant has provided no evidence to establish an unobvious difference between the claimed product and the prior art, but rather has merely argued such alleged difference. Mere arguments cannot take the place of evidence. In re Walters, 168 F.2d 79,80, 77 USPQ 609,610 (CCPA 1948); In re Cole, 326 F.2d. 769,773, 140 USPQ 230,233 (CCPA 1964); In re Schulze, 346 F.2d 600,602, 145 USPQ 716,718 (CCPA 1965); In re Lindner, 457 F.2d 506,508, 173 USPQ 356,358 (CCPA 1972); In re Pearson, 494 F.2d 1399,1405, 181 USPQ 641,646 (CCPA 1974); Meitzner v. Mindick, 549 F.2d 775,782, 193 USPQ 17,22 (CCPA), cert. Denied, 434 U.S. 854 (1977); In re DeBlauwe, 736 F.2d 699,705, 222 USPQ 191,196 (Fed. Cir. 1984).
As noted above, the claims remain rejected under 35 U.S.C. §112(b) for indefiniteness, and under 35 U.S.C. §103 under Voigt as discussed below. The broadest reasonable interpretation of the claim language in view of the instant specification is applied during examination. As discussed in the interview 6/23/2025, Examiner respectfully maintains that Voigt teaches the limitations of the independent claims. In particular, the model and classifiers may be trained using B-mode and color Doppler data, and when provided B-mode data without Doppler information generate flow information [see claim 1 rejection]. In particular, Voigt teaches the following:
“Any machine training may be used for one or more stages. The machine-trained classifier is any one or more classifiers. A single class or binary classifier, collection of different classifiers, cascaded classifiers, hierarchal classifier, multi-class classifier, model-based classifier, classifier based on machine learning, or combinations thereof may be used.”
[0052]
“Any of the input features discussed herein may be used, such as using both B-mode and flow mode features. In one embodiment, the input features from fluid response are not used. The input features from B-mode data are used without flow data features. The features for locations within the global region or bounding box are used and features for other locations are not used.”
[0064]
“an image or quantity is output. The processor outputs to a display, […] The image is from the acquired scan data and/or from the fit model. For example, a combination image is generated where B-mode data shows tissue, flow data indicates fluid, and a mesh, labels, points, coloration, brightness, or lines from the fit model highlight the valve.”
[0094]
“Alternatively or additionally, the image displays a value or other quantification. Any of the quantities calculated in act 48, quantities derived from the fitted model, and/or quantities for the regurgitant jet are displayed. For example, clinical biomarkers, such as PISA and EROA, are displayed. Since the segmentation and/or sampling are performed over time, one or more quantities of dynamic function may be output, such as velocity time integral or regurgitant volume.”
[0102]
As provided above, Voigt provides a model and machine-leant classifier’s which may receive B-mode ultrasound data and generate flow data. 3D B-Mode imaging is not able to capture regurgitant orifice area, color Doppler imaging is used to find the location of regurgitant jets. Rendering both 3D B-Mode and color Doppler flow images together still may not show the regurgitant orifice since the color overlay often obscures the view onto the valve (Voigt [0006]). Accurate anatomical models result from using both B-mode and color Doppler flow data for anatomy delineation, accurate flow quantification may use sampling planes derived from valve anatomy of the model, and detection of regurgitant area on the valve is enhanced by leveraging anatomical models in addition to multi-channel features. 3D+time biomarker quantification (e.g., velocity time integral (VTI) and/or regurgitant volume) is enabled by tracking the regurgitant orifice over time (Voigt [0021]).
Examiner respectfully notes that Applicant’s arguments only address independent claim(s) 1, 16 and 17, and no remarks regarding the subject matter of the dependent claim(s) have been presented. Accordingly, the rejections to dependent claims 2-5, 7-9, 11-15 and 18-19 are modified to address Applicant’s amendments and the new rejection to independent claim(s) 1, 16 and 17 and are sustained. The rejections of claim(s) 1-5, 7-9 and 11-19 under 35 U.S.C. § 102 are maintained.
Claim Objections
Claim 16 is objected to because of the following informalities:
The recitation of “A learning apparatus performing machine learning of a learning model to be used for generating estimated data by an ultrasonic diagnostic apparatus including an ultrasonic probe configured to transmit and receive ultrasonic waves to and from an object for generating estimated data by inputting third data into a model,” in the preamble uses the redundant language of “generating estimated data”, which may be clarified if one instance of the phrase was removed or amended. Appropriate correction is required.
Claim Rejections - 35 USC § 112
35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 1-5, 7-9 and 11-19 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claims 2-5, 7-9, 11-15 and 18-19 are also rejected at least by virtue of dependency upon a rejected base claim.
Regarding claim 1, the limitation(s): “the model has been machine-learned using learning data including pairs […] wherein the model has been machine-learned so that, when data based on a plurality of frames of received signals for B-mode image generation that do not include blood flow information is input into the model, the model estimates one frame of data including blood flow information, and […] wherein the model has been machine-learned so that, when data based on a plurality of frames”, renders the claim indefinite. Similarly, in claim 16 the limitation(s) “wherein the model has been machine-learned using learning data including pairs […] wherein the model has been machine-learned so that, when data based on a plurality of frames of received signals for B-mode generation that do not include blood flow information is input into the model, […] perform machine learning of the learning model by using learning data that includes data, based on a received signal of a reflected ultrasonic wave obtained from the observation region, as input data and blood flow information,”; and in claim 17 the limitation(s) “wherein the model has been machine-learned by one or more processors using learning data including pairs […] wherein the model has been machine-learned so that, when data based on a plurality of frames ” are also rejected, mutatis mutandis, under the same rationale as applied to claim 1. There is insufficient antecedent basis for “an observation region” as recited in the claims. The use of the “an observation region” language in both the ‘first data’ and ‘third data’ clauses results in multiple interpretations of the ‘observation region’, including different or the same ‘region’.
Similarly, the claims do not clearly define how the “model” is trained (what particular processing steps are performed) and generate “estimated data”, failing to distinguish the input ‘third data’ from input ‘data based on a plurality of frames’ acquired. The claim recites multiple instances of training the model and receiving inputs to generate different ‘estimated data’, which renders the timing of the machine-learning process unclear relative to the acquisition of input data and output step(s). It is suggested to clarify the ‘data’ being input to the ‘model’ and ‘estimated data’ output from the ‘model’, and couple processing steps where appropriate. For the purposes of examination, the broadest reasonable interpretation of the claim language includes the same or different input and output ‘data’, and training of the model occurs prior to the input of ‘data’ and output of ‘estimated data’. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-5, 7-9 and 11-19 is/are rejected under pre-AIA 35 U.S.C. 103 as being unpatentable over Voigt et al. (US2015/0366532 A1, 2015-12-24; hereinafter “Voigt”).
Regarding claim 1, Voigt teaches an ultrasonic diagnostic apparatus (“A system for detecting a regurgitant region, the system comprising: an ultrasound scanner configured to scan a heart volume of a patient, the scan providing B-mode and Doppler flow data;” [clm 21]; “The system includes a transducer 18, an ultrasound scanner 10, and a display 16.” [0105]; The system uses ultrasound data to detect a regurgitant orifice, fitting an anatomical model to the valve from the ultrasound data [0007-0011, 0105-0126], [fig. 1-6]), comprising:
an ultrasonic probe configured to transmit and receive ultrasonic waves to and from an object (“an ultrasound scanner configured to scan a heart volume of a patient, the scan providing B-mode and Doppler flow data;” [clm 21]; “The ultrasound scanner 10 uses the transducer 18 to scan a heart volume of a patient.” [0108]; The ultrasound scanner comprising an array of elements (i.e., ultrasound probe) scans a patient during the ultrasound imaging procedure, e.g., transthoracic echocardiography, transesophageal echocardiography, or along the skin surface [0023-0033, 0105-0126], [fig. 1-2, 6]);
at least one memory storing a program (“The ultrasound scanner 10 includes a B-mode detector 20, a flow estimator 22, a processor 12, and a memory 14.” [0105]; “The memory 14 is a buffer, cache, RAM, removable media, hard drive, magnetic, optical, database, or other now known or later developed memory.” [0110]; The memory stores ultrasound data representing a heart or valve volume, and stores flow (e.g., velocity, energy, both) and B-mode ultrasound data [0105-0126], [fig. 6]); and
one or more processors (“a processor” [clm 21]; “The ultrasound scanner 10 includes a B-mode detector 20, a flow estimator 22, a processor 12, and a memory 14.” [0105]; “The processor 12 operates pursuant to stored instructions to perform various acts […] such as controlling scanning, calculating features, detecting anatomy, measuring anatomy, and/or controlling imaging.” [0114]; Processor(s) may control and perform various functions (e.g., detection, computation, segmentation, and fitting acts) of the ultrasound system [0023-0045, 0105-0126], [fig. 1-2, 6]) which, by executing the program, cause the ultrasonic diagnostic apparatus to:
generate estimated data by inputting third data into a model (“Based on the extracted multi-channel features, a discriminative classifier is then trained through a boosting process to detect the valve and/or regurgitant orifice. […] a statistical shape model is further integrated into the framework. This model is used to assist in regurgitant orifice detection, dynamic quantification, and/or segmentation.” [0024]; “A given feature value may be derived from just B-mode or just flow data. […] In other embodiments, only B-mode or only flow mode data is used.” [0042]; “Any of the input features discussed herein may be used, such as using both B-mode and flow mode features. In one embodiment, the input features from fluid response are not used. The input features from B-mode data are used without flow data features.” [0064]; “A patient-specific valve model is fit to the input data to visualize the valve anatomy and to assist therapy planning and procedure simulation. A model is fit to the detected anatomy of the specific patient, so that the fitting causes the model to be patient-specific.” [0067]; “Any model of the valve may be used, such as a theoretical or programmed model.” [0068]; “The classifier may include a plurality of models or classifiers (e.g., detectors) operable together or independently” [0116]; A model based on machine-learnt classifier(s) (i.e., model) may be trained to receive input features from B-mode data without flow data features (i.e., inputting third data) and output detected anatomy, dynamic quantification (i.e., generate estimated data) and segmentation [0037-0093, 0105-0126], [fig. 1-6]),
wherein the model has been machine-learned using learning data including pairs of first data as input data and second data as correct answer data, wherein the first data is based on a plurality of frames of received signals for B-mode image generation that are obtained from an observation region and that do not include blood flow information, and the second data is based on one frame of blood flow information acquired from the observation region by using a color Doppler method (“a processor configured to fit a model of a heart valve over time to the B-mode data using the B-mode data and the Doppler flow data, and use the model to locate a regurgitant region over time; and” [clm 21]; “To detect the orifice, learning-based methods are applied using the multi-channel image features derived from both B-mode and color Doppler flow. The processor detects the regurgitant orifice with a machine-learnt classifier” [0080]; “Any classifier may be applied, such as a model-based classifier […] The classifier is instructions, a matrix, a learned code, or other software and/or hardware for distinguishing between information in a medical image.” [0115]; The model and machine-learnt classifiers may be trained using both B-mode (i.e., first data) and color flow Doppler flow (i.e., second data) images of the cardiac anatomy (i.e., observation region) to detect a valve/orifice and estimate flow data [0037-0093, 0105-0126], [fig. 1-6]), and
wherein the model has been machine-learned so that, when data based on a plurality of frames of received signals for B-mode image generation that do not include blood flow information is input into the model, the model estimates one frame of data including blood flow information (“fluid response to the acoustic energy is estimated. Flow data representing the fluid in the cardiac region is estimated” [0035]; “Any machine training may be used for one or more stages. The machine-trained classifier is any one or more classifiers. A single class or binary classifier, collection of different classifiers, cascaded classifiers, hierarchal classifier, multi-class classifier, model-based classifier, classifier based on machine learning, or combinations thereof may be used.” [0052]; “One or more machine-learnt classifiers are used to identify the anatomic structure or structures. […] different points are sequentially classified.” [0063]; “the input features from fluid response are not used. The input features from B-mode data are used without flow data features. The features for locations within the global region or bounding box are used and features for other locations are not used.” [0064]; “Any process for segmentation may be used. The regurgitant orifice may be associated with a velocity. Any velocities within a range are treated as part of the jet. […] an iso-velocity region is segmented based on the color Doppler data.” [0088]; “The valve anatomy is detected over time (i.e., at each time) or detected once or periodically and tracked the rest of the time. The valve model is fit to detected and/or tracked anatomy” [0092]; After training the classifier(s) the model may receive input data from B-mode data without flow data features and detect valve anatomy, estimate fluid response and flow data (i.e., blood flow information) over time [0037-0093], [fig. 1-6]), and
wherein the third data is based on a plurality of frames of received signals for B-mode image generation that are obtained from an observation region of an object by using the ultrasonic probe and that do not include blood flow information (“A given feature value may be derived from just B-mode or just flow data. […] In other embodiments, only B-mode or only flow mode data is used.” [0042]; “Any of the input features discussed herein may be used, such as using both B-mode and flow mode features. In one embodiment, the input features from fluid response are not used. The input features from B-mode data are used without flow data features.” [0064]; Input features may be from B-mode data without flow data (i.e., third data) and input to the classifier(s) and model [0037-0093, 0105-0126], [fig. 1-6]),
the estimated data is one frame of data including estimated blood flow information about the observation region of the object (“an image or quantity is output. The processor outputs to a display, […] The image is from the acquired scan data and/or from the fit model. For example, a combination image is generated where B-mode data shows tissue, flow data indicates fluid, and a mesh, labels, points, coloration, brightness, or lines from the fit model highlight the valve.” [0094]; “Since the segmentation and/or sampling are performed over time, one or more quantities of dynamic function may be output, such as velocity time integral or regurgitant volume.” [0102]; “The image or quantity is output based on the refined model.” [0104]; “the processor 12 is configured to locate the valve, fit a model to the valve, and use the fit model for further sampling, quantification, and/or regurgitant orifice detection. The detection of the valve anatomy and/or the fitting of the model are performed over time, providing a fit model for each sampled time through all or part of a heart cycle. […] A classifier may be applied, but the input features and/or locations classified are a function of the fit model for that time.” [0121]; A model and classifiers are trained using B-mode and color Doppler data of the patient cardiac region to output flow data (i.e., estimate blood flow information) using input features from B-mode images of the patient without the flow data [0037-0093, 0105-0126], [fig. 1-6]).
Regarding claim 2, Voigt teaches the ultrasonic diagnostic apparatus according to claim 1,
Voigt further teaching wherein the third data includes a received signal obtained by scanning the observation region in order to generate a B-mode image or B-mode image data based on the received signal (“One or more machine-learnt classifiers are used to identify the anatomic structure or structures. […] different points are sequentially classified. The global valve region is searched point by point. For each point, input features for the point and surrounding points are used to classify the probability that the point is part of the anatomy.” [0063]; “In one embodiment, the input features from fluid response are not used. The input features from B-mode data are used without flow data features.” [0064]; “A patient-specific valve model is fit to the input data to visualize the valve anatomy and to assist therapy planning and procedure simulation. A model is fit to the detected anatomy of the specific patient, so that the fitting causes the model to be patient-specific.” [0067]; The classifier may be trained using B-mode and flow data from the same cardiac region [0037-0093, 0105-0126], [fig. 1-6], [see claim 1 rejection]).
Regarding claim 3, Voigt teaches the ultrasonic diagnostic apparatus according to claim 1,
Voigt further teaching wherein the third data includes a received signal obtained by transmitting a plane wave or a diffuse wave or image data based on the received signal (“The scanning transmits acoustic energy. In response to the transmissions, acoustic echoes are received. […] For rapid volume scanning, plane wave or broad transmit beams are formed. Multiple, such as 4, 8, 16, 32, 64, or other number, of receive beams are formed in response to each transmit beam.” [0032]; “In other embodiments, the system is a workstation, computer, or server for detecting using data acquired by a separate system in real-time or using previously acquired patient-specific data stored in a memory” [0105]; “The ultrasound scanner 10 uses the transducer 18 to scan a heart volume of a patient. Electrical and/or mechanical steering allows transmission and reception along different scan lines in the volume. […] In another embodiment, a plane, collimated or diverging transmit waveform is provided for reception along a plurality, large number (e.g., 16-64 receive beams), or all scan lines.” [0108]; Ultrasound data from plane wave beams may be acquired, wherein received ultrasound signals may be stored in memory and used to supplement real-time ultrasound data acquisition [0037-0093, 0105-0126], [fig. 1-6], [see claim 1 rejection]).
Regarding claim 4, Voigt teaches the ultrasonic diagnostic apparatus according to claim 2,
Voigt further teaching wherein the third data includes a plurality of received signals of a reflected ultrasonic wave obtained by scanning the observation region a plurality of times or image data based on the plurality of received signals (“Any format for scanning may be used, such as linear, sector, Vector®, or other format. The distribution of scan lines is in three-dimensions to scan a volume of the cardiac region.” [0031]; “The ultrasound scanner 10 uses the transducer 18 to scan a heart volume of a patient. Electrical and/or mechanical steering allows transmission and reception along different scan lines in the volume. Any scan pattern may be used. For example, a plurality of different planes through the heart is scanned by rocking an array or volume scanning with a matrix array. […] In another embodiment, a plane, collimated or diverging transmit waveform is provided for reception along a plurality, large number (e.g., 16-64 receive beams), or all scan lines.” [0108]; The ultrasound transducer may perform a scan of a three-dimensional volume of the cardiac region, comprising a distribution of scan lines corresponding to a plurality of different planes through the heart [0105-0126], [fig. 1-6]).
Regarding claim 5, Voigt teaches the ultrasonic diagnostic apparatus according to claim 1,
Voigt further teaching wherein the third data includes a part of the received signals for B-mode image generation obtained by performing transmission and reception of an ultrasonic wave a plurality of times on each of a plurality of scan lines of the observation region in order to acquire blood flow information of the observation region, or image data based on the part of received signals (“Any format for scanning may be used, such as linear, sector, Vector®, or other format. The distribution of scan lines is in three-dimensions to scan a volume of the cardiac region.” [0031]; “The ultrasound scanner 10 uses the transducer 18 to scan a heart volume of a patient. Electrical and/or mechanical steering allows transmission and reception along different scan lines in the volume. Any scan pattern may be used. For example, a plurality of different planes through the heart is scanned by rocking an array or volume scanning with a matrix array. […] In another embodiment, a plane, collimated or diverging transmit waveform is provided for reception along a plurality, large number (e.g., 16-64 receive beams), or all scan lines.” [0108]; The ultrasound transducer may perform a scan of a three-dimensional volume of the cardiac region, comprising a distribution of scan lines corresponding to a plurality of different planes through the heart [0105-0126], [fig. 1-6], [see claim 2, 4 rejections]).
Regarding claim 7, Voigt teaches the ultrasonic diagnostic apparatus according to claim 1,
Voigt further teaching wherein the model includes a plurality of models corresponding to a plurality of blood flow velocities (“Different features may be used for different classifiers. The tissue ultrasound features derived from B-mode data and the flow ultrasound features derived from Doppler data (e.g., derived from velocity) are used to locate the valve. Some features may be more determinative of location, rotation, and/or scale than others.” [0049]; “One or more machine-learnt classifiers are used to identify the anatomic structure or structures. Any of the classifiers discussed above, but trained for locating specific or groups of specific anatomic structure of the valve may be used.” [0063]; “Potential aliasing can occur due to blood velocities above the Doppler sampling limit given by the ultrasound probe. […] Automatic dealiasing based on machine learning may also be performed, where aliased signal is automatically detected using image features, and the sampling scale automatically shifted based on the detected aliasing area.” [0077]; “The regurgitant orifice may be associated with a velocity. Any velocities within a range are treated as part of the jet. The resulting spatial distribution may be filtered.” [0088]; Machine-learnt classifiers (i.e., a plurality of models) may be trained to locate specific anatomic structures, wherein machine learning is performed to prevent aliasing due to blood velocity by shifting sampling scale (e.g., respective to classifiers) when detecting a valve [0037-0093, 0105-0126], [fig. 1-6]).
Regarding claim 8, Voigt teaches the ultrasonic diagnostic apparatus according to claim 1, Voigt further teaching wherein the one or more processors cause the ultrasonic diagnostic apparatus to:
extract blood flow information from received signals of a reflected ultrasonic wave obtained by performing transmission/reception of an ultrasonic wave a plurality of times on each of a plurality of scan lines of the observation region and generates Doppler image data based on the blood flow information (“an ultrasound scanner configured to scan a heart volume of a patient, the scan providing B-mode and Doppler flow data; a processor configured to fit a model of a heart valve over time to the B-mode data using the B-mode data and the Doppler flow data,” [clm 21]; “the tissue response to the acoustic energy is detected. The receive beamformed samples are processed to represent the intensity of the echoes from the location. […] In alternative embodiments, Doppler tissue imaging or other mode is used to detect the tissue response.” [0034]; “Electrical and/or mechanical steering allows transmission and reception along different scan lines in the volume. Any scan pattern may be used. For example, a plurality of different planes through the heart is scanned by rocking an array or volume scanning with a matrix array. […] In another embodiment, a plane, collimated or diverging transmit waveform is provided for reception along a plurality, large number (e.g., 16-64 receive beams), or all scan lines.” [0108]; Doppler flow data may be acquired using the ultrasound probe for a plurality of scan lines comprising the volume [0105-0126], [fig. 1-6], [see claim 1, 5 rejections]).
Regarding claim 9, Voigt teaches the ultrasonic diagnostic apparatus according to claim 8,
Voigt further teaching wherein the third data includes a part of received signals for generating the Doppler image data (“One or more machine-learnt classifiers are used to identify the anatomic structure or structures. […] different points are sequentially classified. The global valve region is searched point by point. For each point, input features for the point and surrounding points are used to classify the probability that the point is part of the anatomy.” [0063]; “Any of the input features discussed herein may be used, such as using both B-mode and flow mode features.” [0064]; “The scan may be repeated to provide data for the volume at different times. Ultrasound data representing a volume is provided in response to the scanning. […] The ultrasound data may be of any type, such as B-mode, flow mode (e.g., Doppler mode),” [0109]; The input features may use ultrasound data of any type including Doppler mode [0105-0126], [fig. 1-6], [see claim 1, 5 rejections]).
Regarding claim 11, Voigt teaches the ultrasonic diagnostic apparatus according to claim 8, Voigt further teaching wherein the one or more processors cause the ultrasonic diagnostic apparatus to:
perform control of a display image to be output to a display apparatus, and a display mode in which the display image is updated based on the Doppler image data, instead of based on data estimated and a display mode in which the display image is updated based on the Doppler image data and the estimated data (“a display configured to generate a visualization of the model over time and highlight the regurgitant region without displaying a regurgitant jet.” [clm 21]; “The processor outputs to a display, printer, memory, or network. The image is from the acquired scan data and/or from the fit model. For example, a combination image is generated where B-mode data shows tissue, flow data indicates fluid, and a mesh, labels, points, coloration, brightness, or lines from the fit model highlight the valve.” [0094]; “A sequence of images may be displayed to represent the valve over time. The sequence is rendered from the different volumes throughout a portion of (e.g., simulating closure) or entire heart cycle.” [0100]; A dynamic patient-specific model is determined using both B-Mode and color Doppler, wherein a sequence of images may be presented [0037-0093, 0105-0126], [fig. 1-6], [see claim 10 rejection]).
Regarding claim 12, Voigt teaches the ultrasonic diagnostic apparatus according to claim 11,
Voigt further teaching wherein in the display mode in which the display image is updated based on the Doppler image data and the estimated data, after updating the display image based on the Doppler image data, repeatedly performs processing of updating the display image a prescribed number of times consecutively based on the estimated data (“a processor configured to fit a model of a heart valve over time to the B-mode data using the B-mode data and the Doppler flow data, and use the model to locate a regurgitant region over time; and a display configured to generate a visualization of the model over time and highlight the regurgitant region without displaying a regurgitant jet.” [clm 21]; “A sequence of images may be displayed to represent the valve over time. The sequence is rendered from the different volumes throughout a portion of (e.g., simulating closure) or entire heart cycle.” [0100]; The processor fits a model to B-mode data over time based on acquired B-mode and Doppler flow data (i.e., a prescribed number of times), and the display may present the combination image comprising B-mode image and model overlay (i.e., estimated data) [0037-0093, 0105-0126], [fig. 1-6], [see claim 1, 10 rejections]).
Regarding claim 13, Voigt teaches the ultrasonic diagnostic apparatus according to claim 12, Voigt further teaching wherein the one or more processors cause the ultrasonic diagnostic apparatus to:
change the prescribed number of times in accordance with an input from a user (“The acts are performed in real-time, such as during ultrasound scanning of act 30. The user may view images of act 50 while scanning in act 30 to acquire another dataset representing the cardiac volume. […] Measurements and/or images of automatically detected anatomy may be provided in seconds, such as ten or fewer seconds. Alternatively, the acts are performed as desired by a surgeon regardless of whether a patient is currently at the facility or being scanned.” [0027]; “In alternative embodiments, a semi-automatic process is used where the user confirms or guides the process by indicating one or more locations and/or indicating changes due to proposed treatment.” [0029]; A semi-automatic process may be used wherein the processor performs functions under user guidance to indicate changes (i.e., prescribed number) to the procedure [0023-0045, 0105-0126], [fig. 1-2, 6]).
Regarding claim 14, Voigt teaches the ultrasonic diagnostic apparatus according to claim 11, Voigt further teaching wherein the one or more processors cause the ultrasonic diagnostic apparatus to:
save, when receiving an instruction to save an image from a user, both of or one of the Doppler image data having been acquired at a timing closest to a timing at which the instruction has been received and the estimated data (“The memory 14 stores the ultrasound data, such as ultrasound data representing a heart or valve volume. […] The memory 14 stores flow (e.g., velocity, energy or both) and/or B-mode ultrasound data.” [0111]; “The memory 14 is additionally or alternatively a non-transitory computer readable storage medium with processing instructions. The memory 14 stores data representing instructions executable by the programmed processor 12 for detecting a regurgitant orifice.” [0113]; The displayed images are updated in real-time during ultrasound imaging for a user to view, wherein the user may store information to memory [0105-0126], [fig. 1-2, 6], [see claim 1 rejection]).
Regarding claim 15, Voigt teaches the ultrasonic diagnostic apparatus according to claim 8, Voigt further teaching wherein the one or more processors cause the ultrasonic diagnostic apparatus to:
perform control of a display image to be output to a display apparatus, perform control to display, side by side, an image based on the Doppler image data and an image based on the estimated data (“a display configured to generate a visualization of the model over time and highlight the regurgitant region without displaying a regurgitant jet.” [clm 21]; “The processor outputs to a display, […] a combination image is generated where B-mode data shows tissue, flow data indicates fluid, and a mesh, labels, points, coloration, brightness, or lines from the fit model highlight the valve.” [0094]; “an image of the fit model and the regurgitant point is displayed without overlay of the regurgitant jet. The flow associated with the regurgitant jet is clipped or cropped, such as not including any flow between the model and the viewer. The regurgitant jet is not shown in a same representation with the heart valve, at least for part of the display screen. The regurgitant jet with or without the model may be separately rendered for another part of the display screen.” [0095]; The image of the model may be separately rendered for another part of the display with the combination image, wherein different information and highlighting may be overlain/presented [0094-0126], [fig. 1-6], [see claim 1, 8 rejection]).
Regarding claim 16, Voigt teaches a learning apparatus performing machine learning of a learning model to be used for generating estimated data by an ultrasonic diagnostic apparatus (“A system for detecting a regurgitant region, the system comprising: an ultrasound scanner configured to scan a heart volume of a patient, the scan providing B-mode and Doppler flow data; a processor configured to fit a model of a heart valve over time” [clm 21]; “The system includes a transducer 18, an ultrasound scanner 10, and a display 16.” [0105]; “The processor 12 performs machine learning and/or applies a machine-learnt algorithm.” [0115]; [0105-0126], [fig. 1-6], [see claim 1 rejection]), including
an ultrasonic probe configured to transmit and receive ultrasonic waves to and from an object for generating estimated data by inputting third data into a model (“an ultrasound scanner configured to scan a heart volume of a patient, the scan providing B-mode and Doppler flow data;” [clm 21]; “The ultrasound scanner 10 uses the transducer 18 to scan a heart volume of a patient.” [0108]; [0023-0033, 0105-0126], [fig. 1-2, 6], [see claim 1 rejection]),
wherein the model has been machine-learned using learning data including pairs of first data as input data and second data as correct answer data (“a processor configured to fit a model of a heart valve over time to the B-mode data using the B-mode data and the Doppler flow data, and use the model to locate a regurgitant region over time; and” [clm 21]; “To detect the orifice, learning-based methods are applied using the multi-channel image features derived from both B-mode and color Doppler flow. The processor detects the regurgitant orifice with a machine-learnt classifier” [0080]; “Any classifier may be applied, such as a model-based classifier […] The classifier is instructions, a matrix, a learned code, or other software and/or hardware for distinguishing between information in a medical image.” [0115]; [0037-0093, 0105-0126], [fig. 1-6], [see claim 1 rejection]),
wherein the first data is based on a plurality of frames of received signals for B-mode image generation that are obtained from an observation region and that do not include blood flow information, and the second data is based on one frame of blood flow information acquired from the observation region by using a color Doppler method (“a processor configured to fit a model of a heart valve over time to the B-mode data using the B-mode data and the Doppler flow data, and use the model to locate a regurgitant region over time; and” [clm 21]; “To detect the orifice, learning-based methods are applied using the multi-channel image features derived from both B-mode and color Doppler flow. The processor detects the regurgitant orifice with a machine-learnt classifier” [0080]; “Any classifier may be applied, such as a model-based classifier […] The classifier is instructions, a matrix, a learned code, or other software and/or hardware for distinguishing between information in a medical image.” [0115]; [0037-0093, 0105-0126], [fig. 1-6], [see claim 1 rejection]),
wherein the model has been machine-learned so that, when data based on a plurality of frames of received signals for B-mode generation that do not include blood flow information is input into the model, the model estimates one frame of data including blood flow information (“fluid response to the acoustic energy is estimated. Flow data representing the fluid in the cardiac region is estimated” [0035]; “Any machine training may be used for one or more stages. The machine-trained classifier is any one or more classifiers. A single class or binary classifier, collection of different classifiers, cascaded classifiers, hierarchal classifier, multi-class classifier, model-based classifier, classifier based on machine learning, or combinations thereof may be used.” [0052]; “One or more machine-learnt classifiers are used to identify the anatomic structure or structures. […] different points are sequentially classified.” [0063]; “the input features from fluid response are not used. The input features from B-mode data are used without flow data features. The features for locations within the global region or bounding box are used and features for other locations are not used.” [0064]; “Any process for segmentation may be used. The regurgitant orifice may be associated with a velocity. Any velocities within a range are treated as part of the jet. […] an iso-velocity region is segmented based on the color Doppler data.” [0088]; “The valve anatomy is detected over time (i.e., at each time) or detected once or periodically and tracked the rest of the time. The valve model is fit to detected and/or tracked anatomy” [0092]; [0037-0093], [fig. 1-6], [see claim 1 rejection]), and
wherein the third data is based on a plurality of frames of received signals for B-mode image generation that are obtained from an observation region of an object by using the ultrasonic probe and that do not include blood flow information (“A given feature value may be derived from just B-mode or just flow data. […] In other embodiments, only B-mode or only flow mode data is used.” [0042]; “Any of the input features discussed herein may be used, such as using both B-mode and flow mode features. In one embodiment, the input features from fluid response are not used. The input features from B-mode data are used without flow data features.” [0064]; [0037-0093, 0105-0126], [fig. 1-6], [see claim 1 rejection]),
the estimated data is one frame of data including estimated blood flow information about the observation region of the object (“an image or quantity is output. The processor outputs to a display, […] The image is from the acquired scan data and/or from the fit model. For example, a combination image is generated where B-mode data shows tissue, flow data indicates fluid, and a mesh, labels, points, coloration, brightness, or lines from the fit model highlight the valve.” [0094]; “Since the segmentation and/or sampling are performed over time, one or more quantities of dynamic function may be output, such as velocity time integral or regurgitant volume.” [0102]; “The image or quantity is output based on the refined model.” [0104]; “the processor 12 is configured to locate the valve, fit a model to the valve, and use the fit model for further sampling, quantification, and/or regurgitant orifice detection. The detection of the valve anatomy and/or the fitting of the model are performed over time, providing a fit model for each sampled time through all or part of a heart cycle. […] A classifier may be applied, but the input features and/or locations classified are a function of the fit model for that time.” [0121]; [0037-0093, 0105-0126], [fig. 1-6], [see claim 1 rejection]),
the learning apparatus comprising:
at least one memory storing a program (“The ultrasound scanner 10 includes a B-mode detector 20, a flow estimator 22, a processor 12, and a memory 14.” [0105]; “The memory 14 is a buffer, cache, RAM, removable media, hard drive, magnetic, optical, database, or other now known or later developed memory.” [0110]; [0105-0126], [fig. 6,], [see claim 1 rejection]); and
one or more processors (“a processor” [clm 21]; “The ultrasound scanner 10 includes a B-mode detector 20, a flow estimator 22, a processor 12, and a memory 14.” [0105]; “The processor 12 operates pursuant to stored instructions to perform various acts […] such as controlling scanning, calculating features, detecting anatomy, measuring anatomy, and/or controlling imaging.” [0114]; [0023-0045, 0105-0126], [fig. 1-2, 6], [see claim 1 rejection]) which, by executing the program, causing the learning apparatus to:
perform machine learning of the learning model by using learning data that includes data, based on a received signal of a reflected ultrasonic wave obtained from the observation region, as input data and blood flow information, extracted from a reflected ultrasonic wave obtained by scanning the observation region a plurality of times, as the correct answer data (“fluid response to the acoustic energy is estimated. Flow data representing the fluid in the cardiac region is estimated” [0035]; “Any machine training may be used for one or more stages. The machine-trained classifier is any one or more classifiers. A single class or binary classifier, collection of different classifiers, cascaded classifiers, hierarchal classifier, multi-class classifier, model-based classifier, classifier based on machine learning, or combinations thereof may be used.” [0052]; “One or more machine-learnt classifiers are used to identify the anatomic structure or structures. […] different points are sequentially classified.” [0063]; “the input features from fluid response are not used. The input features from B-mode data are used without flow data features. The features for locations within the global region or bounding box are used and features for other locations are not used.” [0064]; “Any process for segmentation may be used. The regurgitant orifice may be associated with a velocity. Any velocities within a range are treated as part of the jet. […] an iso-velocity region is segmented based on the color Doppler data.” [0088]; “The valve anatomy is detected over time (i.e., at each time) or detected once or periodically and tracked the rest of the time. The valve model is fit to detected and/or tracked anatomy” [0092]; After training the classifier(s) the model may receive input data from B-mode data without flow data features and detect valve anatomy, estimate fluid response and flow data (i.e., blood flow information) over time [0037-0093], [fig. 1-6], [see claim 1 rejection]).
Regarding claim 17, Voigt teaches an image processing method (“A method for detecting a regurgitant point in echocardiography,” [clm 1]; [0007-0011, 0105-0126], [fig. 1-6], [see claim 1 rejection]) comprising:
a receiving step of transmitting an ultrasonic wave to an object and receiving a reflected ultrasonic wave from the object by using an ultrasonic probe (“detecting, by a processor, a valve with a first machine-learnt classifier using input first features from both B-mode