DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/27/2026 has been entered.
Response to Amendment
The amendment filed on 02/27/2026 has been entered. Claims 1, 3-4, 7, 9, 16, 20 and 22 are amended. Claims 6 and 21 are canceled. No new claims are added. Claims 1, 3-5, 7-9, 11-16, 19-20 and 22-23 remain pending.
In the amended claims, the previously raised claim rejections under 35 U.S.C. 112(a) have been properly addressed.
Response to Arguments
35 U.S.C. 101 rejections
On Pages 9-11 of Remarks, Applicant argues that Claim 1 and its dependent claims are not directed toward an abstract idea under Step 2A (Prong I). On Page 9, final paragraph, Applicant argues that the claims “provide a specific method for using machine learning models to evaluate data that addresses a well-known technical problem in the medical field”. Specifically –
First, on Page 10, Applicant cites Example 39 in “2025 Update” and recites amended Claim 1 (Para 2), presumably for a comparison; further in Para 3, argues that “training a neural network is impractical to perform in human mind” (Para 3). Examiner respectfully disagrees. Example 39 in “2025 Update” “does not recite a judicial exception”, while the amended Claim 1 explicitly recites a judicial exception. Amended Claim 1, Line 6, recites “stochastic gradient descent”, which is specific mathematical calculation, like Example 47 in “2025 Update”. This was discussed in previous office action in Pages 6-7; “optimizing … via stochastic gradient descent” in Claim 7 … cover performance of the limitation in the mind, and/or mathematical calculations”.
Second, on Page 11, Para 1, Applicant argues that the limitations of “relative dimensions between one or more anatomical landmarks …” (Claim 1, Lines 11-12) and of “identifying … anatomical landmarks of the fetus absent … based on the corresponding confidence assessment” (Claim 1, Lines 18-20) are also impractical to perform in human mind. Examiner respectfully disagrees. First of all, the limitations of “relative dimensions between one or more anatomical landmarks …” and of “identifying anatomical landmarks of fetus absent … based on the corresponding confidence assessment” are NOT disclosed or suggested in Specification. For the limitation of “relative dimensions”, to a person of ordinary skill in the art (POSITA), e.g. a radiologist or an ultrasound technologist, it is their routine job to measure or quantify some dimensions based on anatomical landmarks in images, so to determine or estimate some relative dimensions based on anatomical landmarks would be achievable by mental process. For the limitation of “identifying”, a POSITA seeing a low or high confidence score of the estimated gestational age would be able to estimate with a relatively high precision what characteristic anatomical landmark might be absent or present in an image, by a mental process.
On Pages 11-12 of Remarks, Applicant argues that Claim 1 and its dependent claims integrate the alleged abstract idea into a practical application under Step 2A (Prong II). On Page 11, Applicant argues that the recited machine learning techniques provide improved performance, by “reducing scanning time” (Page 11) or improving model efficiency with automated ultrasound systems (Page 12). Examiner respectfully disagrees. Any application of computer-based image processing would more or less save some time, and reducing scanning time of “~7 minutes per scan”, even if it’s true, is not substantial. Furthermore, the price for “without the need … to acquire an image … in perfect position” could very possibly be lower confidence for the estimated result. Hence, simply reducing scanning time does not justify significant improvement of the claimed technique.
On Pages 12-13 of Remarks, Applicant argues that Claim 20 and its dependent claims are not directed toward an abstract idea under Step 2A (Prong I). On Page 12, Applicant argues that the claims “provide a specific method for using machine learning models to evaluate data that addresses a well-known technical problem in the medical field” (Para 3). Specifically -
First, on Pages 12-13, Applicant cites Example 39 in “2025 Update” and recites amended Claim 20, presumably for a comparison; further in Para 2 of Page 13, argues that “training a neural network is impractical to perform in human mind”. Examiner respectfully disagrees. Similar to the discussion above for Claim 1, the amended Claim 20 explicitly recites “stochastic gradient descent”, which is a specific mathematical calculation.
Second, on Page 13, Para 3, Applicant argues that the limitations of “filter … values to a predetermined threshold” (Claim 20, Lines 19-20) and of “selecting and aggregating … each estimation … into a final estimation …” (Claim 20, Lines 21-24) are also impractical to perform in human mind. Examiner respectfully disagrees. The limitation of “filter …” can be interpreted as comparing the values to a threshold, which can be performed as a mental process. The limitation of “selecting and aggregating …” is interpreted as collecting the estimates with high confidence and computing a confidence-weighted average, which can involve mental process and mathematical calculation.
On Pages 14-15 of Remarks, Applicant argues that Claim 20 and its dependent claims integrate the alleged abstract idea into a practical application under Step 2A (Prong II). On Page 14, Applicant argues that the recited machine learning techniques provide improved performance, by “reducing scanning time” (Para 2) or improving model efficiency with automated ultrasound systems (Para 3). Examiner respectfully disagrees. See our discussion for Claim 1 above.
Prior-art rejections
On Pages 15-16 of Remarks, Applicant argues that Balicki and the other cited references do not disclose the limitations of “receiving” (Claim 1, Lines 9-12) and “identifying” (Claim 1, Lines 18-20). The limitation of “receiving” is indefinite (see 112(b) below), and is interpreted as “receiving … from the deep learning model, CRL and a head circumference based on the first ultrasound image” and rejected by reference Balicki. The limitation of “identifying” is also taught by reference Balicki as discussed in section of 35 USC § 102.
On Pages 16-17 of Remarks, Applicant argues that Balicki and the other cited references do not disclose the limitations of “filter” and “selecting and aggregating” (Claim 20, Lines 19-24). As discussed in 35 USC § 103, the limitations are taught by references Balicki, Dickie and Gomes.
Claim Objections
Claims 1 and 20 are objected to because of the following informalities:
Claim 1, Lines 3-5, “receiving, by one or more processors, … via one or more ultrasound probes during a scanning session”, should be changed to “receiving during a scanning session, by one or more processors, … via one or more ultrasound probes”.
Claim 20, Lines 4-5, and Lines 8-9, recite “plurality of ultrasound image”, which should be changed to “plurality of ultrasound images”.
Claim 20, Line 21, recites “selecting and aggregating” should be changed to “select and aggregate”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1, 3-5, 7-9, 11-16, 19-20 and 22-23 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1, Lines 10-12, recites “the CRL and the head circumference determined via relative dimensions between one or more anatomical landmarks of the first ultrasound image captured in an arbitrary fetal view”, which is not disclosed in Specification.
Claim 1, Lines 18-20, recites “identifying … anatomical landmarks of the fetus absent … based on …”, which is not disclosed in Specification.
Claim 20, Line 24, recites “the aggregation comprises a confidence-weighted average of the selected estimations of the gestational age”, which is not disclosed in Specification.
Claim 22, Lines 2-3, recites “confidence weighted-average of the obtained estimation”, which is not disclosed in Specification.
Claims 3-5, 7-9, 11-16, 19 and 23 are also rejected under 35 U.S.C. 112(a) because they inherit the deficiencies of the claim(s) they respectively depend upon.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 3-5, 7-9, 11-16, 19-20 and 22-23 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1, Lines 9-12, recites “receiving … from the deep learning model, CRL and a head circumference based on the first ultrasound image, the CRL and the head circumference determined via relative dimension between …”. It is unclear whether the CRL and the head circumference are determined by the “deep learning model”, OR by the “relative dimension between …”. For present purposes of examination, the recited phrase is interpreted to be “receiving … from the deep learning model, CRL and a head circumference based on the first ultrasound image”.
Claim 8 recites “loss function”. To one of ordinary skill in the field of deep learning or artificial neural networks, loss function is only used in the process of model or network training, and is NOT used in implementing a trained model or network. For present purposes of examination, the recited “loss function” is interpreted to refer to “the deep learning model having been trained”.
Claim 20, Lines 14-15 and Line 16, recite “each ultrasound image”. In Claim 20, ultrasound image or ultrasound images are recited multiple times, including Lines 4-5 and 8-9 “plurality of ultrasound image”, Line 9 “prior ultrasound images”, and Line 12 “representative ultrasound images”; therefore, it is unclear which of these the recited “each ultrasound image” refers to. For present purposes of examination, the recited “each ultrasound image” in Lines 14-16 is interpreted to refer to one of “the plurality of ultrasound image” in Lines 4-5 and 8-9.
Claims 3-5, 7, 9, 11-16, 19 and 22-23 are also rejected under 35 U.S.C. 112(b) because they inherit the indefiniteness of the claim(s) they respectively depend upon.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 7 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
Claim 7 recites the same limitations as Claim 1, Lines 5-6, “an artificial neural network having been trained via supervised learning with stochastic gradient descent”. Claim 7 contains terms of “optimizing parameters” and “minimise a loss function”, which are inherent in training artificial neural network using stochastic gradient descent.
Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-5, 7-9, 11-16, 19-20 and 22-23 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
With regard to Claims 1, 3-5, 7-9, 11-16 and 19:
Step 1: the claims are drawn to a method/process, one of the four statutory categories.
Step 2A, Prong One:
The claims recite the limitations of “an artificial neural network … trained via supervised learning with stochastic gradient descent …”, “the CRL and the head circumference determined via relative dimensions …”, “determining … an estimation of the gestational age … and a corresponding confidence assessment …”, “comparing … the corresponding confidence assessment to a predetermined threshold”, and “identifying … the one or more anatomical landmarks of fetus absent … based on the corresponding confidence assessment …” in Claim 1, “optimizing … via the stochastic gradient descent” in Claim 7, “producing a plurality of estimates … and filtering … to select only one or more estimates that meet a predetermined confidence value threshold” in Claim 11, “producing a plurality of estimates … and ranking … based on confidence value” in Claim 12, “arithmetically processing … by averaging the estimations” in Claim 16, and “calculating … from a fetal image obtained at any stage during gestation” in Claim 19. These limitations are, under their broadest reasonable interpretation, limitations that cover performance of the limitation in the mind, and/or mathematical calculations. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind or mathematical calculations, then it falls within the “Mental Processes” grouping or the “Mathematical Concepts” grouping of abstract ideas. Accordingly, the claims recite an abstract idea.
Step 2A, Prong Two:
This judicial exception is not integrated into a practical application. In particular, the claims recite additional elements – receiving ultrasound image or processed result, outputting result to a display device or an indication to user, using a trained deep learning model to analyze ultrasound image in Claim 1, training of a deep learning model in Claim 4, outputting a value range from a deep learning model or the training in Claim 5, configuring loss function to generate the estimation and confidence assessment in Claim 8, the artificial neural network being a convolutional neural network or a vision transformer in Claim 9, and indicating an operative to acquire more images in Claims 14-15. The above-listed additional elements of receiving data and of outputting result or indication are insignificant extra-solution activities. The above-listed additional elements of using and training deep learning model are recited in a very high level of generality so that they amount to no more than mere instructions to apply the exception using a generic approach of artificial intelligence. Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
Step 2B:
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements are either insignificant extra-solution activities, or a generic approach of applying artificial intelligence for image analysis, which cannot provide an inventive concept.
For the reasons set forth above, Claims 1, 3-5, 7-9, 11-16 and 19 are not patent eligible.
With regard to Claims 20, 22 and 23:
Step 1: the claims are drawn to a method/apparatus, one of the four statutory categories.
Step 2A, Prong One:
The claims recite the limitations of “an artificial neural network having been trained via supervised learning with stochastic gradient descent …”, “determine … an estimate of the gestational age … and a corresponding confidence assessment …” and “filter … the corresponding confidence assessment to a predetermined threshold”, and “selecting and aggregating … into a final estimation … wherein the aggregation comprises a confidence-weighted average …” in Claim 20, and “a confidence weighted-average of the obtained estimation …” in Claim 22, which are, under their broadest reasonable interpretation, limitations that cover performance of the limitation in the mind. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind or mathematical calculations, then it falls within the “Mental Processes” grouping or “Mathematical Concepts” grouping of abstract ideas. Accordingly, the claims recite an abstract idea.
Step 2A, Prong Two:
This judicial exception is not integrated into a practical application. In particular, the claims recite the additional elements – one or more ultrasound probes, one or more processors, acquiring and receiving ultrasound images, receiving processed result, outputting via a display device the processed result or indication in Claim 20, providing real-time feedback to the user in Claim 22, and implementing the processing on the ultrasound apparatus or on a separate device that receives the images in Claim 23. The ultrasound probes, the processors, the trained machine learning model, the separate device for image processing, and the display device are recited at a high-level of generality (i.e. as a generic ultrasound system to acquire ultrasound images, as a generic machine learning model or processing device to perform image-processing tasks) such that they amount no more than mere instructions to apply the exception using a generic ultrasound system, a generic processing method and a generic computer. The additional elements of providing real-time feedback to a user and display processed result are routinely used in clinical ultrasound scans, so can be regarded as insignificant extra-solution activities. Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
Step 2B:
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of providing real-time feedback to a user and display processed result are insignificant extra-solution activities. The additional elements of using a generic ultrasound system, a generic processing method and a generic computer amount no more than mere instructions to apply the exception using a generic ultrasound system, method and computer. Mere instructions to apply the exception using a generic ultrasound system, a generic processing method and a generic computer cannot provide an inventive concept.
For the reasons set forth above, Claims 20, 22 and 23 are not patent eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 3-4, 7-9, 11 and 13-15 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Balicki et al (US 20210177374 A1; hereafter Balicki).
With regard to Claim 1, Balicki discloses a method for estimating a gestational age of a fetus (Balicki, Para 0008; “In some embodiments, the method may further involve determining a gestational age and/or a weight estimate based on the measurement.”), the method comprising:
receiving, by one or more processors (a data processor 126, as shown in Fig. 2), a first ultrasound image (Balicki, Para 0020; “… generate a stream of discrete ultrasound image frames 124 from the ultrasound echoes 118 … The image frames 124 … can be communicated to a data processor 126 …”) for input into a deep learning model having been trained with prior ultrasound images obtained via one or more ultrasound probes (Balicki, Para 0024; “… neural network 128 may comprise a deep learning network trained, via imaging data …”; Para 0020; “the data processor 126 can also be coupled, communicatively or otherwise, to a database 127 configured to store various data types, including training data …”) during a scanning session (Balicki, Para 0025; “… the inputs 130 may include current fetal measurements and the corresponding ultrasound images 130a obtained in substantially real time during an ultrasound examination”), the deep learning model including an artificial neural network having been trained via supervised learning with stochastic gradient descent on a set of representative ultrasound images (Balicki, Para 0032; “neural network 128 may comprise a multilayer perceptron (MLP) network configured to perform supervised learning with stochastic dropout.”; Para 0035; “To train neural network 128, 144 and/or 148, training sets which include multiple instances of input arrays and output classifications may be presented to the training algorithm(s) of the neural network(s) (e.g., AlexNet training algorithm, as described by Krizhevsky, A., Sutskever, I. and Hinton, G. E. “ImageNet Classification with Deep Convolutional Neural Networks,” NIPS 2012 or its descendants).”. In the provided reference, “stochastic gradient descent” is disclosed in Page 6), the first ultrasound image comprising a first portion of the fetus (Balicki, Para 0020; “The region 116 may include a developing fetus, as shown, or a variety of other anatomical objects, such as the heart or the lungs …”);
receiving, by the one or more processors from the deep learning model, a crown rump length (CRL) and a head circumference based on the first ultrasound image (Balicki, Para 0025; “One or more of the aforementioned inputs 130 can be received by neural network 128, which is configured to analyze the inputs 130 and generate one or more outputs 132 based on the inputs. Such outputs 132 can include one or more fetal measurements …”; Para 0035; “Fetal measurements obtained via system 100 can include but are not limited to: crown-rump length, head circumference …”) , the CRL and the head circumference determined via relative dimensions between one or more anatomical landmarks of the first ultrasound image captured in an arbitrary fetal view (see section of 112b rejection for Examiner’s interpretation);
determining, by the one or more processors, an estimation of the gestational age of the fetus and a corresponding confidence assessment for the estimation of the gestational age (Balicki, Para 0025; “Such outputs 132 can include … a gestational age estimate and the associated confidence level 132b …”);
comparing, by the one or more processors, the corresponding confidence assessment to a predetermined threshold (Balicki, Para 0026; “To each output 132 of the neural network 128, a confidence threshold 134 may be applied by the data processor 126 to determine whether the quality of a given measurement is satisfactory, or whether re-measurement is necessary.”);
identifying, by the one or more processors, the one or more anatomical landmarks of the fetus absent in the first ultrasound image based on the corresponding confidence assessment being outside the predetermined threshold (Balicki, Para 0026; “… the thresholding result may be conveyed in the form of one or more notifications 140. For example, the data processor 126 can be configured to generate a “Retake Measurement” notification 136 for measurements that do not satisfy the threshold”. In Fig. 2, when “Confidence threshold 134” proceeds to the branch of “Failed”, the process goes to blocks “Retake measurement 136” and notifications 140. Block 140 receives output from block 144, i.e. an image classification network; Para 0027; “an image classification network 144, which may comprise a CNN, can be trained to determine whether a given ultrasound image contains the requisite anatomical landmarks for obtaining a particular measurement.”. In other words, if output of deep learning network 128 has confidence lower than a threshold, user would be notified with at least whether the image contains some anatomical landmarks); and
outputting, by the one or more processors via a display device, at least one of:
the estimation of the gestational age based on the corresponding confidence assessment being within the predetermined threshold (Balicki, Para 0026; “… an “All OK” notification 138 for measurements that do satisfy the threshold 134”), or
an indication, during the scanning session, to obtain a second ultrasound image of the first portion of the fetus, the indication generated based on the corresponding confidence assessment being outside the predetermined threshold (Balicki, Para 0026; “… the data processor 126 can be configured to generate a “Retake Measurement” notification 136 for measurements that do not satisfy the threshold … In addition or alternatively, data processor 126 can be configured to generate a report 142, which may include all or select measurements and associated confidence levels determined by neural network 128.”).
With regard to Claim 3, Balicki discloses the method of Claim 1, and further discloses wherein each fetus of the set of representative ultrasound images of fetuses has a known gestational age at the time of imaging (Balicki, Para 0032; “… neural network 128 may comprise a multilayer perceptron (MLP) network configured to perform supervised learning with stochastic dropout. … To train the MLP, medical expert annotations of various fetal images and/or measurements, along with the corresponding fetal outcomes, e.g., birth weight, normal birth, abnormal birth, can be used.” The disclosed “supervised learning” inherently uses the label or ground truth for the gestational age to be estimated.).
With regard to Claim 4, Balicki discloses the method of Claim 3, and further discloses wherein training of the deep learning model is configured to achieve association between each image and its corresponding gestational age value (Balicki, Para 0032; “… neural network 128 may comprise a multilayer perceptron (MLP) network configured to perform supervised learning with stochastic dropout.” The network 128 receives each image as input, and estimates GA value, so its learning is to necessarily achieve association between an image and its GA value).
With regard to Claim 7, Balicki discloses the method of Claim 1, and further discloses wherein the supervised learning method comprises optimizing parameters of the artificial neural network, via the stochastic gradient descent, to minimise a loss function (Balicki, Para 0035; “To train neural network 128, 144 and/or 148, training sets which include multiple instances of input arrays and output classifications may be presented to the training algorithm(s) of the neural network(s) (e.g., AlexNet training algorithm, as described by Krizhevsky, A., Sutskever, I. and Hinton, G. E. “ImageNet Classification with Deep Convolutional Neural Networks,” NIPS 2012 or its descendants).” In the disclosed reference of Krizhevsky et al, supervised learning via stochastic gradient descent is used to minimize a loss function (Page 6, section of “5 Details of learning”)).
With regard to Claim 8, Balicki discloses the method of Claim 7, and further discloses wherein the loss function is configured to generate the estimation and the corresponding confidence assessment (Balicki, Para 0025; “… neural network 128, which is configured to analyze the inputs 130 and generate one or more outputs 132 based on the inputs. Such outputs 132 can include … a gestational age estimate and the associated confidence level 132b …”.).
With regard to Claim 9, Balicki discloses the method of Claim 1, and further discloses wherein the artificial neural network is a convolutional neural network, a vision transformer, or a variant thereof (Balicki, Para 0017; “the present disclosure may utilize a neural network, for example a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), an autoencoder neural network, or the like”).
With regard to Claim 11, Balicki discloses the method of Claim 1, and further discloses wherein the method comprises producing a plurality of estimates with a plurality of corresponding confidence values (Balicki, Para 0025; “Such outputs 132 can include one or more fetal measurements and an associated confidence level 132a for each measurement, a gestational age estimate and the associated confidence level 132b, and a fetal weight estimate 132c.”), and filtering the plurality of estimates to select only one or more estimates that meet a predetermined confidence value threshold (Balicki, Para 0026; “the data processor 126 can be configured to generate a “Retake Measurement” notification 136 for measurements that do not satisfy the threshold, and an “All OK” notification 138 for measurements that do satisfy the threshold 134.”).
With regard to Claim 13, Balicki discloses the method of Claim 1, and further discloses wherein the method includes directing an operative to obtain one or more specific images of the fetus (Balicki, Para 0038; “… the user interface 160 may be interactive, receiving user input 166 indicating … confirmation that a measurement needs to be reacquired.”).
With regard to Claim 14, Balicki discloses the method of Claim 13, and further discloses wherein the operative is instructed to acquire images of a second portion of the fetus (Balicki, Para 0038; “In some examples, the input 166 may include an instruction to raise or lower threshold 134 or adjust one or more image acquisition settings.” Here the disclosure of adjusting image acquisition settings include acquiring images of a different portion of fetus.).
With regard to Claim 15, Balicki discloses the method of Claim 13, and further discloses wherein the operative is dynamically instructed, while scanning, to acquire additional images in response to the acquired images having wide confidence ranges associated with them (Balicki, Para 0037; “the user interface 160 can be configured to display the ultrasound images 162 in real time as an ultrasound scan is being performed, along with one or more notifications 140 … The notifications 140 can include measurements and associated confidence levels … the notifications 140 along with, in some embodiments, one or more instructions for guiding the user to re-acquire a particular measurement.”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Balicki, in view of Bunn et al (US 20220133260 A1; hereafter Bunn).
With regard to Claim 5, Balicki discloses the method of Claim 4, but does not explicitly and clearly disclose wherein at least one of the deep learning model or the training are configured to output a value range.
Bunn in the same field of endeavor discloses wherein at least one of the deep learning model or the training are configured to output a value range (Bunn, Para 0030; “Quantile regression, and the like, predict a range the actual answer will likely fall into, rather than merely a single value answer. In various embodiments, any regression system that predicts a range or ranges as opposed to a point value may be used in Image Analysis Logic 130” ). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Balicki, as suggested by Bunn, in order to output a value range for GA instead of a point value. One of ordinary skill in the art would have been motivated to make the modification for the benefit of preventing overfitting of trained data that could be imprecisely labelled (Bunn, Para 0030; “Use of a range as an estimation prevents overfitting of the data and is useful when ultrasound images used for training may be mislabeled.’’).
With regard to Claim 12, Balicki discloses the method of Claim 1, but does not explicitly and clearly disclose wherein the method comprises producing a plurality of estimates, with a plurality of corresponding confidence values, and ranking the plurality of estimates based on confidence value.
Bunn in the same field of endeavor discloses wherein the method comprises producing a plurality of estimates, with a plurality of corresponding confidence values, and ranking the plurality of estimates based on confidence value (Bunn, Para 0036; “A regression algorithm outputs one or more values for each percentile chosen. For example, some embodiments use 10%, 25%, 50%, 75%, 90% percentiles for outputs (which represent percentiles of a quantitative prediction), and each of these percentiles may be associated with a probability and/or a confidence measure.” Para 0039; “Calculation Logic 140 may apply a distribution function to the estimates made by Image Analysis Logic 130 and produce a probability distribution therefrom.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Balicki, as suggested by Bunn, in order to produce multiple estimates for GA and rank them based on their confidence values. One of ordinary skill in the art would have been motivated to make the modification for the benefit of selecting a subset of predictions with high confidence and/or combining this subset to determine a final prediction with high confidence (Bunn, Para 0113; “multiple ultrasound sessions are commonly performed on a single pregnancy. Multiple sessions from a single pregnancy could also be combined when performing a prediction”).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Balicki, in view of Gomes et al (arXiv:2203.10139 (2022); hereafter Gomes).
With regard to Claim 16, Balicki discloses the method of Claim 1, but does not explicitly and clearly disclose further comprising arithmetically processing each estimation of the gestational age obtained from a plurality of images by averaging the estimations.
Gomes in the same field of endeavor discloses comprising arithmetically processing each estimation of gestational age obtained from a plurality of images by averaging the estimations (Gomes, Page 24, Para 4; “Predictions for all video clips in each case were aggregated together to generate a single prediction for the case. The gestational age model uses an inverse variance weighting procedure to combine the clip-level predictions into a case-level mean gestational age”. Here the disclosure of “inverse variance weighting procedure” is one type of averaging). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Balicki, as suggested by Gomes, in order to combine gestational age estimates from multiple images by averaging them. One of ordinary skill in the art would have been motivated to make the modification for the benefit of increased estimation precision of gestational age by averaging multiple estimates and therefore reducing the effect of noise.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Balicki, in view of Lee et al (arXiv:2203.11903 (2022); hereafter Lee).
With regard to Claim 19, Balicki discloses the method of Claim 1, including calculating at least one of a gestational age estimate or a confidence value from a fetal image. Balicki does not explicitly and clearly disclose obtaining images at any stage during gestation.
Lee in the same field of endeavor discloses obtaining images at any stage during gestation (Lee, Page 4, Para 1; “Among study visits conducted by sonographers, 63 (9.3%) women had at least one visit during the first trimester, 235 (34.7%) women had at least one visit during the second trimester, and 379 (56.0%) had one or more visits in the third trimester.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Balicki, as suggested by Lee, in order to apply the method to a fetal image obtained at any stage of gestation. One of ordinary skill in the art would have been motivated to make the modification for the benefit of accurately estimating GA for fetus at all stages (e.g. trimesters) of pregnancy. Conventional method of estimating GA based on fetal biometric measurements may not be reliable as expected, as such measurements correlate less with GA as pregnancy progresses (Lee, Page 3, Para 2; “while fetal biometric measurements were generally reproducible, there was increased variance later in pregnancy”; Para 3; “The accuracy and efficiency of biometric measurements is dependent on the skill and experience of the sonographer”).
Claims 20 and 22-23 are rejected under 35 U.S.C. 103 as being unpatentable over Balicki, in view of Dickie et al (US 20220047241 A1; hereafter Dickie) and Gomes.
With regard to Claim 20, Balicki discloses an ultrasound apparatus for estimating a gestational age of a fetus (Balicki, Para 0020; “The ultrasound system 100 …”), the ultrasound apparatus comprising:
an ultrasound image capturing device comprising one or more ultrasound probes (Balicki, Para 0020; “The ultrasound data acquisition unit 110 can include an ultrasound probe …”), the ultrasound image capturing device configured to receive a plurality of ultrasound image comprising a portion of the fetus (Balicki, Para 0020; “The region 116 may include a developing fetus, as shown, or a variety of other anatomical objects, such as the heart or the lungs … the ultrasound data acquisition unit 110 can include a beamformer 120 and a signal processor 122, which can be configured to generate a stream of discrete ultrasound image frames 124”), and
an electronic processing device comprising one or more processors (Balicki, Para 0020; “a data processor 126, e.g., a computational module or circuitry …”), the electronic processing device configured to:
receive, by the one or more processors, the plurality of ultrasound image (Balicki, Para 0020; “The image frames 124 … can be communicated to a data processor 126 …”) for input into a deep learning model having been trained with prior ultrasound images obtained via the one or more ultrasound probes (Balicki, Para 0024; “… neural network 128 may comprise a deep learning network trained, via imaging data …”; Para 0020; “the data processor 126 can also be coupled, communicatively or otherwise, to a database 127 configured to store various data types, including training data …”) , the deep learning model including an artificial neural network having been trained via supervised learning with stochastic gradient descent on a set of representative ultrasound images (Balicki, Para 0032; “neural network 128 may comprise a multilayer perceptron (MLP) network configured to perform supervised learning with stochastic dropout.”; Para 0035; “To train neural network 128, 144 and/or 148, training sets which include multiple instances of input arrays and output classifications may be presented to the training algorithm(s) of the neural network(s) (e.g., AlexNet training algorithm, as described by Krizhevsky, A., Sutskever, I. and Hinton, G. E. “ImageNet Classification with Deep Convolutional Neural Networks,” NIPS 2012 or its descendants).”. In the provided reference, “stochastic gradient descent” is disclosed in Page 6);
receive, by the one or more processors from the deep learning model, a crown rump length (CRL) and a head circumference based on each ultrasound image (Balicki, Para 0025; “One or more of the aforementioned inputs 130 can be received by neural network 128, which is configured to analyze the inputs 130 and generate one or more outputs 132 based on the inputs. Such outputs 132 can include one or more fetal measurements …”; Para 0035; “Fetal measurements obtained via system 100 can include but are not limited to: crown-rump length, head circumference …”);
determine, by the one or more processors, an estimation of the gestational age of the fetus and corresponding confidence assessment value in real-time (Balicki, Para 0025; “Such outputs 132 can include … a gestational age estimate and the associated confidence level 132b …” Para 0024; “… providing a user with a real-time evaluation of measurement accuracy and in some examples, indicating whether one or more measurements should be re-acquired.”);
filter, by the one or more processors, the corresponding confidence assessment values to a predetermined threshold (Balicki, Para 0026; “the data processor 126 can be configured to generate a “Retake Measurement” notification 136 for measurements that do not satisfy the threshold, and an “All OK” notification 138 for measurements that do satisfy the threshold 134.”); and
output, by the one or more processors via a display device (Balicki, Para 0037; “The results … can be displayed to a user via one or more components of system 100.”), at least one of the final estimation of the gestational age based on the corresponding confidence assessment values being within the predetermined threshold or an indication to obtain a second ultrasound image of the portion of the fetus based on the corresponding confidence assessment values being outside the predetermined threshold (Balicki, Para 0026; “… the data processor 126 can be configured to generate a “Retake Measurement” notification 136 for measurements that do not satisfy the threshold, and an “All OK” notification 138 for measurements that do satisfy the threshold 134. … In addition or alternatively, data processor 126 can be configured to generate a report 142, which may include all or select measurements and associated confidence levels determined by neural network 128.”).
Balicki does not clearly and explicitly disclose processing each of the received ultrasound images using a deep learning model, selecting only those estimates with confidence within predetermined threshold, or aggregating multiple estimates into a final estimate using confidence-weighted averaging.
Dickie in the same field of endeavor discloses processing each of the received ultrasound images using a deep learning model (Dickie, Para 0028; “The ultrasound frames may be processed against an artificial intelligence (AI) model to predict a suitable cut line on each of the ultrasound frames”), and selecting only those estimates with confidence within predetermined threshold (Dickie, Para 0069; “… the ultrasound frames for which the confidence level is above a threshold may then be used to generate the 3D fetal representation, with the other ultrasound frames for which the confidence level is below the threshold being discarded or ignored …”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Balicki, as suggested by Dickie, in order to use deep learning model to process each of the received ultrasound images and use only the estimates with high confidence for further analysis. One of ordinary skill in the art would have been motivated to make the modification for the benefit of improved accuracy for the result by processing each of the available images and utilizing only those estimates with high confidence (Dickie, Para 0071; “the resultant 3D fetal representation may less likely include non-fetal anatomy and/or cut off portions of the fetal anatomy than if the full set of ultrasound frames were used.”).
Balicki and Dickie do not clearly and explicitly disclose aggregating multiple estimates into a final estimate using confidence-weighted averaging.
Gomes in the same field of endeavor discloses aggregating multiple estimates into a final estimate using confidence-weighted averaging (Gomes, Page 24, Para 4; “We used the model’s prediction on the final frame of the sequence as the single prediction for the clip. Predictions for all video clips in each case were aggregated together to generate a single prediction for the case. The gestational age model uses an inverse variance weighting procedure 36 to combine the clip-level predictions xc, into a case-level mean gestational age x using estimated variance σc of each clip …”. The disclosed “inverse variance” corresponds to confidence of Application). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Balicki and Dickie, as suggested by Gomes, in order to combine multiple estimates using a confidence-weighted averaging method. One of ordinary skill in the art would have been motivated to make the modification for the benefit of improved accuracy by applying more weight to individual estimates with more confidence.
With regard to Claim 22, Balicki, Dickie and Gomes disclose the ultrasound apparatus of Claim 20, including obtaining a confidence weighted-average of the obtained estimation of gestational age (see discussion of Claim 20 above). Balicki further discloses wherein the ultrasound apparatus is configured to provide real-time feedback to a user on confidence of the obtained estimation of gestational age to direct scanning (Balicki, Para 0037; “the user interface 160 can be configured to display the ultrasound images 162 in real time as an ultrasound scan is being performed, along with one or more notifications 140 … The notifications 140 can include measurements and associated confidence levels … the notifications 140 along with, in some embodiments, one or more instructions for guiding the user to re-acquire a particular measurement.”).
With regard to Claim 23, Balicki, Dickie and Gomes disclose the ultrasound apparatus of Claim 20. Balicki further discloses wherein the electronic processing device is implemented directly on the ultrasound apparatus (Balicki, Para 1 shows that the ultrasound system 100 contains the data processor 126.) or on a separate device which either receives a video feed from the ultrasound apparatus (Balicki, Para 0039; “Various portable devices, e.g., laptops, tablets, smart phones, remote displays and interfaces, or the like, may be used to implement one or more functions of the system 100. Some or all of the data processing may be performed remotely, (e.g., in the cloud). In examples that incorporate such devices, the ultrasound sensor array 112 may be connectable via a USB interface”) or captures a copy of one or more of the ultrasound images via a camera of the separate device.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEI ZHANG whose telephone number is (571)272-7172. The examiner can normally be reached Monday-Friday 8am-5pm E.T..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at (571) 272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/L.Z./ Examiner, Art Unit 3798
/PASCAL M BUI PHO/ Supervisory Patent Examiner, Art Unit 3798
Prosecution Insights